gSla
V1.0.0
下载
功能介绍
    gSla是一款轻量SLA计算工具,只需要1台主机简单5步操作就能轻松搞定SLA,让你随时掌握线上服务质量。

功能:
1、提供以下SLA指标计算:
  - QPS
  - ResponseTime(P95/P99/P999/Avg)
  - ErrorRate
  - 服务可用性
2、可以按服务名、URI进行查询SLA指标。

    本工具已在生产环境使用,业务nginx产生的日志数量10亿左右,每秒产生约25000条日志,经过gSla计算之后的数据保存到influxDB,通过grafana轻松展示SLA相关指标如,QPS、ResponseTime、ErrorRate、服务可用性。(gSla主机配置:4核8G SSD云主机,cpu使用率低于10%,内存使用量500M,influxDB存储数据每天<200M)
版本信息
文件名: gSla-v1.0.0.tar.gz

MD5: 80f14ca72825813e6a022470ca7a8175

下载次数:1
使用说明
1、架构图
2、配置

1)kafka 配置(kafka安装略)

创建kafka topic:
									./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 3 --topic sla-service-log
							    

2)、业务日志收集

将业务日志收集写入到kafka topic,数据格式为json,必须包含project、source、lang、access_time、remote_addr、request_uri、body_bytes_sent、status、upstream_response_time字段。

nginx示例:

									http {
										...
									
										log_format test_log_json '{"access_time":"$time_local", '
											'"project":"test", '
											'"source":"nginx", '
											'"lang":"go", '
											'"remote_addr":"$remote_addr", '
											'"request_uri":"$uri", '
											'"referer":"$http_referer", '
											'"request":"$request", '
											'"status":$status, '
											'"body_bytes_sent":$body_bytes_sent, '
											'"agent":"$http_user_agent", '
											'"x_forwarded":"$http_x_forwarded_for", '
											'"up_addr":"$upstream_addr",'
											'"up_host":"$upstream_http_host",'
											'"upstream_response_time":"$upstream_response_time",'
											'"request_time":"$request_time"'
										'}';
										...
									}
									server{
										...
										access_log  logs/test_access.log test_log_json;
										...
									}
								

通过filebeat收集nginx日志,filebeat配置示例:

							    	filebeat.inputs:
							    	- type: log
							    	  enable: true
							    	  paths:
							    	    - /var/log/nginx/test_access.log
							    	  tail_files: true
							    	output.kafka:
							    	  enabled: true
							    	  hosts: ["192.168.56.101:9092"]
							    	  topic: sla-service-log
							    	  partition.round_robin:
							    	    reachable_only: true
							    	  worker: 2
							    	  required_acks: 1
							    	  compression: gzip
							    	  max_message_bytes: 10000000
							    	  bulk_max_size: 10240
							    	  keep_alive: 60
							    	  channel_buffer_size: 2560
							    	  version: 2.1
							    	  codec.format:
							    	    string: '%{[message]}'
							    	logging.level: warning
							    	logging.to_files: true
							    	logging.files:
							    	  path: /data/log/filebeat/logs
							    	  name: filebeat.log
							    	  keepfiles: 3
							    	  permissions: 0644
							    
启动fileteat: ./filebeat -e -c filebeat.yml

3)、配置gSla

config.yml
									log:
									  level: "info"
									  logFile: "./logs/sla.log"
									  rotationHours: 24
									  maxAgeHours: 72
									sla:
									  tumblingWindow: 15
									kafka:
									  addr: "192.168.56.101:9092"
									  version: "2.1.1"
									  topic: "sla-service-log"
									  parallelism: 2
									influxdb:
									  url: "http://192.168.56.101:8086"
									  username: "sre"
									  password: "sre"
									  timeoutSec: 15
								

4)、influxDB配置(influxDB安装略)

							    	influx> CREATE DATABASE "sli"
							    	influx> CREATE USER "sre" WITH PASSWORD 'sre'
							    	influx> GRANT ALL ON sli TO sre
							    

5)、grafana配置

a)创建influxDB数据源

configure->database->Add data source 选择InfluxDB:
							    	HTTP
							    		URL:http://192.168.56.101:8086
							    		Access:Server (Default)
							    	InfluxDB Details
							    		Database:sli
							    		User:sre
							    		Password:sre
							    		HTTP Method:GET
							    

b)导入grafana图形

									"+" ->import->Upload .json file
								
3、性能测试
主机1台:4核8G 每秒可能处理kafka日志:12万条 (估算约50亿PV)