Hiresaka momba ilay tetikasa ity lahatsoratra ity , izay hamaky ny log nginx, alefaso any amin'ny cluster Clickhouse. Matetika ny ElasticSearch dia ampiasaina amin'ny logs. Ny Clickhouse dia mitaky loharanon-karena vitsy kokoa (espace disk, RAM, CPU). Manoratra angona haingana kokoa ny Clickhouse. Clickhouse dia manindry ny angon-drakitra, izay mahatonga ny angona ao amin'ny kapila ho maivana kokoa. Ny tombony amin'ny Clickhouse dia azo jerena amin'ny sary 2 avy amin'ny tatitra
Raha hijery ny analyse amin'ny alàlan'ny logs, andao hamorona dashboard ho an'ny Grafana.
Iza no mikarakara, tongasoa eto ambany saka.
Mametraka nginx, grafana amin'ny fomba mahazatra.
Mametraka cluster clickhouse miaraka amin'ny ansible-playbook avy amin'ny .
Mamorona tahiry sy tabilao ao amin'ny Clickhouse
Ao amin'io Ny fangatahana SQL amin'ny famoronana angon-drakitra sy tabilao ho an'ny nginx-log-collector ao amin'ny Clickhouse dia nofaritana.
Manao ny fangatahana tsirairay izahay amin'ny mpizara tsirairay ao amin'ny cluster Clickhouse.
Fanamarihana manan-danja. Amin'ity andalana ity, ny logs_cluster dia tokony hosoloina ny anaran'ny cluster avy amin'ny rakitra clickhouse_remote_servers.xml eo anelanelan'ny "remote_servers" sy ny "shard".
ENGINE = Distributed('logs_cluster', 'nginx', 'access_log_shard', rand())Mametraka sy manitsy ny nginx-log-collector-rpm
Nginx-log-collector dia tsy manana rpm. Eto mamorona rpm ho azy. rpm dia hatsangana amin'ny fampiasana
Apetraho ny fonosana rpm nginx-log-collector-rpm
yum -y install yum-plugin-copr
yum copr enable antonpatsev/nginx-log-collector-rpm
yum -y install nginx-log-collector
systemctl start nginx-log-collectorAhitsio config /etc/nginx-log-collector/config.yaml:
.......
upload:
table: nginx.access_log
dsn: http://ip-адрес-кластера-clickhouse:8123/
- tag: "nginx_error:"
format: error # access | error
buffer_size: 1048576
upload:
table: nginx.error_log
dsn: http://ip-адрес-кластера-clickhouse:8123/Fametrahana nginx
General nginx config:
user nginx;
worker_processes auto;
#error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
log_format avito_json escape=json
'{'
'"event_datetime": "$time_iso8601", '
'"server_name": "$server_name", '
'"remote_addr": "$remote_addr", '
'"remote_user": "$remote_user", '
'"http_x_real_ip": "$http_x_real_ip", '
'"status": "$status", '
'"scheme": "$scheme", '
'"request_method": "$request_method", '
'"request_uri": "$request_uri", '
'"server_protocol": "$server_protocol", '
'"body_bytes_sent": $body_bytes_sent, '
'"http_referer": "$http_referer", '
'"http_user_agent": "$http_user_agent", '
'"request_bytes": "$request_length", '
'"request_time": "$request_time", '
'"upstream_addr": "$upstream_addr", '
'"upstream_response_time": "$upstream_response_time", '
'"hostname": "$hostname", '
'"host": "$host"'
'}';
access_log syslog_server=unix:/var/run/nginx_log.sock,nohostname,tag=nginx avito_json; #ClickHouse
error_log syslog_server=unix:/var/run/nginx_log.sock,nohostname,tag=nginx_error; #ClickHouse
#access_log /var/log/nginx/access.log main;
proxy_ignore_client_abort on;
sendfile on;
keepalive_timeout 65;
include /etc/nginx/conf.d/*.conf;
}
Virtual mpampiantrano iray:
vhost1.conf:
upstream backend {
server ip-адрес-сервера-с-stub_http_server:8080;
server ip-адрес-сервера-с-stub_http_server:8080;
server ip-адрес-сервера-с-stub_http_server:8080;
server ip-адрес-сервера-с-stub_http_server:8080;
server ip-адрес-сервера-с-stub_http_server:8080;
}
server {
listen 80;
server_name vhost1;
location / {
proxy_pass http://backend;
}
}Ampio mpampiantrano virtoaly amin'ny rakitra /etc/hosts:
ip-адрес-сервера-с-nginx vhost1HTTP server emulator
Amin'ny maha-emulator mpizara HTTP dia hampiasainay от
nodejs-stub-server dia tsy manana rpm. Eto mamorona rpm ho azy. rpm dia hatsangana amin'ny fampiasana
Mametraka fonosana nodejs-stub-server amin'ny upstream nginx rpm
yum -y install yum-plugin-copr
yum copr enable antonpatsev/nodejs-stub-server
yum -y install stub_http_server
systemctl start stub_http_serverFitsapana adin-tsaina
Ny fitsapana dia atao amin'ny alàlan'ny benchmark Apache.
Mametraka azy:
yum install -y httpd-toolsManomboka manao fitiliana mampiasa benchmark Apache avy amin'ny mpizara 5 samihafa izahay:
while true; do ab -H "User-Agent: 1server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; done
while true; do ab -H "User-Agent: 2server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; done
while true; do ab -H "User-Agent: 3server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; done
while true; do ab -H "User-Agent: 4server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; done
while true; do ab -H "User-Agent: 5server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; doneFametrahana Grafana
Tsy hahita dashboard ao amin'ny tranokala ofisialin'ny Grafana ianao.
Noho izany dia hanao izany amin'ny tanana izahay.
Azonao atao ny mahita ny dashboard voatahiriko .
Mila mamorona faribolana latabatra miaraka amin'ny atiny ihany koa ianao nginx.access_log.
Singlestat Total fangatahana:
SELECT
1 as t,
count(*) as c
FROM $table
WHERE $timeFilter GROUP BY t
Fangatahana tsy nahomby ny Singlestat:
SELECT
1 as t,
count(*) as c
FROM $table
WHERE $timeFilter AND status NOT IN (200, 201, 401) GROUP BY t
Isan-jato tsy nahomby ny Singlestat:
SELECT
1 as t, (sum(status = 500 or status = 499)/sum(status = 200 or status = 201 or status = 401))*100 FROM $table
WHERE $timeFilter GROUP BY t
Fotoana mamaly salan'isa Singlestat:
SELECT
1, avg(request_time) FROM $table
WHERE $timeFilter GROUP BY 1
Fotoana mamaly max Singlestat:
SELECT
1 as t, max(request_time) as c
FROM $table
WHERE $timeFilter GROUP BY t
Sata fanisana:
$columns(status, count(*) as c) from $table
Mba hamoahana angona toy ny pie dia mila mametraka ny plugin ianao ary mamerina ny grafana.
grafana-cli plugins install grafana-piechart-panel
service grafana-server restartSata TOP 5 Pie:
SELECT
1, /* fake timestamp value */
status,
sum(status) AS Reqs
FROM $table
WHERE $timeFilter
GROUP BY status
ORDER BY Reqs desc
LIMIT 5
Hanome fangatahana tsy misy pikantsary aho:
Manisa http_user_agent:
$columns(http_user_agent, count(*) c) FROM $tableSara tsara/ratsy:
$rate(countIf(status = 200) AS good, countIf(status != 200) AS bad) FROM $tablefotoana mamaly:
$rate(avg(request_time) as request_time) FROM $tableFotoana famaliana an-tampony (fotoana famaliana ny 1er ambony):
$rate(avg(arrayElement(upstream_response_time,1)) as upstream_response_time) FROM $tableSata fanisana latabatra ho an'ny vhost rehetra:
$columns(status, count(*) as c) from $tableFijery ankapobeny ny dashboard
Mampitaha avg() sy quantille()
avg ()
quantille ()
famaranana:
Antenaina fa handray anjara amin'ny famolavolana/fitsapana sy fampiasana ny nginx-log-collector ny vondrom-piarahamonina.
Ary rehefa misy mampihatra ny nginx-log-collector dia holazainy aminao ny habetsahan'ny kapila, RAM, CPU.
Fantsona telegrama:
Miliseconds:
Iza no miraharaha ny milisegondra, manorata na mifidy, azafady, amin'ity .
Source: www.habr.com