Ushbu maqolada loyiha muhokama qilinadi nginx jurnallarini o'qiydi, ularni Clickhouse klasteriga yuboring. Odatda ElasticSearch jurnallar uchun ishlatiladi. Clickhouse kamroq resurslarni talab qiladi (disk maydoni, RAM, CPU). Clickhouse ma'lumotlarni tezroq yozadi. Clickhouse ma'lumotlarni siqadi, bu esa diskdagi ma'lumotlarni yanada ixcham qiladi. Clickhouse-ning afzalliklarini hisobotdan 2 ta slaydda ko'rish mumkin
Jurnallar bo'yicha tahlillarni ko'rish uchun Grafana uchun asboblar paneli yarataylik.
Kimga g'amxo'rlik qiladi, mushuk ostida xush kelibsiz.
Nginx, grafana-ni standart tarzda o'rnating.
Clickhouse klasterini ansible-playbook dan o'rnating .
Clickhouse-da ma'lumotlar bazasi va jadvallarni yaratish
Bu erda Clickhouse-da nginx-log-collector uchun ma'lumotlar bazalari va jadvallarni yaratish uchun SQL so'rovlari tasvirlangan.
Biz har bir so'rovni Clickhouse klasterining har bir serverida navbat bilan qilamiz.
Muhim eslatma. Ushbu qatorda logs_cluster "remote_servers" va "shard" o'rtasidagi clickhouse_remote_servers.xml faylidagi klaster nomi bilan almashtirilishi kerak.
ENGINE = Distributed('logs_cluster', 'nginx', 'access_log_shard', rand())Nginx-log-collector-rpm-ni o'rnatish va sozlash
Nginx-log-kollektorida aylanish tezligi yo'q. Bu yerga Buning uchun rpm yarating. rpm yordamida quriladi
nginx-log-collector-rpm rpm paketini o'rnating
yum -y install yum-plugin-copr
yum copr enable antonpatsev/nginx-log-collector-rpm
yum -y install nginx-log-collector
systemctl start nginx-log-collector/etc/nginx-log-collector/config.yaml konfiguratsiyasini tahrirlang:
.......
upload:
table: nginx.access_log
dsn: http://ip-адрес-кластера-clickhouse:8123/
- tag: "nginx_error:"
format: error # access | error
buffer_size: 1048576
upload:
table: nginx.error_log
dsn: http://ip-адрес-кластера-clickhouse:8123/Nginx o'rnatilmoqda
Umumiy nginx konfiguratsiyasi:
user nginx;
worker_processes auto;
#error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
log_format avito_json escape=json
'{'
'"event_datetime": "$time_iso8601", '
'"server_name": "$server_name", '
'"remote_addr": "$remote_addr", '
'"remote_user": "$remote_user", '
'"http_x_real_ip": "$http_x_real_ip", '
'"status": "$status", '
'"scheme": "$scheme", '
'"request_method": "$request_method", '
'"request_uri": "$request_uri", '
'"server_protocol": "$server_protocol", '
'"body_bytes_sent": $body_bytes_sent, '
'"http_referer": "$http_referer", '
'"http_user_agent": "$http_user_agent", '
'"request_bytes": "$request_length", '
'"request_time": "$request_time", '
'"upstream_addr": "$upstream_addr", '
'"upstream_response_time": "$upstream_response_time", '
'"hostname": "$hostname", '
'"host": "$host"'
'}';
access_log syslog_server=unix:/var/run/nginx_log.sock,nohostname,tag=nginx avito_json; #ClickHouse
error_log syslog_server=unix:/var/run/nginx_log.sock,nohostname,tag=nginx_error; #ClickHouse
#access_log /var/log/nginx/access.log main;
proxy_ignore_client_abort on;
sendfile on;
keepalive_timeout 65;
include /etc/nginx/conf.d/*.conf;
}
Virtual xost biri:
vhost1.conf:
upstream backend {
server ip-адрес-сервера-с-stub_http_server:8080;
server ip-адрес-сервера-с-stub_http_server:8080;
server ip-адрес-сервера-с-stub_http_server:8080;
server ip-адрес-сервера-с-stub_http_server:8080;
server ip-адрес-сервера-с-stub_http_server:8080;
}
server {
listen 80;
server_name vhost1;
location / {
proxy_pass http://backend;
}
}/etc/hosts fayliga virtual xostlarni qo'shing:
ip-адрес-сервера-с-nginx vhost1HTTP server emulyatori
HTTP server emulyatori sifatida biz foydalanamiz от
nodejs-stub-serverda rpm yo'q. Bu yerga Buning uchun rpm yarating. rpm yordamida quriladi
Nodejs-stub-server paketini yuqoridagi nginx rpm-ga o'rnating
yum -y install yum-plugin-copr
yum copr enable antonpatsev/nodejs-stub-server
yum -y install stub_http_server
systemctl start stub_http_serverStress testi
Sinov Apache benchmark yordamida amalga oshiriladi.
Uni o'rnating:
yum install -y httpd-toolsBiz 5 xil serverdan Apache benchmarkidan foydalanib sinovni boshlaymiz:
while true; do ab -H "User-Agent: 1server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; done
while true; do ab -H "User-Agent: 2server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; done
while true; do ab -H "User-Agent: 3server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; done
while true; do ab -H "User-Agent: 4server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; done
while true; do ab -H "User-Agent: 5server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; doneGrafana o'rnatilmoqda
Grafana rasmiy veb-saytida asboblar panelini topa olmaysiz.
Shuning uchun biz buni qo'lda qilamiz.
Siz mening saqlangan asboblar panelini topishingiz mumkin .
Bundan tashqari, tarkib bilan jadval o'zgaruvchisini yaratishingiz kerak nginx.access_log.
Yagona umumiy soʻrovlar:
SELECT
1 as t,
count(*) as c
FROM $table
WHERE $timeFilter GROUP BY t
Singlestat bajarilmagan so'rovlar:
SELECT
1 as t,
count(*) as c
FROM $table
WHERE $timeFilter AND status NOT IN (200, 201, 401) GROUP BY t
Yagona statistik muvaffaqiyatsizlik foizi:
SELECT
1 as t, (sum(status = 500 or status = 499)/sum(status = 200 or status = 201 or status = 401))*100 FROM $table
WHERE $timeFilter GROUP BY t
Yagona oʻrtacha javob vaqti:
SELECT
1, avg(request_time) FROM $table
WHERE $timeFilter GROUP BY 1
Singlestat maksimal javob vaqti:
SELECT
1 as t, max(request_time) as c
FROM $table
WHERE $timeFilter GROUP BY t
Hisob holati:
$columns(status, count(*) as c) from $table
Pirog kabi ma'lumotlarni chiqarish uchun siz plaginni o'rnatishingiz va grafanani qayta yuklashingiz kerak.
grafana-cli plugins install grafana-piechart-panel
service grafana-server restartPie TOP 5 holati:
SELECT
1, /* fake timestamp value */
status,
sum(status) AS Reqs
FROM $table
WHERE $timeFilter
GROUP BY status
ORDER BY Reqs desc
LIMIT 5
Keyinchalik men skrinshotsiz so'rovlarni beraman:
http_user_agentni hisoblash:
$columns(http_user_agent, count(*) c) FROM $tableGoodRate/BadRate:
$rate(countIf(status = 200) AS good, countIf(status != 200) AS bad) FROM $tablejavob vaqti:
$rate(avg(request_time) as request_time) FROM $tableYuqori oqimning javob vaqti (birinchi oqimning javob vaqti):
$rate(avg(arrayElement(upstream_response_time,1)) as upstream_response_time) FROM $tableBarcha vhost uchun jadval soni holati:
$columns(status, count(*) as c) from $tableBoshqaruv panelining umumiy ko'rinishi
avg() va quantile() ni solishtirish
avg()
kvantil()
xulosa:
Umid qilamanki, hamjamiyat nginx-log-collector-ni ishlab chiqish/sinov va undan foydalanishda ishtirok etadi.
Va kimdir nginx-log-collector-ni qo'llaganida, u sizga diskni, operativ xotirani, protsessorni qancha saqlaganini aytib beradi.
Telegram kanallari:
Millisoniyalar:
Millisekundlar kimni qiziqtiradi, yozing yoki ovoz bering, iltimos, bu erda .
Manba: www.habr.com