Kab lus no yuav tham txog qhov project , uas yuav nyeem nginx cav, xa lawv mus rau Clickhouse pawg. Feem ntau ElasticSearch yog siv rau cov cav. Clickhouse xav tau kev pab tsawg dua (disk chaw, RAM, CPU). Clickhouse sau cov ntaub ntawv sai dua. Clickhouse compresses cov ntaub ntawv, uas ua rau cov ntaub ntawv ntawm disk txawm ntau compact. Cov txiaj ntsig ntawm Clickhouse tuaj yeem pom hauv 2 slides los ntawm tsab ntawv ceeb toom
Txhawm rau saib analytics los ntawm cov cav, cia peb tsim lub dashboard rau Grafana.
Leej twg mob siab, txais tos hauv qab miv.
Nruab nginx, grafana raws li tus qauv.
Nruab clickhouse pawg nrog ansible-playbook los ntawm .
Tsim ib lub database thiab cov ntxhuav hauv Clickhouse
Hauv no SQL queries rau tsim databases thiab cov ntxhuav rau nginx-log-collector hauv Clickhouse tau piav qhia.
Peb ua txhua qhov kev thov nyob rau hauv lem ntawm txhua tus neeg rau zaub mov ntawm Clickhouse pawg.
Lus ceeb toom. Hauv kab no, logs_cluster yuav tsum tau hloov nrog koj lub npe pawg los ntawm clickhouse_remote_servers.xml cov ntaub ntawv ntawm "remote_servers" thiab "shard".
ENGINE = Distributed('logs_cluster', 'nginx', 'access_log_shard', rand())Txhim kho thiab teeb tsa nginx-log-collector-rpm
Nginx-log-collector tsis muaj rpm. Ntawm no tsim rpm rau nws. rpm yuav tsim siv
Nruab rpm pob nginx-log-collector-rpm
yum -y install yum-plugin-copr
yum copr enable antonpatsev/nginx-log-collector-rpm
yum -y install nginx-log-collector
systemctl start nginx-log-collectorKho kom raug config /etc/nginx-log-collector/config.yaml:
.......
upload:
table: nginx.access_log
dsn: http://ip-адрес-кластера-clickhouse:8123/
- tag: "nginx_error:"
format: error # access | error
buffer_size: 1048576
upload:
table: nginx.error_log
dsn: http://ip-адрес-кластера-clickhouse:8123/Teeb tsa nginx
General nginx config:
user nginx;
worker_processes auto;
#error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
log_format avito_json escape=json
'{'
'"event_datetime": "$time_iso8601", '
'"server_name": "$server_name", '
'"remote_addr": "$remote_addr", '
'"remote_user": "$remote_user", '
'"http_x_real_ip": "$http_x_real_ip", '
'"status": "$status", '
'"scheme": "$scheme", '
'"request_method": "$request_method", '
'"request_uri": "$request_uri", '
'"server_protocol": "$server_protocol", '
'"body_bytes_sent": $body_bytes_sent, '
'"http_referer": "$http_referer", '
'"http_user_agent": "$http_user_agent", '
'"request_bytes": "$request_length", '
'"request_time": "$request_time", '
'"upstream_addr": "$upstream_addr", '
'"upstream_response_time": "$upstream_response_time", '
'"hostname": "$hostname", '
'"host": "$host"'
'}';
access_log syslog_server=unix:/var/run/nginx_log.sock,nohostname,tag=nginx avito_json; #ClickHouse
error_log syslog_server=unix:/var/run/nginx_log.sock,nohostname,tag=nginx_error; #ClickHouse
#access_log /var/log/nginx/access.log main;
proxy_ignore_client_abort on;
sendfile on;
keepalive_timeout 65;
include /etc/nginx/conf.d/*.conf;
}
Virtual host ib:
vhost1.conf:
upstream backend {
server ip-адрес-сервера-с-stub_http_server:8080;
server ip-адрес-сервера-с-stub_http_server:8080;
server ip-адрес-сервера-с-stub_http_server:8080;
server ip-адрес-сервера-с-stub_http_server:8080;
server ip-адрес-сервера-с-stub_http_server:8080;
}
server {
listen 80;
server_name vhost1;
location / {
proxy_pass http://backend;
}
}Ntxiv virtual hosts rau /etc/hosts file:
ip-адрес-сервера-с-nginx vhost1HTTP server emulator
Raws li HTTP server emulator peb yuav siv los ntawm
nodejs-stub-server tsis muaj rpm. Ntawm no tsim rpm rau nws. rpm yuav tsim siv
Nruab nodejs-stub-server pob ntawm cov nginx rpm
yum -y install yum-plugin-copr
yum copr enable antonpatsev/nodejs-stub-server
yum -y install stub_http_server
systemctl start stub_http_serverKev Ntsuas Kev Nyuaj Siab
Kev ntsuam xyuas yog ua tiav siv Apache benchmark.
Nruab nws:
yum install -y httpd-toolsPeb pib sim siv Apache benchmark los ntawm 5 txawv servers:
while true; do ab -H "User-Agent: 1server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; done
while true; do ab -H "User-Agent: 2server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; done
while true; do ab -H "User-Agent: 3server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; done
while true; do ab -H "User-Agent: 4server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; done
while true; do ab -H "User-Agent: 5server" -c 10 -n 10 -t 10 http://vhost1/; sleep 1; doneTeeb tsa Grafana
Koj yuav tsis pom lub dashboard ntawm lub official Grafana lub vev xaib.
Yog li ntawd, peb yuav ua nws ntawm tes.
Koj tuaj yeem pom kuv lub dashboard khaws cia .
Koj kuj yuav tsum tau tsim ib lub rooj sib txawv nrog cov ntsiab lus nginx.access_log.
Singlestat Tag Nrho Cov Thov:
SELECT
1 as t,
count(*) as c
FROM $table
WHERE $timeFilter GROUP BY t
Singlestat thov tsis tau:
SELECT
1 as t,
count(*) as c
FROM $table
WHERE $timeFilter AND status NOT IN (200, 201, 401) GROUP BY t
Singlestat Failing feem pua:
SELECT
1 as t, (sum(status = 500 or status = 499)/sum(status = 200 or status = 201 or status = 401))*100 FROM $table
WHERE $timeFilter GROUP BY t
Singlestat Avg Teb Sijhawm:
SELECT
1, avg(request_time) FROM $table
WHERE $timeFilter GROUP BY 1
Singlestat Max Teb Sijhawm:
SELECT
1 as t, max(request_time) as c
FROM $table
WHERE $timeFilter GROUP BY t
Suav xwm txheej:
$columns(status, count(*) as c) from $table
Txhawm rau tso tawm cov ntaub ntawv zoo li lub ncuav qab zib, koj yuav tsum nruab lub plugin thiab rov rub tawm grafana.
grafana-cli plugins install grafana-piechart-panel
service grafana-server restartPie TOP 5 Status:
SELECT
1, /* fake timestamp value */
status,
sum(status) AS Reqs
FROM $table
WHERE $timeFilter
GROUP BY status
ORDER BY Reqs desc
LIMIT 5
Ntxiv mus kuv yuav muab kev thov yam tsis muaj screenshots:
Suav http_user_agent:
$columns(http_user_agent, count(*) c) FROM $tableGoodRate/BadRate:
$rate(countIf(status = 200) AS good, countIf(status != 200) AS bad) FROM $tablelub sij hawm teb:
$rate(avg(request_time) as request_time) FROM $tableUpstream teb lub sij hawm (lub sij hawm teb ntawm 1st upstream):
$rate(avg(arrayElement(upstream_response_time,1)) as upstream_response_time) FROM $tableTable Count Status rau tag nrho cov vhosts:
$columns(status, count(*) as c) from $tableGeneral saib ntawm lub dashboard
Sib piv avg() thiab quantile()
avg()
quantile()
Xaus:
Vam tias cov zej zog yuav koom nrog hauv kev tsim / sim thiab siv nginx-log-collector.
Thiab thaum ib tus neeg siv nginx-log-collector, nws yuav qhia koj tias nws khaws disk, RAM, CPU npaum li cas.
Telegram channels:
milliseconds:
Leej twg mob siab txog milliseconds, sau lossis pov npav, thov, hauv qhov no .
Tau qhov twg los: www.hab.com