, emebere iji nakọta, gbanwee na zipu data log, metrik na mmemme.
→
N'ịbụ nke edere n'asụsụ Rust, a na-eji ọrụ dị elu na obere oriri RAM tụnyere ihe analogues ya. Na mgbakwunye, a na-akwụ ụgwọ nlebara anya nke ukwuu na ọrụ ndị metụtara izi ezi, ọkachasị, ikike ịchekwa ihe omume anaghị eziga na nchekwa na diski ma tụgharịa faịlụ.
N'usoro ihe owuwu, Vector bụ rawụta mmemme na-enweta ozi site na otu ma ọ bụ karịa isi mmalite, nhọrọ na-etinye n'elu ozi ndị a mgbanwe, na iziga ha otu ma ọ bụ karịa igbapu.
Vector bụ onye nnọchi maka filebeat na logstash, ọ nwere ike rụọ ọrụ na ọrụ abụọ ahụ (nata na zipu ndekọ), nkọwa ndị ọzọ na ha. .
Ọ bụrụ na Logstash a na-arụ agbụ ahụ dị ka ntinye → filter → mmepụta mgbe ahụ na Vector ọ bụ → →
Enwere ike ịhụ ihe atụ na akwụkwọ.
Ntuziaka a bụ ntụzịaka emegharịrị site na . Ntuziaka izizi nwere nhazi geoip. Mgbe ị na-anwale geoip site na netwọk dị n'ime, vector nyere mperi.
Aug 05 06:25:31.889 DEBUG transform{name=nginx_parse_rename_fields type=rename_fields}: vector::transforms::rename_fields: Field did not exist field=«geoip.country_name» rate_limit_secs=30Ọ bụrụ na onye ọ bụla kwesịrị ịhazi geoip, wee rụtụ aka na ntuziaka izizi sitere na .
Anyị ga-ahazi nchikota Nginx (ndekọ nweta) → Vector (Client | Filebeat) → Vector (Server | Logstash) → iche na Clickhouse na iche na Elasticsearch. Anyị ga-etinye sava 4. Agbanyeghị na ị nwere ike iji sava 3 gafere ya.
Usoro ahụ dị ka nke a.
Gbanyụọ Selinux na sava gị niile
sed -i 's/^SELINUX=.*/SELINUX=disabled/g' /etc/selinux/config
rebootAnyị wụnye emulator + ihe nkesa HTTP na sava niile
Dịka emulator nkesa HTTP anyị ga-eji si
Ihe nkesa Nodejs-stub enweghị rpm. mepụta rpm maka ya. A ga-ejikọta rpm site na iji
Tinye ebe nchekwa antonpatsev/nodejs-stub-sava
yum -y install yum-plugin-copr epel-release
yes | yum copr enable antonpatsev/nodejs-stub-serverWụnye nodejs-stub-server, Apache benchmark na ihuenyo ọnụ multiplexer na sava niile.
yum -y install stub_http_server screen mc httpd-tools screenEmeziri m oge nzaghachi stub_http_server na faịlụ /var/lib/stub_http_server/stub_http_server.js ka e nwee ọtụtụ ndekọ.
var max_sleep = 10;Ka anyị malite stub_http_server.
systemctl start stub_http_server
systemctl enable stub_http_serverna sava 3
ClickHouse na-eji usoro ntụziaka SSE 4.2, yabụ ọ gwụla ma akọwapụtaghị ya, nkwado maka ya na ihe nrụpụta eji eme ihe na-aghọ ihe achọrọ sistemụ ọzọ. Nke a bụ iwu iji lelee ma onye nrụpụta ugbu a na-akwado SSE 4.2:
grep -q sse4_2 /proc/cpuinfo && echo "SSE 4.2 supported" || echo "SSE 4.2 not supported"Mbụ ị ga-ejikọta ebe nchekwa gọọmentị:
sudo yum install -y yum-utils
sudo rpm --import https://repo.clickhouse.tech/CLICKHOUSE-KEY.GPG
sudo yum-config-manager --add-repo https://repo.clickhouse.tech/rpm/stable/x86_64Iji wụnye ngwugwu ị ga-eme iwu ndị a:
sudo yum install -y clickhouse-server clickhouse-clientKwe ka ihe nkesa-clickhouse gee kaadị netwọkụ dị na faịlụ /etc/clickhouse-server/config.xml
<listen_host>0.0.0.0</listen_host>Ịgbanwe ọkwa ndekọ site na trace gaa na nbipu
debug
Ntọala mkpakọ ọkọlọtọ:
min_compress_block_size 65536
max_compress_block_size 1048576Iji rụọ ọrụ mkpakọ Zstd, a dụrụ ya ọdụ ka ị ghara imetụ nhazi ahụ aka, kama iji DDL.
Enweghị m ike ịchọta otu esi eji zstd mkpakọ site na DDL na Google. M wee hapụ ya ka ọ dị.
Ndị ọrụ ibe na-eji zstd mkpakọ na Clickhouse, biko kesaa ntuziaka.
Ka ịmalite sava dị ka daemon, gbaa ọsọ:
service clickhouse-server startUgbu a, ka anyị gaa n'ihu ịtọlite Clickhouse
Gaa na Clickhouse
clickhouse-client -h 172.26.10.109 -m172.26.10.109 - IP nke ihe nkesa ebe arụnyere Clickhouse.
Ka anyị mepụta nchekwa data vector
CREATE DATABASE vector;Ka anyị lelee na nchekwa data dị.
show databases;Mepụta tebụl vector.logs.
/* Это таблица где хранятся логи как есть */
CREATE TABLE vector.logs
(
`node_name` String,
`timestamp` DateTime,
`server_name` String,
`user_id` String,
`request_full` String,
`request_user_agent` String,
`request_http_host` String,
`request_uri` String,
`request_scheme` String,
`request_method` String,
`request_length` UInt64,
`request_time` Float32,
`request_referrer` String,
`response_status` UInt16,
`response_body_bytes_sent` UInt64,
`response_content_type` String,
`remote_addr` IPv4,
`remote_port` UInt32,
`remote_user` String,
`upstream_addr` IPv4,
`upstream_port` UInt32,
`upstream_bytes_received` UInt64,
`upstream_bytes_sent` UInt64,
`upstream_cache_status` String,
`upstream_connect_time` Float32,
`upstream_header_time` Float32,
`upstream_response_length` UInt64,
`upstream_response_time` Float32,
`upstream_status` UInt16,
`upstream_content_type` String,
INDEX idx_http_host request_http_host TYPE set(0) GRANULARITY 1
)
ENGINE = MergeTree()
PARTITION BY toYYYYMMDD(timestamp)
ORDER BY timestamp
TTL timestamp + toIntervalMonth(1)
SETTINGS index_granularity = 8192;Anyị na-elele na e kere tebụl. Ka anyị malite clickhouse-client ma rịọ arịrịọ.
Ka anyị gaa na nchekwa data vector.
use vector;
Ok.
0 rows in set. Elapsed: 0.001 sec.Ka anyị leba anya na tebụl.
show tables;
┌─name────────────────┐
│ logs │
└─────────────────────┘Ịwụnye elasticsearch na nkesa nke anọ iji zipu otu data ahụ na Elasticsearch maka iji tụnyere Clickhouse.
Tinye igodo rpm ọha
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearchKa anyị mepụta 2 repo:
/etc/yum.repos.d/elasticsearch.repo
[elasticsearch]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md/etc/yum.repos.d/kibana.repo
[kibana-7.x]
name=Kibana repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-mdWụnye elasticsearch na kibana
yum install -y kibana elasticsearchEbe ọ ga-adị na 1 oyiri, ịkwesịrị ịgbakwunye ihe ndị a na faịlụ /etc/elasticsearch/elasticsearch.yml:
discovery.type: single-nodeKa vector nwee ike izipu data na elasticsearch site na sava ọzọ, ka anyị gbanwee network.host.
network.host: 0.0.0.0Iji jikọọ na kibana, gbanwee paramita server.host na faịlụ /etc/kibana/kibana.yml
server.host: "0.0.0.0"Ochie ma tinye elasticsearch na autostart
systemctl enable elasticsearch
systemctl start elasticsearchna kiban
systemctl enable kibana
systemctl start kibanaNa-ahazi Elasticsearch maka otu ọnụ ụzọ 1 shard, 0 oyiri. O yikarịrị ka ị ga-enwe ụyọkọ nke ọtụtụ sava ma ọ dịghị mkpa ka ị mee nke a.
Maka ndetu n'ọdịniihu, melite ndebiri ndabara:
curl -X PUT http://localhost:9200/_template/default -H 'Content-Type: application/json' -d '{"index_patterns": ["*"],"order": -1,"settings": {"number_of_shards": "1","number_of_replicas": "0"}}' ọnọdụ dị ka nnọchi maka Logstash na sava 2
yum install -y https://packages.timber.io/vector/0.9.X/vector-x86_64.rpm mc httpd-tools screenKa anyị guzobe Vector ka ọ bụrụ nnọchi maka Logstash. Na-edezi faịlụ /etc/vector/vector.toml
# /etc/vector/vector.toml
data_dir = "/var/lib/vector"
[sources.nginx_input_vector]
# General
type = "vector"
address = "0.0.0.0:9876"
shutdown_timeout_secs = 30
[transforms.nginx_parse_json]
inputs = [ "nginx_input_vector" ]
type = "json_parser"
[transforms.nginx_parse_add_defaults]
inputs = [ "nginx_parse_json" ]
type = "lua"
version = "2"
hooks.process = """
function (event, emit)
function split_first(s, delimiter)
result = {};
for match in (s..delimiter):gmatch("(.-)"..delimiter) do
table.insert(result, match);
end
return result[1];
end
function split_last(s, delimiter)
result = {};
for match in (s..delimiter):gmatch("(.-)"..delimiter) do
table.insert(result, match);
end
return result[#result];
end
event.log.upstream_addr = split_first(split_last(event.log.upstream_addr, ', '), ':')
event.log.upstream_bytes_received = split_last(event.log.upstream_bytes_received, ', ')
event.log.upstream_bytes_sent = split_last(event.log.upstream_bytes_sent, ', ')
event.log.upstream_connect_time = split_last(event.log.upstream_connect_time, ', ')
event.log.upstream_header_time = split_last(event.log.upstream_header_time, ', ')
event.log.upstream_response_length = split_last(event.log.upstream_response_length, ', ')
event.log.upstream_response_time = split_last(event.log.upstream_response_time, ', ')
event.log.upstream_status = split_last(event.log.upstream_status, ', ')
if event.log.upstream_addr == "" then
event.log.upstream_addr = "127.0.0.1"
end
if (event.log.upstream_bytes_received == "-" or event.log.upstream_bytes_received == "") then
event.log.upstream_bytes_received = "0"
end
if (event.log.upstream_bytes_sent == "-" or event.log.upstream_bytes_sent == "") then
event.log.upstream_bytes_sent = "0"
end
if event.log.upstream_cache_status == "" then
event.log.upstream_cache_status = "DISABLED"
end
if (event.log.upstream_connect_time == "-" or event.log.upstream_connect_time == "") then
event.log.upstream_connect_time = "0"
end
if (event.log.upstream_header_time == "-" or event.log.upstream_header_time == "") then
event.log.upstream_header_time = "0"
end
if (event.log.upstream_response_length == "-" or event.log.upstream_response_length == "") then
event.log.upstream_response_length = "0"
end
if (event.log.upstream_response_time == "-" or event.log.upstream_response_time == "") then
event.log.upstream_response_time = "0"
end
if (event.log.upstream_status == "-" or event.log.upstream_status == "") then
event.log.upstream_status = "0"
end
emit(event)
end
"""
[transforms.nginx_parse_remove_fields]
inputs = [ "nginx_parse_add_defaults" ]
type = "remove_fields"
fields = ["data", "file", "host", "source_type"]
[transforms.nginx_parse_coercer]
type = "coercer"
inputs = ["nginx_parse_remove_fields"]
types.request_length = "int"
types.request_time = "float"
types.response_status = "int"
types.response_body_bytes_sent = "int"
types.remote_port = "int"
types.upstream_bytes_received = "int"
types.upstream_bytes_send = "int"
types.upstream_connect_time = "float"
types.upstream_header_time = "float"
types.upstream_response_length = "int"
types.upstream_response_time = "float"
types.upstream_status = "int"
types.timestamp = "timestamp"
[sinks.nginx_output_clickhouse]
inputs = ["nginx_parse_coercer"]
type = "clickhouse"
database = "vector"
healthcheck = true
host = "http://172.26.10.109:8123" # Адрес Clickhouse
table = "logs"
encoding.timestamp_format = "unix"
buffer.type = "disk"
buffer.max_size = 104900000
buffer.when_full = "block"
request.in_flight_limit = 20
[sinks.elasticsearch]
type = "elasticsearch"
inputs = ["nginx_parse_coercer"]
compression = "none"
healthcheck = true
# 172.26.10.116 - сервер где установен elasticsearch
host = "http://172.26.10.116:9200"
index = "vector-%Y-%m-%d"Ị nwere ike ịhazigharị ngalaba transforms.nginx_parse_add_defaults.
Ebe ọ bụ na na-eji nhazi ndị a maka obere CDN ma enwere ike inwe ọtụtụ ụkpụrụ na elu elu_*
Dịka ọmụmaatụ:
"upstream_addr": "128.66.0.10:443, 128.66.0.11:443, 128.66.0.12:443"
"upstream_bytes_received": "-, -, 123"
"upstream_status": "502, 502, 200"Ọ bụrụ na nke a abụghị ọnọdụ gị, mgbe ahụ enwere ike ime ka akụkụ a dị mfe
Ka anyị mepụta ntọala ọrụ maka systemd /etc/systemd/system/vector.service
# /etc/systemd/system/vector.service
[Unit]
Description=Vector
After=network-online.target
Requires=network-online.target
[Service]
User=vector
Group=vector
ExecStart=/usr/bin/vector
ExecReload=/bin/kill -HUP $MAINPID
Restart=no
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=vector
[Install]
WantedBy=multi-user.targetMgbe ịmepụtara tebụl, ị nwere ike ịgba ọsọ Vector
systemctl enable vector
systemctl start vectorEnwere ike ịlele ndekọ vector dị ka nke a:
journalctl -f -u vectorEkwesịrị inwe ndenye dị ka nke a na ndekọ
INFO vector::topology::builder: Healthcheck: Passed.
INFO vector::topology::builder: Healthcheck: Passed.Na onye ahịa (ihe nkesa webụ) - nkesa nke mbụ
Na ihe nkesa nwere nginx, ịkwesịrị gbanyụọ ipv6, ebe ọ bụ na tebụl ndekọ na clickhouse na-eji ubi. upstream_addr IPv4, ebe ọ bụ na anaghị m eji ipv6 n'ime netwọk. Ọ bụrụ na agbanyụrụ ipv6, a ga-enwe mperi:
DB::Exception: Invalid IPv4 value.: (while read the value of key upstream_addr)Ikekwe ndị na-agụ akwụkwọ, tinye nkwado ipv6.
Mepụta faịlụ /etc/sysctl.d/98-disable-ipv6.conf
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1Itinye ntọala
sysctl --systemKa anyị tinye nginx.
agbakwunyere faịlụ nginx /etc/yum.repos.d/nginx.repo
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=trueWụnye ngwugwu nginx
yum install -y nginxNke mbụ, anyị kwesịrị ịhazi usoro ndekọ na Nginx na faịlụ /etc/nginx/nginx.conf
user nginx;
# you must set worker processes based on your CPU cores, nginx does not benefit from setting more than that
worker_processes auto; #some last versions calculate it automatically
# number of file descriptors used for nginx
# the limit for the maximum FDs on the server is usually set by the OS.
# if you don't set FD's then OS settings will be used which is by default 2000
worker_rlimit_nofile 100000;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
# provides the configuration file context in which the directives that affect connection processing are specified.
events {
# determines how much clients will be served per worker
# max clients = worker_connections * worker_processes
# max clients is also limited by the number of socket connections available on the system (~64k)
worker_connections 4000;
# optimized to serve many clients with each thread, essential for linux -- for testing environment
use epoll;
# accept as many connections as possible, may flood worker connections if set too low -- for testing environment
multi_accept on;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
log_format vector escape=json
'{'
'"node_name":"nginx-vector",'
'"timestamp":"$time_iso8601",'
'"server_name":"$server_name",'
'"request_full": "$request",'
'"request_user_agent":"$http_user_agent",'
'"request_http_host":"$http_host",'
'"request_uri":"$request_uri",'
'"request_scheme": "$scheme",'
'"request_method":"$request_method",'
'"request_length":"$request_length",'
'"request_time": "$request_time",'
'"request_referrer":"$http_referer",'
'"response_status": "$status",'
'"response_body_bytes_sent":"$body_bytes_sent",'
'"response_content_type":"$sent_http_content_type",'
'"remote_addr": "$remote_addr",'
'"remote_port": "$remote_port",'
'"remote_user": "$remote_user",'
'"upstream_addr": "$upstream_addr",'
'"upstream_bytes_received": "$upstream_bytes_received",'
'"upstream_bytes_sent": "$upstream_bytes_sent",'
'"upstream_cache_status":"$upstream_cache_status",'
'"upstream_connect_time":"$upstream_connect_time",'
'"upstream_header_time":"$upstream_header_time",'
'"upstream_response_length":"$upstream_response_length",'
'"upstream_response_time":"$upstream_response_time",'
'"upstream_status": "$upstream_status",'
'"upstream_content_type":"$upstream_http_content_type"'
'}';
access_log /var/log/nginx/access.log main;
access_log /var/log/nginx/access.json.log vector; # Новый лог в формате json
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}Ka ị ghara imebi nhazi gị ugbu a, Nginx na-enye gị ohere ịnweta ọtụtụ ntụziaka access_log
access_log /var/log/nginx/access.log main; # Стандартный лог
access_log /var/log/nginx/access.json.log vector; # Новый лог в формате jsonEchefula ịgbakwunye iwu iji logrotate maka ndekọ ọhụrụ (ọ bụrụ na faịlụ ndekọ akwụsịghị na .log)
Wepu default.conf na /etc/nginx/conf.d/
rm -f /etc/nginx/conf.d/default.confTinye mebere onye ọbịa /etc/nginx/conf.d/vhost1.conf
server {
listen 80;
server_name vhost1;
location / {
proxy_pass http://172.26.10.106:8080;
}
}Tinye mebere onye ọbịa /etc/nginx/conf.d/vhost2.conf
server {
listen 80;
server_name vhost2;
location / {
proxy_pass http://172.26.10.108:8080;
}
}Tinye mebere onye ọbịa /etc/nginx/conf.d/vhost3.conf
server {
listen 80;
server_name vhost3;
location / {
proxy_pass http://172.26.10.109:8080;
}
}Tinye mebere onye ọbịa /etc/nginx/conf.d/vhost4.conf
server {
listen 80;
server_name vhost4;
location / {
proxy_pass http://172.26.10.116:8080;
}
}Tinye ndị ọbịa mebere (172.26.10.106 ip nke ihe nkesa ebe etinyere nginx) na sava niile na faịlụ /etc/hosts:
172.26.10.106 vhost1
172.26.10.106 vhost2
172.26.10.106 vhost3
172.26.10.106 vhost4Ma ọ bụrụ na ihe niile dị njikere mgbe ahụ
nginx -t
systemctl restart nginxUgbu a ka anyị tinye ya n'onwe anyị
yum install -y https://packages.timber.io/vector/0.9.X/vector-x86_64.rpmKa anyị mepụta faịlụ ntọala maka systemd /etc/systemd/system/vector.service
[Unit]
Description=Vector
After=network-online.target
Requires=network-online.target
[Service]
User=vector
Group=vector
ExecStart=/usr/bin/vector
ExecReload=/bin/kill -HUP $MAINPID
Restart=no
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=vector
[Install]
WantedBy=multi-user.targetMa hazie ngbanwe Filebeat na /etc/vector/vector.toml config. Adreesị IP 172.26.10.108 bụ adreesị IP nke sava log (Vector-Server)
data_dir = "/var/lib/vector"
[sources.nginx_file]
type = "file"
include = [ "/var/log/nginx/access.json.log" ]
start_at_beginning = false
fingerprinting.strategy = "device_and_inode"
[sinks.nginx_output_vector]
type = "vector"
inputs = [ "nginx_file" ]
address = "172.26.10.108:9876"Echefula itinye onye ọrụ vector na otu achọrọ ka o nwee ike ịgụ faịlụ ndekọ. Dịka ọmụmaatụ, nginx na centos na-emepụta ndekọ na ikike otu adm.
usermod -a -G adm vectorKa anyị bido ọrụ vector
systemctl enable vector
systemctl start vectorEnwere ike ịlele ndekọ vector dị ka nke a:
journalctl -f -u vectorEkwesịrị inwe ntinye dị ka nke a na ndekọ
INFO vector::topology::builder: Healthcheck: Passed.Nnwale nrụgide
Anyị na-eme nnwale site na iji benchmark Apache.
Awụnyere ngwungwu httpd-tools na sava niile
Anyị na-amalite ịnwale iji Apache benchmark sitere na sava 4 dị iche iche na ihuenyo. Nke mbụ, anyị na-amalite multiplexer ihuenyo ọnụ, mgbe ahụ anyị na-amalite ịnwale site na iji Apache benchmark. Otu esi arụ ọrụ na ihuenyo ị nwere ike ịchọta .
Site na nkesa nke mbụ
while true; do ab -H "User-Agent: 1server" -c 100 -n 10 -t 10 http://vhost1/; sleep 1; doneSite na nkesa nke mbụ
while true; do ab -H "User-Agent: 2server" -c 100 -n 10 -t 10 http://vhost2/; sleep 1; doneSite na nkesa nke mbụ
while true; do ab -H "User-Agent: 3server" -c 100 -n 10 -t 10 http://vhost3/; sleep 1; doneSite na nkesa nke mbụ
while true; do ab -H "User-Agent: 4server" -c 100 -n 10 -t 10 http://vhost4/; sleep 1; doneKa anyị lelee data na Clickhouse
Gaa na Clickhouse
clickhouse-client -h 172.26.10.109 -mNa-eme ajụjụ SQL
SELECT * FROM vector.logs;
┌─node_name────┬───────────timestamp─┬─server_name─┬─user_id─┬─request_full───┬─request_user_agent─┬─request_http_host─┬─request_uri─┬─request_scheme─┬─request_method─┬─request_length─┬─request_time─┬─request_referrer─┬─response_status─┬─response_body_bytes_sent─┬─response_content_type─┬───remote_addr─┬─remote_port─┬─remote_user─┬─upstream_addr─┬─upstream_port─┬─upstream_bytes_received─┬─upstream_bytes_sent─┬─upstream_cache_status─┬─upstream_connect_time─┬─upstream_header_time─┬─upstream_response_length─┬─upstream_response_time─┬─upstream_status─┬─upstream_content_type─┐
│ nginx-vector │ 2020-08-07 04:32:42 │ vhost1 │ │ GET / HTTP/1.0 │ 1server │ vhost1 │ / │ http │ GET │ 66 │ 0.028 │ │ 404 │ 27 │ │ 172.26.10.106 │ 45886 │ │ 172.26.10.106 │ 0 │ 109 │ 97 │ DISABLED │ 0 │ 0.025 │ 27 │ 0.029 │ 404 │ │
└──────────────┴─────────────────────┴─────────────┴─────────┴────────────────┴────────────────────┴───────────────────┴─────────────┴────────────────┴────────────────┴────────────────┴──────────────┴──────────────────┴─────────────────┴──────────────────────────┴───────────────────────┴───────────────┴─────────────┴─────────────┴───────────────┴───────────────┴─────────────────────────┴─────────────────────┴───────────────────────┴───────────────────────┴──────────────────────┴──────────────────────────┴────────────────────────┴─────────────────┴───────────────────────Chọpụta nha tebụl dị na Clickhouse
select concat(database, '.', table) as table,
formatReadableSize(sum(bytes)) as size,
sum(rows) as rows,
max(modification_time) as latest_modification,
sum(bytes) as bytes_size,
any(engine) as engine,
formatReadableSize(sum(primary_key_bytes_in_memory)) as primary_keys_size
from system.parts
where active
group by database, table
order by bytes_size desc;Ka anyị chọpụta ego ole ndekọ ejirila na Clickhouse.
Nha tebụl ndekọ bụ 857.19 MB.
Ogo nke otu data dị na ndeksi na Elasticsearch bụ 4,5GB.
Ọ bụrụ na ị kọwapụtaghị data na vector na paramita, Clickhouse na-ewe 4500/857.19 = 5.24 ugboro erughị na Elasticsearch.
Na vector, a na-eji mpaghara mkpakọ na ndabara.
Kpaa nkata site na telegram
Kpaa nkata site na telegram
Kpaa nkata telegram site na""
isi: www.habr.com