{"id":54918,"date":"2020-01-07T00:00:00","date_gmt":"2020-01-06T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/ispolzovanie-clickhouse-v-kachestve-zameny-elk-big-query-i-timescaledb"},"modified":"2020-02-18T14:02:58","modified_gmt":"2020-02-18T11:02:58","slug":"ispolzovanie-clickhouse-v-kachestve-zameny-elk-big-query-i-timescaledb","status":"publish","type":"post","link":"https:\/\/prohoster.info\/az\/blog\/administrirovanie\/ispolzovanie-clickhouse-v-kachestve-zameny-elk-big-query-i-timescaledb","title":{"rendered":"Clickhouse-un ELK, Big Query v\u0259 TimescaleDB \u0259v\u0259zi kimi istifad\u0259si","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><noindex><a rel=\"nofollow\" href=\"https:\/\/clickhouse.yandex\/\">Clickhouse<\/a><\/noindex> \u2014 bu, Yandex t\u0259r\u0259find\u0259n yarad\u0131lm\u0131\u015f, analitik sor\u011fular\u0131 onlayn emal etm\u0259k \u00fc\u00e7\u00fcn a\u00e7\u0131q m\u0259nb\u0259li s\u00fctun \u0259sasl\u0131 m\u0259lumat bazas\u0131 sistemidir (OLAP). Bunu Yandex, CloudFlare, VK.com, Badoo v\u0259 d\u00fcnyan\u0131n dig\u0259r xidm\u0259tl\u0259ri h\u0259qiq\u0259t\u0259n b\u00f6y\u00fck m\u0259lumat h\u0259cml\u0259rini saxlamaq \u00fc\u00e7\u00fcn istifad\u0259 edir (saniy\u0259d\u0259 minl\u0259rl\u0259 s\u0259tir daxil etm\u0259 v\u0259 ya diskd\u0259 petabaytlarla m\u0259lumat saxlama).<\/p>\n<p>Tipik \u00abs\u0259tir\u00bb m\u0259lumat bazas\u0131 sisteml\u0259rind\u0259, m\u0259s\u0259l\u0259n, MySQL, Postgres, MS SQL Server il\u0259, m\u0259lumatlar a\u015fa\u011f\u0131dak\u0131 kimi saxlan\u0131l\u0131r: <\/p>\n<p><img decoding=\"async\" alt=\"Clickhouse-un ELK, Big Query v\u0259 TimescaleDB \u0259v\u0259zi kimi istifad\u0259si\" src=\"\/wp-content\/uploads\/2020\/01\/398f260de0152baf3cc11d884fa0d52d.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nBu zaman bir s\u0259tir\u0259 aid olan d\u0259y\u0259rl\u0259r fiziki olaraq yan-yana saxlan\u0131l\u0131r. S\u00fctun \u0259sasl\u0131 m\u0259lumat bazas\u0131 sisteml\u0259rind\u0259 f\u0259rqli s\u00fctunlardan olan d\u0259y\u0259rl\u0259r ayr\u0131-ayr\u0131l\u0131qda saxlan\u0131l\u0131r, bir s\u00fctunun m\u0259lumatlar\u0131 is\u0259 bir yerd\u0259dir:<\/p>\n<p><img decoding=\"async\" alt=\"Clickhouse-un ELK, Big Query v\u0259 TimescaleDB \u0259v\u0259zi kimi istifad\u0259si\" src=\"\/wp-content\/uploads\/2020\/01\/938a2ee3d588ac866ec92bb3ea09c253.png\" style=\"display:block;margin: 0 auto;\" \/><noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<p>S\u00fctun \u0259sasl\u0131 m\u0259lumat bazas\u0131 sisteml\u0259rin\u0259 Vertica, Paraccel (Actian Matrix, Amazon Redshift), Sybase IQ, Exasol, Infobright, InfiniDB, MonetDB (VectorWise, Actian Vector), LucidDB, SAP HANA, Google Dremel, Google PowerDrill, Druid, kdb+ aiddir.<\/p>\n<p>\u015eirk\u0259t \u2013 mail-\u043f\u0440\u043e\u0434\u0430\u043a\u0448\u043d <noindex><a rel=\"nofollow\" href=\"https:\/\/qwintry.com\/\">Qwintry<\/a><\/noindex> 2018-ci ild\u0259n Clickhouse-u hesabatlar haz\u0131rlamaq \u00fc\u00e7\u00fcn istifad\u0259 etm\u0259y\u0259 ba\u015flam\u0131\u015f v\u0259 onun sad\u0259liyi, geni\u015fl\u0259ndiril\u0259 bilm\u0259si, SQL d\u0259st\u0259yi v\u0259 s\u00fcr\u0259ti il\u0259 \u00e7ox heyran qalm\u0131\u015fd\u0131r. Bu m\u0259lumat bazas\u0131n\u0131n i\u015f s\u00fcr\u0259ti sehr\u0259 b\u0259nz\u0259yirdi.<\/p>\n<h3>Kolayl\u0131k<\/h3>\n<p>\nClickhouse, Ubuntu-da yaln\u0131z bir \u0259mrl\u0259 qura\u015fd\u0131r\u0131l\u0131r. \u018fg\u0259r siz SQL bilirsinizs\u0259, Clickhouse-u d\u0259rhal ehtiyaclar\u0131n\u0131z \u00fc\u00e7\u00fcn istifad\u0259 etm\u0259y\u0259 ba\u015flaya bil\u0259rsiniz. Lakin, bu \u00abshow create table\u00bb \u0259m\u0259lini MySQL-d\u0259 yerin\u0259 yetirib SQL-i Clickhouse-a kopyalayaca\u011f\u0131n\u0131z anlam\u0131na g\u0259lmir. <\/p>\n<p>MySQL il\u0259 m\u00fcqayis\u0259d\u0259, bu m\u0259lumat bazas\u0131nda c\u0259dv\u0259l sxeml\u0259rinin t\u0259rifl\u0259rind\u0259 m\u00fchim f\u0259rql\u0259r var, buna g\u00f6r\u0259 d\u0259 rahat i\u015fl\u0259m\u0259k \u00fc\u00e7\u00fcn c\u0259dv\u0259l sxeml\u0259rinin t\u0259rifl\u0259rini d\u0259yi\u015fdirm\u0259k v\u0259 c\u0259dv\u0259l m\u00fch\u0259rrikl\u0259rini \u00f6yr\u0259nm\u0259k \u00fc\u00e7\u00fcn bir q\u0259d\u0259r zamana ehtiyac\u0131n\u0131z olacaq.<\/p>\n<p>Clickhouse \u0259lav\u0259 proqram t\u0259minat\u0131 olmadan m\u00fck\u0259mm\u0259l i\u015fl\u0259yir, lakin \u0259g\u0259r siz replikasiyan\u0131 istifad\u0259 etm\u0259k ist\u0259yirsinizs\u0259, ZooKeeper-i qura\u015fd\u0131rmal\u0131s\u0131n\u0131z. Sor\u011fu performans\u0131n\u0131n analizi m\u00fck\u0259mm\u0259l n\u0259tic\u0259l\u0259r g\u00f6st\u0259rir \u2014 sistem c\u0259dv\u0259ll\u0259ri b\u00fct\u00fcn m\u0259lumatlar\u0131 ehtiva edir v\u0259 b\u00fct\u00fcn m\u0259lumatlar k\u00f6hn\u0259 v\u0259 dar\u0131xd\u0131r\u0131c\u0131 SQL vasit\u0259sil\u0259 \u0259ld\u0259 edil\u0259 bil\u0259r.<\/p>\n<h3>Performans<\/h3>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/clickhouse.yandex\/benchmark.html#%5B%2522100000000%2522,%5B%2522ClickHouse%2522,%2522Vertica%2522,%2522MySQL%2522%5D,%5B%25220%2522,%25221%2522%5D%5D\">Benchmark<\/a><\/noindex> Clickhouse-un Vertica v\u0259 MySQL il\u0259 m\u00fcqayis\u0259si, konfiqurasiyal\u0131 serverd\u0259: iki Intel\u00ae Xeon\u00ae CPU E5-2650 v2 @ 2.60GHz; 128 GiB RAM; 8 6TB SATA HDD, ext4 \u00fcz\u0259rind\u0259 md RAID-5.<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.altinity.com\/blog\/2017\/6\/20\/clickhouse-vs-redshift\">Benchmark<\/a><\/noindex> Clickhouse-un Amazon RedShift bulud m\u0259lumat anbar\u0131 il\u0259 m\u00fcqayis\u0259si.<\/li>\n<li>Blogdan \u00e7\u0131xar\u0131\u015flar <noindex><a rel=\"nofollow\" href=\"https:\/\/blog.cloudflare.com\/how-cloudflare-analyzes-1m-dns-queries-per-second\/\">Cloudflare-in Clickhouse performans\u0131 haqq\u0131nda<\/a><\/noindex>:<\/li>\n<\/ul>\n<p>\n<img decoding=\"async\" alt=\"Clickhouse-un ELK, Big Query v\u0259 TimescaleDB \u0259v\u0259zi kimi istifad\u0259si\" src=\"\/wp-content\/uploads\/2020\/01\/b6ef9fbad8e8c864a227bcffd97ec0fe.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nClickHouse veril\u0259nl\u0259r bazas\u0131 \u00e7ox sad\u0259 dizayna malikdir - klasterd\u0259 b\u00fct\u00fcn d\u00fcy\u00fcnl\u0259r eyni funksionall\u0131\u011fa malikdir v\u0259 yaln\u0131z ZooKeeper-d\u0259n istifad\u0259 ed\u0259r\u0259k koordinasiya olunur. Biz bir ne\u00e7\u0259 d\u00fcy\u00fcnd\u0259n ibar\u0259t ki\u00e7ik bir klaster qurduq v\u0259 test etm\u0259 zaman\u0131 sistemin analitik DB-l\u0259rin benchmarklar\u0131nda g\u00f6st\u0259ril\u0259n \u00fcst\u00fcnl\u00fckl\u0259r\u0259 uy\u011fun olduqca t\u0259sir edici bir performansa sahib oldu\u011funu a\u015fkar etdik. ClickHouse-un \u0259sas\u0131nda duran konsepsiyan\u0131 daha yax\u0131ndan ara\u015fd\u0131rma\u011fa q\u0259rar verdik. Ara\u015fd\u0131rmalar \u00fc\u00e7\u00fcn ilk mane\u0259, al\u0259tl\u0259rin olmamas\u0131 v\u0259 ClickHouse icmas\u0131n\u0131n azl\u0131\u011f\u0131 oldu, buna g\u00f6r\u0259 d\u0259 bu veril\u0259nl\u0259r bazas\u0131n\u0131n dizayn\u0131na d\u0259rinl\u0259\u015fdik ki, onun nec\u0259 i\u015fl\u0259diyini ba\u015fa d\u00fc\u015f\u0259k.<\/p>\n<p>ClickHouse, m\u0259lumatlar\u0131 birba\u015fa Kafka-dan q\u0259bul etmir, \u00e7\u00fcnki bu sad\u0259c\u0259 bir veril\u0259nl\u0259r bazas\u0131d\u0131r, buna g\u00f6r\u0259 d\u0259 biz Go dilind\u0259 \u00f6z adapter xidm\u0259timizi yazd\u0131q. O, Kafka-da kodla\u015fd\u0131r\u0131lm\u0131\u015f mesajlar\u0131 oxuyur, onlar\u0131 TSV-y\u0259 \u00e7evirir v\u0259 ClickHouse-a HTTP interfeysi vasit\u0259sil\u0259 paketl\u0259r \u015f\u0259klind\u0259 g\u00f6nd\u0259rir. Sonra biz bu xidm\u0259ti yenid\u0259n yazd\u0131q ki, Go kitabxanas\u0131n\u0131 ClickHouse-un \u00f6z interfeysi il\u0259 birlikd\u0259 istifad\u0259 ed\u0259k v\u0259 performans\u0131 art\u0131rd\u0131q. Paketl\u0259rin q\u0259bul performans\u0131n\u0131 d\u0259y\u0259rl\u0259ndirdikd\u0259, ClickHouse-un bu performans\u0131n\u0131n paket \u00f6l\u00e7\u00fcs\u00fcnd\u0259n, y\u0259ni eyni anda daxil edil\u0259n s\u0259tir say\u0131ndan \u00e7ox as\u0131l\u0131 oldu\u011funu a\u015fkar etdik. Bunun niy\u0259 ba\u015f verdiyini anlamaq \u00fc\u00e7\u00fcn ClickHouse-un m\u0259lumatlar\u0131 nec\u0259 saxlad\u0131\u011f\u0131n\u0131 ara\u015fd\u0131rd\u0131q.<\/p>\n<p>ClickHouse-un m\u0259lumatlar\u0131 saxlamaq \u00fc\u00e7\u00fcn istifad\u0259 etdiyi \u0259sas m\u00fch\u0259rrik, daha do\u011frusu, c\u0259dv\u0259ll\u0259r \u00fc\u00e7\u00fcn m\u00fch\u0259rrikl\u0259r ail\u0259si MergeTree-dir. Bu m\u00fch\u0259rrik, Google BigTable v\u0259 ya Apache Cassandra-da t\u0259tbiq olunan LSM algoritmin\u0259 konseptual olaraq b\u0259nz\u0259yir, lakin aral\u0131q yadda\u015f c\u0259dv\u0259lini yaratm\u0131r v\u0259 m\u0259lumatlar\u0131 birba\u015fa diskin\u0259 yaz\u0131r. Bu, yazma \u00fc\u00e7\u00fcn \u0259la ke\u00e7id x\u00fcusiyy\u0259tin\u0259 malikdir, \u00e7\u00fcnki daxil edil\u0259n h\u0259r paket yaln\u0131z \u201cbirincil a\u00e7ar\u201d (primary key) il\u0259 s\u0131ralan\u0131r, s\u0131x\u0131l\u0131r v\u0259 disk\u0259 yaz\u0131l\u0131r ki, seqment yarad\u0131ls\u0131n. <\/p>\n<p>Yadda\u015f c\u0259dv\u0259linin olmamas\u0131 v\u0259 ya m\u0259lumatlar\u0131n \u201ct\u0259z\u0259liyi\u201d anlay\u0131\u015f\u0131n\u0131n olmamas\u0131, onlar\u0131n yaln\u0131z \u0259lav\u0259 edil\u0259 bil\u0259c\u0259yini, d\u0259yi\u015fdirilm\u0259si v\u0259 ya silinm\u0259si sistem t\u0259r\u0259find\u0259n d\u0259st\u0259kl\u0259nm\u0259diyini d\u0259 bildirir. Bu g\u00fcn m\u0259lumatlar\u0131 silm\u0259k \u00fc\u00e7\u00fcn yegan\u0259 yol, onlar\u0131 t\u0259qvim aylar\u0131 \u00fczr\u0259 silm\u0259kdir, \u00e7\u00fcnki seqmentl\u0259r he\u00e7 vaxt ay\u0131n s\u0259rh\u0259dini ke\u00e7mir. ClickHouse komandas\u0131 bu funksiyan\u0131 konfiqurasiya oluna bil\u0259n etm\u0259k \u00fcz\u0259rind\u0259 f\u0259al \u00e7al\u0131\u015f\u0131r. Dig\u0259r t\u0259r\u0259fd\u0259n, bu, seqmentl\u0259rin yaz\u0131lmas\u0131n\u0131 v\u0259 birl\u0259\u015fm\u0259sini m\u00fcbahis\u0259siz edir, buna g\u00f6r\u0259 d\u0259 q\u0259bulun ke\u00e7id x\u00fcusiyy\u0259ti paralel daxil edil\u0259nl\u0259r say\u0131n\u0131n artmas\u0131 il\u0259 x\u0259tt \u00fczr\u0259 geni\u015fl\u0259nir, I\/O v\u0259 n\u00fcv\u0259l\u0259rin doyma v\u0259ziyy\u0259tin\u0259 \u00e7atd\u0131\u011f\u0131na q\u0259d\u0259r. <br \/>\nAncaq bu hal sistemin ki\u00e7ik paketl\u0259r \u00fc\u00e7\u00fcn uy\u011fun olmad\u0131\u011f\u0131n\u0131 g\u00f6st\u0259rir, buna g\u00f6r\u0259 d\u0259 Kafka xidm\u0259tl\u0259ri v\u0259 inserterl\u0259r istifad\u0259 olunur. Daha sonra ClickHouse arxa planda daima seqmentl\u0259rin birl\u0259\u015fdirilm\u0259sini h\u0259yata ke\u00e7irir, bel\u0259likl\u0259, bir \u00e7ox ki\u00e7ik m\u0259lumat hiss\u0259l\u0259ri birl\u0259\u015fdiril\u0259r\u0259k daha \u00e7ox sayda yaz\u0131l\u0131r, bu da yazma intensivliyini art\u0131r\u0131r. Bu zaman \u00e7ox sayda ba\u011fl\u0131 olmayan hiss\u0259, birl\u0259\u015fm\u0259 davam ed\u0259rk\u0259n insertl\u0259rin aqressiv \u015f\u0259kild\u0259 t\u0131xanmas\u0131na s\u0259b\u0259b olacaq. Bizim tapd\u0131\u011f\u0131m\u0131z \u0259n yax\u015f\u0131 kompromis, real vaxtda m\u0259lumat q\u0259bul etm\u0259 il\u0259 q\u0259bulun performans\u0131 aras\u0131nda, saniy\u0259d\u0259 m\u0259hdud sayda insertl\u0259ri c\u0259dv\u0259l\u0259 q\u0259bul etm\u0259kdir.<\/p>\n<p>C\u0259dv\u0259ll\u0259rin oxuma performans\u0131n\u0131n a\u00e7ar\u0131 indeksl\u0259\u015fdirm\u0259 v\u0259 m\u0259lumat\u0131n diskd\u0259ki yerl\u0259\u015fimidir. \u0130st\u0259nil\u0259n q\u0259d\u0259r s\u00fcr\u0259tli emal olsa da, motorun diskl\u0259 terabaytlarla m\u0259lumat\u0131 taramas\u0131 v\u0259 yaln\u0131z bir qismini istifad\u0259 etm\u0259si zaman al\u0131r. ClickHouse s\u00fctun \u0259sasl\u0131 bir anbar oldu\u011fu \u00fc\u00e7\u00fcn h\u0259r seqment, h\u0259r s\u0259tir \u00fc\u00e7\u00fcn s\u0131ralanm\u0131\u015f d\u0259y\u0259rl\u0259ri olan h\u0259r s\u00fctun \u00fc\u00e7\u00fcn bir fayl saxlay\u0131r. Bu s\u0259b\u0259bd\u0259n, sor\u011fuda olmayan b\u00fct\u00fcn s\u00fctunlar \u0259vv\u0259lc\u0259 atlan\u0131la bil\u0259r v\u0259 daha sonra bir ne\u00e7\u0259 h\u00fcceyr\u0259 paralel \u015f\u0259kild\u0259 vektorla\u015fd\u0131r\u0131lm\u0131\u015f icra il\u0259 emal oluna bil\u0259r. Tam skan etm\u0259nin qar\u015f\u0131s\u0131n\u0131 almaq \u00fc\u00e7\u00fcn, h\u0259r seqmentin ki\u00e7ik bir indeks fayl\u0131 var. <\/p>\n<p>B\u00fct\u00fcn s\u00fctunlar \"birinci a\u00e7ar\" \u00fczr\u0259 s\u0131raland\u0131\u011f\u0131 \u00fc\u00e7\u00fcn, indeks fayl\u0131 yaln\u0131z h\u0259r N-ci s\u0259trin etiketl\u0259rini (tutulan s\u0259tirl\u0259r) saxlay\u0131r v\u0259 \u00e7ox b\u00f6y\u00fck c\u0259dv\u0259ll\u0259r \u00fc\u00e7\u00fcn bunlar\u0131 yadda\u015fda saxlama\u011fa imkan verir. M\u0259s\u0259l\u0259n, \"h\u0259r 8192-ci s\u0259tri etiketl\u0259m\u0259k\" \u00fc\u00e7\u00fcn standart parametrl\u0259ri qura\u015fd\u0131rmal\u0131s\u0131n\u0131z, bel\u0259likl\u0259, yadda\u015fa asanl\u0131qla yerl\u0259\u015f\u0259 bil\u0259n 1 trilyon s\u0259tirli c\u0259dv\u0259lin \"sade\" indeksl\u0259\u015fm\u0259si yaln\u0131z 122070 xarakter t\u0259l\u0259b ed\u0259c\u0259kdir. <\/p>\n<h3>Sisteminin inki\u015faf\u0131<\/h3>\n<p>\nClickhouse-un inki\u015faf\u0131 v\u0259 t\u0259kmill\u0259\u015fdirilm\u0259si <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/yandex\/ClickHouse\/pulse\">Github repoda<\/a><\/noindex> m\u0259lumatlar\u0131n \"b\u00f6y\u00fcm\u0259\" prosesinin heyr\u0259tamiz s\u00fcr\u0259tl\u0259 gedildiyini v\u0259 s\u00fcbut etm\u0259y\u0259 imkan verir. <\/p>\n<p><img decoding=\"async\" alt=\"Clickhouse-un ELK, Big Query v\u0259 TimescaleDB \u0259v\u0259zi kimi istifad\u0259si\" src=\"\/wp-content\/uploads\/2020\/01\/19211da28e42178d26dff7623689df71.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>Populyarl\u0131q<\/h3>\n<p>\nG\u00f6r\u00fcn\u00fcr ki, Clickhouse-un populyarl\u0131\u011f\u0131 getdikc\u0259 art\u0131r, x\u00fcsusil\u0259 da rus dilli icmada. Ke\u00e7\u0259n ilki High load 2018 konfrans\u0131 (Moskva, 8-9 noyabr 2018) g\u00f6st\u0259rdi ki, vk.com v\u0259 Badoo kimi n\u0259h\u0259ngl\u0259r Clickhouse-dan istifad\u0259 edir, burada m\u0259lumatlar (m\u0259s\u0259l\u0259n, jurnallar) on minl\u0259rl\u0259 serverd\u0259n eyni anda daxil edilir. 40 d\u0259qiq\u0259lik videoda <noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=pbbcMcrQoXw\">Yuri Nasretdinov VKontakt komandas\u0131ndan bunun nec\u0259 h\u0259yata ke\u00e7irildiyi bar\u0259d\u0259 dan\u0131\u015f\u0131r<\/a><\/noindex>. Tezlikl\u0259 materialla i\u015fl\u0259m\u0259k \u00fc\u00e7\u00fcn Habrda transkripti payla\u015faca\u011f\u0131q.<\/p>\n<h3>T\u0259tbiq sah\u0259l\u0259ri<\/h3>\n<p>\nAra\u015fd\u0131rmalara bir m\u00fcdd\u0259t s\u0259rf etdikd\u0259n sonra, ClickHouse-un MySQL, PostgreSQL, ELK, Google Big Query, Amazon RedShift, TimescaleDB, Hadoop, MapReduce, Pinot v\u0259 Druid kimi daha \u0259n\u0259n\u0259vi v\u0259 populyar h\u0259ll\u0259ri tamamil\u0259 \u0259v\u0259z ed\u0259 bil\u0259c\u0259yi v\u0259 ya faydal\u0131 ola bil\u0259c\u0259yi sah\u0259l\u0259r m\u00f6vcud oldu\u011funu d\u00fc\u015f\u00fcn\u00fcr\u0259m. A\u015fa\u011f\u0131da ClickHouse-un qeyd olunan DBMS-l\u0259rin m\u00fcasirl\u0259\u015fdirilm\u0259si v\u0259 ya tam \u0259v\u0259z edilm\u0259si \u00fc\u00e7\u00fcn istifad\u0259 olunmas\u0131 haqq\u0131nda m\u0259lumat verilmi\u015fdir. <\/p>\n<h3>MySQL v\u0259 PostgreSQL-in imkanlar\u0131n\u0131 geni\u015fl\u0259ndirm\u0259k<\/h3>\n<p>\nTezlikl\u0259 Mautic e-po\u00e7t b\u00fclletenl\u0259ri \u00fc\u00e7\u00fcn MySQL-i ClickHouse il\u0259 qism\u0259n \u0259v\u0259z etdik <noindex><a rel=\"nofollow\" href=\"https:\/\/www.mautic.org\/\">Mautic b\u00fclleten<\/a><\/noindex>. Problemin k\u00f6k\u00fc, MySQL-in d\u00fc\u015f\u00fcnc\u0259siz dizayn\u0131 s\u0259b\u0259bind\u0259n g\u00f6nd\u0259rilmi\u015f h\u0259r bir e-po\u00e7tu v\u0259 h\u0259min e-po\u00e7tdak\u0131 h\u0259r bir ke\u00e7idi base64 hash-\u0131 il\u0259 qeyd\u0259 almas\u0131 idi ki, bu da MySQL c\u0259dv\u0259linin (email_stats) b\u00f6y\u00fcm\u0259sin\u0259 s\u0259b\u0259b oldu. Bu xidm\u0259td\u0259n 10 milyon e-po\u00e7t g\u00f6nd\u0259rdikd\u0259n sonra, bu c\u0259dv\u0259l 150 GB fayl sah\u0259si tuturdu v\u0259 MySQL sad\u0259 sor\u011fularda \"yava\u015flay\u0131rd\u0131\". Fayl sah\u0259si problemini h\u0259ll etm\u0259k \u00fc\u00e7\u00fcn, \u0130nnodb c\u0259dv\u0259linin s\u0131x\u0131lmas\u0131ndan u\u011furla istifad\u0259 etdik ki, bu da onu 4 d\u0259f\u0259 ki\u00e7ilm\u0259sin\u0259 s\u0259b\u0259b oldu. Lakin sad\u0259 sor\u011fular\u0131n hans\u0131sa bir s\u0259b\u0259bd\u0259n tam tarama icra etm\u0259li oldu\u011fu halda, 20-30 milyon e-po\u00e7tu yaln\u0131z tarix\u00e7\u0259y\u0259 baxmaq \u00fc\u00e7\u00fcn MySQL-d\u0259 saxlama\u011f\u0131n he\u00e7 bir m\u0259nas\u0131 yoxdur, \u00e7\u00fcnki bu, swap-a v\u0259 I\/O-y\u0259 b\u00f6y\u00fck y\u00fck\u0259 s\u0259b\u0259b olur v\u0259 bu bar\u0259d\u0259 m\u00fcnt\u0259z\u0259m olaraq Zabbix x\u0259b\u0259rdarl\u0131qlar\u0131 al\u0131rd\u0131q.<\/p>\n<p><img decoding=\"async\" alt=\"Clickhouse-un ELK, Big Query v\u0259 TimescaleDB \u0259v\u0259zi kimi istifad\u0259si\" src=\"\/wp-content\/uploads\/2020\/01\/11b3ee4b1a8b7c9457bd3871bd12049f.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nClickhouse iki s\u0131x\u0131lma alqoritmind\u0259n istifad\u0259 edir ki, bu da m\u0259lumatlar\u0131n h\u0259cmini t\u0259xmin\u0259n <noindex><a rel=\"nofollow\" href=\"https:\/\/www.altinity.com\/blog\/2017\/11\/21\/compression-in-clickhouse\">3-4 d\u0259f\u0259  azald\u0131r<\/a><\/noindex>, lakin bu konkret halda veril\u0259nl\u0259r x\u00fcsusil\u0259 \"s\u0131x\u0131lma\u011fa meylli\" idi. <\/p>\n<p><img decoding=\"async\" alt=\"Clickhouse-un ELK, Big Query v\u0259 TimescaleDB \u0259v\u0259zi kimi istifad\u0259si\" src=\"\/wp-content\/uploads\/2020\/01\/d192502073d57967620aa07da0bbfa0c.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>ELK-nin \u0259v\u0259zl\u0259nm\u0259si<\/h3>\n<p>\n\u00d6z t\u0259cr\u00fcb\u0259mizd\u0259n, ELK (ElasticSearch, Logstash v\u0259 Kibana, bu konkret halda ElasticSearch) d\u0259sti loqlar\u0131n saxlanmas\u0131 \u00fc\u00e7\u00fcn laz\u0131m olan resurslardan daha \u00e7oxunu t\u0259l\u0259b edir. ElasticSearch, loqlarda yax\u015f\u0131 tam m\u0259tni axtarmaq \u00fc\u00e7\u00fcn \u0259la bir m\u00fch\u0259rrikdir (amma m\u0259nc\u0259 bu siz\u0259 realda laz\u0131m deyil), lakin niy\u0259 d\u0259-fakto jurnal aparmaq \u00fc\u00e7\u00fcn standart m\u00fch\u0259rrik oldu\u011funu maraq edir\u0259m. Onun q\u0259bul etm\u0259 performans\u0131 Logstash il\u0259 birl\u0259\u015f\u0259r\u0259k, biz\u0259 h\u0259tta nisb\u0259t\u0259n az y\u00fck alt\u0131nda bel\u0259 probleml\u0259r yarad\u0131rd\u0131 v\u0259 daha \u00e7ox RAM v\u0259 disk sah\u0259si t\u0259l\u0259b edirdi. DB olaraq, Clickhouse a\u015fa\u011f\u0131dak\u0131 s\u0259b\u0259bl\u0259rd\u0259n ElasticSearch-d\u0259n \u00fcst\u00fcnd\u00fcr:<\/p>\n<ul>\n<li>SQL dialektinin d\u0259st\u0259yi;<\/li>\n<li>Saxlan\u0131lan m\u0259lumatlar\u0131n daha yax\u015f\u0131 s\u0131x\u0131lmas\u0131;<\/li>\n<li>Tam m\u0259tni axtarmaq \u0259v\u0259zin\u0259 Regex il\u0259 m\u00fcnt\u0259z\u0259m ifad\u0259l\u0259r \u00fczr\u0259 axtar\u0131\u015f\u0131n d\u0259st\u0259yi;<\/li>\n<li>Sor\u011fu planlamas\u0131n\u0131n yax\u015f\u0131la\u015fd\u0131r\u0131lmas\u0131 v\u0259 \u00fcmumi performans\u0131n art\u0131r\u0131lmas\u0131.<\/li>\n<\/ul>\n<p>\nHaz\u0131rda ClickHouse il\u0259 ELK aras\u0131nda m\u00fcqayis\u0259 ed\u0259rk\u0259n \u0259n b\u00f6y\u00fck problem loglar\u0131n transferi \u00fc\u00e7\u00fcn h\u0259ll\u0259rin olmamas\u0131 v\u0259 bu m\u00f6vzuda s\u0259n\u0259din v\u0259 d\u0259rslikl\u0259rin \u00e7at\u0131\u015fmazl\u0131\u011f\u0131d\u0131r. Bu bax\u0131mdan, h\u0259r bir istifad\u0259\u00e7i ELK-ni Digital Ocean r\u0259hb\u0259rliyi il\u0259 qura\u015fd\u0131ra bil\u0259r ki, bu da bel\u0259 texnologiyalar\u0131n s\u00fcr\u0259tli t\u0259tbiqi \u00fc\u00e7\u00fcn \u00e7ox vacibdir. Burada bir veril\u0259nl\u0259r bazas\u0131 m\u00fch\u0259rriki m\u00f6vcuddur, lakin h\u0259l\u0259 ClickHouse \u00fc\u00e7\u00fcn Filebeat yoxdur. B\u0259li, orada var <noindex><a rel=\"nofollow\" href=\"https:\/\/www.fluentd.org\/\">fluentd<\/a><\/noindex> v\u0259 loglarla i\u015fl\u0259m\u0259 \u00fc\u00e7\u00fcn sistem <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/loghouse\">loghouse<\/a><\/noindex>, bir al\u0259t m\u00f6vcuddur <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/Altinity\/clicktail\">clicktail<\/a><\/noindex> log fayllar\u0131n\u0131 ClickHouse-a daxil etm\u0259k \u00fc\u00e7\u00fcn, lakin bunlar daha \u00e7ox vaxt t\u0259l\u0259b edir. Lakin ClickHouse, sad\u0259liyi say\u0259sind\u0259 h\u0259l\u0259 d\u0259 liderdir, buna g\u00f6r\u0259 d\u0259 yeni ba\u015flayanlar bel\u0259 onu asanl\u0131qla qura\u015fd\u0131r\u0131r v\u0259 tam funksional istifad\u0259y\u0259 c\u0259mi 10 d\u0259qiq\u0259 i\u00e7ind\u0259 ba\u015flay\u0131r. <\/p>\n<p>Minimalist h\u0259ll\u0259ri \u00fcst\u00fcn tutaraq, ClickHouse il\u0259 birlikd\u0259 Kafka istifad\u0259 etm\u0259d\u0259n loglar\u0131n \u00f6t\u00fcr\u00fclm\u0259si \u00fc\u00e7\u00fcn \u00e7ox az yadda\u015f s\u0259rf ed\u0259n FluentBit al\u0259tini istifad\u0259 etm\u0259y\u0259 \u00e7al\u0131\u015fd\u0131m. Lakin, FluentBit-dan ClickHouse-a veril\u0259nl\u0259rin \u00e7evrilm\u0259sini t\u0259min ed\u0259n bir proxy qat\u0131n\u0131 olmadan bunu etm\u0259k \u00fc\u00e7\u00fcn ki\u00e7ik uy\u011funsuzluqlar, m\u0259s\u0259l\u0259n <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/fluent\/fluent-bit\/issues\/848\">tarix format\u0131 probleml\u0259ri<\/a><\/noindex>, aradan qald\u0131r\u0131lmal\u0131d\u0131r.<\/p>\n<p>Kibana alternativi olaraq ClickHouse-u backend kimi istifad\u0259 etm\u0259k m\u00fcmk\u00fcnd\u00fcr <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/Vertamedia\/clickhouse-grafana\">Grafana<\/a><\/noindex>. Bildiyim\u0259 g\u00f6r\u0259, burada \u00e7ox sayda m\u0259lumat n\u00f6qt\u0259sinin renderl\u0259nm\u0259si il\u0259 ba\u011fl\u0131 performans probleml\u0259ri yarana bil\u0259r, x\u00fcsusil\u0259 k\u00f6hn\u0259 versiyalar\u0131 il\u0259 Grafana. Qwintry-d\u0259 biz bunu indiy\u0259 q\u0259d\u0259r s\u0131namam\u0131\u015f\u0131q, lakin zaman-zaman ClickHouse d\u0259st\u0259k kanal\u0131nda bunu haqq\u0131nda \u015fikay\u0259tl\u0259r g\u0259lir. <\/p>\n<h3>Google Big Query v\u0259 Amazon RedShift-in (b\u00f6y\u00fck \u015firk\u0259tl\u0259r \u00fc\u00e7\u00fcn h\u0259ll) \u0259v\u0259zl\u0259nm\u0259si<\/h3>\n<p>\nBigQuery \u00fc\u00e7\u00fcn ideal istifad\u0259 variant\u0131 1 TB JSON m\u0259lumat\u0131n\u0131 y\u00fckl\u0259m\u0259k v\u0259 onun \u00fcz\u0259rind\u0259 analitik sor\u011fular yerin\u0259 yetirm\u0259kdir. Big Query - \u00f6l\u00e7m\u0259mi \u00e7\u0259tin qiym\u0259tl\u0259ndiril\u0259n m\u00fck\u0259mm\u0259l bir m\u0259hsuldur. Bu, ClickHouse-dan daha m\u00fcr\u0259kk\u0259b bir proqramd\u0131r, daxili klasterd\u0259 i\u015fl\u0259yir, lakin m\u00fc\u015ft\u0259ri bax\u0131m\u0131ndan ClickHouse il\u0259 \u00e7ox \u015feyd\u0259 ortaqd\u0131r. BigQuery, h\u0259r bir SELECT \u00fc\u00e7\u00fcn \u00f6d\u0259m\u0259y\u0259 ba\u015flad\u0131\u011f\u0131n\u0131z zaman tez <\/p>\n<p>ClickHouse, hesablama bax\u0131m\u0131ndan daha baha ba\u015fa g\u0259l\u0259n sor\u011fular yerin\u0259 yetirdiyiniz zaman \u0259n yax\u015f\u0131 se\u00e7imdir. H\u0259r g\u00fcn yerin\u0259 yetirdiyiniz SELECT sor\u011fular\u0131n\u0131n say\u0131 n\u0259 q\u0259d\u0259r \u00e7oxdursa, Big Query-ni ClickHouse-a d\u0259yi\u015fm\u0259kd\u0259 m\u0259nas\u0131 o q\u0259d\u0259r art\u0131r, \u00e7\u00fcnki bel\u0259 bir d\u0259yi\u015fiklik siz\u0259 \u00e7oxsayl\u0131 terabaytlar\u0131n emal\u0131nda minl\u0259rl\u0259 dollar q\u0259na\u0259t etm\u0259y\u0259 k\u00f6m\u0259k ed\u0259 bil\u0259r. Bu, Big Query-d\u0259 emal\u0131 nisb\u0259t\u0259n ucuz olan saxlan\u0131lan veril\u0259nl\u0259r \u00fc\u00e7\u00fcn ke\u00e7\u0259rli deyil.<\/p>\n<p>Altinity \u015firk\u0259tinin t\u0259sis\u00e7il\u0259rind\u0259n Aleksandr Zaytsev-in m\u0259qal\u0259sind\u0259 <noindex><a rel=\"nofollow\" href=\"https:\/\/www.altinity.com\/blog\/2017\/10\/23\/migration-to-clickhouse\">\u00abClickHouse-a ke\u00e7id\u00bb<\/a><\/noindex> bu tip bazalar\u0131n migrasiyas\u0131n\u0131n \u00fcst\u00fcnl\u00fckl\u0259ri haqq\u0131nda dan\u0131\u015f\u0131l\u0131r.<\/p>\n<h3>TimescaleDB-nin \u0259v\u0259zl\u0259nm\u0259si<\/h3>\n<p>\nTimescaleDB, PostgreSQL geni\u015fl\u0259nm\u0259si, ad\u0259t\u0259n veril\u0259nl\u0259r bazas\u0131nda zaman seriyalar\u0131 il\u0259 i\u015fl\u0259m\u0259yi optimalla\u015fd\u0131r\u0131r.<noindex><a rel=\"nofollow\" href=\"https:\/\/docs.timescale.com\/v1.0\/introduction\">https:\/\/docs.timescale.com\/v1.0\/introduction<\/a><\/noindex>, <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/zabbix\/blog\/458530\/\">https:\/\/habr.com\/ru\/company\/zabbix\/blog\/458530\/<\/a><\/noindex>). <\/p>\n<p>H\u0259tta ClickHouse zaman seriyalar\u0131 sah\u0259sind\u0259 ciddi r\u0259qib olmasa da, onun s\u00fctun strukturuna v\u0259 vektor icras\u0131na g\u00f6r\u0259 analitik sor\u011fular\u0131n bir \u00e7ox hallarda emal\u0131n\u0131 TimescaleDB-d\u0259n daha s\u00fcr\u0259tli edir. Eyni zamanda, ClickHouse\u2019un partiya m\u0259lumatlar\u0131n\u0131n q\u0259bulu performans\u0131 t\u0259xmin\u0259n 3 d\u0259f\u0259 \u00fcst\u00fcnd\u00fcr v\u0259 20 d\u0259f\u0259 daha az disk sah\u0259si istifad\u0259 edir, bu da b\u00f6y\u00fck h\u0259cmli tarixi m\u0259lumatlar\u0131n emal\u0131 \u00fc\u00e7\u00fcn olduqca \u0259h\u0259miyy\u0259tlidir.<noindex><a rel=\"nofollow\" href=\"https:\/\/www.altinity.com\/blog\/ClickHouse-for-time-series\">https:\/\/www.altinity.com\/blog\/ClickHouse-for-time-series<\/a><\/noindex>.<\/p>\n<p>ClickHouse'dan f\u0259rqli olaraq, TimescaleDB-d\u0259 bir az disk sah\u0259sinin q\u0259na\u0259t edilm\u0259sinin yegan\u0259 yolu ZFS v\u0259 ya ox\u015far fayl sisteml\u0259rinin istifad\u0259sidir.<\/p>\n<p>G\u0259l\u0259c\u0259k ClickHouse yenil\u0259m\u0259l\u0259ri ehtimal ki, delta s\u0131xla\u015fd\u0131rma funksiyas\u0131n\u0131 ke\u00e7ir\u0259c\u0259k ki, bu da onu zaman seriyalar\u0131 m\u0259lumatlar\u0131n\u0131n emal\u0131 v\u0259 saxlanmas\u0131 \u00fc\u00e7\u00fcn daha \u0259lveri\u015fli ed\u0259c\u0259k. TimescaleDB, \u00ab\u00e7\u0131plaq\u00bb ClickHouse-dan daha yax\u015f\u0131 se\u00e7im ola bil\u0259r, a\u015fa\u011f\u0131dak\u0131 hallarda:<\/p>\n<ul>\n<li>\u00e7ox az \u0259m\u0259liyyat x\u0259rci (&lt;3 GB);<\/li>\n<li>b\u00f6y\u00fck fraksiyalara y\u0131\u011fmaq ist\u0259m\u0259diyiniz \u00e7oxlu ki\u00e7ik INSERT;<\/li>\n<li>daha yax\u015f\u0131 uyum, vahidlik v\u0259 ACID t\u0259l\u0259bl\u0259ri;<\/li>\n<li>PostGIS d\u0259st\u0259yi;<\/li>\n<li>m\u00f6vcud PostgreSQL c\u0259dv\u0259ll\u0259ri il\u0259 inteqrasiya, \u00e7\u00fcnki TimescaleDB, \u0259slind\u0259, PostgreSQL-dir.<\/li>\n<\/ul>\n<p><\/p>\n<h3>Hadoop v\u0259 MapReduce sisteml\u0259ri il\u0259 r\u0259qab\u0259t<\/h3>\n<p>\nHadoop v\u0259 dig\u0259r MapReduce m\u0259hsullar\u0131 \u00e7oxsayl\u0131 m\u00fcr\u0259kk\u0259b hesablamalar ed\u0259 bil\u0259r, lakin ad\u0259t\u0259n b\u00f6y\u00fck gecikm\u0259l\u0259rl\u0259 i\u015fl\u0259yirl\u0259r. ClickHouse bu problemi yar\u0131m-terabayt m\u0259lumatlar\u0131 emal etm\u0259kl\u0259 v\u0259 dem\u0259k olar ki, ani n\u0259tic\u0259 verm\u0259kl\u0259 d\u00fcz\u0259ldir. Bel\u0259likl\u0259, ClickHouse h\u0131zl\u0131, interaktiv analitik t\u0259dqiqatlar \u00fc\u00e7\u00fcn daha s\u0259m\u0259r\u0259li bir se\u00e7imdir ki, bu da m\u0259lumat emal\u0131 m\u00fct\u0259x\u0259ssisl\u0259rinin mara\u011f\u0131n\u0131 c\u0259lb etm\u0259lidir.<\/p>\n<h3>Pinot v\u0259 Druid il\u0259 r\u0259qab\u0259t<\/h3>\n<p>\nClickHouse'un \u0259n yax\u0131n r\u0259qibl\u0259ri s\u00fctun strukturuna g\u00f6r\u0259 d\u00fcz miqyaslanan a\u00e7\u0131q m\u0259nb\u0259li m\u0259hsul olan Pinot v\u0259 Druid-dir. Bu sisteml\u0259rin m\u00fcqayis\u0259sin\u0259 dair \u0259la bir i\u015f, m\u0259qal\u0259d\u0259 d\u0259rc edilmi\u015fdir. <noindex><a rel=\"nofollow\" href=\"https:\/\/medium.com\/@leventov\/comparison-of-the-open-source-olap-systems-for-big-data-ClickHouse-druid-and-pinot-8e042a5ed1c7\">Roman Leventov<\/a><\/noindex> 2018-ci ilin 1 fevral\u0131nda. <\/p>\n<p><img decoding=\"async\" alt=\"Clickhouse-un ELK, Big Query v\u0259 TimescaleDB \u0259v\u0259zi kimi istifad\u0259si\" src=\"\/wp-content\/uploads\/2020\/01\/fe58814aa97b287b70c2f724b72b448a.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nBu m\u0259qal\u0259 yenil\u0259m\u0259y\u0259 ehtiyac duyur \u2013 burada ClickHouse-un UPDATE v\u0259 DELETE \u0259m\u0259liyyatlar\u0131n\u0131 d\u0259st\u0259kl\u0259m\u0259diyi qeyd edilir ki, bu da son versiyalara tamamil\u0259 uy\u011fun deyil.<\/p>\n<p>Bizim bu DBMS-l\u0259rl\u0259 kifay\u0259t q\u0259d\u0259r t\u0259cr\u00fcb\u0259miz yoxdur, amma Druid v\u0259 Pinot-un i\u015f\u0259 sal\u0131nmas\u0131 \u00fc\u00e7\u00fcn t\u0259l\u0259b olunan infrastrukturun m\u00fcr\u0259kk\u0259bliyini he\u00e7 b\u0259y\u0259nmir\u0259m \u2014 bu, \u0259traf\u0131nda Java olan bir \u00e7ox \"h\u0259r\u0259k\u0259tli hiss\u0259l\u0259r\"-d\u0259n ibar\u0259tdir.<\/p>\n<p>Druid v\u0259 Pinot, inki\u015faf\u0131 Apache'nin GitHub layih\u0259l\u0259rinin s\u0259hif\u0259l\u0259rind\u0259 \u0259trafl\u0131 i\u015f\u0131qland\u0131r\u0131lan Apache'nin inkubator layih\u0259l\u0259ridir. Pinot, 2018-ci ilin oktyabr\u0131nda inkubatora daxil olmu\u015f, Druid is\u0259 8 ay \u0259vv\u0259ldir - fevralda yaranm\u0131\u015fd\u0131r.<\/p>\n<p>AFS-in nec\u0259 i\u015fl\u0259diyin\u0259 dair m\u0259lumat\u0131n olmamas\u0131 m\u0259nd\u0259 m\u00fc\u0259yy\u0259n, b\u0259lk\u0259 d\u0259, g\u00fcl\u00fcnc suallar yarad\u0131r. Maraql\u0131d\u0131r, Pinot-un m\u00fc\u0259llifl\u0259ri Apache Fondunun Druid-\u0259 daha meylli oldu\u011funu g\u00f6r\u00fcb-g\u00f6rm\u0259dil\u0259r v\u0259 bu r\u0259qib\u0259 m\u00fcnasib\u0259t, q\u0131sqancl\u0131q hissi do\u011furubmu? \u018fg\u0259r ilkini d\u0259st\u0259kl\u0259y\u0259n sponsorlar bird\u0259n-bir\u0259 ikincini maraqland\u0131rma\u011fa ba\u015flasa, Druid-in inki\u015faf\u0131 l\u0259ngiy\u0259c\u0259kmi v\u0259 Pinot-un inki\u015faf\u0131 s\u00fcr\u0259tl\u0259n\u0259c\u0259kmi?<\/p>\n<h3>ClickHouse-un \u00e7at\u0131\u015fmazl\u0131qlar\u0131<\/h3>\n<p>\nK\u00f6n\u00fcll\u00fc olmay\u0131\u015f\u0131: g\u00f6r\u00fcn\u00fcr ki, bu h\u0259l\u0259 d\u0259 maraql\u0131 bir texnologiyad\u0131r, amma h\u0259r halda, dig\u0259r s\u00fctunlu DB-l\u0259rd\u0259 bel\u0259 bir \u015fey yoxdur. <\/p>\n<p>Ki\u00e7ik daxil etm\u0259l\u0259r y\u00fcks\u0259k s\u00fcr\u0259tl\u0259 pis i\u015fl\u0259yir: daxil etm\u0259l\u0259r b\u00f6y\u00fck hiss\u0259l\u0259r\u0259 b\u00f6l\u00fcnm\u0259lidir, \u00e7\u00fcnki ki\u00e7ik daxil etm\u0259l\u0259rin performans\u0131 h\u0259r s\u0259tird\u0259ki s\u00fctunlar\u0131n say\u0131na m\u00fct\u0259nasib olaraq azal\u0131r. ClickHouse-da veril\u0259nl\u0259r diskl\u0259rd\u0259 bel\u0259 saxlan\u0131l\u0131r - h\u0259r s\u00fctun 1 fayl v\u0259 ya daha \u00e7oxunu g\u00f6st\u0259rir, ona g\u00f6r\u0259 d\u0259 100 s\u00fctun olan 1 s\u0259tri daxil etm\u0259k \u00fc\u00e7\u00fcn \u0259n az\u0131 100 fayl\u0131 a\u00e7\u0131b yazmaq laz\u0131md\u0131r. Daxil etm\u0259l\u0259ri tamponla\u015fd\u0131rmaq \u00fc\u00e7\u00fcn vasit\u0259\u00e7i t\u0259l\u0259b olunur (\u0259g\u0259r m\u00fc\u015ft\u0259ri \u00f6z\u00fc tamponla\u015fd\u0131rma t\u0259min etmirs\u0259) - bu ad\u0259t\u0259n Kafka v\u0259 ya h\u0259r hans\u0131 bir s\u0131ra idar\u0259etm\u0259 sistemidir. H\u0259m\u00e7inin, daha sonra b\u00f6y\u00fck m\u0259lumat hiss\u0259l\u0259rini MergeTree c\u0259dv\u0259ll\u0259rin\u0259 k\u00f6\u00e7\u00fcrm\u0259k \u00fc\u00e7\u00fcn Buffer table m\u00fch\u0259rriki istifad\u0259 edil\u0259 bil\u0259r.<\/p>\n<p>C\u0259dv\u0259dl\u0259rin birl\u0259\u015fm\u0259l\u0259ri serverin \u0259m\u0259liyyat yadda\u015f\u0131 il\u0259 m\u0259hdudla\u015f\u0131r, amma \u0259n az\u0131ndan onlar\u0131n var! M\u0259s\u0259l\u0259n, Druid v\u0259 Pinot-da \u00fcmumiyy\u0259tl\u0259 bel\u0259 birl\u0259\u015fm\u0259l\u0259r yoxdur, \u00e7\u00fcnki b\u00f6l\u00fc\u015fd\u00fcr\u00fclm\u00fc\u015f sisteml\u0259rd\u0259 b\u00f6y\u00fck m\u0259lumat hiss\u0259l\u0259rinin d\u00fcy\u00fcnl\u0259r aras\u0131nda h\u0259r\u0259k\u0259tini h\u0259yata ke\u00e7irm\u0259k \u00e7\u0259tindir. <\/p>\n<h3>Sonu\u00e7lar<\/h3>\n<p>\nG\u0259l\u0259c\u0259k ill\u0259rd\u0259 ClickHouse-u Qwintry-d\u0259 geni\u015f t\u0259tbiq etm\u0259yi planla\u015fd\u0131r\u0131r\u0131q, \u00e7\u00fcnki bu DB m\u00fck\u0259mm\u0259l performans, a\u015fa\u011f\u0131 x\u0259rc, miqyaslanma v\u0259 sad\u0259lik balans\u0131n\u0131 t\u0259min edir. Dey\u0259rdim ki, ClickHouse icmas\u0131 onu ki\u00e7ik v\u0259 orta sisteml\u0259rd\u0259 daha \u00e7ox istifad\u0259 etm\u0259k \u00fc\u00e7\u00fcn yollar tapd\u0131qca tez yay\u0131mlanma\u011fa ba\u015flayacaq.<\/p>\n<h3>Bir az reklam \ud83d\ude42<\/h3>\n<p>\nBiziml\u0259 qald\u0131\u011f\u0131n\u0131z \u00fc\u00e7\u00fcn t\u0259\u015f\u0259kk\u00fcr edirik. Bizim m\u0259qal\u0259l\u0259rimizi b\u0259y\u0259nirsiniz? Daha maraql\u0131 materiallar g\u00f6rm\u0259k ist\u0259yirsiniz? Bizi d\u0259st\u0259kl\u0259yin, sifari\u015f ver\u0259r\u0259k v\u0259 ya tan\u0131\u015flar\u0131n\u0131za t\u00f6vsiy\u0259 ed\u0259r\u0259k. <noindex><a rel=\"nofollow\" href=\"https:\/\/ua-hosting.company\/cloudvps\/nl\">\u0130nki\u015faf\u00e7\u0131lar \u00fc\u00e7\u00fcn bulud VPS-d\u0259n $4.99-dan<\/a><\/noindex>, <b>Sizin \u00fc\u00e7\u00fcn icad etdiyimiz b\u0259nz\u0259rsiz entry-level serverl\u0259rin analoqu:<\/b> <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/company\/ua-hosting\/blog\/347386\/\">VPS (KVM) E5-2697 v3 (6 Core) 10GB DDR4 480GB SSD 1Gbps haqq\u0131nda b\u00fct\u00fcn h\u0259qiq\u0259t $19-dan, v\u0259 ya serveri nec\u0259 d\u00fczg\u00fcn b\u00f6lm\u0259k olur?<\/a><\/noindex> (RAID1 v\u0259 RAID10 il\u0259 variantlar, 24 n\u00fcv\u0259y\u0259 q\u0259d\u0259r v\u0259 40GB DDR4-a q\u0259d\u0259r m\u00f6vcuddur).<\/p>\n<p><b>Dell R730xd, Amsterdamdak\u0131 Equinix Tier IV data m\u0259rk\u0259zind\u0259 iki d\u0259f\u0259 ucuz?<\/b> Yaln\u0131z bizd\u0259 <b><noindex><a rel=\"nofollow\" href=\"https:\/\/ua-hosting.company\/serversnl\">2 x Intel TetraDeca-Core Xeon 2x E5-2697v3 2.6GHz 14C 64GB DDR4 4x960GB SSD 1Gbps 100 TB yaln\u0131z 199 $-dan<\/a><\/noindex> Niderlandda! <b>Dell R420 \u2014 2x E5-2430 2.2GHz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB \u2014 99 $-dan!<\/b><\/b> Bunun haqq\u0131nda oxuyun <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/company\/ua-hosting\/blog\/329618\/\">Dell R730xd E5-2650 v4 serverl\u0259rind\u0259n istifad\u0259 etm\u0259kl\u0259 korporativ s\u0259viyy\u0259li infrastruktur nec\u0259 qurmaq olar - 9000 avro d\u0259y\u0259rind\u0259 olanlar\u0131 c\u00fczdanla \u0259ld\u0259 etm\u0259k?<\/a><\/noindex><br \/>\n<br \/>M\u0259nb\u0259: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/ua-hosting\/blog\/483112\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>Clickhouse \u2014 \u044d\u0442\u043e \u0441\u0442\u043e\u043b\u0431\u0446\u043e\u0432\u0430\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u0430 \u0443\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0431\u0430\u0437\u0430\u043c\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 \u0434\u043b\u044f \u043e\u043d\u043b\u0430\u0439\u043d \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0438 \u0430\u043d\u0430\u043b\u0438\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0445 \u0437\u0430\u043f\u0440\u043e\u0441\u043e\u0432 (OLAP) \u0441 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u043c \u0438\u0441\u0445\u043e\u0434\u043d\u044b\u043c \u043a\u043e\u0434\u043e\u043c, \u0441\u043e\u0437\u0434\u0430\u043d\u043d\u0430\u044f \u042f\u043d\u0434\u0435\u043a\u0441\u043e\u043c. \u0415\u0435 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044e\u0442 \u042f\u043d\u0434\u0435\u043a\u0441, CloudFlare, VK.com, Badoo \u0438 \u0434\u0440\u0443\u0433\u0438\u0435 \u0441\u0435\u0440\u0432\u0438\u0441\u044b \u043f\u043e \u0432\u0441\u0435\u043c\u0443 \u043c\u0438\u0440\u0443 \u0434\u043b\u044f \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0434\u0435\u0439\u0441\u0442\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0431\u043e\u043b\u044c\u0448\u0438\u0445 \u043e\u0431\u044a\u0435\u043c\u043e\u0432 \u0434\u0430\u043d\u043d\u044b\u0445 (\u0432\u0441\u0442\u0430\u0432\u043a\u0430 \u0442\u044b\u0441\u044f\u0447 \u0441\u0442\u0440\u043e\u043a \u0432 \u0441\u0435\u043a\u0443\u043d\u0434\u0443 \u0438\u043b\u0438 \u043f\u0435\u0442\u0430\u0431\u0430\u0439\u0442\u044b \u0434\u0430\u043d\u043d\u044b\u0445, \u0445\u0440\u0430\u043d\u044f\u0449\u0438\u0445\u0441\u044f \u043d\u0430 \u0434\u0438\u0441\u043a\u0435). \u0412 \u043e\u0431\u044b\u0447\u043d\u043e\u0439, \u00ab\u0441\u0442\u0440\u043e\u043a\u043e\u0432\u043e\u0439\u00bb \u0421\u0423\u0411\u0414, \u043f\u0440\u0438\u043c\u0435\u0440\u0430\u043c\u0438 \u043a\u043e\u0442\u043e\u0440\u044b\u0445 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-54918","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.0.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"Clickhouse \u2014 \u044d\u0442\u043e \u0441\u0442\u043e\u043b\u0431\u0446\u043e\u0432\u0430\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u0430 \u0443\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0431\u0430\u0437\u0430\u043c\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 \u0434\u043b\u044f \u043e\u043d\u043b\u0430\u0439\u043d \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0438 \u0430\u043d\u0430\u043b\u0438\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0445 \u0437\u0430\u043f\u0440\u043e\u0441\u043e\u0432 (OLAP) \u0441 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u043c \u0438\u0441\u0445\u043e\u0434\u043d\u044b\u043c \u043a\u043e\u0434\u043e\u043c.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/az\/blog\/administrirovanie\/ispolzovanie-clickhouse-v-kachestve-zameny-elk-big-query-i-timescaledb\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.0.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"az_AZ\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0418\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435 Clickhouse \u0432 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0435 \u0437\u0430\u043c\u0435\u043d\u044b ELK, Big Query \u0438 TimescaleDB | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"Clickhouse \u2014 \u044d\u0442\u043e \u0441\u0442\u043e\u043b\u0431\u0446\u043e\u0432\u0430\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u0430 \u0443\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0431\u0430\u0437\u0430\u043c\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 \u0434\u043b\u044f \u043e\u043d\u043b\u0430\u0439\u043d \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0438 \u0430\u043d\u0430\u043b\u0438\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0445 \u0437\u0430\u043f\u0440\u043e\u0441\u043e\u0432 (OLAP) \u0441 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u043c \u0438\u0441\u0445\u043e\u0434\u043d\u044b\u043c \u043a\u043e\u0434\u043e\u043c.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/az\/blog\/administrirovanie\/ispolzovanie-clickhouse-v-kachestve-zameny-elk-big-query-i-timescaledb\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-01-06T21:00:00+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-02-18T11:02:58+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47 Clickhouse-u ELK, Big Query v\u0259 TimescaleDB-nin \u0259v\u0259zind\u0259 istifad\u0259 | ProHoster","description":"Clickhouse - a\u00e7\u0131q m\u0259nb\u0259li analitik sor\u011fular\u0131n (OLAP) onlayn i\u015fl\u0259nm\u0259si \u00fc\u00e7\u00fcn s\u00fctunlu veril\u0259nl\u0259r bazas\u0131 idar\u0259etm\u0259 sistemidir.","canonical_url":"https:\/\/prohoster.info\/az\/blog\/administrirovanie\/ispolzovanie-clickhouse-v-kachestve-zameny-elk-big-query-i-timescaledb","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"az_AZ","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0418\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435 Clickhouse \u0432 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0435 \u0437\u0430\u043c\u0435\u043d\u044b ELK, Big Query \u0438 TimescaleDB | ProHoster","og:description":"Clickhouse \u2014 \u044d\u0442\u043e \u0441\u0442\u043e\u043b\u0431\u0446\u043e\u0432\u0430\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u0430 \u0443\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0431\u0430\u0437\u0430\u043c\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 \u0434\u043b\u044f \u043e\u043d\u043b\u0430\u0439\u043d \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0438 \u0430\u043d\u0430\u043b\u0438\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0445 \u0437\u0430\u043f\u0440\u043e\u0441\u043e\u0432 (OLAP) \u0441 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u043c \u0438\u0441\u0445\u043e\u0434\u043d\u044b\u043c \u043a\u043e\u0434\u043e\u043c.","og:url":"https:\/\/prohoster.info\/az\/blog\/administrirovanie\/ispolzovanie-clickhouse-v-kachestve-zameny-elk-big-query-i-timescaledb","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-01-06T21:00:00+00:00","article:modified_time":"2020-02-18T11:02:58+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"54918","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-24 13:09:54","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 19:55:36","updated":"2026-01-24 13:09:54","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/az\/wp-json\/wp\/v2\/posts\/54918","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/az\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/az\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/az\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/az\/wp-json\/wp\/v2\/comments?post=54918"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/az\/wp-json\/wp\/v2\/posts\/54918\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/az\/wp-json\/wp\/v2\/media?parent=54918"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/az\/wp-json\/wp\/v2\/categories?post=54918"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/az\/wp-json\/wp\/v2\/tags?post=54918"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}