{"id":80775,"date":"2020-05-08T13:42:47","date_gmt":"2020-05-08T11:42:47","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah"},"modified":"2020-05-08T13:42:47","modified_gmt":"2020-05-08T11:42:47","slug":"clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah","status":"publish","type":"post","link":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah","title":{"rendered":"ClickHouse pro pokro\u010dil\u00e9 u\u017eivatele v ot\u00e1zk\u00e1ch a odpov\u011bd\u00edch","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>V dubnu se in\u017een\u00fd\u0159i Avito chystali na online setk\u00e1n\u00ed s hlavn\u00edm v\u00fdvoj\u00e1\u0159em ClickHouse Alexejem Milovidovem a Kirillem Shvakovem, v\u00fdvoj\u00e1\u0159em v Golangu z firmy Integros. Diskutovali jsme o tom, jak vyu\u017e\u00edv\u00e1me syst\u00e9m spr\u00e1vy datab\u00e1z\u00ed a jak\u00e9 probl\u00e9my n\u00e1m vznikaj\u00ed. <\/p>\n<p><\/p>\n<p>Na z\u00e1klad\u011b setk\u00e1n\u00ed jsme p\u0159ipravili \u010dl\u00e1nek s odpov\u011b\u010fmi expert\u016f na na\u0161e a div\u00e1ck\u00e9 ot\u00e1zky t\u00fdkaj\u00edc\u00ed se z\u00e1lohov\u00e1n\u00ed, reshardingu dat, extern\u00edch slovn\u00edk\u016f, Golang ovlada\u010de a aktualizace verz\u00ed ClickHouse. M\u016f\u017ee b\u00fdt u\u017eite\u010dn\u00fd pro v\u00fdvoj\u00e1\u0159e, kte\u0159\u00ed ji\u017e aktivn\u011b pracuj\u00ed se SDB 'Yandexu' a zaj\u00edmaj\u00ed se o jej\u00ed sou\u010dasnost a budoucnost. Odpov\u011bdi Alexeje Milovidova jsou defaultn\u00ed, pokud nen\u00ed uvedeno jinak. <\/p>\n<p><\/p>\n<p>Pozor, pod \u0159ezem je hodn\u011b textu. Douf\u00e1me, \u017ee obsah s ot\u00e1zkami v\u00e1m pom\u016f\u017ee se orientovat.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"ClickHouse pro pokro\u010dil\u00e9 u\u017eivatele v ot\u00e1zk\u00e1ch a odpov\u011bd\u00edch\" src=\"\/wp-content\/uploads\/2020\/05\/242b1d8d002fe115614435c242297fd2.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2 id=\"soderzhanie\">Obsah<\/h2>\n<p><\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"#old-data\">ClickHouse se neust\u00e1le aktualizuje, ale na\u0161e data nikoliv. Co s t\u00edm d\u011blat?<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#backup-best-practicies\">Jak\u00e9 jsou nejlep\u0161\u00ed praktiky pro z\u00e1lohov\u00e1n\u00ed dat z ClickHouse v sou\u010dasnosti?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#replication\">Bude mo\u017en\u00e9 organizovat kontrolovan\u00e9 zpo\u017ed\u011bn\u00ed replik v exekuc\u00edch?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#soooo-changeable\">Co d\u011blat, pokud se struktura tabulky zm\u011bnila?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#resharding-best-practices\">Jak\u00e9 jsou nyn\u00ed nejlep\u0161\u00ed praktiky v reshardingu dat?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#clickhouse-copier\">V ClickHouse existuje utilita clickhouse-copier. M\u016f\u017eete o n\u00ed \u0159\u00edct n\u011bco v\u00edc?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#resharding-tool\">M\u011bli jste pilotn\u00ed projekt, kter\u00fd se naz\u00fdval reshading. Co s n\u00edm?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#move-to-slow-disk\">Je mo\u017en\u00e9 slou\u010dit v\u0161echny \u010d\u00e1sti dat dohromady p\u0159ed p\u0159esunem na pomal\u00e9 disky?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#up-to-date\">Jak se p\u0159est\u011bhovat na nov\u00e9 verze ClickHouse, pokud nen\u00ed mo\u017en\u00e9 p\u0159edem ov\u011b\u0159it kompatibilitu?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#kill-query\">Kill query by m\u011bl zab\u00edjet dotazy, ale ned\u011bl\u00e1 to. Pro\u010d?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#reading-time\">Jak vypo\u010d\u00edtat \u010das odezvy p\u0159i \u010dtec\u00edm zat\u00ed\u017een\u00ed?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#pimp-my-clickhouse\">Co nastavit v ClickHouse, aby se v\u00edce dat ve\u0161lo do cache?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#storage-configuration\">Jak lze nastavit storage_configuration pro ulo\u017een\u00ed do opera\u010dn\u00ed pam\u011bti?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#low-cardinality\">Do jak\u00e9ho po\u010dtu unik\u00e1tn\u00edch hodnot je Low Cardinality efektivn\u00ed?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#fulltext-search\">Jak\u00e9 jsou nejlep\u0161\u00ed praktiky pro fulltextov\u00e9 vyhled\u00e1v\u00e1n\u00ed v tabulce s p\u011bti miliardami \u0159\u00e1dk\u016f?<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#hello-and-welcome\">Jak nejl\u00e9pe organizovat p\u0159\u00edstup v ClickHouse pro velk\u00e9 mno\u017estv\u00ed u\u017eivatel\u016f?<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#smorgasbord\">Lze vr\u00e1tit v\u00fdsledky jednoho dotazu deseti klient\u016fm?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#asynchronous\">Jak se vypo\u0159\u00e1dat s asynchronn\u00edmi operacemi a materializovan\u00fdmi pohledy?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#dashboard\">V ClickHouse je hodn\u011b log\u016f. Jak mohu vid\u011bt v\u0161echno, co se d\u011bje se serverem v dan\u00e9m okam\u017eiku?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#zen\">Jak zasahovat do slu\u010dov\u00e1n\u00ed, aby server nespadl do OOM?<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#go\">Jak bude prob\u00edhat v\u00fdvoj Golang ovlada\u010de pro ClickHouse?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#lazy-load\">Extern\u00ed slovn\u00edk se nenahr\u00e1v\u00e1 po restartu s povolen\u00fdm nastaven\u00edm lazy_load. Co d\u011blat?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#reload-dictionaries\">Jak si po\u010d\u00ednat v p\u0159\u00edpad\u011b, \u017ee system reload dictionaries nenahr\u00e1v\u00e1 \u017e\u00e1dn\u00fd z mnoha slovn\u00edk\u016f, pokud alespo\u0148 jeden z nich sel\u017ee s chybou?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#connection\">Existuje zp\u016fsob, jak konfigurovat \u00fadaje v&nbsp;konfigura\u010dn\u00edm souboru ClickHouse, ani\u017e by se zobrazovaly p\u0159i&nbsp;chyb\u00e1ch?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#zoom-backgrounds\">Bonus: pozad\u00ed pro Zoom z&nbsp;poseden\u00ed<\/a><\/noindex><\/li>\n<\/ul>\n<p><\/p>\n<blockquote><p>Pokud se v\u00e1m nechce \u010d\u00edst text, m\u016f\u017eete se pod\u00edvat na z\u00e1znam posezen\u00ed <noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=n1tm4j4W8ZQ&amp;t=8147s\">na&nbsp;na\u0161em YouTube kan\u00e1lu<\/a><\/noindex>. \u010casov\u00e9 k\u00f3dy jsou v&nbsp;prvn\u00edm koment\u00e1\u0159i pod&nbsp;videem.<\/p><\/blockquote>\n<p><\/p>\n<h2 id=\"anchorold-dataanchorclickhouse-postoyanno-obnovlyaetsya-a-nashi-dannyenbsp-net-chto-snbspetim-delat\"><noindex><a rel=\"nofollow\" name=\"old-data\"><\/a><\/noindex>ClickHouse se neust\u00e1le aktualizuje, ale na\u0161e data se&nbsp;neaktualizuj\u00ed. Co s&nbsp;t\u00edm d\u011blat?<\/h2>\n<p><\/p>\n<blockquote><p>ClickHouse se neust\u00e1le aktualizuje, ale na\u0161e data, kter\u00e1 byla optimalizov\u00e1na v\u00fdrazu final, se neaktualizuj\u00ed a le\u017e\u00ed v&nbsp;z\u00e1loze. <\/p>\n<p>P\u0159edstavme si, \u017ee do\u0161lo k n\u011bjak\u00e9mu probl\u00e9mu a data byla ztracena. Rozhodli jsme se obnovit a zjistili jsme, \u017ee star\u00e9 partition, kter\u00e9 le\u017e\u00ed na z\u00e1lohovac\u00edch serverech, se velmi siln\u011b rozch\u00e1zej\u00ed s&nbsp;aktu\u00e1ln\u011b pou\u017e\u00edvanou verz\u00ed ClickHouse. Co v&nbsp;tak\u00e9 situaci d\u011blat, a je to mo\u017en\u00e9?<\/p><\/blockquote>\n<p>Situace, kdy jste obnovili data ze z\u00e1lohy ve&nbsp;star\u00e9m form\u00e1tu a na&nbsp;nov\u00e9 verzi se nep\u0159ipojuj\u00ed, nen\u00ed mo\u017en\u00e1. Db\u00e1me na&nbsp;to, aby form\u00e1t dat v&nbsp;ClickHouse v\u017edy z\u016fstal zp\u011btn\u011b kompatibiln\u00ed. To je mnohem d\u016fle\u017eit\u011bj\u0161\u00ed ne\u017e zp\u011btn\u00e1 kompatibilita funk\u010dnosti, pokud se zm\u011bnilo chov\u00e1n\u00ed n\u011bjak\u00e9 z\u0159\u00eddka pou\u017e\u00edvan\u00e9 funkce. Data, kter\u00e1 jsou ulo\u017eena na&nbsp;disku, by v\u017edy m\u011bla b\u00fdt novou verz\u00ed ClickHouse \u010diteln\u00e1. To je z\u00e1kon. <\/p>\n<p><\/p>\n<h2 id=\"anchorbackup-best-practiciesanchorkakie-luchshie-praktiki-est-nanbspdannyy-moment-ponbsprezervnomu-kopirovaniyu-dannyh-iznbspclickhouse\"><noindex><a rel=\"nofollow\" name=\"backup-best-practicies\"><\/a><\/noindex>Jak\u00e9 jsou nejlep\u0161\u00ed praktiky pro z\u00e1lohov\u00e1n\u00ed dat z ClickHouse v sou\u010dasnosti?<\/h2>\n<p><\/p>\n<blockquote><p>Jak d\u011blat z\u00e1lohy s&nbsp;ohledem na to, \u017ee m\u00e1me operace optimize final, obrovskou datab\u00e1zi na&nbsp;terabajty a data, kter\u00e1 se aktualizuj\u00ed, dejme tomu, za&nbsp;posledn\u00ed t\u0159i dny, a pot\u00e9 s&nbsp;n\u00edmi \u017e\u00e1dn\u00e9 procedury neprob\u00edhaj\u00ed? <\/p>\n<p>M\u016f\u017eeme si vy\u0161perkovat vlastn\u00ed \u0159e\u0161en\u00ed a napsat na&nbsp;bash: shroma\u017e\u010fuj takto a takto tyto z\u00e1lohy. Mo\u017en\u00e1, \u017ee nic vym\u00fd\u0161let nemus\u00edme a kolo u\u017e bylo d\u00e1vno vynalezeno? <\/p><\/blockquote>\n<p>Nejprve k&nbsp;nejlep\u0161\u00edm praktik\u00e1m. Moji kolegov\u00e9 v\u017edy doporu\u010duj\u00ed v&nbsp;odezv\u011b na&nbsp;dotazy ohledn\u011b z\u00e1loh p\u0159ipom\u00ednat slu\u017ebu \u201eYandex.Cloud\u201c, kde je tento \u00fakol ji\u017e vy\u0159e\u0161en. Tak\u017ee ji vyu\u017eijte, pokud m\u00e1te mo\u017enost. <\/p>\n<p><\/p>\n<p>Pln\u00e9 \u0159e\u0161en\u00ed, sto procent integrovan\u00e9 v&nbsp;ClickHouse pro z\u00e1lohy neexistuje. Existuj\u00ed ur\u010dit\u00e9 \u0161ablony, kter\u00e9 lze pou\u017e\u00edt. Abychom z\u00edskali \u00fapln\u00e9 \u0159e\u0161en\u00ed, budeme muset bu\u010f trochu ru\u010dn\u011b popracovat, nebo vytvo\u0159it ob\u00e1lky ve&nbsp;form\u011b skript\u016f.<\/p>\n<p><\/p>\n<p>Za\u010dnu od&nbsp;nejjednodu\u0161\u0161\u00edch \u0159e\u0161en\u00ed a skon\u010d\u00edm t\u011bmi nejn\u00e1ro\u010dn\u011bj\u0161\u00edmi v&nbsp;z\u00e1vislosti na objemu dat a velikosti clusteru. \u010c\u00edm v\u011bt\u0161\u00ed cluster \u2014 t\u00edm slo\u017eit\u011bj\u0161\u00ed se \u0159e\u0161en\u00ed st\u00e1v\u00e1.<\/p>\n<p><\/p>\n<p>Pokud tabulka s&nbsp;daty zab\u00edr\u00e1 pouze n\u011bkolik gigabajt\u016f, m\u016f\u017eete z\u00e1lohu prov\u00e9st takto: <\/p>\n<p><\/p>\n<ol>\n<li>Ulo\u017eit definice tabulek, tedy metadata \u2014 <strong>show create table<\/strong>.<\/li>\n<li>Vytvo\u0159it dump pomoc\u00ed klienta ClickHouse \u2014 <strong>select<\/strong> * <strong>z tabulky<\/strong> do&nbsp;souboru. Standardn\u011b dostanete soubor ve&nbsp;form\u00e1tu TabSeparated. Pokud chcete b\u00fdt efektivn\u011bj\u0161\u00ed \u2014 m\u016f\u017eete pou\u017e\u00edt form\u00e1t Native. <\/li>\n<\/ol>\n<p><\/p>\n<p>Pokud objem dat je v\u011bt\u0161\u00ed, z\u00e1loha zabere v\u00edce \u010dasu a m\u00edsta. Tomu se \u0159\u00edk\u00e1 logick\u00e1 z\u00e1loha, kter\u00e1 nen\u00ed v\u00e1zan\u00e1 na&nbsp;datov\u00fd form\u00e1t ClickHouse. Pokud existuje, m\u016f\u017eete v&nbsp;nejhor\u0161\u00edm p\u0159\u00edpad\u011b vz\u00edt z\u00e1lohu a na\u010d\u00edst ji do&nbsp;MySQL pro&nbsp;obnoven\u00ed. <\/p>\n<p><\/p>\n<p>Pro pokro\u010dilej\u0161\u00ed p\u0159\u00edpady m\u00e1 ClickHouse zabudovanou mo\u017enost vytvo\u0159it snapshot parti\u00ed v&nbsp;lok\u00e1ln\u00edm souborov\u00e9m syst\u00e9mu. Tato mo\u017enost je k&nbsp;dispozici ve&nbsp;form\u011b dotazu <strong>alter table freeze partition<\/strong>. Nebo jednodu\u0161e <strong>alter table freeze<\/strong> \u2014 to je snapshot cel\u00e9 tabulky. <\/p>\n<p><\/p>\n<p>Snapshot bude vytvo\u0159en konzistentn\u00edm pro&nbsp;jednu tabulku na&nbsp;jednom shardu, tedy vytvo\u0159it konzistentn\u00ed snapshot cel\u00e9ho clusteru t\u00edmto zp\u016fsobem nelze. Ale pro&nbsp;v\u011bt\u0161inu \u00faloh takov\u00e1 pot\u0159eba nen\u00ed, a sta\u010d\u00ed na&nbsp;ka\u017ed\u00e9m shardu prov\u00e9st dotaz a z\u00edskat konzistentn\u00ed snapshot. Vytv\u00e1\u0159\u00ed se ve&nbsp;form\u011b hardlink\u016f a proto nezab\u00edr\u00e1 dal\u0161\u00ed m\u00edsto. D\u00e1le tento snapshot zkop\u00edrujete na&nbsp;z\u00e1lohovac\u00ed server nebo do&nbsp;\u00falo\u017ei\u0161t\u011b, kter\u00e9 pou\u017e\u00edv\u00e1te pro&nbsp;z\u00e1lohy.<\/p>\n<p><\/p>\n<p>Obnovit takovou z\u00e1lohu je docela snadn\u00e9. Nejprve&nbsp;\u2014 vytvo\u0159\u00edte tabulky podle&nbsp;existuj\u00edc\u00edch definic tabulek. D\u00e1le zkop\u00edrujete ulo\u017een\u00e9 snapshoty parti\u00ed do&nbsp;Directory-Detached pro&nbsp;dan\u00e9 tabulky a vykon\u00e1te dotaz <strong>attach partition<\/strong>. Takov\u00e9 \u0159e\u0161en\u00ed je zcela vhodn\u00e9 pro&nbsp;nejv\u00e1\u017en\u011bj\u0161\u00ed objemy dat. <\/p>\n<p><\/p>\n<p>N\u011bkdy je zapot\u0159eb\u00ed n\u011bco je\u0161t\u011b slo\u017eit\u011bj\u0161\u00edho&nbsp;\u2014 v&nbsp;t\u011bch p\u0159\u00edpadech, kdy m\u00e1te des\u00edtky nebo dokonce stovky terabajt\u016f na&nbsp;ka\u017ed\u00e9m serveru a stovky server\u016f. Zde je \u0159e\u0161en\u00ed, kter\u00e9 jsem vybledl od&nbsp;kolegy z&nbsp;\u201eYandex.Metriky\u201c. Necht\u011bl bych to doporu\u010dovat ka\u017ed\u00e9mu \u2014 p\u0159e\u010dt\u011bte si a rozhodn\u011bte se sami, zda je to vhodn\u00e9. <\/p>\n<p><\/p>\n<p>Nejprve je t\u0159eba vytvo\u0159it n\u011bkolik server\u016f s velk\u00fdmi diskov\u00fdmi z\u00e1sobn\u00edky. Pot\u00e9 na t\u011bchto serverech nasadit n\u011bkolik ClickHouse server\u016f a nastavit je tak, aby fungovaly jako dal\u0161\u00ed replika pro stejn\u00e9 shardov\u00e9 skupiny. A d\u00e1le vyu\u017e\u00edvat na t\u011bchto serverech souborov\u00fd syst\u00e9m nebo n\u011bjak\u00fd n\u00e1stroj, kter\u00fd umo\u017e\u0148uje vytv\u00e1\u0159et snapshoty. Existuj\u00ed dv\u011b mo\u017enosti. Prvn\u00ed mo\u017enost\u00ed jsou LVM snapshoty, druhou mo\u017enost\u00ed je ZFS na Linuxu. <\/p>\n<p><\/p>\n<p>Po tomto je t\u0159eba ka\u017ed\u00fd den vytv\u00e1\u0159et snapshot, ten bude ulo\u017een a zabere ur\u010dit\u00e9 m\u00edsto. Samoz\u0159ejm\u011b, pokud se data m\u011bn\u00ed, objem m\u00edsta bude postupem \u010dasu nar\u016fstat. Tento snapshot lze kdykoli z\u00edskat a obnovit data, co\u017e je pon\u011bkud zvl\u00e1\u0161tn\u00ed \u0159e\u0161en\u00ed. Nav\u00edc je t\u0159eba omezit tyto repliky v konfiguraci, aby se nesna\u017eily st\u00e1t v\u016fdci.<\/p>\n<p><\/p>\n<h2 id=\"anchorreplicationanchormozhno-li-budet-organizovat-kontroliruemoe-otstavanie-replik-vnbspvalah\"><noindex><a rel=\"nofollow\" name=\"replication\"><\/a><\/noindex>Bude mo\u017en\u00e9 organizovat kontrolovan\u00e9 zpo\u017ed\u011bn\u00ed replik v exekuc\u00edch?<\/h2>\n<p><\/p>\n<blockquote><p>V tomto roce pl\u00e1nujete d\u011blat vlny v ClickHouse. Bude mo\u017en\u00e9 v nich organizovat kontrolovan\u00e9 zpo\u017ed\u011bn\u00ed replik? R\u00e1di bychom se t\u00edmto zp\u016fsobem ochr\u00e1nili p\u0159ed negativn\u00edmi sc\u00e9n\u00e1\u0159i s alteracemi a jin\u00fdmi zm\u011bnami. <\/p>\n<p>Je mo\u017en\u00e9 ud\u011blat n\u011bjak\u00e9 roll back pro alterace? Nap\u0159\u00edklad ve st\u00e1vaj\u00edc\u00ed vln\u011b vz\u00edt a \u0159\u00edci, \u017ee do tohoto okam\u017eiku aplikujte zm\u011bny, a od tohoto okam\u017eiku p\u0159esta\u0148te aplikovat zm\u011bny?<\/p>\n<p>Pokud do na\u0161eho clusteru p\u0159i\u0161el t\u00fdm a po\u0161kodil ho, m\u00e1me podm\u00ednkovou repliku se zpo\u017ed\u011bn\u00edm jedn\u00e9 hodiny, kde m\u016f\u017eeme \u0159\u00edci, \u017ee ji nyn\u00ed budeme pou\u017e\u00edvat, ale posledn\u00edch deset minut zm\u011bn v n\u00ed nebudeme aplikovat? <\/p><\/blockquote>\n<p>Nejprve o kontrolovan\u00e9m zpo\u017ed\u011bn\u00ed replik. Takov\u00fd po\u017eadavek m\u011bl u\u017eivatel a my jsme vytvo\u0159ili issue na GitHubu s prosbou: \u201ePokud to n\u011bkdo pot\u0159ebuje, dejte like, dejte srd\u00ed\u010dko.\u201c Nikdo nedal a issue byla uzav\u0159ena. Nicm\u00e9n\u011b u\u017e nyn\u00ed lze tuto mo\u017enost z\u00edskat, pokud nastav\u00edte ClickHouse. Pravda, pouze od verze 20.3.<\/p>\n<p><\/p>\n<p>ClickHouse neust\u00e1le v pozad\u00ed prov\u00e1d\u00ed slou\u010den\u00ed dat \u2014 merge. Kdy\u017e je merge proveden, ur\u010dit\u00fd soubor datov\u00fdch blok\u016f je nahrazen v\u011bt\u0161\u00edm blokem. P\u0159i\u010dem\u017e bloky dat, kter\u00e9 byly d\u0159\u00edve, z\u016fst\u00e1vaj\u00ed na disku po ur\u010ditou dobu.<\/p>\n<p><\/p>\n<p>Za prv\u00e9, z\u016fst\u00e1vaj\u00ed uchov\u00e1ny, dokud existuj\u00ed dotazy select, kter\u00e9 je vyu\u017e\u00edvaj\u00ed, aby bylo mo\u017en\u00e9 zajistit neblokuj\u00edc\u00ed provoz. Dotazy select v klidu \u010dtou ze star\u00fdch kus\u016f.<\/p>\n<p><\/p>\n<p>Za druh\u00e9, existuje tak\u00e9 \u010dasov\u00fd pr\u00e1h \u2013 star\u00e9 kusy dat le\u017e\u00ed na disku osm minut. T\u011bchto osm minut lze nastavit a p\u0159etvo\u0159it dokonce na jeden den. To si vy\u017e\u00e1d\u00e1 m\u00edsto na disku: v z\u00e1vislosti na toku dat zjist\u00edte, \u017ee za posledn\u00ed den se data nejen zdvojn\u00e1sob\u00ed, ale m\u016f\u017ee jich b\u00fdt p\u011btkr\u00e1t v\u00edce. Ale v p\u0159\u00edpad\u011b v\u00e1\u017en\u00e9ho probl\u00e9mu budete moci zastavit server ClickHouse a v\u0161e vy\u0159e\u0161it.<\/p>\n<p><\/p>\n<p>Nyn\u00ed vyvst\u00e1v\u00e1 ot\u00e1zka, jak to chr\u00e1n\u00ed p\u0159ed alteracemi. Zde je t\u0159eba se pod\u00edvat hloub\u011bji, proto\u017ee ve star\u00fdch verz\u00edch ClickHouse alter fungoval tak, \u017ee jednodu\u0161e m\u011bnil kusy. Existuje kus dat s n\u011bjak\u00fdmi soubory, a my d\u011bl\u00e1me nap\u0159\u00edklad <strong>alter drop column<\/strong>. V tom p\u0159\u00edpad\u011b je tento sloupec fyzicky odstran\u011bn ze v\u0161ech kus\u016f.<\/p>\n<p><\/p>\n<p>Ale od verze 20.3 byl mechanismus alterac\u00ed zcela zm\u011bn\u011bn a nyn\u00ed jsou kusy dat v\u017edy immutable. V\u016fbec se nem\u011bn\u00ed \u2013 alterace nyn\u00ed funguj\u00ed podobn\u011b jako merge. M\u00edsto toho, abychom m\u011bnili kus na m\u00edst\u011b, vytv\u00e1\u0159\u00edme nov\u00fd. V nov\u00e9m kusu se soubory, kter\u00e9 se nezm\u011bnily, st\u00e1vaj\u00ed hardlinky, a pokud jsme odstranili n\u011bjak\u00fd sloupec, jednodu\u0161e bude chyb\u011bt v nov\u00e9m kusu. Star\u00fd kus se odstran\u00ed automaticky za osm minut, a tady je mo\u017en\u00e9 upravit nastaven\u00ed, o kter\u00fdch se mluvilo v\u00fd\u0161e. <\/p>\n<p><\/p>\n<p>Stejn\u011b tak to plat\u00ed pro alty typu mutac\u00ed. Kdy\u017e ud\u011bl\u00e1te <strong>alter delete<\/strong> nebo <strong>alter update<\/strong>, nezm\u011bn\u00ed kus, ale vytvo\u0159\u00ed nov\u00fd. A pot\u00e9 odstran\u00ed star\u00fd.<\/p>\n<p><\/p>\n<h2 id=\"anchorsoooo-changeableanchorkak-byt-esli-struktura-tablicy-pomenyalas\"><noindex><a rel=\"nofollow\" name=\"soooo-changeable\"><\/a><\/noindex>Co d\u011blat, pokud se struktura tabulky zm\u011bnila?<\/h2>\n<p><\/p>\n<blockquote><p>Jak obnovit z\u00e1lohu, kter\u00e1 byla vytvo\u0159ena se starou strukturou? A druh\u00e1 ot\u00e1zka se t\u00fdk\u00e1 p\u0159\u00edpadu se snapshoty a prost\u0159edky souborov\u00e9ho syst\u00e9mu. Je zde Btrfs m\u00edsto ZFS na Linux LVM?<\/p><\/blockquote>\n<p>Pokud d\u011bl\u00e1te <strong>attach partition<\/strong> partice s jinou strukturou, tak v\u00e1m ClickHouse \u0159ekne, \u017ee to nelze. \u0158e\u0161en\u00ed je n\u00e1sleduj\u00edc\u00ed. Prvn\u00ed \u2013 vytvo\u0159it do\u010dasnou tabulku typu MergeTree se starou strukturou, p\u0159ipojit tam data pomoc\u00ed attach, prov\u00e9st dotaz alter. Pot\u00e9 lze bu\u010f zkop\u00edrovat nebo p\u0159esunout tato data a prov\u00e9st attach znovu, nebo pou\u017e\u00edt dotaz <strong>alter table move partition<\/strong>.<\/p>\n<p><\/p>\n<p>Druh\u00fd dotaz je, zda lze pou\u017e\u00edt Btrfs. Pokud m\u00e1te LVM, sta\u010d\u00ed sn\u00edmky LVM a souborov\u00fd syst\u00e9m m\u016f\u017ee b\u00fdt ext4, to nen\u00ed rozhoduj\u00edc\u00ed. Co se t\u00fd\u010de Btrfs, v\u0161echno z\u00e1vis\u00ed na va\u0161ich zku\u0161enostech s jeho pou\u017e\u00edv\u00e1n\u00edm. Je to vysp\u011bl\u00fd souborov\u00fd syst\u00e9m, ale n\u011bkte\u0159\u00ed maj\u00ed st\u00e1le pochyby o tom, jak to ve skute\u010dnosti bude fungovat v konkr\u00e9tn\u00edm sc\u00e9n\u00e1\u0159i. Nedoporu\u010doval bych to pou\u017e\u00edvat, pokud nem\u00e1te Btrfs v produk\u010dn\u00edm prost\u0159ed\u00ed.<\/p>\n<p><\/p>\n<h2 id=\"anchorresharding-best-practicesanchorkakie-seychas-luchshie-praktiki-vnbspreshardinge-dannyh\"><noindex><a rel=\"nofollow\" name=\"resharding-best-practices\"><\/a><\/noindex>Jak\u00e9 jsou nyn\u00ed nejlep\u0161\u00ed praktiky v reshardingu dat?<\/h2>\n<p><\/p>\n<p>Dotaz na p\u0159ehodnocen\u00ed shardingu je slo\u017eit\u00fd a mnohovrstevn\u00fd. M\u016f\u017eeme na n\u011bj odpov\u011bd\u011bt n\u011bkolika zp\u016fsoby. M\u016f\u017eeme vz\u00edt jednu stranu a \u0159\u00edci, \u017ee v ClickHouse nen\u00ed vestav\u011bn\u00e1 mo\u017enost p\u0159ehodnocen\u00ed shardingu. Ob\u00e1v\u00e1m se v\u0161ak, \u017ee tato odpov\u011b\u010f nikoho neuspokoj\u00ed. Tak\u017ee m\u016f\u017eeme p\u0159ij\u00edt s jin\u00fdm \u00fahlem pohledu a \u0159\u00edci, \u017ee v ClickHouse existuje mnoho zp\u016fsob\u016f, jak p\u0159ehodnotit data. <\/p>\n<p><\/p>\n<p>Pokud se na clusteru za\u010d\u00edn\u00e1 vy\u010derp\u00e1vat m\u00edsto nebo nezvl\u00e1d\u00e1 z\u00e1t\u011b\u017e, p\u0159id\u00e1v\u00e1te nov\u00e9 servery. Tyto servery jsou v\u0161ak z defaultu pr\u00e1zdn\u00e9, nemaj\u00ed \u017e\u00e1dn\u00e1 data a nevznik\u00e1 na nich \u017e\u00e1dn\u00e1 z\u00e1t\u011b\u017e. Je t\u0159eba p\u0159esunout data, aby byla rovnom\u011brn\u011b rozlo\u017eena do nov\u011b roz\u0161\u00ed\u0159en\u00e9ho clusteru.<\/p>\n<p><\/p>\n<p>Prvn\u00ed zp\u016fsob, jak to ud\u011blat, je zkop\u00edrovat \u010d\u00e1st partitions na nov\u00e9 servery pomoc\u00ed dotazu. <strong>alter table fetch partition<\/strong>Nap\u0159\u00edklad, pokud jste m\u011bli partitions podle m\u011bs\u00edc\u016f, vezmete prvn\u00ed m\u011bs\u00edc roku 2017 a zkop\u00edrujete na nov\u00fd server, pak t\u0159et\u00ed m\u011bs\u00edc zkop\u00edrujete na n\u011bjak\u00fd jin\u00fd nov\u00fd server. A takto pokra\u010dujete, dokud to nebude v\u00edce m\u00e9n\u011b rovnom\u011brn\u00e9.<\/p>\n<p><\/p>\n<p>P\u0159enos je mo\u017en\u00e9 prov\u00e1d\u011bt pouze pro ty partitions, kter\u00e9 se nem\u011bn\u00ed p\u0159i z\u00e1pisu. Pro nov\u00e9 partitions bude nutn\u00e9 vypnout z\u00e1pis, proto\u017ee jejich p\u0159enos nen\u00ed atom\u00e1rn\u00ed. Jinak dostanete duplicity nebo mezery v datech. Nicm\u00e9n\u011b, tento zp\u016fsob je praktick\u00fd a funguje dost efektivn\u011b. Po s\u00edti se p\u0159en\u00e1\u0161ej\u00ed ji\u017e hotov\u00e9 komprimovan\u00e9 partitions, tedy data se neznovu komprimuj\u00ed ani k\u00f3duj\u00ed.<\/p>\n<p><\/p>\n<p>Tato metoda m\u00e1 jednu nev\u00fdhodu, kter\u00e1 z\u00e1vis\u00ed na sch\u00e9matu shardingu, zda jste toto sch\u00e9ma sharding vytvo\u0159ili, jak\u00fd byl v\u00e1\u0161 kl\u00ed\u010d shardingu. Ve va\u0161em p\u0159\u00edkladu pro p\u0159\u00edpad s metrikami je kl\u00ed\u010d shardingu hash cesty. Kdy\u017e prov\u00e1d\u00edte select na distribuovan\u00e9 tabulce, jde to okam\u017eit\u011b na v\u0161echny shardy clusteru a z\u00edsk\u00e1v\u00e1 z nich data. <\/p>\n<p><\/p>\n<p>To znamen\u00e1, \u017ee pro v\u00e1s vlastn\u011b nez\u00e1le\u017e\u00ed na tom, jak\u00e1 data se nach\u00e1zej\u00ed na kter\u00e9m shardu. Hlavn\u00ed je, \u017ee data podle jedn\u00e9 cesty jsou na jednom shardu, ale na kter\u00e9m konkr\u00e9tn\u011b, na tom nez\u00e1le\u017e\u00ed. V tomto p\u0159\u00edpad\u011b se pro p\u0159enos hotov\u00fdch parti\u00ed skv\u011ble hod\u00ed, proto\u017ee p\u0159i selectech, a\u0165 u\u017e p\u0159ed nebo po p\u0159eshardingov\u00e1n\u00ed, sch\u00e9ma hodnoty moc neznamen\u00e1 \u2014 budete dost\u00e1vat kompletn\u00ed data.<\/p>\n<p><\/p>\n<p>Ale existuj\u00ed tak\u00e9 slo\u017eit\u011bj\u0161\u00ed p\u0159\u00edpady. Pokud se na \u00farovni logiky aplikace zakl\u00e1d\u00e1te na speci\u00e1ln\u00edm sch\u00e9matu shardingu, kde je tento klient um\u00edst\u011bn na takov\u00e9m shardu, a \u017e\u00e1dost m\u016f\u017ee b\u00fdt zasl\u00e1na p\u0159\u00edmo tam, a ne do distribuovan\u00e9 tabulky. Nebo pou\u017e\u00edv\u00e1te dostate\u010dn\u011b \u010derstvou verzi ClickHouse a povolili jste nastaven\u00ed <strong>optimize skip unused shards<\/strong>. V tomto p\u0159\u00edpad\u011b b\u011bhem dotazu select bude v\u00fdraz v sekci where analyzov\u00e1n a bude vypo\u010d\u00edt\u00e1no, na kter\u00e9 shardy je pot\u0159eba j\u00edt podle sch\u00e9matu shardingu. To funguje, pokud jsou data rozlo\u017eena p\u0159esn\u011b podle tohoto sch\u00e9matu shardingu. Pokud jste je p\u0159eorganizovali ru\u010dn\u011b, shoda se m\u016f\u017ee zm\u011bnit.<\/p>\n<p><\/p>\n<p>Tak\u017ee to je prvn\u00ed zp\u016fsob. A \u010dek\u00e1m na va\u0161i odpov\u011b\u010f, zda je tento zp\u016fsob vhodn\u00fd, nebo pokra\u010dujeme d\u00e1l.<\/p>\n<p><\/p>\n<p><strong>Vladimir Kolobajev, lead system administrator ve slu\u017eb\u011b Avito<\/strong>: Alexej, ten zp\u016fsob, kter\u00fd jste zm\u00ednil, se moc dob\u0159e nehod\u00ed, kdy\u017e je t\u0159eba rozlo\u017eit z\u00e1t\u011b\u017e v\u010detn\u011b \u010dten\u00ed. M\u016f\u017eeme vz\u00edt m\u011bs\u00ed\u010dn\u00ed partii a p\u0159edchoz\u00ed m\u011bs\u00edc p\u0159en\u00e9st na jinou nodu, ale kdy\u017e p\u0159ijde \u017e\u00e1dost o tato data, budeme zat\u011b\u017eovat pouze ji. A cht\u011blo by se zat\u011b\u017eovat cel\u00fd cluster, proto\u017ee v opa\u010dn\u00e9m p\u0159\u00edpad\u011b bude po ur\u010ditou dobu ve\u0161ker\u00e1 z\u00e1t\u011b\u017e na \u010dten\u00ed zpracov\u00e1v\u00e1na dv\u011bma shardy.<\/p>\n<p><\/p>\n<p><strong>Alexei Milovidov:<\/strong> Odpov\u011b\u010f je zde zvl\u00e1\u0161tn\u00ed \u2014 ano, \u0161patn\u011b, ale m\u016f\u017ee to i vyj\u00edt. Vysv\u011btl\u00edm, jak p\u0159esn\u011b. Je t\u0159eba se pod\u00edvat na sc\u00e9n\u00e1\u0159 z\u00e1t\u011b\u017ee, kter\u00e1 p\u0159ich\u00e1z\u00ed za va\u0161imi daty. Pokud se jedn\u00e1 o data monitorov\u00e1n\u00ed, t\u00e9m\u011b\u0159 s jistotou lze \u0159\u00edci, \u017ee p\u0159ev\u00e1\u017en\u00e1 v\u011bt\u0161ina dotaz\u016f jde za \u010derstv\u00fdmi daty. <\/p>\n<p><\/p>\n<p>Nasadili jste nov\u00e9 servery, p\u0159esunuli star\u00e9 partitiony, ale tak\u00e9 jste zm\u011bnili zp\u016fsob, jak\u00fdm se zapisuj\u00ed nov\u00e9 \u00fadaje. Nov\u00e9 \u00fadaje budou rozprost\u0159eny po&nbsp;cel\u00e9m clusteru. T\u00edm p\u00e1dem u\u017e za p\u011bt minut budou dotazy za posledn\u00edch p\u011bt minut rovnom\u011brn\u011b zat\u011b\u017eovat cluster, za den budou dotazy za 24 hodin rovnom\u011brn\u011b zat\u011b\u017eovat cluster. Dotazy za p\u0159edchoz\u00ed m\u011bs\u00edc v\u0161ak bohu\u017eel p\u016fjdou pouze na \u010d\u00e1st server\u016f clusteru.<\/p>\n<p><\/p>\n<p>Ale \u010dasto nebudete m\u00edt dotazy pr\u00e1v\u011b za \u00fanor 2019. Pravd\u011bpodobn\u011b, pokud u\u017e dotazy sm\u011b\u0159uj\u00ed do roku 2019, budou se t\u00fdkat cel\u00e9ho roku 2019 \u2014 za velk\u00fd \u010dasov\u00fd interval, ne za n\u011bjak\u00fd mal\u00fd rozsah. Takov\u00e9 dotazy tak\u00e9 dok\u00e1\u017eou rovnom\u011brn\u011b zat\u011b\u017eovat cluster. Ale celkov\u011b je va\u0161e pozn\u00e1mka naprosto spr\u00e1vn\u00e1, \u017ee se jedn\u00e1 o ad hoc \u0159e\u0161en\u00ed, kter\u00e9 \u00fapln\u011b rovnom\u011brn\u011b nerozprost\u00ed \u00fadaje.<\/p>\n<p><\/p>\n<p>M\u00e1m je\u0161t\u011b n\u011bkolik bod\u016f k odpov\u011bdi na ot\u00e1zku. Jeden z nich se t\u00fdk\u00e1 toho, jak od za\u010d\u00e1tku nastavit sch\u00e9ma shardov\u00e1n\u00ed tak, aby bylo m\u00e9n\u011b bolestiv\u00e9 p\u0159i p\u0159eshardov\u00e1n\u00ed. To nen\u00ed v\u017edy mo\u017en\u00e9.<\/p>\n<p><\/p>\n<p>Nap\u0159\u00edklad m\u00e1te \u00fadaje o monitorov\u00e1n\u00ed. \u00dadaje o monitorov\u00e1n\u00ed rostou ze t\u0159\u00ed d\u016fvod\u016f. Prvn\u00ed \u2014 akumulace historick\u00fdch \u00fadaj\u016f. Druh\u00e1 \u2014 n\u00e1r\u016fst provozu. A t\u0159et\u00ed \u2014 zvy\u0161uj\u00edc\u00ed se po\u010det polo\u017eek, kter\u00e9 spadaj\u00ed pod monitorov\u00e1n\u00ed. Objevuj\u00ed se nov\u00e9 mikroservisy a metriky, kter\u00e9 je t\u0159eba uchovat. <\/p>\n<p><\/p>\n<p>Je mo\u017en\u00e9, \u017ee z nich nejv\u011bt\u0161\u00ed n\u00e1r\u016fst souvis\u00ed pr\u00e1v\u011b s t\u0159et\u00edm d\u016fvodem \u2014 to je zv\u00fd\u0161en\u00e9 pou\u017e\u00edv\u00e1n\u00ed monitorov\u00e1n\u00ed. A v tomto p\u0159\u00edpad\u011b je d\u016fle\u017eit\u00e9 se pod\u00edvat na charakter z\u00e1t\u011b\u017ee, jak\u00e9 jsou hlavn\u00ed dotazy na select. Hlavn\u00ed dotazy na select pravd\u011bpodobn\u011b sm\u011b\u0159uj\u00ed na ur\u010dit\u00e9 podmno\u017estv\u00ed metrik.<\/p>\n<p><\/p>\n<p>Nap\u0159\u00edklad vyu\u017eit\u00ed CPU na n\u011bkter\u00fdch serverech n\u011bjakou slu\u017ebou. Vych\u00e1z\u00ed to tak, \u017ee existuje ur\u010dit\u00e9 podmno\u017estv\u00ed kl\u00ed\u010d\u016f, podle kter\u00fdch tyto \u00fadaje z\u00edsk\u00e1v\u00e1te. A samotn\u00fd dotaz na tyto \u00fadaje je pravd\u011bpodobn\u011b dost jednoduch\u00fd a vykon\u00e1v\u00e1 se b\u011bhem des\u00edtek milisekund. Pou\u017e\u00edv\u00e1 se pro monitorovac\u00ed slu\u017eby, pro dashboardy. Douf\u00e1m, \u017ee to ch\u00e1pu spr\u00e1vn\u011b.<\/p>\n<p><\/p>\n<p><strong>Vladimir Kolobajev:<\/strong> Jde o to, \u017ee velmi \u010dasto apelujeme na historick\u00e1 data, proto\u017ee v re\u00e1ln\u00e9m \u010dase porovn\u00e1v\u00e1me aktu\u00e1ln\u00ed stav s historick\u00fdm. A pro n\u00e1s je d\u016fle\u017eit\u00e9 m\u00edt rychl\u00fd p\u0159\u00edstup k velk\u00e9mu objemu dat, a ClickHouse s t\u00edm skv\u011ble vych\u00e1z\u00ed.<\/p>\n<p><\/p>\n<p>M\u00e1te naprostou pravdu, v\u011bt\u0161inu \u017e\u00e1dost\u00ed o \u010dten\u00ed za\u017e\u00edv\u00e1me za posledn\u00ed den, jako ka\u017ed\u00e1 monitorovac\u00ed syst\u00e9m. Ale i tak je zat\u00ed\u017een\u00ed historick\u00fdch dat tak\u00e9 pom\u011brn\u011b vysok\u00e9. Poch\u00e1z\u00ed v\u011bt\u0161inou od alertn\u00edho syst\u00e9mu, kter\u00fd ka\u017ed\u00fdch t\u0159icet vte\u0159in oslovuje ClickHouse: \u201eDej mi data za posledn\u00edch \u0161est t\u00fddn\u016f. A te\u010f mi z nich vytvo\u0159 n\u011bjak\u00fd klouzav\u00fd pr\u016fm\u011br, a porovnejme aktu\u00e1ln\u00ed hodnotu s historickou.\u201c <\/p>\n<p><\/p>\n<p>Cht\u011bl bych \u0159\u00edci, \u017ee m\u00e1me pro takov\u00e9 velmi \u010derstv\u00e9 dotazy je\u0161t\u011b jednu malou tabulku, ve kter\u00e9 uchov\u00e1v\u00e1me pouze dvoudenn\u00ed data, a hlavn\u00ed dotazy sm\u011b\u0159uj\u00ed do n\u00ed. Do v\u011bt\u0161\u00ed shardovan\u00e9 tabulky pos\u00edl\u00e1me pouze velk\u00e9 historick\u00e9 dotazy.<\/p>\n<p><\/p>\n<p><strong>Alexei Milovidov:<\/strong> Bohu\u017eel, pro v\u00e1\u0161 sc\u00e9n\u00e1\u0159 se to ukazuje jako \u0161patn\u011b aplikovateln\u00e9, ale pop\u00ed\u0161u v\u00e1m dv\u011b \u0161patn\u00e9 a slo\u017eit\u00e9 sch\u00e9mata shardov\u00e1n\u00ed, kter\u00e1 by se nem\u011bla pou\u017e\u00edvat, ale kter\u00e1 jsou pou\u017eita v slu\u017eb\u011b m\u00fdch p\u0159\u00e1tel. <\/p>\n<p><\/p>\n<p>Existuje hlavn\u00ed cluster se ud\u00e1lostmi \u201eYandex.Metriky\u201c. Ud\u00e1losti \u2013 to jsou zobrazen\u00ed str\u00e1nek, kliknut\u00ed a p\u0159echody. V\u011bt\u0161ina dotaz\u016f sm\u011b\u0159uje na konkr\u00e9tn\u00ed webov\u00e9 str\u00e1nky. Otev\u00edr\u00e1te slu\u017ebu \u201eYandex.Metrika\u201c, m\u00e1te web \u2013 avito.ru, vstoup\u00edte do zpr\u00e1vy a prob\u00edh\u00e1 dotaz na va\u0161em webu.<\/p>\n<p><\/p>\n<p>Ale existuj\u00ed i dal\u0161\u00ed dotazy \u2013 analytick\u00e9 a glob\u00e1ln\u00ed, kter\u00e9 prov\u00e1d\u011bj\u00ed intern\u00ed analytici. Pro po\u0159\u00e1dek poznamen\u00e1m, \u017ee intern\u00ed analytici prov\u00e1d\u011bj\u00ed dotazy pouze na slu\u017eb\u00e1ch \u201eYandexu\u201c. Nicm\u00e9n\u011b, i slu\u017eby \u201eYandexu\u201c zauj\u00edmaj\u00ed zna\u010dn\u00fd pod\u00edl v\u0161ech dat. To jsou dotazy ne na konkr\u00e9tn\u00ed m\u011b\u0159i\u010de, ale na \u0161ir\u0161\u00ed filtrace.<\/p>\n<p><\/p>\n<p>Jak uspo\u0159\u00e1dat data tak, aby efektivn\u011b fungovala jak pro jeden m\u011b\u0159i\u010d, tak pro glob\u00e1ln\u00ed dotazy? Obt\u00ed\u017enost spo\u010d\u00edv\u00e1 tak\u00e9 v tom, \u017ee po\u010det dotaz\u016f v ClickHouse na cluster \u201eMetriky\u201c \u2013 n\u011bkolik tis\u00edc za sekundu. P\u0159itom netrivi\u00e1ln\u00ed dotazy, nap\u0159\u00edklad n\u011bkolik tis\u00edc za sekundu, jeden server ClickHouse nezvl\u00e1dne.<\/p>\n<p><\/p>\n<p>Velikost clusteru&nbsp;\u2014 \u0161est set s&nbsp;n\u011b\u010d\u00edm servery. Pokud nad&nbsp;t\u00edmto clusterem jednodu\u0161e nat\u00e1hneme Distributed tabulku a pos\u00edl\u00e1me tam n\u011bkolik tis\u00edc dotaz\u016f, bude to je\u0161t\u011b hor\u0161\u00ed, ne\u017e je pos\u00edlat na&nbsp;jeden server. Na druhou stranu, varianta s&nbsp;t\u00edm, \u017ee jsou data rovnom\u011brn\u011b rozprost\u0159ena, a my jdeme a dotazujeme ze&nbsp;v\u0161ech server\u016f, se okam\u017eit\u011b zam\u00edt\u00e1.<\/p>\n<p><\/p>\n<p>Existuje diametr\u00e1ln\u011b opa\u010dn\u00e1 varianta. P\u0159edstavte si, \u017ee data budeme shardovat podle&nbsp;web\u016f, a dotaz pro&nbsp;jeden web p\u016fjde na&nbsp;jeden shard. Nyn\u00ed cluster bude schopn\u00fd zvl\u00e1dnout deset tis\u00edc dotaz\u016f za&nbsp;sekundu, ale na&nbsp;jednom shardu n\u011bjak\u00fd jeden dotaz bude fungovat p\u0159\u00edli\u0161 pomalu. Nebude ji\u017e \u0161k\u00e1lovat podle&nbsp;propustnosti. Zvl\u00e1\u0161\u0165 pokud je to web avito.ru. Nebudu odhalovat tajemstv\u00ed, kdy\u017e \u0159eknu, \u017ee Avito&nbsp;\u2014 to je jeden z&nbsp;nejnav\u0161t\u011bvovan\u011bj\u0161\u00edch web\u016f v&nbsp;runetu. A zpracovat ho na&nbsp;jednom shardu by bylo \u0161\u00edlenstv\u00ed.<\/p>\n<p><\/p>\n<p>Proto je sch\u00e9ma shardov\u00e1n\u00ed uspo\u0159\u00e1d\u00e1no slo\u017eit\u011bji. Cel\u00fd cluster je rozd\u011blen na&nbsp;n\u011bkolik clusterk\u016f, kter\u00e9 naz\u00fdv\u00e1me vrstvami. Uvnit\u0159 ka\u017ed\u00e9ho clusterku je od&nbsp;deseti do&nbsp;n\u011bkolika des\u00edtek shard\u016f. A celkem existuje t\u0159icet dev\u011bt takov\u00fdch clusterk\u016f. <\/p>\n<p><\/p>\n<p>Jak se to v\u0161echno \u0161k\u00e1luje? Po\u010det clusterk\u016f se nem\u011bn\u00ed&nbsp;\u2014 jak bylo p\u0159ed n\u011bkolika lety t\u0159icet dev\u011bt, tak to z\u016fstalo. Ale uvnit\u0159 ka\u017ed\u00e9ho z&nbsp;nich postupn\u011b zvy\u0161ujeme po\u010det shard\u016f s&nbsp;jak se \u00fadaje hromad\u00ed. A sch\u00e9ma shardov\u00e1n\u00ed je takov\u00e9&nbsp;\u2014 rozd\u011blen\u00ed na&nbsp;tyto clusterky se prov\u00e1d\u00ed podle&nbsp;web\u016f, a abychom pochopili, kter\u00fd web je na&nbsp;jak\u00e9m clusteru, pou\u017e\u00edv\u00e1 se v\u016fbec samostatn\u00e1 metabase v&nbsp;MySQL. Jeden web&nbsp;\u2014 na&nbsp;jednom clusterku. A uvnit\u0159 n\u011bho prob\u00edh\u00e1 shardov\u00e1n\u00ed podle&nbsp;identifik\u00e1tor\u016f n\u00e1v\u0161t\u011bvn\u00edk\u016f.<\/p>\n<p><\/p>\n<p>P\u0159i zapisov\u00e1n\u00ed je rozd\u011blujeme podle zbytku po d\u011blen\u00ed identifik\u00e1toru n\u00e1v\u0161t\u011bvn\u00edka. Ale p\u0159i p\u0159id\u00e1v\u00e1n\u00ed nov\u00e9ho shardu se sch\u00e9ma shardov\u00e1n\u00ed m\u011bn\u00ed, pokra\u010dujeme v d\u011blen\u00ed, ale se zbytkem po d\u011blen\u00ed jin\u00fdm \u010d\u00edslem. To znamen\u00e1, \u017ee jeden n\u00e1v\u0161t\u011bvn\u00edk je skute\u010dn\u011b um\u00edst\u011bn na n\u011bkolika serverech, a na to se nelze spol\u00e9hat. Bylo to ud\u011bl\u00e1no v\u00fdhradn\u011b za \u00fa\u010delem lep\u0161\u00edho komprese dat. P\u0159i dotazech p\u0159istupujeme k Distributed tabulce, kter\u00e1 proch\u00e1z\u00ed clusterem a obrac\u00ed se na des\u00edtky server\u016f. Takov\u00fd je ten hloup\u00fd syst\u00e9m.<\/p>\n<p><\/p>\n<p>Ale m\u016fj p\u0159\u00edb\u011bh by nebyl \u00fapln\u00fd, kdybych ne\u0159ekl, \u017ee jsme se od tohoto sch\u00e9matu vzdali. V nov\u00e9m sch\u00e9matu jsme v\u0161e zm\u011bnili a v\u0161echna data zkop\u00edrovali pomoc\u00ed clickhouse-copier.<\/p>\n<p><\/p>\n<p>V nov\u00e9m sch\u00e9matu jsou v\u0161echny weby rozd\u011bleny do dvou kategori\u00ed \u2014 velk\u00e9 a mal\u00e9. Nezn\u00e1m pr\u00e1h, podle kter\u00e9ho byl vybr\u00e1n, ale v\u00fdsledkem je, \u017ee velk\u00e9 weby jsou zapisov\u00e1ny do jednoho clusteru, kde je 120 shard\u016f po t\u0159ech replik\u00e1ch v ka\u017ed\u00e9m \u2014 tedy 360 server\u016f. A sch\u00e9ma shardov\u00e1n\u00ed je takov\u00e9, \u017ee ka\u017ed\u00fd dotaz m\u00ed\u0159\u00ed rovnou na v\u0161echny shardy. Pokud se te\u010f v 'Yandex.Metrice' otev\u0159ete jakoukoli str\u00e1nku zpr\u00e1vy pro avito.ru, dotaz p\u016fjde na 120 server\u016f. Velk\u00fdch web\u016f v ru-netu je m\u00e1lo. A dotazy se tedy nez\u00edsk\u00e1vaj\u00ed tis\u00edce za sekundu, ale dokonce m\u00e9n\u011b ne\u017e sto. V\u0161echno to bez probl\u00e9m\u016f zvl\u00e1dne Distributed tabulka, kterou ka\u017ed\u00fd z nich zpracov\u00e1v\u00e1 120 servery.<\/p>\n<p><\/p>\n<p>Druh\u00fd cluster je pro mal\u00e9 weby. Zde je sch\u00e9ma shardov\u00e1n\u00ed podle identifik\u00e1toru webu a ka\u017ed\u00fd dotaz jde p\u0159esn\u011b na jeden shard.<\/p>\n<p><\/p>\n<h2 id=\"anchorclickhouse-copieranchorv-clickhouse-est-utilita-clickhouse-copier-mozhete-pronbspneyo-rasskazat\"><noindex><a rel=\"nofollow\" name=\"clickhouse-copier\"><\/a><\/noindex>V ClickHouse existuje utilita clickhouse-copier. M\u016f\u017eete o n\u00ed \u0159\u00edct n\u011bco v\u00edc?<\/h2>\n<p><\/p>\n<p>Ihned \u0159eknu, \u017ee toto \u0159e\u0161en\u00ed je objemn\u011bj\u0161\u00ed a o n\u011bco m\u00e9n\u011b v\u00fdkonn\u00e9. V\u00fdhodou je, \u017ee rozkl\u00e1d\u00e1 data zcela podle sch\u00e9matu, kter\u00e9 uvedete. Ale nev\u00fdhodou n\u00e1stroje je, \u017ee v\u016fbec neprov\u00e1d\u00ed p\u0159eshardov\u00e1n\u00ed. Kop\u00edruje data z jednoho sch\u00e9matu clusteru do jin\u00e9ho sch\u00e9matu clusteru.<\/p>\n<p><\/p>\n<p>To znamen\u00e1, \u017ee pro jej\u00ed provozov\u00e1n\u00ed mus\u00edte m\u00edt dva clustery. Mohou b\u00fdt um\u00edst\u011bny na stejn\u00fdch serverech, ale p\u0159esto nebudou data p\u0159esouv\u00e1na inkrement\u00e1ln\u011b, ale budou zkop\u00edrov\u00e1na. <\/p>\n<p><\/p>\n<p>Nap\u0159\u00edklad bylo \u010dty\u0159i servery, stalo se jich osm. Vytv\u00e1\u0159\u00edte na v\u0161ech serverech novou distribuovanou tabulku, nov\u00e9 m\u00edstn\u00ed tabulky a spou\u0161t\u00edte clickhouse-copier, p\u0159i\u010dem\u017e mu ur\u010dujete pracovn\u00ed sch\u00e9ma, \u017ee by m\u011bl \u010d\u00edst odtud, p\u0159ijmout novou shardingovou sch\u00e9ma a p\u0159en\u00e1\u0161et data tam. A na star\u00fdch serverech budete pot\u0159ebovat m\u00edsto o jeden a p\u016fl kr\u00e1t v\u00edce, ne\u017e je nyn\u00ed, proto\u017ee star\u00e1 data mus\u00ed z\u016fstat na nich a nav\u00edc na n\u011b p\u0159ijde polovina z t\u011bchto star\u00fdch dat. Pokud jste p\u0159edem zv\u00e1\u017eili, \u017ee je t\u0159eba data p\u0159echardit a m\u00edsto je, pak je tento zp\u016fsob vhodn\u00fd.<\/p>\n<p><\/p>\n<p>Jak funguje clickhouse-copier? Rozd\u011bluje celou pr\u00e1ci na sadu \u00faloh pro zpracov\u00e1n\u00ed jedn\u00e9 partie jedn\u00e9 tabulky na jednom shardu. V\u0161echny tyto \u00falohy mohou b\u00fdt prov\u00e1d\u011bny paraleln\u011b a clickhouse-copier m\u016f\u017ee b\u00fdt spu\u0161t\u011bn na r\u016fzn\u00fdch stroj\u00edch v n\u011bkolika instanc\u00edch, ale to, co d\u011bl\u00e1 pro jednu partii, nen\u00ed nic jin\u00e9ho ne\u017e insert select. Data se \u010dtou, decomprimuj\u00ed, p\u0159eorganizov\u00e1vaj\u00ed, znovu komprimuj\u00ed, zapisuj\u00ed n\u011bkam, p\u0159et\u0159\u00ed\u010fuj\u00ed se. To je t\u011b\u017e\u0161\u00ed \u0159e\u0161en\u00ed.<\/p>\n<p><\/p>\n<h2 id=\"anchorresharding-toolanchoru-vas-byla-pilotnaya-shtuka-kotoraya-nazyvalas-resharding-chto-snbspney\"><noindex><a rel=\"nofollow\" name=\"resharding-tool\"><\/a><\/noindex>M\u011bli jste pilotn\u00ed projekt, kter\u00fd se naz\u00fdval reshading. Co s n\u00edm?<\/h2>\n<p><\/p>\n<blockquote><p>M\u011bli jste u\u017e v roce 2017 pilotn\u00ed projekt, kter\u00fd se jmenoval resharding. Dokonce existuje mo\u017enost v ClickHouse. Ch\u00e1pu, \u017ee to neusp\u011blo. M\u016f\u017eete \u0159\u00edct, pro\u010d se to stalo? Vypad\u00e1 to p\u0159itom velmi aktu\u00e1ln\u011b.<\/p><\/blockquote>\n<p>Cel\u00fd probl\u00e9m je v tom, \u017ee p\u0159i pot\u0159eb\u011b p\u0159echardit data na m\u00edst\u011b je zapot\u0159eb\u00ed velmi slo\u017eit\u00e9 synchronizace, aby se to provedlo atom\u00e1rn\u011b. Kdy\u017e jsme se za\u010dali d\u00edvat na to, jak je tato synchronizace uspo\u0159\u00e1d\u00e1na, uk\u00e1zalo se, \u017ee existuj\u00ed fundament\u00e1ln\u00ed probl\u00e9my. A tyto fundament\u00e1ln\u00ed probl\u00e9my nejsou jen teoretick\u00e9, ale tak\u00e9 se hned za\u010daly projevovat v praxi t\u00edm, \u017ee se to d\u00e1 vysv\u011btlit velmi jednodu\u0161e \u2014 nic nefunguje.<\/p>\n<p><\/p>\n<h2 id=\"anchormove-to-slow-diskanchormozhno-li-slivat-vse-chasti-dannyh-voedino-perednbspperemescheniem-nanbspmedlennye-diski\"><noindex><a rel=\"nofollow\" name=\"move-to-slow-disk\"><\/a><\/noindex>Je mo\u017en\u00e9 slou\u010dit v\u0161echny \u010d\u00e1sti dat dohromady p\u0159ed p\u0159esunem na pomal\u00e9 disky?<\/h2>\n<p><\/p>\n<blockquote><p>Dotaz na TTL s mo\u017enost\u00ed p\u0159esunout na pomal\u00fd disk v kontextu merge. Existuje zp\u016fsob, krom\u011b cron, jak slou\u010dit v\u0161echny \u010d\u00e1sti do jedn\u00e9 p\u0159ed p\u0159esunem na pomal\u00e9 disky?<\/p><\/blockquote>\n<p>Odpov\u011b\u010f na ot\u00e1zku, zda je mo\u017en\u00e9 automaticky slou\u010dit v\u0161echny kousky do jedn\u00e9 p\u0159ed jejich p\u0159esunem \u2014 nen\u00ed. Mysl\u00edm, \u017ee nen\u00ed pot\u0159eba. Lze to jednodu\u0161e ne slou\u010dit v\u0161echny \u010d\u00e1sti do jedn\u00e9, ale prost\u011b po\u010d\u00edtat s t\u00edm, \u017ee budou automaticky p\u0159esunuty na pomal\u00e9 disky. <\/p>\n<p><\/p>\n<p>M\u00e1me dv\u011b krit\u00e9ria pravidla p\u0159esunu. Prvn\u00ed \u2013 na z\u00e1klad\u011b zapln\u011bn\u00ed. Pokud je na aktu\u00e1ln\u00ed \u00farovni \u00falo\u017ei\u0161t\u011b m\u00e9n\u011b ne\u017e ur\u010dit\u00fd procento voln\u00e9ho m\u00edsta, vybereme jeden blok a p\u0159esuneme ho na pomalej\u0161\u00ed \u00falo\u017ei\u0161t\u011b. Sp\u00ed\u0161e ne na pomalej\u0161\u00ed, ale na n\u00e1sleduj\u00edc\u00ed \u2013 jak to nastav\u00edte.<\/p>\n<p><\/p>\n<p>Druh\u00e9 krit\u00e9rium \u2013 podle velikosti. T\u00fdk\u00e1 se p\u0159esunu velk\u00fdch blok\u016f. M\u016f\u017eete nastavit pr\u00e1h podle voln\u00e9ho m\u00edsta na rychl\u00e9m disku a data se budou p\u0159esouvat automaticky.<\/p>\n<p><\/p>\n<h2 id=\"anchorup-to-dateanchorkak-pereezzhat-nanbspnovye-versii-clickhouse-esli-net-vozmozhnosti-zaranee-proverit-sovmestimost\"><noindex><a rel=\"nofollow\" name=\"up-to-date\"><\/a><\/noindex>Jak se p\u0159est\u011bhovat na nov\u00e9 verze ClickHouse, pokud nen\u00ed mo\u017en\u00e9 p\u0159edem ov\u011b\u0159it kompatibilitu?<\/h2>\n<p><\/p>\n<blockquote><p>Toto t\u00e9ma se pravideln\u011b diskutuje <noindex><a rel=\"nofollow\" href=\"https:\/\/teleg.run\/clickhouse_ru\">v telegr\u00e1mov\u00e9m chatu ClickHouse<\/a><\/noindex> s ohledem na r\u016fzn\u00e9 verze, a p\u0159esto. Jak bezpe\u010dn\u00e9 je aktualizovat z verze 19.11 na 19.16 a nap\u0159\u00edklad z 19.16 na 20.3. Jak nejl\u00e9pe p\u0159ech\u00e1zet na nov\u00e9 verze, ani\u017e bychom m\u011bli mo\u017enost p\u0159edem ov\u011b\u0159it kompatibilitu v p\u00edse\u010dn\u00e9 sf\u00e9\u0159e?<\/p><\/blockquote>\n<p>Tady je n\u011bkolik \u201ezlat\u00fdch\u201c pravidel. Prvn\u00ed \u2013 <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/ClickHouse\/ClickHouse\/blob\/master\/CHANGELOG.md\">\u010d\u00edst changelog<\/a><\/noindex>. Je to velk\u00e9, ale existuj\u00ed zde jednotliv\u00e9 body o zp\u011btn\u011b nekompatibiln\u00edch zm\u011bn\u00e1ch. Nem\u011bli byste tyto body br\u00e1t jako \u010dervenou vlajku. Obvykle se jedn\u00e1 o drobn\u00e9 nekompatibility spojen\u00e9 s ur\u010ditou hrani\u010dn\u00ed funkcionalitou, kterou pravd\u011bpodobn\u011b nepou\u017e\u00edv\u00e1te.<\/p>\n<p><\/p>\n<p>Druh\u00e9 \u2013 pokud nen\u00ed mo\u017en\u00e9 ov\u011b\u0159it kompatibilitu v p\u00edse\u010dn\u00e9 sf\u00e9\u0159e a chcete se aktualizovat p\u0159\u00edmo v produkci, doporu\u010den\u00ed je \u2013 ned\u011blejte to. Nejprve vytvo\u0159te p\u00edse\u010dnou sf\u00e9ru a ov\u011b\u0159te. Pokud nem\u00e1te testovac\u00ed prost\u0159ed\u00ed, pravd\u011bpodobn\u011b nem\u00e1te p\u0159\u00edli\u0161 velkou firmu, tak\u017ee m\u016f\u017eete zkop\u00edrovat \u010d\u00e1st dat na sv\u016fj notebook a na n\u011bm se ujistit, \u017ee v\u0161e spr\u00e1vn\u011b funguje. M\u016f\u017eete dokonce zprovoznit n\u011bkolik replik lok\u00e1ln\u011b na sv\u00e9m po\u010d\u00edta\u010di. Nebo m\u016f\u017eete n\u011bkde pobl\u00ed\u017e zprovoznit novou verzi a nahr\u00e1t tam \u010d\u00e1st dat \u2013 tedy vytvo\u0159it improvizovan\u00e9 testovac\u00ed prost\u0159ed\u00ed. <\/p>\n<p><\/p>\n<p>Dal\u0161\u00ed pravidlo \u2013 neaktualizovat se v pr\u016fb\u011bhu t\u00fddne po vyd\u00e1n\u00ed verze kv\u016fli odchyt\u00e1v\u00e1n\u00ed chyb v produkci a n\u00e1sledn\u00fdm rychl\u00fdm oprav\u00e1m. Poj\u010fme se pod\u00edvat na \u010d\u00edslov\u00e1n\u00ed verz\u00ed ClickHouse, abychom se neztratili. <\/p>\n<p><\/p>\n<p>Existuje verze 20.3.4. \u010c\u00edslo 20 ozna\u010duje rok vyd\u00e1n\u00ed \u2014 2020. Z hlediska toho, co je uvnit\u0159, to nem\u00e1 \u017e\u00e1dn\u00fd v\u00fdznam, tak\u017ee se t\u00edm nebudeme zab\u00fdvat. D\u00e1le \u2014 20.3. Druhou \u010d\u00edslici \u2014 v tomto p\u0159\u00edpad\u011b 3 \u2014 zvy\u0161ujeme poka\u017ed\u00e9, kdy\u017e vyd\u00e1me verzi s n\u011bjakou novou funkcionalitou. Pokud chceme do ClickHouse p\u0159idat n\u011bjakou mo\u017enost, mus\u00edme toto \u010d\u00edslo zv\u00fd\u0161it. Tak\u017ee ve verzi 20.4 bude ClickHouse fungovat je\u0161t\u011b l\u00e9pe. T\u0159et\u00ed \u010d\u00edslice \u2014 20.3.4. Zde 4 je po\u010det patch-releases, ve kter\u00fdch jsme nep\u0159id\u00e1vali \u017e\u00e1dn\u00e9 nov\u00e9 mo\u017enosti, ale opravovali n\u011bjak\u00e9 chyby. A 4 znamen\u00e1, \u017ee jsme to ud\u011blali \u010dty\u0159ikr\u00e1t.<\/p>\n<p><\/p>\n<p>Nem\u011bli byste si myslet, \u017ee je to n\u011bco hrozn\u00e9ho. Obvykle si u\u017eivatel m\u016f\u017ee nainstalovat nejnov\u011bj\u0161\u00ed verzi a ta bude fungovat bez jak\u00fdchkoli probl\u00e9m\u016f s uptime po dobu jednoho roku. Ale p\u0159edstavte si, \u017ee u n\u011bjak\u00e9 funkce pro zpracov\u00e1n\u00ed bitmap, kterou p\u0159idali na\u0161i \u010d\u00edn\u0161t\u00ed kolegov\u00e9, server spadne p\u0159i p\u0159ed\u00e1n\u00ed nespr\u00e1vn\u00fdch argument\u016f. Mus\u00edme to opravit. Vyd\u00e1me novou patch-verzi a ClickHouse se stane stabiln\u011bj\u0161\u00edm.<\/p>\n<p><\/p>\n<p>Pokud v\u00e1m ClickHouse b\u011b\u017e\u00ed v produkci a vych\u00e1z\u00ed nov\u00e1 verze ClickHouse s nov\u00fdmi funkcemi \u2014 nap\u0159\u00edklad 20.4.1 \u2014 neusp\u011bchejte s jej\u00edm nasazen\u00edm do produkce hned prvn\u00ed den. Pro\u010d ji vlastn\u011b pot\u0159ebujete? Pokud je\u0161t\u011b ClickHouse nepou\u017e\u00edv\u00e1te, m\u016f\u017eete jej nainstalovat, a pravd\u011bpodobn\u011b v\u0161e p\u016fjde dob\u0159e. Ale pokud ClickHouse u\u017e stabiln\u011b funguje, sledujte patchy a aktualizace \u2014 jak\u00e9 probl\u00e9my opravujeme.<\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> Cht\u011bl bych p\u0159idat n\u011bco m\u00e1lo o testovac\u00edch prost\u0159ed\u00edch. V\u0161ichni se hodn\u011b boj\u00ed testovac\u00edch prost\u0159ed\u00ed a n\u011bjak podivn\u011b si mysl\u00ed, \u017ee pokud m\u00e1te velmi velk\u00fd cluster ClickHouse, pak i testovac\u00ed prost\u0159ed\u00ed mus\u00ed b\u00fdt alespo\u0148 stejn\u011b velk\u00e9 nebo alespo\u0148 desetkr\u00e1t men\u0161\u00ed. To v\u016fbec nen\u00ed pravda.<\/p>\n<p><\/p>\n<p>Mohu \u0159\u00edct na sv\u00e9m p\u0159\u00edkladu. M\u00e1m projekt a tam je ClickHouse. Na\u0161e testovac\u00ed prost\u0159ed\u00ed pro n\u011bj je mal\u00fd virtu\u00e1ln\u00ed server v Hetzneru za dvacet eur, kde je v\u0161echno nasazeno. Abychom toho dos\u00e1hli, m\u00e1me kompletn\u00ed automatizaci v Ansible, a proto v z\u00e1sad\u011b nez\u00e1le\u017e\u00ed na tom, zda to nasad\u00edme na fyzick\u00e9 servery nebo jednodu\u0161e na virtu\u00e1ln\u00ed stroje.<\/p>\n<p><\/p>\n<p>Co lze ud\u011blat? Bylo by dobr\u00e9 vytvo\u0159it v&nbsp;dokumentaci ClickHouse p\u0159\u00edklad, jak si nastavit mal\u00fd cluster&nbsp;\u2014 v&nbsp;Dockeru, v&nbsp;LXC, eventu\u00e1ln\u011b vytvo\u0159it Ansible playbook, proto\u017ee ka\u017ed\u00fd m\u00e1 jin\u00fd deployment. To by velmi usnadnilo pr\u00e1ci. Kdy\u017e si b\u011bhem p\u011bti minut rozjede\u0161 cluster, je mnohem jednodu\u0161\u0161\u00ed se v&nbsp;n\u011b\u010dem zorientovat. To je mnohem pohodln\u011bj\u0161\u00ed, proto\u017ee nasazen\u00ed do&nbsp;produk\u010dn\u00ed verze, kterou jsi nevyzkou\u0161el&nbsp;\u2014 to je cesta do&nbsp;nikam. N\u011bkdy to funguje, n\u011bkdy ne. A proto spol\u00e9hat se na&nbsp;\u00fasp\u011bch&nbsp;\u2014 nen\u00ed dobr\u00e9.<\/p>\n<p><\/p>\n<p><strong>Maksim Kotyakov, senior backend engineer Avito:<\/strong> Trochu dopln\u00edm o&nbsp;testovac\u00edch prost\u0159ed\u00edch na z\u00e1klad\u011b probl\u00e9m\u016f velk\u00fdch spole\u010dnost\u00ed. M\u00e1me pln\u011b funk\u010dn\u00ed akcepta\u010dn\u00ed cluster ClickHouse, kter\u00fd je p\u0159esnou kopi\u00ed datov\u00fdch sch\u00e9mat a nastaven\u00ed toho, co m\u00e1me v&nbsp;produkci. Tento cluster je nasazen ve&nbsp;velmi omezen\u00fdch kontejnerech s&nbsp;minimem zdroj\u016f. Zapisujeme tam ur\u010dit\u00fd procento z&nbsp;produktivn\u00edch dat, na\u0161t\u011bst\u00ed m\u00e1me mo\u017enost replikovat data do&nbsp;Kafky. V\u0161echno je synchronizov\u00e1no a \u0161k\u00e1lov\u00e1no&nbsp;\u2014 jak co se schopnost\u00ed t\u00fd\u010de, tak podle toku, a v&nbsp;teorii by se to m\u011blo chovat podle metrik jako produkce. V\u0161echny potenci\u00e1ln\u011b problematick\u00e9 v\u011bci nejd\u0159\u00edve prob\u00edhaj\u00ed na tomto standu a n\u011bkolik dn\u00ed tam z\u016fst\u00e1vaj\u00ed, dokud nejsou p\u0159ipraven\u00e9. Ale samoz\u0159ejm\u011b, toto \u0159e\u0161en\u00ed je n\u00e1kladn\u00e9, t\u011b\u017ek\u00e9 a s&nbsp;n\u00edzk\u00fdmi n\u00e1klady na&nbsp;\u00fadr\u017ebu. <\/p>\n<p><\/p>\n<p><strong>Alexei Milovidov:<\/strong> Pov\u00edm v\u00e1m, jak vypad\u00e1 testovac\u00ed prost\u0159ed\u00ed na\u0161ich p\u0159\u00e1tel z&nbsp;\"Yandex.Metriky\". Jeden cluster m\u011bl p\u0159es 600 server\u016f, druh\u00fd 360, a existuje je\u0161t\u011b t\u0159et\u00ed a n\u011bkolik dal\u0161\u00edch cluster\u016f. Testovac\u00ed prost\u0159ed\u00ed pro&nbsp;jeden z&nbsp;nich&nbsp;\u2014 to jsou prost\u011b dva shardy s&nbsp;dv\u011bma replikami v&nbsp;ka\u017ed\u00e9m. Pro\u010d dva shardy? Aby byl v\u00edc ne\u017e jeden. A tak\u00e9 repliky, aby tam byly. Prost\u011b n\u011bjak\u00e9 minim\u00e1ln\u00ed mno\u017estv\u00ed, kter\u00e9 si m\u016f\u017ee\u0161 dovolit.<\/p>\n<p><\/p>\n<p>Toto testovac\u00ed prost\u0159ed\u00ed umo\u017e\u0148uje zkontrolovat funk\u010dnost dotaz\u016f a zda se n\u011bco z\u00e1sadn\u00edho neporouchalo. Ale \u010dasto se objevuj\u00ed probl\u00e9my zcela jin\u00e9ho charakteru, kdy v\u0161echno funguje, ale jsou zde n\u011bjak\u00e9 men\u0161\u00ed zm\u011bny v&nbsp;z\u00e1t\u011b\u017ei.<\/p>\n<p><\/p>\n<p>D\u00e1m p\u0159\u00edklad. Rozhodli jsme se nainstalovat novou verzi ClickHouse. Byla uvoln\u011bna na testovac\u00ed prost\u0159ed\u00ed, pro\u0161ly automatizovan\u00e9 testy v samotn\u00e9 \u201eYandex.Metrice\u201c, kter\u00e9 porovn\u00e1vaj\u00ed data ze star\u00e9 verze a nov\u00e9 verze, a projede se cel\u00fd proces. A samoz\u0159ejm\u011b, zelen\u00e9 testy na\u0161eho CI. Jinak bychom tuto verzi ani nenab\u00eddli.<\/p>\n<p><\/p>\n<p>V\u0161e je v po\u0159\u00e1dku. Za\u010d\u00edn\u00e1me nasazovat do produkce. Dost\u00e1v\u00e1m zpr\u00e1vu, \u017ee na grafech se z\u00e1t\u011b\u017e zv\u00fd\u0161ila n\u011bkolikr\u00e1t. Vrac\u00edme verzi zp\u011bt. D\u00edv\u00e1m se na graf a vid\u00edm: z\u00e1t\u011b\u017e se skute\u010dn\u011b zv\u00fd\u0161ila n\u011bkolikr\u00e1t b\u011bhem nasazen\u00ed a zp\u011bt klesla, kdy\u017e jsme ji nasadili. Potom jsme za\u010dali vracet verzi. A z\u00e1t\u011b\u017e se p\u0159esn\u011b tak zv\u00fd\u0161ila a p\u0159esn\u011b tak zp\u011bt klesla. Tak\u017ee z\u00e1v\u011br je takov\u00fd \u2014 z\u00e1t\u011b\u017e vzrostla v souvislosti s nasazen\u00edm, nic p\u0159ekvapiv\u00e9ho.<\/p>\n<p><\/p>\n<p>D\u00e1le bylo t\u011b\u017ek\u00e9 p\u0159esv\u011bd\u010dit kolegy, aby skute\u010dn\u011b nainstalovali novou verzi. \u0158\u00edk\u00e1m: \u201eV\u0161e je v po\u0159\u00e1dku, nasazujte. Dr\u017ete palce, v\u0161echno bude fungovat. Te\u010f se z\u00e1t\u011b\u017e na grafech zv\u00fd\u0161ila, ale v\u0161e je v po\u0159\u00e1dku. Dr\u017ete se.\u201c Tak jsme to ud\u011blali a v\u0161ichni \u2014 verze byla nasazena do produkce. Ale p\u0159i ka\u017ed\u00e9m nasazen\u00ed vznikaj\u00ed podobn\u00e9 probl\u00e9my.<\/p>\n<p><\/p>\n<h2 id=\"anchorkill-queryanchorkill-query-dolzhen-ubivat-zaprosy-no-on-etogo-ne-delaet-pochemu\"><noindex><a rel=\"nofollow\" name=\"kill-query\"><\/a><\/noindex>Kill query by m\u011bl zab\u00edjet dotazy, ale ned\u011bl\u00e1 to. Pro\u010d?<\/h2>\n<p><\/p>\n<blockquote><p>P\u0159i\u0161el za mnou u\u017eivatel, n\u011bjak\u00fd analytik, a vytvo\u0159il ur\u010dit\u00fd dotaz, kter\u00fd polo\u017eil m\u016fj cluster ClickHouse. N\u011bjakou uzel nebo cel\u00fd cluster \u2014 v z\u00e1vislosti na tom, do jak\u00e9 repliky nebo shardu dotaz dopadl. Vid\u00edm, \u017ee v\u0161echny zdroje CPU na tomto serveru jsou pln\u00e9, v\u0161e \u010derven\u00e9. P\u0159itom samotn\u00fd ClickHouse na dotazy odpov\u00edd\u00e1. A j\u00e1 p\u00ed\u0161u: \u201eUka\u017e mi, pros\u00edm, seznam proces\u016f, kter\u00fd dotaz zp\u016fsobil tento chaos.\u201c<\/p>\n<p>Nach\u00e1z\u00edm tento dotaz a p\u00ed\u0161u mu kill. A vid\u00edm, \u017ee se nic ned\u011bje. M\u016fj server je pln\u00fd, ClickHouse mi d\u00e1l poskytuje n\u011bjak\u00e9 p\u0159\u00edkazy, ukazuje, \u017ee server \u017eije, a v\u0161echno je skv\u011bl\u00e9. Ale m\u00e1m degradaci ve v\u0161ech u\u017eivatelsk\u00fdch dotazech, za\u010d\u00edn\u00e1 degradace z\u00e1pis\u016f do ClickHouse a m\u016fj kill dotaz nefunguje. Pro\u010d? Myslel jsem, \u017ee kill dotaz by m\u011bl zab\u00edjet dotazy, ale to se ned\u011bje.<\/p><\/blockquote>\n<p>Te\u010f bude docela podivn\u00e1 odpov\u011b\u010f. Jde o to, \u017ee kill dotaz nezab\u00edj\u00ed dotazy. <\/p>\n<p><\/p>\n<p>Kill query nastavuje malou vlaje\u010dku s n\u00e1zvem \u201echci, aby tento dotaz byl zru\u0161en\u201c. A samotn\u00fd dotaz p\u0159i zpracov\u00e1n\u00ed ka\u017ed\u00e9ho bloku se d\u00edv\u00e1 na tuto vlaje\u010dku. Pokud je nastavena, dotaz p\u0159estane fungovat. To znamen\u00e1, \u017ee nikdo dotaz nezab\u00edj\u00ed, on si mus\u00ed v\u0161e zkontrolovat a zastavit se. A m\u011blo by to fungovat ve v\u0161ech p\u0159\u00edpadech, kdy se dotaz nach\u00e1z\u00ed ve stavu zpracov\u00e1n\u00ed blok\u016f dat. Zpracuje dal\u0161\u00ed datov\u00fd blok, zkontroluje vlaje\u010dku a zastav\u00ed se.<\/p>\n<p><\/p>\n<p>To nefunguje v p\u0159\u00edpadech, kdy je dotaz zablokov\u00e1n na n\u011bjak\u00e9 operaci. Pravda, pravd\u011bpodobn\u011b to nen\u00ed v\u00e1\u0161 p\u0159\u00edpad, proto\u017ee podle va\u0161ich slov vyu\u017e\u00edv\u00e1 hromadu serverov\u00fdch zdroj\u016f. Mo\u017en\u00e1, \u017ee to nefunguje v p\u0159\u00edpad\u011b extern\u00edho t\u0159\u00edd\u011bn\u00ed a je\u0161t\u011b v n\u011bkter\u00fdch detailech. Ale obecn\u011b by to tak b\u00fdt nem\u011blo, to je chyba. A jedin\u00e9, co mohu doporu\u010dit, je aktualizovat ClickHouse.<\/p>\n<p><\/p>\n<h2 id=\"anchorreading-timeanchorkak-rasschitat-vremya-otveta-pri-chitayuschey-nagruzke\"><noindex><a rel=\"nofollow\" name=\"reading-time\"><\/a><\/noindex>Jak vypo\u010d\u00edtat dobu odezvy p\u0159i \u010dtec\u00ed z\u00e1t\u011b\u017ei?<\/h2>\n<p><\/p>\n<blockquote><p>Existuje tabulka, ve kter\u00e9 jsou uchov\u00e1v\u00e1ny agreg\u00e1ty podle item \u2013 r\u016fzn\u00e9 po\u010d\u00edtadla. Po\u010det \u0159\u00e1dk\u016f je p\u0159ibli\u017en\u011b sto milion\u016f. Lze o\u010dek\u00e1vat p\u0159edv\u00eddateln\u00fd \u010das odezvy, pokud zah\u00e1j\u00edme 1K RPS na 1K item\u016f? <\/p><\/blockquote>\n<p>Soud\u011b podle kontextu, mluv\u00edme o \u010dtec\u00ed z\u00e1t\u011b\u017ei, proto\u017ee na z\u00e1pis nejsou \u017e\u00e1dn\u00e9 probl\u00e9my \u2013 a\u0165 u\u017e tis\u00edce, nebo statis\u00edce, n\u011bkdy i n\u011bkolik milion\u016f \u0159\u00e1dk\u016f lze vkl\u00e1dat. <\/p>\n<p><\/p>\n<p>\u010ctec\u00ed dotazy maj\u00ed r\u016fzn\u00e9 podoby. Ve select 1 m\u016f\u017ee ClickHouse vykon\u00e1vat n\u011bkolik des\u00edtek tis\u00edc dotaz\u016f za sekundu, tak\u017ee i dotazy podle jednoho kl\u00ed\u010de u\u017e budou vy\u017eadovat ur\u010dit\u00e9 zdroje. A takov\u00e9 c\u00edlen\u00e9 dotazy budou slo\u017eit\u011bj\u0161\u00ed ne\u017e v jak\u00fdchkoliv key-value datab\u00e1z\u00edch, proto\u017ee pro ka\u017ed\u00e9 \u010dten\u00ed je t\u0159eba p\u0159e\u010d\u00edst datov\u00fd blok podle indexu. Index u n\u00e1s adresuje ne ka\u017ed\u00fd z\u00e1znam, ale ka\u017ed\u00fd rozsah. To znamen\u00e1, \u017ee bude nutn\u00e9 p\u0159e\u010d\u00edst cel\u00fd rozsah \u2013 to je 8192 \u0159\u00e1dk\u016f ve v\u00fdchoz\u00edm nastaven\u00ed. A bude nutn\u00e9 dekomprimovat komprimovan\u00fd datov\u00fd blok z 64 kB na 1 MB. Obvykle takov\u00e9 c\u00edlen\u00e9 dotazy trvaj\u00ed od n\u011bkolika milisekund. Ale to je nejjednodu\u0161\u0161\u00ed p\u0159\u00edpad.<\/p>\n<p><\/p>\n<p>Poj\u010fme se pokusit ud\u011blat jednoduchou aritmetiku. Pokud vyn\u00e1sob\u00edme n\u011bkolik milisekund tis\u00edcem, dostaneme n\u011bkolik sekund. Jako by nebylo mo\u017en\u00e9 dr\u017eet tis\u00edc po\u017eadavk\u016f za sekundu, ale ve skute\u010dnosti je to mo\u017en\u00e9, proto\u017ee m\u00e1me n\u011bkolik procesorov\u00fdch jader. Tak\u017ee v principu m\u016f\u017ee ClickHouse n\u011bkdy zvl\u00e1dnout 1000 RPS, ale p\u0159i kr\u00e1tk\u00fdch, konkr\u00e9tn\u00edch dotazech.<\/p>\n<p><\/p>\n<p>Pokud je pot\u0159eba \u0161k\u00e1lovat cluster ClickHouse podle po\u010dtu jednoduch\u00fdch dotaz\u016f, doporu\u010duji to nejjednodu\u0161\u0161\u00ed - zv\u00fd\u0161it po\u010det replik a pos\u00edlat dotazy na n\u00e1hodnou replik. Pokud jedna replikace zvl\u00e1dne p\u011bt set dotaz\u016f za sekundu, co\u017e je zcela re\u00e1ln\u00e9, pak t\u0159i replikace zvl\u00e1dnou patn\u00e1ct set.<\/p>\n<p><\/p>\n<p>N\u011bkdy je mo\u017en\u00e9 ClickHouse konfigurovat na maxim\u00e1ln\u00ed po\u010det konkr\u00e9tn\u00edch \u010dten\u00ed. Co k tomu pot\u0159ebujeme? Prvn\u00ed - sn\u00ed\u017eit granularitu indexu. P\u0159i tom by se m\u011bla sni\u017eovat ne na jednotku, ale s v\u00fdpo\u010dtem, \u017ee po\u010det z\u00e1znam\u016f v indexu bude n\u011bkolik milion\u016f nebo des\u00edtek milion\u016f na server. Pokud m\u00e1 tabulka sto milion\u016f \u0159\u00e1dk\u016f, jako granularitu lze nastavit 64.<\/p>\n<p><\/p>\n<p>M\u016f\u017eeme sn\u00ed\u017eit velikost komprimovan\u00e9ho bloku. K tomu existuj\u00ed nastaven\u00ed. <strong>min compress block size<\/strong>, <strong>max compress block size<\/strong>M\u016f\u017eete je zmen\u0161it, p\u0159eorganizovat data, a pak budou konkr\u00e9tn\u00ed dotazy rychlej\u0161\u00ed. Ale st\u00e1le ClickHouse nen\u00ed datab\u00e1ze typu key-value. Velk\u00e9 mno\u017estv\u00ed mal\u00fdch dotaz\u016f je antipattern z\u00e1t\u011b\u017ee.<\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> D\u00e1m radu pro p\u0159\u00edpad, \u017ee tam jsou b\u011b\u017en\u00e9 \u00fa\u010detn\u00ed polo\u017eky. To je dost standardn\u00ed situace, kdy v ClickHouse ukl\u00e1daj\u00ed n\u011bjak\u00fd \u010d\u00edta\u010d. M\u00e1m u\u017eivatele, je z ur\u010dit\u00e9 zem\u011b, a je\u0161t\u011b n\u011bjak\u00e9 t\u0159et\u00ed pole, a pot\u0159ebujete inkrement\u00e1ln\u011b n\u011bco zvy\u0161ovat. Vezmete MySQL, vytvo\u0159\u00edte unik\u00e1tn\u00ed kl\u00ed\u010d - v MySQL je to duplicate key, a v PostgreSQL je to conflict - a p\u0159id\u00e1v\u00e1te pluskem. To bude fungovat mnohem l\u00e9pe. <\/p>\n<p><\/p>\n<p>Kdy\u017e m\u00e1te m\u00e1lo dat, nem\u00e1 smysl pou\u017e\u00edvat ClickHouse. Existuj\u00ed b\u011b\u017en\u00e9 datab\u00e1ze a s t\u00edm se dob\u0159e vyrovn\u00e1vaj\u00ed. <\/p>\n<p><\/p>\n<h2 id=\"anchorpimp-my-clickhouseanchorchto-podtyunit-v-clickhouse-chtoby-bolshe-dannyh-bylo-vnbspkeshe\"><noindex><a rel=\"nofollow\" name=\"pimp-my-clickhouse\"><\/a><\/noindex>Co v ClickHouse vyladit, aby v\u00edce dat bylo v cache?<\/h2>\n<p><\/p>\n<blockquote><p>P\u0159edstavme si situaci - na serverech je 256 GB RAM, v ka\u017edodenn\u00ed rutin\u011b ClickHouse bere p\u0159ibli\u017en\u011b 60-80 GB, v maximu a\u017e 130. Co lze zapnout a vyladit, aby v\u00edce dat bylo v cache a t\u00edm p\u00e1dem byly m\u00e9n\u011b p\u0159\u00edstupy na disk?<\/p><\/blockquote>\n<p>Obecn\u011b plat\u00ed, \u017ee page cache opera\u010dn\u00edho syst\u00e9mu si s touto \u00falohou dob\u0159e porad\u00ed. Kdy\u017e otev\u0159ete t\u0159eba top a pod\u00edv\u00e1te se na cached nebo free \u2014 tam tak\u00e9 uvid\u00edte, kolik je ulo\u017eeno v cache. M\u016f\u017eete si v\u0161imnout, \u017ee cel\u00e1 voln\u00e1 pam\u011b\u0165 je vyu\u017eita pro cache. A tato data se p\u0159i \u010dten\u00ed na\u010d\u00edtaj\u00ed ne z disku, ale z opera\u010dn\u00ed pam\u011bti. M\u016f\u017eu \u0159\u00edct, \u017ee cache je efektivn\u011b vyu\u017e\u00edv\u00e1na, proto\u017ee se ukl\u00e1daj\u00ed pr\u00e1v\u011b komprimovan\u00e1 data.<\/p>\n<p><\/p>\n<p>Nicm\u00e9n\u011b, pokud chcete je\u0161t\u011b v\u00edce urychlit n\u011bkter\u00e9 jednoduch\u00e9 dotazy, je mo\u017enost v ClickHouse zapnout cache pro dekomprimovan\u00e1 data. Tomu se \u0159\u00edk\u00e1 <strong>uncompressed cache<\/strong>. V konfigura\u010dn\u00edm souboru config.xml nastavte velikost uncompressed cache na po\u017eadovanou hodnotu \u2014 doporu\u010duji maxim\u00e1ln\u011b polovinu voln\u00e9 opera\u010dn\u00ed pam\u011bti, proto\u017ee zbytek p\u016fjde na page cache. <\/p>\n<p><\/p>\n<p>Krom\u011b toho existuj\u00ed dv\u011b nastaven\u00ed na \u00farovni dotazu. Prvn\u00ed nastaven\u00ed \u2014 <strong>use uncompressed cache<\/strong> \u2014 to zahrnuje jej\u00ed pou\u017eit\u00ed. Doporu\u010duje se ji zapnout pro v\u0161echny dotazy krom\u011b t\u011b\u017ek\u00fdch, kter\u00e9 mohou p\u0159e\u010d\u00edst v\u0161echny dostupn\u00e9 \u00fadaje a tento cache vypr\u00e1zdnit. Druh\u00e1 nastaven\u00ed je n\u011bco jako maxim\u00e1ln\u00ed po\u010det \u0159\u00e1dk\u016f, kter\u00e9 lze cache pou\u017e\u00edt. Automaticky omezuje velk\u00e9 dotazy, aby je obe\u0161ly.<\/p>\n<p><\/p>\n<h2 id=\"anchorstorage-configurationanchorkak-mozhno-nastroit-storage_configuration-dlya-hraneniya-v-operativke\"><noindex><a rel=\"nofollow\" name=\"storage-configuration\"><\/a><\/noindex>Jak lze nastavit storage_configuration pro ulo\u017een\u00ed do opera\u010dn\u00ed pam\u011bti?<\/h2>\n<p><\/p>\n<blockquote><p>V nov\u00e9 dokumentaci ClickHouse jsem nalezl \u010d\u00e1st souvisej\u00edc\u00ed <noindex><a rel=\"nofollow\" href=\"https:\/\/clickhouse.tech\/docs\/en\/single\/#table_engine-mergetree-multiple-volumes\">s data storage<\/a><\/noindex>. V popisu je p\u0159\u00edklad s rychl\u00fdm SSD. <\/p>\n<p>Zaj\u00edmav\u00e9 je, jak stejn\u00e9 nakonfigurovat s memory hot volume. A je\u0161t\u011b jedna ot\u00e1zka. Jak funguje select s takovou organizac\u00ed dat, jestli na\u010d\u00edt\u00e1 cel\u00fd set nebo jen ten, kter\u00fd le\u017e\u00ed na disku, a jestli se tato data komprimuj\u00ed v pam\u011bti? A jak funguje sekce prewhere v takov\u00e9 organizaci dat?<\/p><\/blockquote>\n<p>Toto nastaven\u00ed ovliv\u0148uje ukl\u00e1d\u00e1n\u00ed datov\u00fdch blok\u016f, ale jejich form\u00e1t se nem\u011bn\u00ed.<br \/>\nPod\u00edvejme se na to podrobn\u011bji. <\/p>\n<p><\/p>\n<p>Ulo\u017een\u00ed dat v opera\u010dn\u00ed pam\u011bti lze nakonfigurovat. V\u0161echno, co se konfiguruje pro disk, je jeho cesta. Vytvo\u0159\u00edte tmpfs odd\u00edl, kter\u00fd je p\u0159ipojen k ur\u010dit\u00e9 cest\u011b v souborov\u00e9m syst\u00e9mu. Tuto cestu uvedete jako cestu pro ukl\u00e1d\u00e1n\u00ed dat pro nejv\u00edce hot odd\u00edl, kam za\u010d\u00ednaj\u00ed p\u0159ich\u00e1zet a zapisovat se datov\u00e9 bloky, v\u0161echno funguje dob\u0159e. <\/p>\n<p><\/p>\n<p>Ale nedoporu\u010duji to d\u011blat kv\u016fli n\u00edzk\u00e9 spolehlivosti, i kdy\u017e pokud m\u00e1te alespo\u0148 t\u0159i repliky v r\u016fzn\u00fdch datov\u00fdch centrech, m\u016f\u017ee to fungovat. V p\u0159\u00edpad\u011b pot\u0159eby lze data obnovit. P\u0159edstavme si, \u017ee server byl n\u00e1hle vypnut a znovu zapnut. Odd\u00edl se op\u011bt p\u0159ipojil, ale je tam pr\u00e1zdno. Server ClickHouse p\u0159i startu vid\u00ed, \u017ee mu tyto \u010d\u00e1sti chyb\u00ed, i kdy\u017e podle metadat ZooKeeper by m\u011bly b\u00fdt dostupn\u00e9. Kontroluje, na kter\u00fdch replik\u00e1ch se nach\u00e1zej\u00ed, \u017e\u00e1d\u00e1 je a stahuje. Takto budou data obnovena. <\/p>\n<p><\/p>\n<p>V tomto smyslu se ukl\u00e1d\u00e1n\u00ed dat v opera\u010dn\u00ed pam\u011bti princip\u00e1ln\u011b neli\u0161\u00ed od ukl\u00e1d\u00e1n\u00ed na disku, proto\u017ee p\u0159i z\u00e1pisu dat na disk se nejprve dost\u00e1vaj\u00ed do str\u00e1nkov\u00e9ho cache a fyzicky se zapisuj\u00ed zpo\u017ed\u011bn\u011b. To z\u00e1vis\u00ed na zp\u016fsobu p\u0159ipojen\u00ed souborov\u00e9ho syst\u00e9mu. Pro jistotu v\u0161ak uv\u00e1d\u00edm, \u017ee ClickHouse p\u0159i vkl\u00e1d\u00e1n\u00ed neprov\u00e1d\u00ed fsync.<\/p>\n<p><\/p>\n<p>Data v opera\u010dn\u00ed pam\u011bti jsou p\u0159itom uchov\u00e1v\u00e1na ve stejn\u00e9m form\u00e1tu jako na disku. Dotaz select vyb\u00edr\u00e1 \u010d\u00e1sti, kter\u00e9 je t\u0159eba \u010d\u00edst, stejn\u00fdm zp\u016fsobem, v kusech vyb\u00edr\u00e1 pot\u0159ebn\u00e9 datov\u00e9 intervaly a \u010dte je. A prewhere funguje zcela stejn\u011b, bez ohledu na to, zda byla data v opera\u010dn\u00ed pam\u011bti nebo na disku.<\/p>\n<p><\/p>\n<h2 id=\"anchorlow-cardinalityanchordo-kakogo-kolichestva-unikalnyh-znacheniy-effektiven-low-cardinality\"><noindex><a rel=\"nofollow\" name=\"low-cardinality\"><\/a><\/noindex>Do jak\u00e9ho po\u010dtu unik\u00e1tn\u00edch hodnot je Low Cardinality efektivn\u00ed?<\/h2>\n<p><\/p>\n<p>Low Cardinality je chyt\u0159e navr\u017eeno. Vytv\u00e1\u0159\u00ed datov\u00e9 slovn\u00edky, ale ty jsou m\u00edstn\u00ed. Za prv\u00e9, slovn\u00edky jsou vlastn\u00ed pro ka\u017ed\u00fd kus, za druh\u00e9, i uvnit\u0159 jednoho kusu mohou b\u00fdt r\u016fzn\u00e9 pro ka\u017ed\u00fd interval. Kdy\u017e po\u010det unik\u00e1tn\u00edch hodnot dos\u00e1hne prahov\u00e9 hodnoty \u2014 mysl\u00edm, \u017ee jeden milion \u2014 slovn\u00edk se jednodu\u0161e odlo\u017e\u00ed a vytvo\u0159\u00ed se nov\u00fd.<\/p>\n<p><\/p>\n<p>Odpov\u011b\u010f v z\u00e1sad\u011b zn\u00ed: pro ka\u017ed\u00fd lok\u00e1ln\u00ed interval \u2014 \u0159ekn\u011bme pro ka\u017ed\u00fd den \u2014 je Low Cardinality efektivn\u00ed a\u017e do milionu unik\u00e1tn\u00edch hodnot. Pot\u00e9 se pou\u017eije fallback, p\u0159i kter\u00e9m se vyu\u017e\u00edv\u00e1 v\u00edce r\u016fzn\u00fdch slovn\u00edk\u016f, nikoli jeden. Bude fungovat zhruba stejn\u011b jako b\u011b\u017en\u00fd sloupec typu string, mo\u017en\u00e1 s m\u00edrn\u011b ni\u017e\u0161\u00ed efektivitou, ale nedojde k v\u00fdrazn\u00e9mu poklesu v\u00fdkonu. <\/p>\n<p><\/p>\n<h2 id=\"anchorfulltext-searchanchorkakie-luchshie-praktiki-ponbsppolnotekstovomu-poisku-ponbsptablice-snbsppyatyu-milliardami-strok\"><noindex><a rel=\"nofollow\" name=\"fulltext-search\"><\/a><\/noindex>Jak\u00e9 jsou nejlep\u0161\u00ed praktiky pro fulltextov\u00e9 vyhled\u00e1v\u00e1n\u00ed v tabulce s p\u011bti miliardami \u0159\u00e1dk\u016f?<\/h2>\n<p><\/p>\n<p>Existuj\u00ed r\u016fzn\u00e9 mo\u017enosti odpov\u011bdi. Prvn\u00ed je \u0159\u00edci, \u017ee ClickHouse nen\u00ed syst\u00e9m ur\u010den\u00fd pro fulltextov\u00e9 vyhled\u00e1v\u00e1n\u00ed. Pro to existuj\u00ed speci\u00e1ln\u00ed syst\u00e9my, nap\u0159\u00edklad <noindex><a rel=\"nofollow\" href=\"https:\/\/www.elastic.co\/enterprise-search\">Elasticsearch<\/a><\/noindex> a <noindex><a rel=\"nofollow\" href=\"http:\/\/sphinxsearch.com\/\">Sphinx<\/a><\/noindex>. Nicm\u00e9n\u011b, \u010d\u00edm d\u00e1l \u010dast\u011bji potk\u00e1v\u00e1m lidi, kte\u0159\u00ed \u0159\u00edkaj\u00ed, \u017ee p\u0159ech\u00e1zej\u00ed z Elasticsearch na ClickHouse.<\/p>\n<p><\/p>\n<p>Pro\u010d se to d\u011bje? Vysv\u011btluj\u00ed to t\u00edm, \u017ee Elasticsearch p\u0159est\u00e1v\u00e1 zvl\u00e1dat z\u00e1t\u011b\u017e p\u0159i ur\u010dit\u00e9m objemu, zejm\u00e9na pokud jde o vytv\u00e1\u0159en\u00ed index\u016f. Indexy se st\u00e1vaj\u00ed p\u0159\u00edli\u0161 objemn\u00fdmi, a pokud pouze p\u0159esunete data do ClickHouse, zjist\u00edte, \u017ee jsou ulo\u017eena n\u011bkolikr\u00e1t efektivn\u011bji co do objemu. Vyhled\u00e1vac\u00ed dotazy \u010dasto nebyly takov\u00e9, \u017ee by bylo pot\u0159eba naj\u00edt ve v\u0161ech datech n\u011bjakou fr\u00e1zi s ohledem na morfologii, ale zcela jin\u00e9. Nap\u0159\u00edklad naj\u00edt v posledn\u00edch n\u011bkolika hodin\u00e1ch v log\u00e1ch n\u011bjakou subsekvenci byt\u016f.<\/p>\n<p><\/p>\n<p>V tomto p\u0159\u00edpad\u011b vytv\u00e1\u0159\u00edte v ClickHouse index, jeho\u017e prvn\u00edm polem bude datum s \u010dasem. A nejv\u011bt\u0161\u00ed omezen\u00ed dat bude p\u0159esn\u011b podle rozsahu dat. V r\u00e1mci zvolen\u00e9ho rozsahu dat je mo\u017en\u00e9 prov\u00e1d\u011bt fulltextov\u00e9 vyhled\u00e1v\u00e1n\u00ed i bruteforce metodou pomoc\u00ed like. Oper\u00e1tor like v ClickHouse je nejefektivn\u011bj\u0161\u00ed oper\u00e1tor like, kter\u00fd m\u016f\u017eete naj\u00edt. Pokud najdete lep\u0161\u00ed \u2014 \u0159ekn\u011bte mi. <\/p>\n<p><\/p>\n<p>Ale p\u0159esto like je full scan. A full scan m\u016f\u017ee b\u00fdt pomal\u00fd nejen podle CPU, ale i podle disku. Pokud m\u00e1te nap\u0159\u00edklad jeden terabajt dat za den a hled\u00e1te n\u011bjak\u00e9 slovo za den, budete muset proskenovat terabajt. A ten je rozhodn\u011b na b\u011b\u017en\u00fdch pevn\u00fdch disc\u00edch, co\u017e v d\u016fsledku znamen\u00e1, \u017ee se dostanou do takov\u00e9ho zat\u00ed\u017een\u00ed, \u017ee se na server nedostanete p\u0159es SSH.<\/p>\n<p><\/p>\n<p>V tomto p\u0159\u00edpad\u011b jsem ochoten navrhnout je\u0161t\u011b jeden mal\u00fd trik. Je to sp\u00ed\u0161e experiment\u00e1ln\u00ed \u2014 m\u016f\u017ee fungovat, ale tak\u00e9 nemus\u00ed. V ClickHouse existuj\u00ed fulltextov\u00e9 indexy ve form\u011b trigramov\u00fdch Bloom filtr\u016f. Na\u0161i kolegov\u00e9 ze spole\u010dnosti Arenadata ji\u017e tyto indexy otestovali a \u010dasto funguj\u00ed p\u0159esn\u011b tak, jak bylo zam\u00fd\u0161leno.<\/p>\n<p><\/p>\n<p>Abychom je spr\u00e1vn\u011b vyu\u017eili, je t\u0159eba dob\u0159e rozum\u011bt tomu, jak funguj\u00ed: co p\u0159edstavuje trigramov\u00fd Bloom filtr a jak zvolit jeho velikost. Mohu \u0159\u00edci, \u017ee pomohou pro dotazy na n\u011bjak\u00e9 vz\u00e1cn\u00e9 fr\u00e1ze, pod\u0159et\u011bzce, kter\u00e9 se v datech m\u00e1lokdy vyskytuj\u00ed. V tomto p\u0159\u00edpad\u011b budou podle index\u016f vybr\u00e1ny podrozsahy a bude p\u0159e\u010dteno m\u00e9n\u011b dat.<\/p>\n<p><\/p>\n<p>Ned\u00e1vno se v ClickHouse objevily je\u0161t\u011b pokro\u010dilej\u0161\u00ed funkce pro fulltextov\u00e9 vyhled\u00e1v\u00e1n\u00ed. To zahrnuje vyhled\u00e1v\u00e1n\u00ed mnoha pod\u0159et\u011bzc\u016f najednou, v\u010detn\u011b variant s ohledem na velikost p\u00edsmen, bez ohledu na velikost p\u00edsmen, s podporou UTF-8 nebo pouze pro ASCII. Zvolte ten nejefektivn\u011bj\u0161\u00ed, kter\u00fd pot\u0159ebujete. <\/p>\n<p><\/p>\n<p>Objevilo se tak\u00e9 vyhled\u00e1v\u00e1n\u00ed n\u011bkolika regul\u00e1rn\u00edch v\u00fdraz\u016f najednou. Nemus\u00edte ps\u00e1t X like jeden pod\u0159et\u011bzec or X like druh\u00fd pod\u0159et\u011bzec. Prost\u011b to napi\u0161te a v\u0161e se provede co nejefektivn\u011bji.<\/p>\n<p><\/p>\n<p>T\u0159et\u00ed \u2013 nyn\u00ed existuje p\u0159ibl\u00ed\u017een\u00ed pro vyhled\u00e1v\u00e1n\u00ed regul\u00e1rn\u00edch v\u00fdraz\u016f a p\u0159ibl\u00ed\u017een\u00ed pro vyhled\u00e1v\u00e1n\u00ed pod\u0159et\u011bzc\u016f. Pokud n\u011bkdo napsal slovo s p\u0159eklepem, bude hled\u00e1no podle maxim\u00e1ln\u00ed shody.<\/p>\n<p><\/p>\n<h2 id=\"anchorhello-and-welcomeanchorkak-luchshe-organizovat-dostup-vnbspclickhouse-dlyanbspbolshogo-kolichestva-polzovateley\"><noindex><a rel=\"nofollow\" name=\"hello-and-welcome\"><\/a><\/noindex>Jak nejl\u00e9pe organizovat p\u0159\u00edstup v ClickHouse pro velk\u00e9 mno\u017estv\u00ed u\u017eivatel\u016f?<\/h2>\n<p><\/p>\n<blockquote><p>Pov\u011bzte mi, jak nejl\u00e9pe uspo\u0159\u00e1dat p\u0159\u00edstup pro velk\u00e9 mno\u017estv\u00ed u\u017eivatel\u016f a analytik\u016f. Jak vytvo\u0159it frontu, prioritizovat dotazy max concurrent queries a jak\u00e9 n\u00e1stroje pou\u017e\u00edt?<\/p><\/blockquote>\n<p>Pokud je cluster dostate\u010dn\u011b velk\u00fd, m\u016f\u017ee b\u00fdt dobr\u00fdm \u0159e\u0161en\u00edm zvednout je\u0161t\u011b dva servery, kter\u00e9 se stanou vstupn\u00edm bodem pro analytiky. To znamen\u00e1, \u017ee nepust\u00edte analytiky k jednotliv\u00fdm shard\u016fm clusteru, ale jednodu\u0161e vytvo\u0159\u00edte dva pr\u00e1zdn\u00e9 servery, bez dat, a na nich nastav\u00edte pr\u00e1va p\u0159\u00edstupu. P\u0159itom nastaven\u00ed u\u017eivatel\u016f p\u0159i distribuovan\u00fdch dotazech se p\u0159en\u00e1\u0161ej\u00ed na vzd\u00e1len\u00e9 servery. To znamen\u00e1, \u017ee v\u0161e nastav\u00edte na t\u011bchto dvou serverech a nastaven\u00ed m\u00e1 efekt na cel\u00e9m clusteru.<\/p>\n<p><\/p>\n<p>V principe jsou tyto servery bez dat, ale objem RAM na nich je velmi d\u016fle\u017eit\u00fd pro vykon\u00e1v\u00e1n\u00ed dotaz\u016f. Disk m\u016f\u017ee b\u00fdt tak\u00e9 pou\u017eit pro do\u010dasn\u00e1 data, pokud je zapnut\u00e1 extern\u00ed agregace nebo extern\u00ed t\u0159\u00edd\u011bn\u00ed.<\/p>\n<p><\/p>\n<p>Je d\u016fle\u017eit\u00e9 pod\u00edvat se na nastaven\u00ed, kter\u00e9 souvis\u00ed se v\u0161emi mo\u017en\u00fdmi limity. Pokud nyn\u00ed jako analytik vstoup\u00edm do clustru \u201eYandex.Metriky\u201c a zad\u00e1m dotaz <strong>select count from hits<\/strong>, hned mi vyhod\u00ed v\u00fdjimku, \u017ee dotaz nemohu prov\u00e9st. Maxim\u00e1ln\u00ed po\u010det \u0159\u00e1dk\u016f, kter\u00e9 mohu prozkoumat, je sto miliard, a celkov\u011b jich v clusteru je pades\u00e1t bilion\u016f v jedn\u00e9 tabulce. To je prvn\u00ed omezen\u00ed. <\/p>\n<p><\/p>\n<p>P\u0159edpokl\u00e1dejme, \u017ee odstran\u00edm omezen\u00ed po\u010dtu \u0159\u00e1dk\u016f a dotaz provedu znovu. Potom uvid\u00edm n\u00e1sleduj\u00edc\u00ed v\u00fdjimku \u2013 nastaven\u00ed je zapnuto. <strong>force index by date<\/strong>Nemohu splnit po\u017eadavek, pokud jsem nevymezil \u010dasov\u00fd rozsah. Nen\u00ed pot\u0159eba spol\u00e9hat na to, \u017ee analytici to uvedou ru\u010dn\u011b. Typick\u00fd p\u0159\u00edpad je naps\u00e1n jako \u010dasov\u00fd rozsah, kde datum ud\u00e1losti je mezi t\u00fddny. A pak se prost\u011b n\u011bkde ud\u011blala chyba v z\u00e1vork\u00e1ch, a m\u00edsto and vzniklo or \u2014 or URL match. Pokud nejsou \u017e\u00e1dn\u00e1 omezen\u00ed, za\u010dne skenovat sloupec URL a prost\u011b spot\u0159ebov\u00e1v\u00e1 ohromn\u00e9 mno\u017estv\u00ed zdroj\u016f.<\/p>\n<p><\/p>\n<p>Nav\u00edc m\u00e1 ClickHouse dv\u011b nastaven\u00ed priorit. Bohu\u017eel jsou velmi primitivn\u00ed. Jedno se naz\u00fdv\u00e1 jednodu\u0161e <strong>priority<\/strong>. Pokud priority \u2260 0 a jsou prov\u00e1d\u011bny dotazy s n\u011bjakou prioritou, ale z\u00e1rove\u0148 se prov\u00e1d\u00ed dotaz s prioritou, kter\u00e1 m\u00e1 men\u0161\u00ed hodnotu, co\u017e znamen\u00e1 vy\u0161\u0161\u00ed prioritu, pak dotaz s hodnotou priority v\u011bt\u0161\u00ed, co\u017e ozna\u010duje ni\u017e\u0161\u00ed prioritu, prost\u011b se pozastav\u00ed a nebude fungovat v\u016fbec v tuto dobu.<\/p>\n<p><\/p>\n<p>Je to velmi hrub\u00e9 nastaven\u00ed a nehod\u00ed se pro p\u0159\u00edpady, kdy je v clusteru trval\u00e9 zat\u00ed\u017een\u00ed. Ale pokud m\u00e1te kr\u00e1tk\u00e9, impulsivn\u00ed d\u016fle\u017eit\u00e9 dotazy a v\u011bt\u0161inu \u010dasu je cluster ne\u010dinn\u00fd, takov\u00e9 nastaven\u00ed bude vyhovovat.<\/p>\n<p><\/p>\n<p>Dal\u0161\u00ed nastaven\u00ed priorit se naz\u00fdv\u00e1 <strong>priorita vl\u00e1kna OS<\/strong>. Jen nastavuje pro v\u0161echny vl\u00e1kna vykon\u00e1v\u00e1n\u00ed dotazu hodnotu nice pro pl\u00e1nova\u010d Linuxu. Funguje to tak tak, ale p\u0159ece jen funguje. Pokud nastav\u00edte nejni\u017e\u0161\u00ed hodnotu nice \u2014 co\u017e je nejvy\u0161\u0161\u00ed hodnota, a tedy nejni\u017e\u0161\u00ed priorita \u2014 a pro dotazy s vysokou prioritou nastav\u00edte -19, pak CPU za\u010dne spot\u0159ebov\u00e1vat dotazy s n\u00edzkou prioritou p\u0159ibli\u017en\u011b \u010dty\u0159ikr\u00e1t m\u00e9n\u011b ne\u017e dotazy s vysokou prioritou. <\/p>\n<p><\/p>\n<p>Je\u0161t\u011b je pot\u0159eba nastavit maxim\u00e1ln\u00ed \u010das na prov\u00e1d\u011bn\u00ed dotazu \u2014 \u0159ekn\u011bme, p\u011bt minut. Minim\u00e1ln\u00ed rychlost prov\u00e1d\u011bn\u00ed dotazu \u2014 to je to nejlep\u0161\u00ed. Toto nastaven\u00ed existuje u\u017e dlouho a je pot\u0159eba, aby se neprov\u00e1d\u011bl pouh\u00fd v\u00fdrok, \u017ee ClickHouse nezpomalil, ale aby to bylo vynuceno.<\/p>\n<p><\/p>\n<p>P\u0159edstavte si, \u017ee nastavujete: pokud n\u011bjak\u00fd dotaz zpracov\u00e1v\u00e1 m\u00e9n\u011b ne\u017e jeden milion \u0159\u00e1dk\u016f za sekundu \u2014 to tak nelze. To po\u0161kozuje na\u0161i dobrou pov\u011bst, na\u0161i dobrou datab\u00e1zi. Poj\u010fme to prost\u011b zak\u00e1zat. Ve skute\u010dnosti existuj\u00ed dv\u011b nastaven\u00ed. Jedno se naz\u00fdv\u00e1 <strong>minim\u00e1ln\u00ed rychlost vykon\u00e1v\u00e1n\u00ed<\/strong> \u2014 v \u0159\u00e1dc\u00edch za sekundu, a druh\u00e1 se naz\u00fdv\u00e1 timeout p\u0159ed kontrolou minim\u00e1ln\u00ed rychlosti vykon\u00e1n\u00ed \u2014 ve v\u00fdchoz\u00edm nastaven\u00ed patn\u00e1ct sekund. To znamen\u00e1, \u017ee patn\u00e1ct sekund m\u016f\u017ee b\u00fdt, a pak, pokud se to zpomaluje, tak prost\u011b vyhod\u00ed v\u00fdjimku \u2014 p\u0159eru\u0161\u00ed po\u017eadavek.<\/p>\n<p><\/p>\n<p>Tak\u00e9 je t\u0159eba nastavit kv\u00f3ty. V ClickHouse existuje vestav\u011bn\u00e1 funkce kv\u00f3t, kter\u00e1 sleduje spot\u0159ebu zdroj\u016f. Bohu\u017eel v\u0161ak ne \u017eelezn\u00fdch zdroj\u016f, jako je CPU, disky, ale logick\u00fdch \u2014 po\u010det zpracovan\u00fdch po\u017eadavk\u016f, \u0159\u00e1dk\u016f a p\u0159e\u010dten\u00fdch bajt\u016f. A lze nastavit nap\u0159\u00edklad maxim\u00e1ln\u011b sto po\u017eadavk\u016f b\u011bhem p\u011bti minut a tis\u00edc po\u017eadavk\u016f za hodinu.<\/p>\n<p><\/p>\n<p>Pro\u010d je to d\u016fle\u017eit\u00e9? Proto\u017ee \u010d\u00e1st analytick\u00fdch po\u017eadavk\u016f budou prov\u00e1d\u011bt ru\u010dn\u011b p\u0159\u00edmo z klienta ClickHouse. A v\u0161echno bude v po\u0159\u00e1dku. Ale pokud m\u00e1te ve sv\u00e9m podniku pokro\u010dil\u00e9 analytiky, nap\u00ed\u0161ou skript, a ve skriptu m\u016f\u017ee b\u00fdt chyba. A tato chyba m\u016f\u017ee v\u00e9st k tomu, \u017ee po\u017eadavek bude vykon\u00e1v\u00e1n v nekone\u010dn\u00e9 smy\u010dce. A od toho je t\u0159eba se chr\u00e1nit.<\/p>\n<p><\/p>\n<h2 id=\"anchorsmorgasbordanchormozhno-li-otdat-rezultaty-odnogo-zaprosa-desyati-klientam\"><noindex><a rel=\"nofollow\" name=\"smorgasbord\"><\/a><\/noindex>Lze vr\u00e1tit v\u00fdsledky jednoho dotazu deseti klient\u016fm?<\/h2>\n<p><\/p>\n<blockquote><p>M\u00e1me n\u011bkolik u\u017eivatel\u016f, kte\u0159\u00ed r\u00e1di p\u0159ich\u00e1zej\u00ed s velmi velk\u00fdmi po\u017eadavky ve stejnou chv\u00edli. Po\u017eadavek je velk\u00fd, vykon\u00e1v\u00e1 se v podstat\u011b rychle, ale kv\u016fli tomu, \u017ee je simult\u00e1nn\u011b mnoho takov\u00fdch po\u017eadavk\u016f, vznik\u00e1 v\u00fdrazn\u00fd probl\u00e9m. Lze prov\u00e9st stejn\u00fd po\u017eadavek, kter\u00fd p\u0159i\u0161el desetkr\u00e1t za sebou, pouze jednou a v\u00fdsledek vr\u00e1tit deseti klient\u016fm?<\/p><\/blockquote>\n<p>Probl\u00e9m je v tom, \u017ee v sou\u010dasnosti nem\u00e1me v\u00fdsledk\u016f z cache nebo cache mezipam\u011bti. Existuje page cache opera\u010dn\u00edho syst\u00e9mu, kter\u00e1 umo\u017en\u00ed ne\u010det dat z disku znovu, ale bohu\u017eel se data st\u00e1le mus\u00ed dekomprimovat, deserializovat a zpracovat znovu. <\/p>\n<p><\/p>\n<p>R\u00e1di bychom se tomu n\u011bjak\u00fdm zp\u016fsobem vyhnuli, a\u0165 u\u017e cacheov\u00e1n\u00edm mezipam\u011b\u0165ov\u00fdch dat, nebo uspo\u0159\u00e1d\u00e1n\u00edm podobn\u00fdch po\u017eadavk\u016f do fronty a p\u0159id\u00e1n\u00edm cache v\u00fdsledk\u016f. V sou\u010dasnosti m\u00e1me v v\u00fdvoji jeden pull request, kter\u00fd p\u0159id\u00e1v\u00e1 cache po\u017eadavk\u016f, ale pouze pro poddotazy v sekc\u00edch in a join \u2014 tedy \u0159e\u0161en\u00ed nen\u00ed \u00fapln\u00e9.<\/p>\n<p><\/p>\n<p>P\u0159esto se u n\u00e1s tak\u00e9 objevuje takov\u00e1 situace. Zvl\u00e1\u0161t\u011b typick\u00fdm p\u0159\u00edkladem jsou dotazy s&nbsp;paginac\u00ed. Existuje zpr\u00e1va, v&nbsp;n\u00ed\u017e je n\u011bkolik str\u00e1nek, a jde dotaz limit 10. Pot\u00e9 tot\u00e9\u017e, ale limit 10,10. Pak je\u0161t\u011b dal\u0161\u00ed str\u00e1nka. A je tu ot\u00e1zka, pro\u010d to ka\u017edou chv\u00edli po\u010d\u00edt\u00e1me? Ale nyn\u00ed neexistuje \u0159e\u0161en\u00ed, a vyhnout se tomu nelze.<\/p>\n<p><\/p>\n<p>Existuje alternativn\u00ed \u0159e\u0161en\u00ed, kter\u00e9 se instaluje jako sidecar vedle ClickHouse \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/Vertamedia\/chproxy\">ClickHouse Proxy<\/a><\/noindex>.<\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> V ClickHouse Proxy je vestav\u011bn\u00fd rate limiter a vestav\u011bn\u00fd cache v\u00fdsledk\u016f. Je tam mnoho nastaven\u00ed, proto\u017ee se \u0159e\u0161il podobn\u00fd probl\u00e9m. Proxy umo\u017e\u0148uje limitovat dotazy t\u00edm, \u017ee je \u0159ad\u00ed do fronty, a nastavovat, jak dlouho cache dotaz\u016f \u017eije. Pokud byly dotazy opravdu stejn\u00e9, Proxy je vr\u00e1t\u00ed mnohokr\u00e1t, zat\u00edmco do ClickHouse zajde jen jednou.<\/p>\n<p><\/p>\n<p>V Nginx tak\u00e9 existuje cache ve zdarma verzi a to tak\u00e9 bude fungovat. U Nginx dokonce existuj\u00ed nastaven\u00ed, \u017ee pokud dotazy p\u0159i\u0161ly sou\u010dasn\u011b, bude zpo\u017e\u010fovat ostatn\u00ed, dokud nebude jeden spln\u011bn. Ale pr\u00e1v\u011b v ClickHouse Proxy je nastaven\u00ed provedeno mnohem l\u00e9pe. Bylo navr\u017eeno p\u0159\u00edmo pro ClickHouse, p\u0159esn\u011b pro tyto dotazy, tak\u017ee je v\u00edce vhodn\u00e9. No a je snadn\u00e9 ho nastavit. <\/p>\n<p><\/p>\n<h2 id=\"anchorasynchronousanchorkak-byt-snbspasinhronnymi-operaciyami-i-materializovannymi-predstavleniyami\"><noindex><a rel=\"nofollow\" name=\"asynchronous\"><\/a><\/noindex>Jak se vypo\u0159\u00e1dat s asynchronn\u00edmi operacemi a materializovan\u00fdmi pohledy?<\/h2>\n<p><\/p>\n<blockquote><p>Existuje takov\u00fd probl\u00e9m, \u017ee operace s&nbsp;replacing engine jsou asynchronn\u00ed \u2014 nejprve se zapisuj\u00ed data, pot\u00e9 doch\u00e1z\u00ed k jejich slou\u010den\u00ed. Pokud pod tabulkou \u017eije materializovan\u00e1 tabulka s n\u011bjak\u00fdmi agreg\u00e1ty, pak se do n\u00ed z\u00e1znamy duplikuj\u00ed. A pokud neexistuje n\u011bjak\u00e1 slo\u017eit\u00e1 logika, data budou duplikov\u00e1na. Co s t\u00edm lze ud\u011blat?<\/p>\n<p>Existuje z\u0159ejm\u00e9 \u0159e\u0161en\u00ed \u2014 implementovat trigger na ur\u010dit\u00fd typ materializovan\u00fdch pohled\u016f p\u0159i asynchronn\u00ed operaci slou\u010den\u00ed. Existuj\u00ed n\u011bjak\u00e9 \"st\u0159\u00edbrn\u00e9 kulky\", pl\u00e1ny na implementaci podobn\u00e9 funkcionality?<\/p><\/blockquote>\n<p>Je t\u0159eba pochopit, jak funguje deduplikace. To, o \u010dem nyn\u00ed budu mluvit, nesouvis\u00ed s&nbsp;ot\u00e1zkou, ale pro ka\u017ed\u00fd p\u0159\u00edpad za to stoj\u00ed p\u0159ipomenout.<\/p>\n<p><\/p>\n<p>P\u0159i vkl\u00e1d\u00e1n\u00ed do replikovan\u00e9 tabulky prob\u00edh\u00e1 deduplikace cel\u00fdch vlo\u017een\u00fdch blok\u016f. Pokud znovu vlo\u017e\u00edte stejn\u00fd blok, kter\u00fd obsahuje stejn\u00e9 mno\u017estv\u00ed t\u011bch sam\u00fdch \u0159\u00e1dk\u016f ve stejn\u00e9m po\u0159ad\u00ed, budou data deduplikov\u00e1na. Na insert dostanete \u201eOk\u201c, ale ve skute\u010dnosti bude zaps\u00e1na jen jedna d\u00e1vka dat a duplicita se nevytvo\u0159\u00ed.<\/p>\n<p><\/p>\n<p>To je d\u016fle\u017eit\u00e9 pro p\u0159esnost. Pokud b\u011bhem vkl\u00e1d\u00e1n\u00ed z\u00edsk\u00e1te \u201eOk\u201c, znamen\u00e1 to, \u017ee va\u0161e data byla vlo\u017eena. Pokud jste dostali chybu z ClickHouse, znamen\u00e1 to, \u017ee nebyla vlo\u017eena a je t\u0159eba vlo\u017een\u00ed zopakovat. Ale pokud b\u011bhem vkl\u00e1d\u00e1n\u00ed do\u0161lo k p\u0159eru\u0161en\u00ed spojen\u00ed, nev\u00edte, zda byla data vlo\u017eena \u010di nikoli. Jedinou mo\u017enost\u00ed je vlo\u017een\u00ed zopakovat. Pokud byla data skute\u010dn\u011b vlo\u017eena a vy je vlo\u017e\u00edte znovu, prob\u00edh\u00e1 deduplikace blok\u016f. Ta je nutn\u00e1 k zamezen\u00ed duplicity. <\/p>\n<p><\/p>\n<p>A tak\u00e9 je d\u016fle\u017eit\u00e9, jak to funguje pro materializovan\u00e9 pohledy. Pokud byla data deduplikov\u00e1na p\u0159i vlo\u017een\u00ed do hlavn\u00ed tabulky, pak do materializovan\u00e9ho pohledu nep\u016fjdou.<\/p>\n<p><\/p>\n<p>Te\u010f k ot\u00e1zce. M\u00e1te slo\u017eit\u011bj\u0161\u00ed situaci, proto\u017ee zapisujete duplicity jednotliv\u00fdch \u0159\u00e1dk\u016f. To znamen\u00e1, \u017ee cel\u00e1 d\u00e1vka nen\u00ed zdvojen\u00e1, ale konkr\u00e9tn\u00ed \u0159\u00e1dky, a ty se v pozad\u00ed slou\u010d\u00ed. Opravu, data se slou\u010d\u00ed v hlavn\u00ed tabulce, a do materializovan\u00e9ho pohledu p\u016fjdou neslou\u010den\u00e9, a p\u0159i mergech se nic s materializovan\u00fdmi pohledy nestane. Proto\u017ee materializovan\u00fd pohled je nic jin\u00e9ho ne\u017e spou\u0161t\u011b\u010d pro insert. P\u0159i jin\u00fdch operac\u00edch se s n\u00edm nic nav\u00edc ned\u011bje.<\/p>\n<p><\/p>\n<p>A tady v\u00e1s bohu\u017eel nemohu pot\u011b\u0161it. Je t\u0159eba hledat konkr\u00e9tn\u00ed \u0159e\u0161en\u00ed pro tento p\u0159\u00edpad. Nap\u0159\u00edklad, lze v materializovan\u00e9m pohledu tak\u00e9 prov\u00e9st jeho nahrazen\u00ed, a zp\u016fsob deduplikace mo\u017en\u00e1 bude fungovat stejn\u011b. Ale bohu\u017eel, ne v\u017edy. Pokud je agreguj\u00edc\u00ed, pak to nep\u016fjde. <\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> I&nbsp;remember that we also had our share of makeshift solutions back in the day. There was an issue with ad impressions, and there are some data points that we can display in&nbsp;real-time \u2014 it\u2019s just impressions. They rarely duplicate, but if that happens, we\u2019ll still consolidate them later. And there were things that couldn\u2019t be duplicated&nbsp;\u2014 clicks and all that history. But we still wanted to show them almost immediately.<\/p>\n<p><\/p>\n<p>How were materialized views created? There were views where data is written directly&nbsp;\u2014 it goes into&nbsp;raw data and is written to&nbsp;views. At&nbsp;one point, the data there wasn't very accurate, they were duplicated and so on. And there\u2019s a second part of the table where they look absolutely the same as materialized views; in&nbsp;other words, they are structurally identical. Occasionally, we recalculate the data, count the data without duplicates, and write to those tables. <\/p>\n<p><\/p>\n<p>We accessed it via API \u2014 working with&nbsp;ClickHouse manually won\u2019t work. And the API checks: when I have the date of the last addition to&nbsp;the table, where the data is guaranteed to be correct and calculated, and it makes a request to&nbsp;one table and the other table. From&nbsp;one, it selects up&nbsp;to a certain point in&nbsp;time, and from&nbsp;the other, it picks up what hasn\u2019t been counted yet. And it works, but not using just one ClickHouse.<\/p>\n<p><\/p>\n<p>If you have some API&nbsp;\u2014 for&nbsp;analysts, for&nbsp;users&nbsp;\u2014 then in principle, that\u2019s an option. You always recalculate, always tally up. You can do this once a&nbsp;day or at&nbsp;some other time. You choose the range you don\u2019t need and isn\u2019t critical for&nbsp;yourself.<\/p>\n<p><\/p>\n<h2 id=\"anchordashboardanchorv-clickhouse-mnogo-logov-kak-ya-mogu-videt-vsyo-chto-proishodit-s-serverom-vnbspmomente\"><noindex><a rel=\"nofollow\" name=\"dashboard\"><\/a><\/noindex>In ClickHouse, there are many logs. How can I see everything happening with the server in&nbsp;real-time?<\/h2>\n<p><\/p>\n<blockquote><p>There are a significant number of various logs in ClickHouse, and this number is increasing. In&nbsp;new versions, some of them are even enabled by default, while in&nbsp;old versions, they need to be activated upon upgrading. Nevertheless, there are always more and more of them. I would like to see, ultimately, what\u2019s currently happening with my server, possibly on some consolidated dashboard. <\/p>\n<p>Nem\u00e1te v t\u00fdmu ClickHouse nebo v t\u00fdmech va\u0161ich p\u0159\u00e1tel n\u011bkoho, kdo by podporoval n\u011bjakou funk\u010dnost hotov\u00fdch dashboard\u016f, kter\u00e9 by tyto logy zobrazovaly jako hotov\u00fd produkt? Koneckonc\u016f, jen tak se d\u00edvat na logy v ClickHouse \u2014 to je fajn. Ale bylo by skv\u011bl\u00e9 m\u00edt to u\u017e ve form\u011b p\u0159ipraven\u00e9ho dashboardu. Z toho bych m\u011bl radost. <\/p><\/blockquote>\n<p>Dashboardy existuj\u00ed, pravda, nejsou standardizovan\u00e9. U n\u00e1s ve firm\u011b pou\u017e\u00edv\u00e1 ClickHouse asi 60 t\u00fdm\u016f a nejpodivn\u011bj\u0161\u00ed je, \u017ee mnoz\u00ed z nich maj\u00ed dashboardy, kter\u00e9 si sami vytvo\u0159ili, a jsou m\u00edrn\u011b odli\u0161n\u00e9. N\u011bkter\u00e9 t\u00fdmy pou\u017e\u00edvaj\u00ed intern\u00ed instalaci \"Yandex.Cloud\". Tam jsou n\u011bkter\u00e9 hotov\u00e9 zpr\u00e1vy, i kdy\u017e ne v\u0161echny pot\u0159ebn\u00e9. Jin\u00e9 maj\u00ed vlastn\u00ed. <\/p>\n<p><\/p>\n<p>Moji kolegov\u00e9 z \"Metariky\" maj\u00ed sv\u016fj dashboard v Grafana a j\u00e1 m\u00e1m sv\u016fj na jejich clusteru. Sleduji tam v\u011bci jako hit rate pro cache. A je to je\u0161t\u011b slo\u017eit\u011bj\u0161\u00ed, proto\u017ee pou\u017e\u00edv\u00e1me r\u016fzn\u00e9 n\u00e1stroje. Sv\u016fj dashboard jsem vytvo\u0159il na velmi star\u00e9m n\u00e1stroji, kter\u00fd se naz\u00fdv\u00e1 Graphite-web. Je naprosto o\u0161kliv\u00fd. A dodnes ho pou\u017e\u00edv\u00e1m, i kdy\u017e Grafana by byla pravd\u011bpodobn\u011b pohodln\u011bj\u0161\u00ed a hez\u010d\u00ed. <\/p>\n<p><\/p>\n<p>Z\u00e1kladn\u00ed v\u011bc na dashboardech je stejn\u00e1. Jsou to syst\u00e9mov\u00e9 metriky na clusteru: CPU, pam\u011b\u0165, disk, s\u00ed\u0165. Dal\u0161\u00ed \u2014 po\u010det sou\u010dasn\u00fdch dotaz\u016f, po\u010det sou\u010dasn\u00fdch slou\u010den\u00ed, po\u010det dotaz\u016f za sekundu, maxim\u00e1ln\u00ed po\u010det kus\u016f pro partition tabulek MergeTree, zpo\u017ed\u011bn\u00ed replikace, velikost fronty replikace, po\u010det vlo\u017een\u00fdch \u0159\u00e1dk\u016f za sekundu, po\u010det vlo\u017een\u00fdch blok\u016f za sekundu. To v\u0161e, co se nedost\u00e1v\u00e1 z log\u016f, ale z metrik.<\/p>\n<p><\/p>\n<p><strong>Vladimir Kolobajev:<\/strong> Alexi, cht\u011bl bych trochu korigovat. Existuje Grafana. Grafana m\u00e1 datov\u00fd zdroj, kter\u00fd je ClickHouse. To znamen\u00e1, \u017ee mohu z Grafany pos\u00edlat dotazy p\u0159\u00edmo do ClickHouse. V ClickHouse je tabulka s logy, ta je pro v\u0161echny stejn\u00e1. Chci v Grafan\u011b p\u0159istupovat k t\u00e9to tabulce log\u016f a vid\u011bt ty dotazy, kter\u00e9 m\u016fj server generuje. Bylo by skv\u011bl\u00e9 m\u00edt takov\u00fd dashboard.<\/p>\n<p><\/p>\n<p>Vytvo\u0159il jsem ho s\u00e1m. Ale m\u00e1m ot\u00e1zku \u2014 pokud je to v\u0161echno standardizovan\u00e9 a Grafana se pou\u017e\u00edv\u00e1 v\u0161emi, pro\u010d v \"Yandexu\" neexistuje takov\u00fd ofici\u00e1ln\u00ed dashboard?<\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> Ve skute\u010dnosti datasource, kter\u00fd jde k ClickHouse, nyn\u00ed podporuje Altinity. A chci jen poskytnout sm\u011br, kam se pod\u00edvat a koho tla\u010dit. M\u016f\u017eete se jich zeptat, proto\u017ee \u201eYandex\u201c koneckonc\u016f d\u011bl\u00e1 ClickHouse, ne p\u0159\u00edb\u011bh kolem n\u011bj. Altinity je hlavn\u00ed spole\u010dnost, kter\u00e1 nyn\u00ed propaguje ClickHouse. Neodlo\u017e\u00ed ho, ale budou ho podporovat. Proto\u017ee v z\u00e1sad\u011b, abyste nahr\u00e1li dashboard na str\u00e1nku Grafana, sta\u010d\u00ed se zaregistrovat a nahr\u00e1t ho \u2013 \u017e\u00e1dn\u00e9 zvl\u00e1\u0161tn\u00ed probl\u00e9my nejsou. <\/p>\n<p><\/p>\n<p><strong>Alexei Milovidov:<\/strong> Za posledn\u00ed rok se v ClickHouse p\u0159idalo mnoho funkc\u00ed pro profilaci dotaz\u016f. Existuj\u00ed metriky pro ka\u017ed\u00fd dotaz podle vyu\u017eit\u00ed zdroj\u016f. A ned\u00e1vno byl p\u0159id\u00e1n je\u0161t\u011b n\u00edzko\u00farov\u0148ov\u00fd profiler dotaz\u016f, aby bylo mo\u017en\u00e9 sledovat, kde dotaz str\u00e1v\u00ed ka\u017edou milisekundu. Ale abych mohl vyu\u017e\u00edt tuto funkci, mus\u00edm otev\u0159\u00edt konzolov\u00e9ho klienta a napsat dotaz, na kter\u00fd neust\u00e1le zapom\u00edn\u00e1m. Ulo\u017eil jsem si ho n\u011bkam, ale neust\u00e1le zapom\u00edn\u00e1m, kam p\u0159esn\u011b. <\/p>\n<p><\/p>\n<p>Cht\u011bl bych, aby existoval n\u00e1stroj, ve kter\u00e9m by bylo jednodu\u0161e naps\u00e1no \u2013 tady jsou va\u0161e n\u00e1ro\u010dn\u00e9 dotazy, rozd\u011blen\u00e9 podle t\u0159\u00edd dotaz\u016f. Kdy\u017e na n\u011bjak\u00fd kliknu, \u0159ekli by mi, pro\u010d je n\u00e1ro\u010dn\u00fd. V sou\u010dasnosti takov\u00e9 \u0159e\u0161en\u00ed neexistuje. A opravdu je dost podivn\u00e9, \u017ee kdy\u017e se m\u011b lid\u00e9 ptaj\u00ed: \u201eExistuj\u00ed n\u011bjak\u00e9 hotov\u00e9 dashboardy pro Grafana?\u201c, \u0159\u00edk\u00e1m: \u201ePod\u00edvejte se na str\u00e1nku Grafana, tam je komunita 'Dashboardy', a tam je dashboard od D\u00edmky, je tam dashboard od Kostyana. Co to je, nev\u00edm, s\u00e1m jsem to nepou\u017e\u00edval.\u201c<\/p>\n<p><\/p>\n<h2 id=\"anchorzenanchorkak-vozdeystvovat-na-merdzhi-chtoby-server-ne-padal-vnbspoom\"><noindex><a rel=\"nofollow\" name=\"zen\"><\/a><\/noindex>Jak ovlivnit slu\u010dov\u00e1n\u00ed, aby server nespadl do OOM?<\/h2>\n<p><\/p>\n<blockquote><p>M\u00e1m tabulku, ve kter\u00e9 je jen jedna partition, je to ReplacingMergeTree. Do n\u00ed zapisuj\u00ed data po dobu \u010dty\u0159 let. Musel jsem v n\u00ed prov\u00e9st alter a smazat ur\u010dit\u00e9 \u00fadaje.<\/p>\n<p>Ud\u011blalo se to, a b\u011bhem zpracov\u00e1n\u00ed tohoto dotazu byla spot\u0159ebov\u00e1na ve\u0161ker\u00e1 pam\u011b\u0165 na v\u0161ech serverech clusteru a v\u0161echny servery clusteru se shodn\u011b dostaly do OOM. Pot\u00e9 se v\u0161echny dohromady znovu spustily, za\u010daly prov\u00e1d\u011bt slu\u010dov\u00e1n\u00ed t\u00e9to same operace, tohoto bloku dat, a znovu spadly do OOM. Pot\u00e9 se znovu spustily a znovu spadly. A tohle se neopakovalo.<\/p>\n<p>Pak se uk\u00e1zalo, \u017ee to byla skute\u010dn\u011b chyba, kterou kluci opravili. To je skv\u011bl\u00e9, moc d\u011bkuji. Ale z\u016fst\u00e1v\u00e1 tu nep\u0159\u00edjemn\u00fd pocit. A te\u010f, kdy\u017e p\u0159em\u00fd\u0161l\u00edm o tom, jak ud\u011blat n\u011bjak\u00e9 slou\u010den\u00ed v tabulce, objevuje se mi ot\u00e1zka \u2014 pro\u010d nemohu n\u011bjak ovlivnit tyto slou\u010den\u00ed? Nap\u0159\u00edklad omezit je podle mno\u017estv\u00ed vy\u017eadovan\u00e9 opera\u010dn\u00ed pam\u011bti, nebo obecn\u011b podle jejich po\u010dtu, kter\u00e9 bude zpracov\u00e1vat konkr\u00e9tn\u011b tato tabulka.<\/p>\n<p>M\u00e1m tabulku, kter\u00e1 se naz\u00fdv\u00e1 \u201eMetriky\u201c, pros\u00edm, zpracuj mi ji ve dvou vl\u00e1knech. Nepot\u0159ebuj m\u00edt deset nebo p\u011bt slou\u010den\u00ed sou\u010dasn\u011b, ud\u011blej to ve dvou. Mysl\u00edm, \u017ee na dv\u011b mi pam\u011b\u0165 sta\u010d\u00ed, ale na zpracov\u00e1n\u00ed deseti mo\u017en\u00e1 ne. Pro\u010d obavy p\u0159etrv\u00e1vaj\u00ed? Proto\u017ee tabulka roste a jednou se setk\u00e1m se situac\u00ed, kdy u\u017e ne kv\u016fli chyb\u011b, ale proto, \u017ee data se budou m\u011bnit v takov\u00e9m mno\u017estv\u00ed, \u017ee mi prost\u011b na serveru nebude sta\u010dit pam\u011b\u0165. A tehdy server spadne do OOM p\u0159i slou\u010den\u00ed. Muta\u010dn\u00ed operaci mohu zru\u0161it, ale slou\u010den\u00ed u\u017e ne.<\/p><\/blockquote>\n<p>V\u00edte, p\u0159i slou\u010den\u00edch server nespadne do OOM, proto\u017ee p\u0159i slou\u010den\u00ed se pou\u017e\u00edv\u00e1 mno\u017estv\u00ed pam\u011bti pouze na jeden mal\u00fd rozsah dat. Tak\u017ee v\u0161e bude v po\u0159\u00e1dku bez ohledu na objem dat.<\/p>\n<p><\/p>\n<p><strong>Vladimir Kolobajev:<\/strong> Dob\u0159e. Je tu takov\u00fd moment, \u017ee pot\u00e9, co opravili chybu, st\u00e1hl jsem si novou verzi a na jin\u00e9, men\u0161\u00ed tabulce, kde je hodn\u011b partition, jsem ud\u011blal podobnou operaci. A b\u011bhem slou\u010den\u00ed bylo na serveru sp\u00e1leno p\u0159ibli\u017en\u011b 100 GB opera\u010dn\u00ed pam\u011bti. M\u011bl jsem 150 GB obsazeno, 100 GB se spot\u0159ebovalo a z\u016fstalo mi 50 GB voln\u00e9ho m\u00edsta, tak\u017ee jsem nespadl do OOM.<\/p>\n<p><\/p>\n<p>Co m\u011b v tuto chv\u00edli chr\u00e1n\u00ed p\u0159ed t\u00edm, abych nespadl do OOM, pokud opravdu spot\u0159ebov\u00e1v\u00e1 100 GB opera\u010dn\u00ed pam\u011bti? Co d\u011blat v situaci, kdy m\u00e1 n\u00e1hle pam\u011b\u0165 na slou\u010den\u00ed dojde?<\/p>\n<p><\/p>\n<p><strong>Alexei Milovidov:<\/strong> Existuje probl\u00e9m, \u017ee spot\u0159eba opera\u010dn\u00ed pam\u011bti p\u0159i merge nen\u00ed omezena. A druh\u00fd probl\u00e9m spo\u010d\u00edv\u00e1 v tom, \u017ee pokud byl n\u011bjak\u00fd merge napl\u00e1nov\u00e1n, je nutn\u00e9 jej prov\u00e9st, proto\u017ee je zaznamen\u00e1n v logu replikace. Log replikace jsou ty akce, kter\u00e9 jsou pot\u0159eba k tomu, aby se replikace dostala do konzistentn\u00edho stavu. Pokud se neprovedou ru\u010dn\u00ed manipulace, kter\u00e9 tento log replikace vr\u00e1t\u00ed zp\u011bt, merge bude muset b\u00fdt, tak \u010di onak, proveden.<\/p>\n<p><\/p>\n<p>Samoz\u0159ejm\u011b by nebylo na \u0161kodu m\u00edt omezen\u00ed na opera\u010dn\u00ed pam\u011bti, kter\u00e9 \u201epro ka\u017ed\u00fd p\u0159\u00edpad\u201c chr\u00e1n\u00ed pr\u00e1v\u011b p\u0159ed OOM. To nepom\u016f\u017ee merge prov\u00e9st, za\u010dne znovu, dojde k n\u011bjak\u00e9mu prahu, vyhod\u00ed v\u00fdjimku a pak za\u010dne znovu \u2013 nic dobr\u00e9ho z toho nevzejde. Ale v z\u00e1sad\u011b by bylo u\u017eite\u010dn\u00e9 toto omezen\u00ed zav\u00e9st.<\/p>\n<p><\/p>\n<h2 id=\"anchorgoanchorkak-budet-proishodit-razrabotka-golang-drayvera-dlya-clickhouse\"><noindex><a rel=\"nofollow\" name=\"go\"><\/a><\/noindex>Jak se bude vyv\u00edjet Golang driver pro ClickHouse?<\/h2>\n<p><\/p>\n<blockquote><p>Golang driver, kter\u00fd napsal Kirill Shvakov, je nyn\u00ed ofici\u00e1ln\u011b podporov\u00e1n t\u00fdmem ClickHouse. On <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/ClickHouse\/clickhouse-go\">v repozit\u00e1\u0159i ClickHouse<\/a><\/noindex>, je te\u010f velk\u00fd a prav\u00fd.<\/p>\n<p>Mal\u00e1 pozn\u00e1mka. Existuje \u00fa\u017easn\u00e9 a v\u0161emi milovan\u00e9 \u00falo\u017ei\u0161t\u011b norm\u00e1ln\u00edch forem nekone\u010dn\u00e9ho po\u0159ad\u00ed \u2014 to je Vertica. Maj\u00ed tak\u00e9 sv\u016fj ofici\u00e1ln\u00ed python ovlada\u010d, kter\u00fd je podporov\u00e1n v\u00fdvoj\u00e1\u0159i Vertica. A n\u011bkolikr\u00e1t se stalo, \u017ee verze \u00falo\u017ei\u0161t\u011b a verze ovlada\u010de se dost podstatn\u011b li\u0161ily, a ovlada\u010d v ur\u010dit\u00e9m okam\u017eiku p\u0159estal fungovat. Druh\u00fd bod. Podpora tohoto ofici\u00e1ln\u00edho ovlada\u010de, m\u00e1m dojem, funguje syst\u00e9mem \u201enipple\u201c \u2014 nap\u00ed\u0161ete jim issue a to vis\u00ed v\u011b\u010dn\u011b.<\/p>\n<p>M\u00e1m dv\u011b ot\u00e1zky. Aktu\u00e1ln\u011b je Golang ovlada\u010d Kirilla t\u00e9m\u011b\u0159 v\u00fdchoz\u00edm zp\u016fsobem, jak komunikovat z Golangu s ClickHouse. Snad jen n\u011bkdo st\u00e1le komunikuje p\u0159es http rozhran\u00ed, proto\u017ee mu to tak vyhovuje. Jak bude prob\u00edhat v\u00fdvoj tohoto ovlada\u010de? Bude se synchronizovat s n\u011bjak\u00fdmi breaking changes v samotn\u00e9m \u00falo\u017ei\u0161ti? A jak\u00fd je postup p\u0159i \u0159e\u0161en\u00ed issue? <\/p><\/blockquote>\n<p><strong>Kirill Shvakov:<\/strong> Prvn\u00ed v\u011bc je, jak to v\u0161e funguje byrokraticky. Tento moment nebyl projedn\u00e1n, tak\u017ee na to nemohu odpov\u011bd\u011bt.<\/p>\n<p><\/p>\n<p>Abych mohl odpov\u011bd\u011bt na ot\u00e1zku ohledn\u011b issue, je pot\u0159ebn\u00e1 mal\u00e1 historie ovlada\u010de. Pracoval jsem ve firm\u011b, kde bylo hodn\u011b dat. Byla to reklamn\u00ed platforma s obrovsk\u00fdm mno\u017estv\u00edm ud\u00e1lost\u00ed, kter\u00e9 bylo pot\u0159eba n\u011bkde uchov\u00e1vat. A v ur\u010dit\u00e9m okam\u017eiku se objevil ClickHouse. Nalili jsme tam data a na za\u010d\u00e1tku to \u0161lo dob\u0159e, ale pak ClickHouse spadl. V tu chv\u00edli jsme se rozhodli, \u017ee ho nepot\u0159ebujeme. <\/p>\n<p><\/p>\n<p>O rok pozd\u011bji jsme se k my\u0161lence pou\u017e\u00edv\u00e1n\u00ed ClickHouse vr\u00e1tili a pot\u0159ebovali jsme n\u011bjak\u00fdm zp\u016fsobem zapisovat data. Z\u00e1kladn\u00ed situace byla takov\u00e1 - hardware byl velmi slab\u00fd, prost\u0159edk\u016f m\u00e1lo. Ale v\u017edy jsme takto pracovali, a proto jsme se pod\u00edvali na nativn\u00ed protokol.<\/p>\n<p><\/p>\n<p>Jeliko\u017e jsme pracovali na Go, bylo jasn\u00e9, \u017ee pot\u0159ebujeme ovlada\u010d na Go. D\u011blal jsem ho prakticky na pln\u00fd \u00favazek \u2013 to byla moje pracovn\u00ed \u00faloha. Do ur\u010dit\u00e9ho okam\u017eiku jsme ho dovedli dokonce, a v podstat\u011b nikdo nep\u0159edpokl\u00e1dal, \u017ee by ho n\u011bkdo krom\u011b n\u00e1s pou\u017e\u00edval. Pak p\u0159i\u0161el CloudFlare s naprosto stejn\u00fdm probl\u00e9mem a n\u011bjakou dobu jsme s nimi pracovali velmi hladce, proto\u017ee m\u011bli stejn\u00e9 \u00fakoly. A d\u011blali jsme to jak v ClickHouse samotn\u00e9m, tak v ovlada\u010di. <\/p>\n<p><\/p>\n<p>V ur\u010dit\u00e9m okam\u017eiku jsem se na to prost\u011b vyka\u0161lal, proto\u017ee m\u00e9 aktivity z pohledu ClickHouse a pr\u00e1ce se malinko zm\u011bnily. Proto probl\u00e9my nejsou \u0159e\u0161eny. Ob\u010das do repozit\u00e1\u0159e commituje lidi, kte\u0159\u00ed n\u011bco pot\u0159ebuj\u00ed. Pak se pod\u00edv\u00e1m na pull request a n\u011bkdy n\u011bco s\u00e1m oprav\u00edm, ale to se st\u00e1v\u00e1 z\u0159\u00eddka.<\/p>\n<p><\/p>\n<p>Cht\u011bl bych se vr\u00e1tit k ovlada\u010di. P\u0159ed n\u011bkolika lety, kdy\u017e to v\u0161echno za\u010d\u00ednalo, byl ClickHouse tak\u00e9 jin\u00fd a m\u011bl jin\u00e9 mo\u017enosti. Te\u010f u\u017e rozum\u00edm, jak ovlada\u010d p\u0159epracovat, aby byl dobr\u00fd. Pokud se to stane, verze 2 bude v ka\u017ed\u00e9m p\u0159\u00edpad\u011b nekompatibiln\u00ed kv\u016fli nahromad\u011bn\u00fdm workaround\u016fm. <\/p>\n<p><\/p>\n<p>Jak to organizovat, nev\u00edm. M\u00e1m s\u00e1m m\u00e1lo \u010dasu. Pokud se n\u011bkdo rozhodne dod\u011blat ovlada\u010d, mohu jim pomoci a \u0159\u00edct, co d\u011blat. Ale aktivn\u00ed \u00fa\u010dast \"Yandxu\" na rozvoji projektu zat\u00edm nebyla nijak prob\u00edr\u00e1na. <\/p>\n<p><\/p>\n<p><strong>Alexei Milovidov:<\/strong> Ve skute\u010dnosti zat\u00edm neexistuje \u017e\u00e1dn\u00e1 byrokracie ohledn\u011b t\u011bchto ovlada\u010d\u016f. Jedin\u011b to, \u017ee byly p\u0159esunuty do ofici\u00e1ln\u00ed organizace, co\u017e znamen\u00e1, \u017ee tento ovlada\u010d je pova\u017eov\u00e1n za ofici\u00e1ln\u00ed v\u00fdchoz\u00ed \u0159e\u0161en\u00ed pro Go. Existuj\u00ed n\u011bjak\u00e9 jin\u00e9 ovlada\u010de, ale ty jsou odd\u011blen\u00e9. <\/p>\n<p><\/p>\n<p>V r\u00e1mci n\u00e1s neprob\u00edh\u00e1 \u017e\u00e1dn\u00fd v\u00fdvoj pro tyto ovlada\u010de. Ot\u00e1zka je, zda budeme moci najmout jin\u00e9ho \u010dlov\u011bka, ne p\u0159\u00edmo pro tento ovlada\u010d, ale pro rozvoj v\u0161ech komunitn\u00edch ovlada\u010d\u016f, nebo zda najdeme n\u011bkoho zven\u010d\u00ed. <\/p>\n<p><\/p>\n<h2 id=\"anchorlazy-loadanchorvneshniy-slovar-ne-podnimaetsya-posle-perezagruzki-snbspvklyuchennoy-nastroykoy-lazy_load-chto-delat\"><noindex><a rel=\"nofollow\" name=\"lazy-load\"><\/a><\/noindex>Extern\u00ed slovn\u00edk se po restartu nezvy\u0161uje s povolen\u00fdm nastaven\u00edm lazy_load. Co d\u011blat?<\/h2>\n<p><\/p>\n<blockquote><p>M\u00e1me povolen\u00e9 nastaven\u00ed lazy_load, a po restartu serveru se slovn\u00edk s\u00e1m nezved\u00e1. Zved\u00e1 se pouze pot\u00e9, co se u\u017eivatel obr\u00e1t\u00ed na tento slovn\u00edk. A p\u0159i prvn\u00edm p\u0159\u00edstupu vyhazuje chybu. Lze n\u011bjak automaticky prost\u0159ednictv\u00edm ClickHouse na\u010d\u00edtat slovn\u00edky, nebo je nutn\u00e9 v\u017edy sami kontrolovat jejich p\u0159ipravenost, aby u\u017eivatel\u00e9 nedost\u00e1vali chyby?<\/p>\n<p>Mo\u017en\u00e1 m\u00e1me starou verzi ClickHouse, a proto se slovn\u00edk automaticky nenab\u00eddl. M\u016f\u017ee to b\u00fdt?<\/p><\/blockquote>\n<p>Za prv\u00e9, slovn\u00edky lze vynutit na\u010d\u00edst pomoc\u00ed dotazu. <strong>system reload dictionaries<\/strong>. Za druh\u00e9, pokud jde o chybu \u2014 pokud je slovn\u00edk ji\u017e na\u010dten, dotazy budou pracovat s t\u011bmi daty, kter\u00e1 byla na\u010dtena. Pokud slovn\u00edk je\u0161t\u011b nebyl na\u010dten, na\u010dte se p\u0159\u00edmo b\u011bhem dotazu.<\/p>\n<p><\/p>\n<p>Pro t\u011b\u017ek\u00e9 slovn\u00edky to nen\u00ed moc pohodln\u00e9. Nap\u0159\u00edklad, je pot\u0159eba dot\u00e1hnout milion \u0159\u00e1dk\u016f z MySQL. N\u011bkdo provede jednoduch\u00fd select, ale tento select bude \u010dekat na on\u011bch milion \u0159\u00e1dk\u016f. Existuj\u00ed dv\u011b \u0159e\u0161en\u00ed. Prvn\u00ed \u2014 vypnout lazy_load. Druh\u00e9 \u2014 kdy\u017e server nab\u00edh\u00e1, p\u0159ed t\u00edm, ne\u017e se na n\u011bj zvedne zat\u00ed\u017een\u00ed, ud\u011blat <strong>system reload dictionary<\/strong> nebo jednodu\u0161e prov\u00e9st dotaz, kter\u00fd pou\u017e\u00edv\u00e1 slovn\u00edk. Pak se slovn\u00edk na\u010dte. Je pot\u0159eba sami kontrolovat dostupnost slovn\u00edk\u016f se zapnut\u00fdm nastaven\u00edm lazy_load, proto\u017ee ClickHouse je automaticky nepodporuje.<\/p>\n<p><\/p>\n<p>Na posledn\u00ed ot\u00e1zku odpov\u011b\u010f \u2014 bu\u010f je verze star\u00e1, nebo je pot\u0159eba ladit. <\/p>\n<p><\/p>\n<h2 id=\"anchorreload-dictionariesanchorkak-byt-snbsptem-chto-system-reload-dictionaries-ne-podgruzhaet-ni-odin-iznbspmnozhestva-slovarey-esli-hotya-by-odin-iznbspnih-padaet-snbsposhibkoy\"><noindex><a rel=\"nofollow\" name=\"reload-dictionaries\"><\/a><\/noindex>Jak si po\u010d\u00ednat v p\u0159\u00edpad\u011b, \u017ee system reload dictionaries nenahr\u00e1v\u00e1 \u017e\u00e1dn\u00fd z mnoha slovn\u00edk\u016f, pokud alespo\u0148 jeden z nich sel\u017ee s chybou?<\/h2>\n<p><\/p>\n<blockquote><p>Je\u0161t\u011b je dotaz ohledn\u011b system reload dictionaries. M\u00e1me dva slovn\u00edky \u2014 jeden se nenach\u00e1z\u00ed, druh\u00fd se na\u010d\u00edt\u00e1. System reload dictionaries v takov\u00e9m p\u0159\u00edpad\u011b nena\u010dte \u017e\u00e1dn\u00fd slovn\u00edk a je t\u0159eba konkr\u00e9tn\u00ed slovn\u00edk na\u010d\u00edst pomoc\u00ed system reload dictionary podle jeho n\u00e1zvu. To tak\u00e9 souvis\u00ed s verz\u00ed ClickHouse?<\/p><\/blockquote>\n<p>Chci v\u00e1s pot\u011b\u0161it. Toto chov\u00e1n\u00ed se m\u011bnilo. Pokud aktualizujete ClickHouse, tak se to tak\u00e9 zm\u011bn\u00ed. Pokud v\u00e1m aktu\u00e1ln\u00ed chov\u00e1n\u00ed nevyhovuje <strong>system reload dictionaries<\/strong>, aktualizujte se, a budeme doufat, \u017ee se to zm\u011bn\u00ed k lep\u0161\u00edmu.<\/p>\n<p><\/p>\n<h2 id=\"anchorconnectionanchorest-li-sposob-konfigurirovat-rekvizity-vnbspkonfige-clickhouse-no-ne-svetit-ih-prinbsposhibkah\"><noindex><a rel=\"nofollow\" name=\"connection\"><\/a><\/noindex>Existuje zp\u016fsob, jak konfigurovat \u00fadaje v&nbsp;konfigura\u010dn\u00edm souboru ClickHouse, ani\u017e by se zobrazovaly p\u0159i&nbsp;chyb\u00e1ch?<\/h2>\n<p><\/p>\n<blockquote><p>Dal\u0161\u00ed ot\u00e1zka t\u00fdkaj\u00edc\u00ed se chyb souvisej\u00edc\u00edch se slovn\u00edkem, konkr\u00e9tn\u011b p\u0159ihla\u0161ovac\u00ed \u00fadaje. P\u0159ihla\u0161ovac\u00ed \u00fadaje jsme zadali v konfiguraci ClickHouse ke slovn\u00edku, a p\u0159i chyb\u011b tyto \u00fadaje a heslo dost\u00e1v\u00e1me jako odpov\u011b\u010f. <\/p>\n<p>Tuto chybu jsme vy\u0159e\u0161ili vy\u010dlen\u011bn\u00edm p\u0159ihla\u0161ovac\u00edch \u00fadaj\u016f do konfigurace ODBC. Existuje zp\u016fsob, jak nakonfigurovat p\u0159ihla\u0161ovac\u00ed \u00fadaje v konfiguraci ClickHouse, ale nesv\u011bd\u010dit tyto \u00fadaje p\u0159i chyb\u00e1ch?<\/p><\/blockquote>\n<p>Zde je skute\u010dn\u011b \u0159e\u0161en\u00ed \u2014 uv\u00e1d\u011bt tyto credentials v odbc.ini, a v samotn\u00e9m ClickHouse uv\u00e1d\u011bt pouze ODBC Data Source Name. Pro ostatn\u00ed zdroje slovn\u00edk\u016f to nebude mo\u017en\u00e9 \u2014 ani pro slovn\u00edk z MySQL, ani pro ostatn\u00ed byste nem\u011bli vid\u011bt heslo p\u0159i zpr\u00e1v\u011b o chyb\u011b. Tak\u00e9 se na ODBC pod\u00edv\u00e1m \u2014 pokud n\u011bco existuje, m\u011blo by to prost\u011b zmizet.<\/p>\n<p><\/p>\n<h2 id=\"anchorzoom-backgroundsanchorbonus-fony-dlya-zuma-snbspposidelok\"><noindex><a rel=\"nofollow\" name=\"zoom-backgrounds\"><\/a><\/noindex>Bonus: pozad\u00ed pro Zoom z&nbsp;poseden\u00ed<\/h2>\n<p><\/p>\n<p>Klepnut\u00edm na&nbsp;obr\u00e1zek se otev\u0159ou bonusov\u00e9 pozad\u00ed z&nbsp;poseden\u00ed pro&nbsp;nejtvrd\u0161\u00ed \u010dten\u00e1\u0159e. Uhas\u00edme po\u017e\u00e1r spole\u010dn\u011b s&nbsp;maskoty technologi\u00ed Avito, konzultujeme s&nbsp;kolegy z&nbsp; m\u00edstnosti spr\u00e1vce nebo star\u00e9 po\u010d\u00edta\u010dov\u00e9 klubovny a po\u0159\u00e1d\u00e1me daily pod&nbsp;mostem na&nbsp;pozad\u00ed graffiti.<\/p>\n<p><\/p>\n<p><noindex><a rel=\"nofollow\" href=\"http:\/\/amp.gs\/KvUr\"><img decoding=\"async\" alt=\"ClickHouse pro pokro\u010dil\u00e9 u\u017eivatele v ot\u00e1zk\u00e1ch a odpov\u011bd\u00edch\" src=\"\/wp-content\/uploads\/2020\/05\/38b2ea076283285d934913c395863eb1.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/p>\n<p>Zdroj: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/avito\/blog\/500678\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0412 \u0430\u043f\u0440\u0435\u043b\u0435 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b \u0410\u0432\u0438\u0442\u043e \u0441\u043e\u0431\u0438\u0440\u0430\u043b\u0438\u0441\u044c \u043d\u0430&nbsp;\u043e\u043d\u043b\u0430\u0439\u043d-\u043f\u043e\u0441\u0438\u0434\u0435\u043b\u043a\u0438 \u0441 \u0433\u043b\u0430\u0432\u043d\u044b\u043c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c ClickHouse \u0410\u043b\u0435\u043a\u0441\u0435\u0435\u043c \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432\u044b\u043c \u0438 \u041a\u0438\u0440\u0438\u043b\u043b\u043e\u043c \u0428\u0432\u0430\u043a\u043e\u0432\u044b\u043c, Golang-\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c \u0438\u0437&nbsp;\u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Integros. \u041e\u0431\u0441\u0443\u0436\u0434\u0430\u043b\u0438, \u043a\u0430\u043a \u043c\u044b \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u0441\u0438\u0441\u0442\u0435\u043c\u0443 \u0443\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0431\u0430\u0437\u0430\u043c\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 \u0438 \u043a\u0430\u043a\u0438\u0435 \u0441\u043b\u043e\u0436\u043d\u043e\u0441\u0442\u0438 \u0443&nbsp;\u043d\u0430\u0441 \u0432\u043e\u0437\u043d\u0438\u043a\u0430\u044e\u0442. \u041f\u043e&nbsp;\u043c\u043e\u0442\u0438\u0432\u0430\u043c \u0432\u0441\u0442\u0440\u0435\u0447\u0438 \u043c\u044b \u0441\u043e\u0431\u0440\u0430\u043b\u0438 \u0441\u0442\u0430\u0442\u044c\u044e \u0441&nbsp;\u043e\u0442\u0432\u0435\u0442\u0430\u043c\u0438 \u044d\u043a\u0441\u043f\u0435\u0440\u0442\u043e\u0432 \u043d\u0430&nbsp;\u043d\u0430\u0448\u0438 \u0438 \u0437\u0440\u0438\u0442\u0435\u043b\u044c\u0441\u043a\u0438\u0435 \u0432\u043e\u043f\u0440\u043e\u0441\u044b \u043f\u0440\u043e&nbsp;\u0431\u044d\u043a\u0430\u043f\u044b, \u0440\u0435\u0448\u0430\u0440\u0434\u0438\u043d\u0433 \u0434\u0430\u043d\u043d\u044b\u0445, \u0432\u043d\u0435\u0448\u043d\u0438\u0435 \u0441\u043b\u043e\u0432\u0430\u0440\u0438, Golang-\u0434\u0440\u0430\u0439\u0432\u0435\u0440 \u0438 \u043e\u0431\u043d\u043e\u0432\u043b\u0435\u043d\u0438\u0435 \u0432\u0435\u0440\u0441\u0438\u0439 ClickHouse. \u041e\u043d\u0430 \u043c\u043e\u0436\u0435\u0442 \u0431\u044b\u0442\u044c [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":80776,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-80775","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.3 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0412 \u0430\u043f\u0440\u0435\u043b\u0435 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b \u0410\u0432\u0438\u0442\u043e \u0441\u043e\u0431\u0438\u0440\u0430\u043b\u0438\u0441\u044c \u043d\u0430 \u043e\u043d\u043b\u0430\u0439\u043d-\u043f\u043e\u0441\u0438\u0434\u0435\u043b\u043a\u0438 \u0441 \u0433\u043b\u0430\u0432\u043d\u044b\u043c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c ClickHouse \u0410\u043b\u0435\u043a\u0441\u0435\u0435\u043c \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432\u044b\u043c \u0438 \u041a\u0438\u0440\u0438\u043b\u043b\u043e\u043c \u0428\u0432\u0430\u043a\u043e\u0432\u044b\u043c, Golang-\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Integros.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah\" \/>\n\t\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.3\" \/>\n\t\t<meta property=\"og:locale\" content=\"cs_CZ\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47ClickHouse \u0434\u043b\u044f \u043f\u0440\u043e\u0434\u0432\u0438\u043d\u0443\u0442\u044b\u0445 \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u0435\u0439 \u0432 \u0432\u043e\u043f\u0440\u043e\u0441\u0430\u0445 \u0438 \u043e\u0442\u0432\u0435\u0442\u0430\u0445 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0412 \u0430\u043f\u0440\u0435\u043b\u0435 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b \u0410\u0432\u0438\u0442\u043e \u0441\u043e\u0431\u0438\u0440\u0430\u043b\u0438\u0441\u044c \u043d\u0430 \u043e\u043d\u043b\u0430\u0439\u043d-\u043f\u043e\u0441\u0438\u0434\u0435\u043b\u043a\u0438 \u0441 \u0433\u043b\u0430\u0432\u043d\u044b\u043c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c ClickHouse \u0410\u043b\u0435\u043a\u0441\u0435\u0435\u043c \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432\u044b\u043c \u0438 \u041a\u0438\u0440\u0438\u043b\u043b\u043e\u043c \u0428\u0432\u0430\u043a\u043e\u0432\u044b\u043c, Golang-\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Integros.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-05-08T11:42:47+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-05-08T11:42:47+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47ClickHouse pro pokro\u010dil\u00e9 u\u017eivatele v ot\u00e1zk\u00e1ch a odpov\u011bd\u00edch | ProHoster","description":"V dubnu in\u017een\u00fd\u0159i Avito pl\u00e1novali online setk\u00e1n\u00ed s hlavn\u00edm v\u00fdvoj\u00e1\u0159em ClickHouse Alexejem Milovidovem a Kirillem \u0160vakovem, v\u00fdvoj\u00e1\u0159em v Golangu z firmy Integros.","canonical_url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"cs_CZ","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47ClickHouse \u0434\u043b\u044f \u043f\u0440\u043e\u0434\u0432\u0438\u043d\u0443\u0442\u044b\u0445 \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u0435\u0439 \u0432 \u0432\u043e\u043f\u0440\u043e\u0441\u0430\u0445 \u0438 \u043e\u0442\u0432\u0435\u0442\u0430\u0445 | ProHoster","og:description":"\u0412 \u0430\u043f\u0440\u0435\u043b\u0435 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b \u0410\u0432\u0438\u0442\u043e \u0441\u043e\u0431\u0438\u0440\u0430\u043b\u0438\u0441\u044c \u043d\u0430 \u043e\u043d\u043b\u0430\u0439\u043d-\u043f\u043e\u0441\u0438\u0434\u0435\u043b\u043a\u0438 \u0441 \u0433\u043b\u0430\u0432\u043d\u044b\u043c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c ClickHouse \u0410\u043b\u0435\u043a\u0441\u0435\u0435\u043c \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432\u044b\u043c \u0438 \u041a\u0438\u0440\u0438\u043b\u043b\u043e\u043c \u0428\u0432\u0430\u043a\u043e\u0432\u044b\u043c, Golang-\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Integros.","og:url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-05-08T11:42:47+00:00","article:modified_time":"2020-05-08T11:42:47+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"80775","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 16:11:22","updated":"2022-09-28 05:48:13","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/80775","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/comments?post=80775"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/80775\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/media\/80776"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/media?parent=80775"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/categories?post=80775"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/tags?post=80775"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}