{"id":80775,"date":"2020-05-08T13:42:47","date_gmt":"2020-05-08T11:42:47","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah"},"modified":"2020-05-08T13:42:47","modified_gmt":"2020-05-08T11:42:47","slug":"clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah","title":{"rendered":"ClickHouse p\u00ebr p\u00ebrdoruesit e avancuar n\u00eb pyetje dhe p\u00ebrgjigje","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>N\u00eb prill, inxhinier\u00ebt e Avito kishin planifikuar nj\u00eb takim online me kryetarin e zhvilluesve t\u00eb ClickHouse, Aleksej Milovidov, dhe Kirill Shvakov, zhvilluesin e Golang nga kompania Integros. Diskutuan se si e p\u00ebrdorim sistemin e menaxhimit t\u00eb bazave t\u00eb dh\u00ebnash dhe cilat v\u00ebshtir\u00ebsi hasim. <\/p>\n<p><\/p>\n<p>P\u00ebr motive t\u00eb takimit, ne p\u00ebrgatit\u00ebm nj\u00eb artikull me p\u00ebrgjigjet e ekspert\u00ebve n\u00eb pyetje tona dhe ato t\u00eb shikuesve mbi backup-et, reshardimin e t\u00eb dh\u00ebnave, fjalor\u00ebt e jasht\u00ebm, driver-in Golang dhe p\u00ebrdit\u00ebsimin e versioneve t\u00eb ClickHouse. Ky informacion mund t\u00eb jet\u00eb i dobish\u00ebm p\u00ebr zhvilluesit q\u00eb tashm\u00eb punojn\u00eb aktivisht me DB-n\u00eb \u2018Yandex\u2019 dhe q\u00eb ndihen t\u00eb interesuar p\u00ebr t\u00eb ardhmen dhe aktualitetin e saj. Si zakonisht, p\u00ebrgjigjet e Aleksej Milovidov, p\u00ebrve\u00e7 n\u00ebse p\u00ebrmendet ndryshe. <\/p>\n<p><\/p>\n<p>Kujdes, ka shum\u00eb tekst m\u00eb posht\u00eb. Shpresojm\u00eb q\u00eb p\u00ebrmbajtja me pyetje do t'ju ndihmoj\u00eb p\u00ebr orientim.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"ClickHouse p\u00ebr p\u00ebrdoruesit e avancuar n\u00eb pyetje dhe p\u00ebrgjigje\" src=\"\/wp-content\/uploads\/2020\/05\/242b1d8d002fe115614435c242297fd2.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2 id=\"soderzhanie\">P\u00ebrmbajtja<\/h2>\n<p><\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"#old-data\">ClickHouse vazhdon t\u00eb p\u00ebrdit\u00ebsohet, nd\u00ebrsa t\u00eb dh\u00ebnat tona \u2014 jo. \u00c7far\u00eb duhet t\u00eb b\u00ebjm\u00eb p\u00ebr k\u00ebt\u00eb?<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#backup-best-practicies\">Cilat jan\u00eb praktikat m\u00eb t\u00eb mira p\u00ebr momentin p\u00ebr backup-in e t\u00eb dh\u00ebnave nga ClickHouse?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#replication\">A do t\u00eb jet\u00eb e mundur t\u00eb organizoni nj\u00eb vones\u00eb t\u00eb kontrolluar t\u00eb replika n\u00eb volka?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#soooo-changeable\">\u00c7far\u00eb t\u00eb b\u00ebjm\u00eb n\u00ebse struktura e tabel\u00ebs ka ndryshuar?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#resharding-best-practices\">Cilat jan\u00eb praktikat m\u00eb t\u00eb mira aktualisht n\u00eb reshardimin e t\u00eb dh\u00ebnave?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#clickhouse-copier\">N\u00eb ClickHouse ekziston utilita clickhouse-copier. A mund t\u00eb flisni p\u00ebr t\u00eb?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#resharding-tool\">Keni pasur nj\u00eb nxjerrje pilot q\u00eb quhej reshardim. \u00c7fare ndodhi me t\u00eb?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#move-to-slow-disk\">A \u00ebsht\u00eb e mundur t\u00eb bashkoni t\u00eb gjitha pjes\u00ebt e t\u00eb dh\u00ebnave para se t\u00eb zhvendosni n\u00eb disqe t\u00eb ngadalta?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#up-to-date\">Si t\u00eb kalojm\u00eb n\u00eb versione t\u00eb reja t\u00eb ClickHouse, n\u00ebse nuk ka mund\u00ebsi t\u00eb kontrollohet paraprakisht p\u00ebr kompatibilitetin?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#kill-query\">Kill query duhet t\u00eb ndal\u00eb k\u00ebrkesat, por kjo nuk ndodh. Pse?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#reading-time\">Si t\u00eb llogarisim koh\u00ebn e p\u00ebrgjigjes n\u00ebn ngarkes\u00eb lexuese?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#pimp-my-clickhouse\">\u00c7far\u00eb duhet rregulluar n\u00eb ClickHouse p\u00ebr t\u00eb pasur m\u00eb shum\u00eb t\u00eb dh\u00ebna n\u00eb ke\u0161?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#storage-configuration\">Si mund t\u00eb konfiguroj storage_configuration p\u00ebr ruajtjen n\u00eb RAM?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#low-cardinality\">Derisa sa unik\u00eb vlera jan\u00eb efektive Low Cardinality?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#fulltext-search\">Cilat jan\u00eb praktikat m\u00eb t\u00eb mira p\u00ebr k\u00ebrkimin e plot\u00eb n\u00eb nj\u00eb tabel\u00eb me pes\u00eb miliard\u00eb rreshta?<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#hello-and-welcome\">Si \u00ebsht\u00eb m\u00eb mir\u00eb t\u00eb organizoni qasjen n\u00eb ClickHouse p\u00ebr nj\u00eb num\u00ebr t\u00eb madh p\u00ebrdoruesish?<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#smorgasbord\">A \u00ebsht\u00eb e mundur t\u00eb jepen rezultatet e nj\u00eb k\u00ebrkese n\u00eb dhjet\u00eb klient\u00eb?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#asynchronous\">Si t\u00eb veprojm\u00eb me operacionet asinkrone dhe pamjet e materializuara?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#dashboard\">ClickHouse ka shum\u00eb ditar\u00eb. Si mund t\u00eb shoh gjith\u00e7ka q\u00eb ndodh me serverin n\u00eb momentin e duhur?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#zen\">Si t\u00eb ndikojm\u00eb n\u00eb bashk\u00ebngjitjet, n\u00eb m\u00ebnyr\u00eb q\u00eb serveri t\u00eb mos shkoj\u00eb n\u00eb OOM?<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#go\">Si do t\u00eb zhvillohet driver-i Golang p\u00ebr ClickHouse?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#lazy-load\">Fjalori i jasht\u00ebm nuk ngrihet pas rinisjes me funksionin e aktivizuar lazy_load. \u00c7far\u00eb duhet t\u00eb b\u00ebjm\u00eb?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#reload-dictionaries\">Si t\u00eb veprojm\u00eb me faktin q\u00eb system reload dictionaries nuk ngarkon asnj\u00eb nga shum\u00eb fjalor\u00ebt, n\u00ebse t\u00eb pakt\u00ebn nj\u00ebra prej tyre d\u00ebshtoi me nj\u00eb gabim?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#connection\">A ka ndonj\u00eb m\u00ebnyr\u00eb p\u00ebr t\u00eb konfiguruar kredencialet n\u00eb konfigurimin ClickHouse, por pa i ekspozuar ato gjat\u00eb gabimeve?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#zoom-backgrounds\">Bonus: sfondet p\u00ebr Zoom nga takimet<\/a><\/noindex><\/li>\n<\/ul>\n<p><\/p>\n<blockquote><p>N\u00ebse nuk d\u00ebshironi t\u00eb lexoni, mund t\u00eb shikoni regjistrimin e takimeve. <noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=n1tm4j4W8ZQ&amp;t=8147s\">n\u00eb kanal ton\u00eb n\u00eb youtube<\/a><\/noindex>. Koha \u00ebsht\u00eb n\u00eb komentarin e par\u00eb n\u00ebn videon.<\/p><\/blockquote>\n<p><\/p>\n<h2 id=\"anchorold-dataanchorclickhouse-postoyanno-obnovlyaetsya-a-nashi-dannyenbsp-net-chto-snbspetim-delat\"><noindex><a rel=\"nofollow\" name=\"old-data\"><\/a><\/noindex>ClickHouse vazhdon t\u00eb p\u00ebrdit\u00ebsohet, nd\u00ebrsa t\u00eb dh\u00ebnat tona \u2014 jo. \u00c7far\u00eb duhet t\u00eb b\u00ebjm\u00eb p\u00ebr k\u00ebt\u00eb?<\/h2>\n<p><\/p>\n<blockquote><p>ClickHouse vazhdon t\u00eb p\u00ebrdit\u00ebsohet, nd\u00ebrsa t\u00eb dh\u00ebnat tona, t\u00eb cilat ishin p\u00ebrpunuar me optimize final, nuk p\u00ebrdit\u00ebsohen dhe q\u00ebndrojn\u00eb n\u00eb rezerv\u00eb. <\/p>\n<p>Supozoni se ndodhi ndonj\u00eb problem dhe t\u00eb dh\u00ebnat u humb\u00ebn. Ne vendos\u00ebm t\u00eb rikuperohemi dhe u zbulua se partit\u00eb e vjetra, q\u00eb ruhen n\u00eb server\u00ebt e backup-it, ndahen shum\u00eb nga versioni aktual i ClickHouse. \u00c7far\u00eb t\u00eb b\u00ebjm\u00eb n\u00eb nj\u00eb situat\u00eb t\u00eb till\u00eb, dhe \u00ebsht\u00eb e mundur?<\/p><\/blockquote>\n<p>Nj\u00eb situat\u00eb n\u00eb t\u00eb cil\u00ebn rikuperoni t\u00eb dh\u00ebnat nga nj\u00eb backup n\u00eb nj\u00eb format t\u00eb vjet\u00ebr, nd\u00ebrsa n\u00eb versionin e ri ato nuk lidh\u00ebn, nuk ndodh. Ne kujdesemi q\u00eb formatet e t\u00eb dh\u00ebnave n\u00eb ClickHouse t\u00eb mbeten gjithmon\u00eb prapa kompatib\u00ebl. Kjo \u00ebsht\u00eb shum\u00eb m\u00eb e r\u00ebnd\u00ebsishme se prapa kompatibiliteti p\u00ebr funksionalitetin, n\u00ebse \u00ebsht\u00eb ndryshuar sjellja e ndonj\u00eb funksioni t\u00eb rrall\u00eb. T\u00eb dh\u00ebnat q\u00eb jan\u00eb ruajtur n\u00eb disk, versioni i ri i ClickHouse gjithmon\u00eb duhet t\u00eb jet\u00eb n\u00eb gjendje t'i lexoj\u00eb. Kjo \u00ebsht\u00eb nj\u00eb ligj. <\/p>\n<p><\/p>\n<h2 id=\"anchorbackup-best-practiciesanchorkakie-luchshie-praktiki-est-nanbspdannyy-moment-ponbsprezervnomu-kopirovaniyu-dannyh-iznbspclickhouse\"><noindex><a rel=\"nofollow\" name=\"backup-best-practicies\"><\/a><\/noindex>Cilat jan\u00eb praktikat m\u00eb t\u00eb mira p\u00ebr momentin p\u00ebr backup-in e t\u00eb dh\u00ebnave nga ClickHouse?<\/h2>\n<p><\/p>\n<blockquote><p>Si t\u00eb b\u00ebjm\u00eb kopje rezerv\u00eb duke marr\u00eb parasysh q\u00eb kemi operacione optimize final, nj\u00eb databaz\u00eb t\u00eb madhe n\u00eb terabajt\u00eb, dhe t\u00eb dh\u00ebna q\u00eb p\u00ebrdit\u00ebsohen, le t\u00eb themi, gjat\u00eb tri dit\u00ebve t\u00eb fundit, dhe pastaj nuk ndodhin procedura m\u00eb me to? <\/p>\n<p>Ne mund t\u00eb b\u00ebjm\u00eb nj\u00eb zgjidhje ton\u00ebn dhe n\u00eb bash\u00eb t\u00eb shkruajm\u00eb: mblidh k\u00ebshtu e k\u00ebshtu k\u00ebto kopje rezerv\u00eb. Ndoshta nuk duhet t\u00eb b\u00ebjm\u00eb asgj\u00eb, dhe bi\u00e7ikleta \u00ebsht\u00eb shpikur prej koh\u00ebsh? <\/p><\/blockquote>\n<p>S\u00eb pari p\u00ebr praktikat m\u00eb t\u00eb mira. Koleg\u00ebt e mi gjithmon\u00eb rekomandojn\u00eb si p\u00ebrgjigje p\u00ebr pyetjet rreth backupeve t\u00eb p\u00ebrmendin sh\u00ebrbimin 'Yandex.Cloud', ku kjo detyr\u00eb \u00ebsht\u00eb zgjidhur tashm\u00eb. Pra, p\u00ebrfitoni nga ai, n\u00ebse keni mund\u00ebsi. <\/p>\n<p><\/p>\n<p>Nuk ka nj\u00eb zgjidhje t\u00eb plot\u00eb, krejt\u00ebsisht t\u00eb integruar n\u00eb ClickHouse, p\u00ebr backupe. Ka disa p\u00ebrgatitje q\u00eb mund t\u00eb p\u00ebrdoren. P\u00ebr t\u00eb marr\u00eb nj\u00eb zgjidhje t\u00eb plot\u00eb, do t\u00eb duhet t\u00eb punoni pak manualisht ose t\u00eb b\u00ebni mb\u00ebshtjellje n\u00eb form\u00ebn e skripteve.<\/p>\n<p><\/p>\n<p>Do t\u00eb filloj me zgjidhjet m\u00eb t\u00eb thjeshta dhe do t\u00eb p\u00ebrfundoj me ato m\u00eb t\u00eb avancuara n\u00eb var\u00ebsi t\u00eb volumit t\u00eb t\u00eb dh\u00ebnave dhe madh\u00ebsis\u00eb s\u00eb klasterit. Sa m\u00eb i madh clasteri, aq m\u00eb e v\u00ebshtir\u00eb b\u00ebhet zgjidhja.<\/p>\n<p><\/p>\n<p>N\u00ebse tabela me t\u00eb dh\u00ebnat z\u00eb vet\u00ebm disa gigabajt, backup mund t\u00eb b\u00ebhet k\u00ebshtu: <\/p>\n<p><\/p>\n<ol>\n<li>Ruani definicionin e tabelave, dmth metadatat \u2014 <strong>show create table<\/strong>.<\/li>\n<li>T\u00eb b\u00ebhet dump me ndihm\u00ebn e klientit t\u00eb ClickHouse \u2014 <strong>select<\/strong> * <strong>nga tabela<\/strong> n\u00eb skedarin. Sipas parazgjedhjes do t\u00eb merrni nj\u00eb skedar n\u00eb formatin TabSeparated. N\u00ebse d\u00ebshironi m\u00eb efikasitet, mund t\u00eb p\u00ebrdorni formatin Native. <\/li>\n<\/ol>\n<p><\/p>\n<p>N\u00ebse volumi i t\u00eb dh\u00ebnave \u00ebsht\u00eb m\u00eb i madh, backup do t\u00eb marr\u00eb m\u00eb shum\u00eb koh\u00eb dhe hap\u00ebsir\u00eb. Kjo quhet backup logjik, ai nuk \u00ebsht\u00eb i lidhur me formatin e t\u00eb dh\u00ebnave ClickHouse. N\u00ebse \u00ebsht\u00eb aty, at\u00ebher\u00eb n\u00eb rastin m\u00eb ekstrem do t\u00eb jeni n\u00eb gjendje ta merrni back-up dhe ta ngarkoni n\u00eb MySQL p\u00ebr rikuperim. <\/p>\n<p><\/p>\n<p>P\u00ebr raste m\u00eb t\u00eb avancuara, ClickHouse ka nj\u00eb mund\u00ebsi t\u00eb integruar p\u00ebr t\u00eb krijuar nj\u00eb snapshot t\u00eb partive n\u00eb sistemin lokal t\u00eb skedar\u00ebve. Kjo mund\u00ebsi \u00ebsht\u00eb e disponueshme n\u00eb form\u00ebn e nj\u00eb k\u00ebrkese. <strong>alter table freeze partition<\/strong>. Ose thjesht <strong>alter table freeze<\/strong> \u2014 ky \u00ebsht\u00eb snapshot i t\u00ebr\u00eb tabel\u00ebs. <\/p>\n<p><\/p>\n<p>Nj\u00eb snapshot do t\u00eb krijohet n\u00eb m\u00ebnyr\u00eb konsistente p\u00ebr nj\u00eb tabel\u00eb n\u00eb nj\u00eb shard, pra \u00ebsht\u00eb e paqart\u00eb t\u00eb krijosh nj\u00eb snapshot konsistent p\u00ebr t\u00eb gjith\u00eb klasterin n\u00eb k\u00ebt\u00eb m\u00ebnyr\u00eb. Megjithat\u00eb, p\u00ebr shumic\u00ebn e detyrave nuk ka nevoj\u00eb p\u00ebr k\u00ebt\u00eb, dhe \u00ebsht\u00eb e mjaftueshme q\u00eb t\u00eb kryhet nj\u00eb k\u00ebrkes\u00eb n\u00eb secilin shard dhe t\u00eb marr\u00eb nj\u00eb snapshot konsistent. Ai krijohet si hard links dhe prandaj nuk z\u00eb hap\u00ebsir\u00eb shtes\u00eb. M\u00eb pas, ky snapshot e kopjoni n\u00eb serverin e backup-it ose n\u00eb depozitat q\u00eb p\u00ebrdorni p\u00ebr backup-et.<\/p>\n<p><\/p>\n<p>T\u00eb rikuperosh nj\u00eb backup t\u00eb till\u00eb \u00ebsht\u00eb mjaft e leht\u00eb. E para \u2014 krijoni tabelat sipas p\u00ebrkufizimeve ekzistuese t\u00eb tabelave. M\u00eb pas, kopjoni snapshot-at e ruajtur t\u00eb particioneve n\u00eb Directory-Detached p\u00ebr k\u00ebto tabela dhe kryeni k\u00ebrkes\u00ebn. <strong>attach partition<\/strong>. Ky zgjidhje \u00ebsht\u00eb plot\u00ebsisht e p\u00ebrshtatshme p\u00ebr volumin m\u00eb serioz t\u00eb t\u00eb dh\u00ebnave. <\/p>\n<p><\/p>\n<p>Ndonj\u00ebher\u00eb k\u00ebrkohet di\u00e7ka akoma m\u00eb t\u00eb avancuar \u2014 n\u00eb ato raste kur keni dhjet\u00ebra ose madje qindra terabajt n\u00eb \u00e7do server dhe qindra server\u00eb. K\u00ebtu ka nj\u00eb zgjidhje, e cila e kam m\u00ebsuar nga koleg\u00ebt n\u00eb \"Yandex.Metrics\". Nuk do t\u00eb rekomandoja at\u00eb p\u00ebr t\u00eb gjith\u00eb \u2014 lexoni dhe vendosni vet\u00eb n\u00ebse ju p\u00ebrshtatet ose jo. <\/p>\n<p><\/p>\n<p>S\u00eb pari, duhet t\u00eb krijoni disa servera me raftet e m\u00ebdha t\u00eb disqeve. M\u00eb pas, n\u00eb k\u00ebta server\u00eb do t\u00eb ngrini disa server\u00eb ClickHouse dhe t'i konfiguroni ata n\u00eb m\u00ebnyr\u00eb q\u00eb t\u00eb funksionojn\u00eb si nj\u00eb replik\u00eb tjet\u00ebr p\u00ebr t\u00eb nj\u00ebjtat shards. M\u00eb pas, p\u00ebrdorni nj\u00eb sistem skedar\u00ebsh ose nj\u00eb mjet q\u00eb lejon krijimin e snapshot-eve. K\u00ebtu ka dy mund\u00ebsi. Mund\u00ebsia e par\u00eb \u2014 jan\u00eb snapshot-et LVM, mund\u00ebsia e dyt\u00eb \u2014 \u00ebsht\u00eb ZFS n\u00eb Linux. <\/p>\n<p><\/p>\n<p>Pas k\u00ebsaj, \u00e7do dit\u00eb duhet t\u00eb krijoni nj\u00eb snapshot, ai do t\u00eb q\u00ebndroj\u00eb dhe do t\u00eb z\u00ebr\u00eb ndonj\u00eb hap\u00ebsir\u00eb. Sigurisht, n\u00ebse t\u00eb dh\u00ebnat ndryshojn\u00eb, at\u00ebher\u00eb me kalimin e koh\u00ebs, volumi i hap\u00ebsir\u00ebs do t\u00eb rritet. Ky snapshot mund t\u00eb nxirret n\u00eb \u00e7do moment dhe t\u00eb rikuperoni t\u00eb dh\u00ebnat, \u00ebsht\u00eb nj\u00eb zgjidhje e \u00e7uditshme. Plus, gjithashtu duhet t'i kufizoni k\u00ebto replika n\u00eb konfigurim, n\u00eb m\u00ebnyr\u00eb q\u00eb ato t\u00eb mos p\u00ebrpiqen t\u00eb b\u00ebhen lider\u00eb.<\/p>\n<p><\/p>\n<h2 id=\"anchorreplicationanchormozhno-li-budet-organizovat-kontroliruemoe-otstavanie-replik-vnbspvalah\"><noindex><a rel=\"nofollow\" name=\"replication\"><\/a><\/noindex>A do t\u00eb jet\u00eb e mundur t\u00eb organizoni nj\u00eb vones\u00eb t\u00eb kontrolluar t\u00eb replika n\u00eb volka?<\/h2>\n<p><\/p>\n<blockquote><p>K\u00ebt\u00eb vit planifikoni t\u00eb b\u00ebni vala n\u00eb ClickHouse. A do t\u00eb jet\u00eb e mundur t\u00eb organizoni nj\u00eb vones\u00eb t\u00eb kontrolluar t\u00eb replikave n\u00eb to? Ne do t\u00eb doja ta p\u00ebrdorim k\u00ebt\u00eb p\u00ebr t\u00eb siguruar veten nga skenar\u00ebt negativ\u00eb me alterimet dhe ndryshime t\u00eb tjera. <\/p>\n<p>A \u00ebsht\u00eb e mundur t\u00eb b\u00ebni ndonj\u00eb roll back p\u00ebr alteret? P\u00ebr shembull, n\u00eb vallen ekzistuese t\u00eb themi q\u00eb deri n\u00eb k\u00ebt\u00eb moment aplikoni ndryshimet, kurse nga ky moment ndaloni aplikimin e ndryshimeve?<\/p>\n<p>N\u00ebse n\u00eb klasterin ton\u00eb ka ardhur nj\u00eb ekip dhe e ka prishur at\u00eb, at\u00ebher\u00eb ne kemi nj\u00eb replik\u00eb t\u00eb kushtuar me nj\u00eb vones\u00eb prej nj\u00eb ore, ku mund t\u00eb themi se le t\u00eb p\u00ebrdorim pik\u00ebrisht at\u00eb n\u00eb k\u00ebt\u00eb moment, por nuk do t\u00eb aplikojm\u00eb ndryshimet e fundit t\u00eb dhjet\u00eb minutave n\u00eb t\u00eb. <\/p><\/blockquote>\n<p>S\u00eb pari, p\u00ebrsa i p\u00ebrket kontrolletit t\u00eb vones\u00ebs s\u00eb kopjeve. Ky ishte nj\u00eb k\u00ebrkes\u00eb nga p\u00ebrdoruesit dhe ne krijuam nj\u00eb \u00e7\u00ebshtje n\u00eb GitHub me k\u00ebrkes\u00ebn: \"N\u00ebse dikujt i nevojitet, jepni nj\u00eb like, jepni nj\u00eb zem\u00ebr\". Askush nuk e b\u00ebri k\u00ebt\u00eb dhe \u00e7\u00ebshtja u mbyll. Megjithat\u00eb, tani \u00ebsht\u00eb e mundur t\u00eb merrni nj\u00eb mund\u00ebsi t\u00eb till\u00eb duke e konfiguruar ClickHouse. E v\u00ebrteta \u00ebsht\u00eb se kjo \u00ebsht\u00eb e mundur vet\u00ebm q\u00eb nga versione 20.3.<\/p>\n<p><\/p>\n<p>ClickHouse vazhdon t\u00eb kryej\u00eb shumicimin e t\u00eb dh\u00ebnave n\u00eb sfond \u2014 merxh. Kur merxh b\u00ebhet, nj\u00eb grup i caktuar t\u00eb dh\u00ebnash z\u00ebvend\u00ebsohet me nj\u00eb grup m\u00eb t\u00eb madh. Megjithat\u00eb, grupet e t\u00eb dh\u00ebnave q\u00eb ishin m\u00eb par\u00eb vazhdojn\u00eb t\u00eb q\u00ebndrojn\u00eb n\u00eb disk p\u00ebr nj\u00eb koh\u00eb t\u00eb caktuar.<\/p>\n<p><\/p>\n<p>S\u00eb pari, ato vazhdojn\u00eb t\u00eb ruajn\u00eb deri n\u00eb momentin kur ka k\u00ebrkesa select q\u00eb i p\u00ebrdorin ato, p\u00ebr t\u00eb siguruar nj\u00eb funksionim pa bllokime. K\u00ebrkesat select lexojn\u00eb qet\u00ebsisht nga grupet e vjetra.<\/p>\n<p><\/p>\n<p>S\u00eb dyti, ka nj\u00eb prag kohor \u2014 grupet e vjetra t\u00eb t\u00eb dh\u00ebnave q\u00ebndrojn\u00eb n\u00eb disk p\u00ebr tet\u00eb minuta. K\u00ebto tet\u00eb minuta mund t\u00eb konfigurohen dhe mund t\u00eb shnd\u00ebrrohen edhe n\u00eb nj\u00eb dit\u00eb. Kjo do t\u00eb kushtoj\u00eb hap\u00ebsir\u00eb n\u00eb disk: n\u00eb var\u00ebsi t\u00eb fluksit t\u00eb t\u00eb dh\u00ebnave, mund t\u00eb ndodhi q\u00eb n\u00eb dit\u00ebn e fundit t\u00eb dh\u00ebnat jo vet\u00ebm q\u00eb do t\u00eb dyfishohen, por mund t\u00eb b\u00ebhen pes\u00eb her\u00eb m\u00eb shum\u00eb. Por n\u00ebse keni nj\u00eb problem serioz, mund ta ndaloni serverin ClickHouse dhe t\u00eb zgjidhni gjith\u00e7ka.<\/p>\n<p><\/p>\n<p>Tani shkaktohet pyetja, si kjo mbron nga alter-at. K\u00ebtu duhen par\u00eb gj\u00ebrat m\u00eb thell\u00eb, sepse n\u00eb versionet e vjetra t\u00eb ClickHouse, alter punonte n\u00eb nj\u00eb m\u00ebnyr\u00eb ku thjesht nd\u00ebrronte grupet. Ka nj\u00eb grup t\u00eb dh\u00ebnash me disa skedar\u00eb, dhe ne b\u00ebjm\u00eb, p\u00ebr shembull, <strong>alter drop column<\/strong>. Pastaj ky kolon shkat\u00ebrrohet fizikisht nga t\u00eb gjith\u00eb grupet.<\/p>\n<p><\/p>\n<p>Por duke filluar nga versioni 20.3, mekanizmi i alter-eve \u00ebsht\u00eb ndryshuar plot\u00ebsisht, dhe tani grupet e t\u00eb dh\u00ebnave jan\u00eb gjithmon\u00eb t\u00eb pandryshueshme. Ato nuk ndryshojn\u00eb fare \u2014 alter-at tani funksionojn\u00eb m\u00eb shum\u00eb si merxh. N\u00eb vend q\u00eb t\u00eb nd\u00ebrroni nj\u00eb grup n\u00eb vend, ne krijojm\u00eb nj\u00eb t\u00eb ri. N\u00eb grupin e ri, skedar\u00ebt q\u00eb nuk jan\u00eb nd\u00ebrruar b\u00ebhen hardlink, dhe n\u00ebse ne kemi fshir\u00eb nj\u00eb kolon\u00eb, ajo thjesht do t\u00eb mungoj\u00eb n\u00eb grupin e ri. Grupi i vjet\u00ebr do t\u00eb fshihet automatikisht pas tet\u00eb minutash, dhe k\u00ebtu mund t\u00eb rregullojm\u00eb parametrat e p\u00ebrmendur m\u00eb par\u00eb. <\/p>\n<p><\/p>\n<p>E nj\u00ebjta gj\u00eb vlen edhe p\u00ebr alteret e tipit t\u00eb mutacioneve. Kur b\u00ebni <strong>alter delete<\/strong> ose <strong>alter update<\/strong>, ai nuk ndryshon nj\u00eb pjes\u00eb, por krijon nj\u00eb t\u00eb re. Pastaj fshin t\u00eb vjetr\u00ebn.<\/p>\n<p><\/p>\n<h2 id=\"anchorsoooo-changeableanchorkak-byt-esli-struktura-tablicy-pomenyalas\"><noindex><a rel=\"nofollow\" name=\"soooo-changeable\"><\/a><\/noindex>\u00c7far\u00eb t\u00eb b\u00ebjm\u00eb n\u00ebse struktura e tabel\u00ebs ka ndryshuar?<\/h2>\n<p><\/p>\n<blockquote><p>Si t\u00eb ngrem\u00eb nj\u00eb backup q\u00eb \u00ebsht\u00eb b\u00ebr\u00eb me skem\u00ebn e vjet\u00ebr? Dhe pyetja tjet\u00ebr lidhet me rastin me snapshot-et dhe mjetet e sistemit t\u00eb skedar\u00ebve. A \u00ebsht\u00eb i p\u00ebrshtatsh\u00ebm Btrfs n\u00eb vend t\u00eb ZFS p\u00ebr Linux LVM?<\/p><\/blockquote>\n<p>N\u00ebse krijoni <strong>attach partition<\/strong> particionet me nj\u00eb struktur\u00eb tjet\u00ebr, at\u00ebher\u00eb ClickHouse do t'ju thot\u00eb q\u00eb kjo nuk \u00ebsht\u00eb e mundur. Zgjidhja \u00ebsht\u00eb si vijon. E para \u2014 krijoni nj\u00eb tavolin\u00eb p\u00ebrkohshme t\u00eb tipit MergeTree me struktur\u00ebn e vjet\u00ebr, bashk\u00ebngjitni aty t\u00eb dh\u00ebnat me an\u00eb t\u00eb attach, b\u00ebni nj\u00eb k\u00ebrkes\u00eb alter. M\u00eb pas mund t\u00eb kopjoni ose transferoni k\u00ebto t\u00eb dh\u00ebna dhe t\u00eb b\u00ebni p\u00ebrs\u00ebri attach, ose t\u00eb p\u00ebrdorni nj\u00eb k\u00ebrkes\u00eb <strong>alter table move partition<\/strong>.<\/p>\n<p><\/p>\n<p>Tani \u00ebsht\u00eb pyetja e dyt\u00eb \u2014 a \u00ebsht\u00eb e mundur t\u00eb p\u00ebrdorim Btrfs. N\u00eb fillim, n\u00ebse keni LVM, mjafton LVM snapshots, dhe sistemi i skedave mund t\u00eb jet\u00eb edhe ext4, kjo nuk ka r\u00ebnd\u00ebsi. Me Btrfs, gjith\u00e7ka varet nga p\u00ebrvoja juaj me p\u00ebrdorimin e tij. \u00cbsht\u00eb nj\u00eb sistem i skedar\u00ebve t\u00eb pjekur, por akoma ka disa dyshime se si do t\u00eb funksionoj\u00eb n\u00eb praktik\u00eb n\u00eb nj\u00eb skenar t\u00eb caktuar. Nuk do ta rekomandonim k\u00ebt\u00eb n\u00ebse nuk keni Btrfs n\u00eb prodhim.<\/p>\n<p><\/p>\n<h2 id=\"anchorresharding-best-practicesanchorkakie-seychas-luchshie-praktiki-vnbspreshardinge-dannyh\"><noindex><a rel=\"nofollow\" name=\"resharding-best-practices\"><\/a><\/noindex>Cilat jan\u00eb praktikat m\u00eb t\u00eb mira aktualisht n\u00eb reshardimin e t\u00eb dh\u00ebnave?<\/h2>\n<p><\/p>\n<p>Pyetja p\u00ebr ri-shardimin \u00ebsht\u00eb e komplikuar dhe shum\u00eb-facet. K\u00ebtu mund t\u00eb p\u00ebrgjigjemi me disa variante. Mund t\u00eb fillojm\u00eb duke th\u00ebn\u00eb k\u00ebshtu \u2014 n\u00eb ClickHouse, nuk ka mund\u00ebsi t\u00eb integruara p\u00ebr ri-shardim. Por kam frik\u00eb se kjo p\u00ebrgjigje nuk do t'i k\u00ebnaq\u00eb ask\u00ebnd. Prandaj, mund t\u00eb fillojm\u00eb nga nj\u00eb k\u00ebndv\u00ebshtrim tjet\u00ebr dhe t\u00eb themi se n\u00eb ClickHouse ka shum\u00eb m\u00ebnyra p\u00ebr t\u00eb ri-sharduar t\u00eb dh\u00ebnat. <\/p>\n<p><\/p>\n<p>N\u00ebse ka mbaruar vendi n\u00eb klaster apo ai nuk po p\u00ebrballon ngarkes\u00ebn, ju shtoni server\u00eb t\u00eb rinj. Por k\u00ebta server\u00eb jan\u00eb me default t\u00eb zbraz\u00ebt, nuk kan\u00eb t\u00eb dh\u00ebna, nuk ka ngarkes\u00eb. Ju duhet t\u00eb transferoni t\u00eb dh\u00ebnat, n\u00eb m\u00ebnyr\u00eb q\u00eb ato t\u00eb shp\u00ebrndahen n\u00eb m\u00ebnyr\u00eb t\u00eb barabart\u00eb n\u00eb klasterin e ri, t\u00eb zgjeruar.<\/p>\n<p><\/p>\n<p>Forma e par\u00eb p\u00ebr ta b\u00ebr\u00eb k\u00ebt\u00eb \u00ebsht\u00eb t\u00eb kopjoni nj\u00eb pjes\u00eb t\u00eb particioneve n\u00eb server\u00eb t\u00eb rinj me an\u00eb t\u00eb nj\u00eb k\u00ebrkese. <strong>alter table fetch partition<\/strong>P\u00ebr shembull, n\u00ebse keni pasur particione sipas muajve, merrni muajin e par\u00eb t\u00eb vitit 2017 dhe kopjoni at\u00eb n\u00eb nj\u00eb server t\u00eb ri, pastaj kopjoni muajin e tret\u00eb n\u00eb nj\u00eb server tjet\u00ebr. Dhe k\u00ebshtu vazhdoni derisa t\u00eb b\u00ebhet m\u00eb shum\u00eb-k\u00ebshtu e barabart\u00eb.<\/p>\n<p><\/p>\n<p>Transferimi mund t\u00eb b\u00ebhet vet\u00ebm p\u00ebr ato particione q\u00eb nuk ndryshojn\u00eb gjat\u00eb shkrimit. P\u00ebr particionet e fresk\u00ebta, duhet t\u00eb \u00e7aktivizoni shkrimin, sepse transferimi i tyre nuk \u00ebsht\u00eb atomar. P\u00ebrndryshe, do t\u00eb merrni kopje t\u00eb dyfishta ose mungesa n\u00eb t\u00eb dh\u00ebna. Megjithat\u00eb, ky metod\u00eb \u00ebsht\u00eb praktike dhe funksionon mjaft efikas. Particionet e kompresuara d\u00ebrgohen p\u00ebrmes rrjetit, dometh\u00ebn\u00eb t\u00eb dh\u00ebnat nuk rishkruhen dhe nuk rikodohen.<\/p>\n<p><\/p>\n<p>Kjo metod\u00eb ka nj\u00eb disavantazh, dhe ai varet nga skema e shardimit, n\u00ebse jeni mb\u00ebshtetur n\u00eb k\u00ebt\u00eb skem\u00eb shardimi, cili ka qen\u00eb \u00e7el\u00ebsi juaj i shardimit. N\u00eb shembullin tuaj p\u00ebr rastin me metrikat, \u00e7el\u00ebsi i shardimit \u00ebsht\u00eb hash nga rruga. Kur b\u00ebn nj\u00eb select n\u00eb tabel\u00ebn e Shp\u00ebrndar\u00eb, ai shkon direkt n\u00eb t\u00eb gjitha shardet e klasterit dhe merr t\u00eb dh\u00ebnat nga aty. <\/p>\n<p><\/p>\n<p>Kjo do t\u00eb thot\u00eb q\u00eb faktikisht p\u00ebr ju nuk ka r\u00ebnd\u00ebsi se cilat t\u00eb dh\u00ebna ndodhen n\u00eb cilin shard. E r\u00ebnd\u00ebsishme \u00ebsht\u00eb q\u00eb t\u00eb dh\u00ebnat t\u00eb jen\u00eb t\u00eb vendosura n\u00eb nj\u00eb itinerar t\u00eb caktuar n\u00eb nj\u00eb shard, nd\u00ebrsa cilin sakt\u00ebsisht, nuk ka r\u00ebnd\u00ebsi. N\u00eb k\u00ebt\u00eb rast, transferimi i particioneve t\u00eb gatshme \u00ebsht\u00eb i p\u00ebrshtatsh\u00ebm, sepse gjat\u00eb k\u00ebrkesave select, ju gjithashtu \u2014 pa marr\u00eb parasysh n\u00ebse \u00ebsht\u00eb p\u00ebrshar\u00ebzimi i m\u00ebparsh\u00ebm apo pas, skema e t\u00eb dh\u00ebnave nuk ka r\u00ebnd\u00ebsi t\u00eb madhe \u2014 do t\u00eb merrni t\u00eb dh\u00ebna t\u00eb plota.<\/p>\n<p><\/p>\n<p>Por ndodhin edhe raste m\u00eb t\u00eb komplikuara. N\u00ebse n\u00eb nivelin e logjik\u00ebs s\u00eb aplikacionit planifikoni nj\u00eb skem\u00eb t\u00eb ve\u00e7ant\u00eb sharding, q\u00eb ky klient ndodhet n\u00eb nj\u00eb shard t\u00eb caktuar, dhe k\u00ebrkesa mund t\u00eb d\u00ebrgohet menj\u00ebher\u00eb aty, p\u00ebrve\u00e7se n\u00eb tabel\u00ebn Distributed. Ose p\u00ebrdorni nj\u00eb version mjaft t\u00eb ri t\u00eb ClickHouse dhe keni aktivizuar konfigurimin <strong>optimize skip unused shards<\/strong>. N\u00eb k\u00ebt\u00eb rast, gjat\u00eb k\u00ebrkes\u00ebs select, shprehja n\u00eb seksionin where do t\u00eb analizohet dhe do t\u00eb llogaritet se n\u00eb cilat shards duhet t\u00eb shkoni sipas skem\u00ebs s\u00eb sharding. Kjo funksionon me kushtin q\u00eb t\u00eb dh\u00ebnat t\u00eb jen\u00eb vendosur n\u00eb p\u00ebrputhje me k\u00ebt\u00eb skem\u00eb sharding. N\u00ebse i keni transferuar manualisht, p\u00ebrputhshm\u00ebria mund t\u00eb ndryshoj\u00eb.<\/p>\n<p><\/p>\n<p>Pra, ky \u00ebsht\u00eb m\u00ebnyra num\u00ebr nj\u00eb. Dhe pres p\u00ebrgjigjen tuaj, n\u00ebse kjo m\u00ebnyr\u00eb \u00ebsht\u00eb e p\u00ebrshtatshme, ose shkojm\u00eb m\u00eb tej.<\/p>\n<p><\/p>\n<p><strong>Vladimir Kolobaev, administrator sistemi lead n\u00eb Avito<\/strong>: Alexey, m\u00ebnyra q\u00eb p\u00ebrm\u00ebnd\u00ebt nuk \u00ebsht\u00eb shum\u00eb e p\u00ebrshtatshme, kur \u00ebsht\u00eb e nevojshme t\u00eb shp\u00ebrndahen ngarkesat edhe n\u00eb lexim. Ne mund t\u00eb marrim nj\u00eb particion, q\u00eb \u00ebsht\u00eb mujor dhe mund ta transferojm\u00eb muajin e kaluar n\u00eb nj\u00eb nod tjet\u00ebr, por kur t\u00eb vij\u00eb k\u00ebrkesa p\u00ebr k\u00ebto t\u00eb dh\u00ebna, ne do ta ngarkojm\u00eb vet\u00ebm at\u00eb. A do t\u00eb donim t\u00eb ngarkonim t\u00eb gjith\u00eb grupin, sepse, n\u00eb t\u00eb kund\u00ebrt, p\u00ebr nj\u00eb koh\u00eb t\u00eb gjat\u00eb, e gjith\u00eb ngarkesa e leximit do t\u00eb p\u00ebrpunsohet nga dy shards.<\/p>\n<p><\/p>\n<p><strong>Alexey Milovidov:<\/strong> P\u00ebrgjigjja k\u00ebtu \u00ebsht\u00eb e \u00e7uditshme \u2014 po, \u00ebsht\u00eb keq, por ndoshta do t\u00eb funksionoj\u00eb. Do ta shpjegoj si. Duhet t\u00eb shikoni skenarin e ngarkes\u00ebs q\u00eb shoq\u00ebron t\u00eb dh\u00ebnat tuaja. N\u00ebse jan\u00eb t\u00eb dh\u00ebna monitorimi, pothuajse me siguri mund t\u00eb themi se shumica e k\u00ebrkesave jan\u00eb p\u00ebr t\u00eb dh\u00ebna t\u00eb fresk\u00ebta. <\/p>\n<p><\/p>\n<p>Keni vendosur server\u00eb t\u00eb rinj, keni transferuar particionet e vjetra, por gjithashtu keni ndryshuar m\u00ebnyr\u00ebn se si regjistrohen t\u00eb dh\u00ebnat e fresk\u00ebta. T\u00eb dh\u00ebnat e fresk\u00ebta do t\u00eb shp\u00ebrndahen n\u00eb t\u00eb gjith\u00eb grupin. Si rezultat, pas pes\u00eb minutash, k\u00ebrkesat p\u00ebr pes\u00eb minutat e fundit do t\u00eb ngarkojn\u00eb n\u00eb m\u00ebnyr\u00eb t\u00eb barabart\u00eb grupin, dhe pas nj\u00eb dite k\u00ebrkesat p\u00ebr 24 or\u00eb do t\u00eb ngarkojn\u00eb n\u00eb m\u00ebnyr\u00eb t\u00eb barabart\u00eb grupin. Nd\u00ebrsa k\u00ebrkesat p\u00ebr muajin e kaluar, fatkeq\u00ebsisht, do t\u00eb shkojn\u00eb vet\u00ebm n\u00eb nj\u00eb pjes\u00eb t\u00eb server\u00ebve t\u00eb grupit.<\/p>\n<p><\/p>\n<p>Por shum\u00eb shpesh nuk do t\u00eb keni k\u00ebrkesa p\u00ebr shkurtin e vitit 2019. M\u00eb s\u00eb shumti, n\u00ebse k\u00ebrkesat jan\u00eb p\u00ebr vitin 2019, ato do t\u00eb jen\u00eb p\u00ebr t\u00ebr\u00eb vitin 2019 \u2014 p\u00ebr nj\u00eb interval t\u00eb madh kohor, e jo p\u00ebr ndonj\u00eb diapazon t\u00eb vog\u00ebl. K\u00ebto d\u00ebshmi mund t\u00eb ngarkojn\u00eb gjithashtu uniform\u00eb klasterin. Por n\u00eb p\u00ebrgjith\u00ebsi, v\u00ebrejtja juaj \u00ebsht\u00eb plot\u00ebsisht e sakt\u00eb, se kjo \u00ebsht\u00eb nj\u00eb zgjidhje ad hoc, e cila nuk e shp\u00ebrndan t\u00eb dh\u00ebnat plot\u00ebsisht n\u00eb m\u00ebnyr\u00eb uniforme.<\/p>\n<p><\/p>\n<p>Kam disa pika tjera p\u00ebr t'u p\u00ebrgjigjur n\u00eb pyetje. Nj\u00ebra prej tyre ka t\u00eb b\u00ebj\u00eb me m\u00ebnyr\u00ebn si inicialisht t\u00eb b\u00ebhet skema e sharding-ut n\u00eb m\u00ebnyr\u00eb q\u00eb t\u00eb ket\u00eb sa m\u00eb pak dhimbje nga rishardimi. Kjo nuk \u00ebsht\u00eb gjithmon\u00eb e mundur.<\/p>\n<p><\/p>\n<p>P\u00ebr shembull, keni t\u00eb dh\u00ebna monitorimi. T\u00eb dh\u00ebnat e monitorimit rriten p\u00ebr tri arsye. E para \u2014 akumulimi i t\u00eb dh\u00ebnave historike. E dyta \u2014 rritja e trafikut. Dhe e treta \u2014 rritja e numrit t\u00eb atyre gj\u00ebrave q\u00eb bien n\u00ebn monitorim. Po shfaqen mikrosh\u00ebrbime dhe metrika t\u00eb reja q\u00eb duhet t\u00eb ruhen. <\/p>\n<p><\/p>\n<p>Mund\u00ebsisht, rritja m\u00eb e madhe lidhet me arsyen e tret\u00eb \u2014 kjo \u00ebsht\u00eb rritja e p\u00ebrdorimit t\u00eb monitorimit. N\u00eb k\u00ebt\u00eb rast, \u00ebsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb shikoni karakterin e ngarkes\u00ebs, se cilat jan\u00eb k\u00ebrkesat kryesore p\u00ebr select. K\u00ebrkesat kryesore p\u00ebr select, me shum\u00eb mund\u00ebsi do t\u00eb vijn\u00eb nga nj\u00eb n\u00ebngrup i caktuar metrikash.<\/p>\n<p><\/p>\n<p>P\u00ebr shembull, p\u00ebrdorimi i CPU-s\u00eb nga disa servera p\u00ebr nj\u00eb sh\u00ebrbim t\u00eb caktuar. Pra, ka nj\u00eb n\u00ebngrup t\u00eb caktuar \u00e7el\u00ebsash q\u00eb ju e merrni k\u00ebto t\u00eb dh\u00ebna. Dhe vet\u00eb k\u00ebrkesa p\u00ebr k\u00ebto t\u00eb dh\u00ebna, me shum\u00eb mund\u00ebsi, \u00ebsht\u00eb mjaft e thjesht\u00eb dhe ekzekutohet p\u00ebr disa dhjet\u00ebra milisekonda. P\u00ebrdoret p\u00ebr sh\u00ebrbimet e monitorimit, p\u00ebr dashboards. Shpresoj q\u00eb e kuptoj si\u00e7 duhet.<\/p>\n<p><\/p>\n<p><strong>Vladimir Kolobaev:<\/strong> \u00c7\u00ebshtja \u00ebsht\u00eb se ne shpesh i referohemi t\u00eb dh\u00ebnave historike, pasi n\u00eb koh\u00eb reale krahasojm\u00eb pozitat aktuale me ato historike. dhe p\u00ebr ne \u00ebsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb kemi qasje t\u00eb shpejt\u00eb n\u00eb nj\u00eb volum t\u00eb madh t\u00eb dh\u00ebnash, dhe ClickHouse e b\u00ebn k\u00ebt\u00eb mjaft mir\u00eb.<\/p>\n<p><\/p>\n<p>Keni absolutisht t\u00eb drejt\u00eb, shumica e k\u00ebrkesave p\u00ebr lexim i hasim n\u00eb dit\u00ebn e fundit, sikurse \u00e7do sistem monitorimi. Por edhe p\u00ebr t\u00eb dh\u00ebnat historike \u00ebsht\u00eb nj\u00eb ngarkes\u00eb mjaft e madhe. Kjo kryesisht vjen nga sistemi i alertimit, i cili \u00e7do tridhjet\u00eb sekonda shkon dhe i thot\u00eb ClickHouse: \"M\u00eb jep t\u00eb dh\u00ebnat p\u00ebr gjasht\u00eb jav\u00ebt e fundit. Dhe tani, ndihmoje t\u00eb krijoj nj\u00eb mesatare l\u00ebviz\u00ebse dhe le t\u00eb krahasojm\u00eb vler\u00ebn aktuale me at\u00eb historike.\" <\/p>\n<p><\/p>\n<p>Do t\u00eb doja t\u00eb thoja se p\u00ebr k\u00ebto k\u00ebrkesa shum\u00eb t\u00eb fresk\u00ebta ekziston nj\u00eb tabel\u00eb e vog\u00ebl tjet\u00ebr, n\u00eb t\u00eb cil\u00ebn ruajm\u00eb vet\u00ebm dy dit\u00eb t\u00eb dh\u00ebnash, dhe k\u00ebrkesat kryesore shkojn\u00eb atje. N\u00eb tabel\u00ebn e madhe t\u00eb sharduar d\u00ebrgojm\u00eb vet\u00ebm k\u00ebrkesat e m\u00ebdha historike.<\/p>\n<p><\/p>\n<p><strong>Alexey Milovidov:<\/strong> Fatke, p\u00ebr fat t\u00eb keq, ky skenar \u00ebsht\u00eb shum\u00eb i pap\u00ebrshtatsh\u00ebm, por do t'ju jap nj\u00eb p\u00ebrshkrim t\u00eb dy skemave t\u00eb k\u00ebqija dhe t\u00eb komplikuara t\u00eb sharding-ut q\u00eb nuk duhen p\u00ebrdorur, por q\u00eb p\u00ebrdoren n\u00eb sh\u00ebrbimin e miqve t\u00eb mi. <\/p>\n<p><\/p>\n<p>Ekziston nj\u00eb klaster kryesor me ngjarjet e \u2018Yandex.Metrika\u2019. Ngjarjet jan\u00eb shikime faqesh, klikime dhe kalime. Shumica e k\u00ebrkesave shkojn\u00eb p\u00ebr nj\u00eb website konkret. Ju hapni sh\u00ebrbimin \u2018Yandex.Metrika\u2019, keni nj\u00eb website \u2013 avito.ru, hyni n\u00eb raport dhe b\u00ebni k\u00ebrkes\u00eb p\u00ebr website-in tuaj.<\/p>\n<p><\/p>\n<p>Por ka edhe k\u00ebrkesat e tjera \u2013 analitike dhe globale, t\u00eb cilat i b\u00ebjn\u00eb analist\u00ebt e brendsh\u00ebm. P\u00ebr \u00e7do rast, tregoj se analist\u00ebt e brendsh\u00ebm b\u00ebjn\u00eb k\u00ebrkesa vet\u00ebm p\u00ebr sh\u00ebrbimet e \u2018Yandexit\u2019. Megjithat\u00eb, edhe sh\u00ebrbimet e \u2018Yandexit\u2019 z\u00ebn\u00eb nj\u00eb pjes\u00eb t\u00eb konsiderueshme t\u00eb t\u00eb dh\u00ebnave. K\u00ebto jan\u00eb k\u00ebrkesa jo p\u00ebr numrat konkret\u00eb, por p\u00ebr nj\u00eb filtrimin m\u00eb t\u00eb gjer\u00eb.<\/p>\n<p><\/p>\n<p>Si mund t\u00eb organizoni t\u00eb dh\u00ebnat n\u00eb nj\u00eb m\u00ebnyr\u00eb q\u00eb gjithashtu t\u00eb funksionojn\u00eb efektivisht p\u00ebr nj\u00eb num\u00ebr konkret dhe k\u00ebrkesat globale? Kompleksiteti \u00ebsht\u00eb gjithashtu n\u00eb at\u00eb se numri i k\u00ebrkesave n\u00eb ClickHouse p\u00ebr klasterin \u2018Metrika\u2019 \u00ebsht\u00eb disa mij\u00ebra n\u00eb sekond\u00eb. N\u00eb k\u00ebt\u00eb rast, k\u00ebrkesat jo triviale, p\u00ebr shembull, disa mij\u00ebra n\u00eb sekond\u00eb nj\u00eb server ClickHouse nuk mund t'i mbaj\u00eb.<\/p>\n<p><\/p>\n<p>Madh\u00ebsia e klasterit \u00ebsht\u00eb mbi sixhent e disa server\u00eb. N\u00ebse mbi k\u00ebt\u00eb klaster vet\u00ebm t\u00eb vendosim nj\u00eb tabel\u00eb t\u00eb shp\u00ebrndar\u00eb dhe t\u00eb d\u00ebrgojm\u00eb disa mij\u00ebra k\u00ebrkesa atje, do b\u00ebhet akoma m\u00eb keq se sa d\u00ebrgimi i tyre n\u00eb nj\u00eb server. Nga ana tjet\u00ebr, varianti me t\u00eb dh\u00ebnat q\u00eb jan\u00eb shp\u00ebrndar\u00eb barabart\u00eb dhe ne k\u00ebrkojm\u00eb nga t\u00eb gjith\u00eb server\u00ebt, e eliminojm\u00eb menj\u00ebher\u00eb.<\/p>\n<p><\/p>\n<p>Ekziston nj\u00eb variant diametralisht i kund\u00ebrt. Imagjinoni, n\u00ebse ne do t\u00eb shardonim t\u00eb dh\u00ebnat sipas website-ve, dhe k\u00ebrkesa p\u00ebr nj\u00eb website do t\u00eb shkonte n\u00eb nj\u00eb shard. Tani klasteri me t\u00eb v\u00ebrtet\u00eb do t\u00eb mund t\u00eb mbaj\u00eb dhjet\u00eb mij\u00eb k\u00ebrkesa n\u00eb sekond\u00eb, por n\u00eb nj\u00eb shard nj\u00eb k\u00ebrkes\u00eb do t\u00eb punoj\u00eb shum\u00eb ngadal\u00eb. N\u00eb k\u00ebt\u00eb rast, nuk do t\u00eb ishte n\u00eb gjendje t\u00eb zgjerohej sipas kapacitetit t\u00eb kalimit. Sidomos n\u00ebse \u00ebsht\u00eb website avito.ru. Nuk do t\u00eb zbulonim nj\u00eb sekret n\u00ebse them se Avito \u00ebsht\u00eb nj\u00eb nga website-t m\u00eb t\u00eb vizituara n\u00eb runet. T\u00eb p\u00ebrpunosh k\u00ebt\u00eb n\u00eb nj\u00eb shard do t\u00eb ishte marr\u00ebzi.<\/p>\n<p><\/p>\n<p>Prandaj, skema e sharding-ut \u00ebsht\u00eb dizajnuar n\u00eb nj\u00eb m\u00ebnyr\u00eb m\u00eb t\u00eb zgjuar. I gjith\u00eb klasteri \u00ebsht\u00eb i ndar\u00eb n\u00eb disa klasterk\u00eb, t\u00eb cilat ne i quajm\u00eb shtresa. Brenda \u00e7do klasterke ka nga dhjet\u00eb deri n\u00eb disa dhjet\u00ebra sharde. N\u00eb total jan\u00eb tridhjet\u00eb e n\u00ebnt\u00eb t\u00eb till\u00eb. <\/p>\n<p><\/p>\n<p>Si t\u00eb shkall\u00ebzohet gjith\u00e7ka? Numri i klaster\u00ebve nuk ndryshon \u2013 si\u00e7 ishin tridhjet\u00eb e n\u00ebnt\u00eb disa vite m\u00eb par\u00eb, ashtu ka mbetur. Por brenda secilit prej tyre ne gradualisht rritim numrin e shard\u00ebve p\u00ebr sa koh\u00eb q\u00eb grumbullojm\u00eb t\u00eb dh\u00ebna. Dhe skema e shardimit n\u00eb p\u00ebrgjith\u00ebsi \u00ebsht\u00eb e till\u00eb \u2013 ndarja n\u00eb k\u00ebto klaster\u00eb b\u00ebhet sipas faqeve t\u00eb internetit, dhe p\u00ebr t\u00eb kuptuar se cila faqe \u00ebsht\u00eb n\u00eb cilin klaster, p\u00ebrdoret nj\u00eb baz\u00eb e ve\u00e7ant\u00eb n\u00eb MySQL. Nj\u00eb faqe \u2013 n\u00eb nj\u00eb klaster. Dhe brenda saj, shardimi ndodh sipas identifikuesve t\u00eb vizitor\u00ebve.<\/p>\n<p><\/p>\n<p>Gjat\u00eb shkrimit ne i ndajm\u00eb sipas mbetjes nga ndarja e identifikuesit t\u00eb vizitorit. Por kur shtojm\u00eb nj\u00eb shard t\u00eb ri, skema e shardimit ndryshon, ne vazhdojm\u00eb t\u00eb ndajm\u00eb, por sipas mbetjes nga ndarja me nj\u00eb num\u00ebr tjet\u00ebr. Kjo do t\u00eb thot\u00eb se nj\u00eb vizitor tashm\u00eb ndodhet n\u00eb disa servera, dhe nuk mund t\u00eb mb\u00ebshtetesh n\u00eb k\u00ebt\u00eb. Kjo \u00ebsht\u00eb b\u00ebr\u00eb ekskluzivisht p\u00ebr t\u00eb siguruar q\u00eb t\u00eb dh\u00ebnat t\u00eb kompresohen m\u00eb mir\u00eb. Nd\u00ebrsa gjat\u00eb k\u00ebrkesave ne shkojm\u00eb n\u00eb tabel\u00ebn Distribuar, q\u00eb shikon n\u00eb klaster dhe i drejtohet dhjet\u00ebra server\u00ebve. K\u00ebshtu \u00ebsht\u00eb kjo skem\u00eb e \u00e7uditshme.<\/p>\n<p><\/p>\n<p>Por tregimi im do t\u00eb ishte i pap\u00ebrsosur n\u00ebse nuk do t\u00eb thosha se kjo skem\u00eb ne e braktis\u00ebm. N\u00eb skem\u00ebn e re ne e kemi ndryshuar gjith\u00e7ka dhe t\u00eb gjitha t\u00eb dh\u00ebnat i kemi kopjuar me ndihm\u00ebn e clickhouse-copier.<\/p>\n<p><\/p>\n<p>N\u00eb skem\u00ebn e re, t\u00eb gjitha faqet ndahen n\u00eb dy kategori - t\u00eb m\u00ebdha dhe t\u00eb vogla. Nuk e di se si \u00ebsht\u00eb zgjedhur kufiri, por si rezultat, faqet e m\u00ebdha regjistrohen n\u00eb nj\u00eb klaster, ku ka 120 shard\u00eb me tre replika n\u00eb secilin - dometh\u00ebn\u00eb 360 server\u00eb. Dhe skema e shardimit \u00ebsht\u00eb e till\u00eb q\u00eb \u00e7do k\u00ebrkes\u00eb shkon menj\u00ebher\u00eb n\u00eb t\u00eb gjith\u00eb shard\u00ebt. N\u00ebse tani hapni ndonj\u00eb faqe raporti n\u00eb \u2018Yandex.Metrika\u2019 p\u00ebr avito.ru, k\u00ebrkesa do t\u00eb shkoj\u00eb n\u00eb 120 server\u00eb. Ka shum\u00eb pak faqe t\u00eb m\u00ebdha n\u00eb runet. Dhe k\u00ebrkesat del se nuk jan\u00eb nj\u00eb mij\u00eb n\u00eb sekond\u00eb, por m\u00eb pak se nj\u00ebqind. T\u00eb gjitha k\u00ebto qet\u00ebsisht p\u00ebrpunohen nga tabela Distribuar, q\u00eb \u00e7do nj\u00eb prej tyre e trajton me 120 servera.<\/p>\n<p><\/p>\n<p>Dhe klasteri i dyt\u00eb - p\u00ebr faqet e vogla. K\u00ebtu skema e shardimit \u00ebsht\u00eb sipas identifikuesit t\u00eb faqes, dhe \u00e7do k\u00ebrkes\u00eb shkon sakt\u00ebsisht n\u00eb nj\u00eb shard.<\/p>\n<p><\/p>\n<h2 id=\"anchorclickhouse-copieranchorv-clickhouse-est-utilita-clickhouse-copier-mozhete-pronbspneyo-rasskazat\"><noindex><a rel=\"nofollow\" name=\"clickhouse-copier\"><\/a><\/noindex>N\u00eb ClickHouse ekziston utilita clickhouse-copier. A mund t\u00eb flisni p\u00ebr t\u00eb?<\/h2>\n<p><\/p>\n<p>T\u00eb them menj\u00ebher\u00eb se kjo zgjidhje \u00ebsht\u00eb m\u00eb e r\u00ebnd\u00eb dhe disi m\u00eb pak produktive. Avantazhi \u00ebsht\u00eb se ajo shp\u00ebrndan t\u00eb dh\u00ebnat plot\u00ebsisht sipas skem\u00ebs q\u00eb do t\u00eb tregoni. Por disavantazhi i utilitetit \u00ebsht\u00eb se ajo nuk b\u00ebn fare riparimin e shard\u00ebve. Ajo kopjon t\u00eb dh\u00ebnat nga nj\u00eb skem\u00eb klasteri n\u00eb nj\u00eb skem\u00eb tjet\u00ebr klasteri.<\/p>\n<p><\/p>\n<p>Kjo do t\u00eb thot\u00eb se p\u00ebr pun\u00ebn e saj duhet t\u00eb keni dy klaster\u00eb. Ata mund t\u00eb jen\u00eb t\u00eb vendosur n\u00eb servera t\u00eb nj\u00ebjt\u00eb, por megjithat\u00eb, t\u00eb dh\u00ebnat nuk do t\u00eb l\u00ebvizin incremental, por do t\u00eb kopjohen. <\/p>\n<p><\/p>\n<p>P\u00ebr shembull, kishte kat\u00ebr server\u00eb, tani ka tet\u00eb. Ju krijoni nj\u00eb tabel\u00eb t\u00eb re Distributed n\u00eb t\u00eb gjitha server\u00ebt, tabele lokale t\u00eb reja dhe filloni clickhouse-copier, duke caktuar n\u00eb t\u00eb skem\u00ebn e pun\u00ebs, se ai duhet t\u00eb lexoj\u00eb nga atje, t\u00eb pranoj\u00eb skem\u00ebn e re t\u00eb shardimit dhe t\u00eb transferoj\u00eb t\u00eb dh\u00ebnat atje. Dhe p\u00ebr ju n\u00eb server\u00ebt e vjet\u00ebr do t\u00eb k\u00ebrkohet hap\u00ebsir\u00eb 1.5 her\u00eb m\u00eb shum\u00eb se sa ka tani, sepse t\u00eb dh\u00ebnat e vjetra duhet t\u00eb q\u00ebndrojn\u00eb aty, dhe sip\u00ebr tyre do t\u00eb vij\u00eb gjysma e atyre t\u00eb dh\u00ebnave t\u00eb vjetra. N\u00ebse keni menduar m\u00eb par\u00eb p\u00ebr at\u00eb q\u00eb t\u00eb dh\u00ebnat duhet t\u00eb rishardonohen dhe ka hap\u00ebsir\u00eb, at\u00ebher\u00eb ky m\u00ebnyr\u00eb do t\u00eb funksionoj\u00eb.<\/p>\n<p><\/p>\n<p>Si funksionon brenda clickhouse-copier? Ai ndan t\u00eb gjith\u00eb pun\u00ebn n\u00eb nj\u00eb grup detyrash p\u00ebr p\u00ebrpunimin e nj\u00eb particionit t\u00eb nj\u00eb tabele n\u00eb nj\u00eb shard. T\u00eb gjitha k\u00ebto detyra mund t\u00eb ekzekutohen paralelisht, dhe clickhouse-copier mund t\u00eb nis\u00eb n\u00eb makina t\u00eb ndryshme n\u00eb disa instance, por ajo q\u00eb b\u00ebn p\u00ebr nj\u00eb partit\u00eb \u00ebsht\u00eb asgj\u00eb tjet\u00ebr ve\u00e7se nj\u00eb insert select. T\u00eb dh\u00ebnat lexohen, shkrijn\u00eb, riorganizohen, pastaj p\u00ebrs\u00ebri kompresohen, regjistrohen diku, riorganizohen. Ky \u00ebsht\u00eb nj\u00eb zgjidhje m\u00eb e r\u00ebnd\u00eb.<\/p>\n<p><\/p>\n<h2 id=\"anchorresharding-toolanchoru-vas-byla-pilotnaya-shtuka-kotoraya-nazyvalas-resharding-chto-snbspney\"><noindex><a rel=\"nofollow\" name=\"resharding-tool\"><\/a><\/noindex>Keni pasur nj\u00eb nxjerrje pilot q\u00eb quhej reshardim. \u00c7fare ndodhi me t\u00eb?<\/h2>\n<p><\/p>\n<blockquote><p>Ishte nj\u00eb projekt pilot q\u00eb quhej rishardim n\u00eb vitin 2017. Ka madje nj\u00eb opsion n\u00eb ClickHouse. E kuptoj, nuk ka funksionuar. A mund t\u00eb na tregoni se pse ndodhi k\u00ebshtu? Dukej se ishte shum\u00eb aktuale.<\/p><\/blockquote>\n<p>I gjith\u00eb problemi \u00ebsht\u00eb se, kur \u00ebsht\u00eb e nevojshme t\u00eb rishardohet t\u00eb dh\u00ebnat, k\u00ebrkohet nj\u00eb sinkronizim mjaft kompleks p\u00ebr ta realizuar at\u00eb n\u00eb m\u00ebnyr\u00eb atomike. Kur filluam t\u00eb shohim se si \u00ebsht\u00eb organizuar kjo sinkronizim, b\u00ebhet e qart\u00eb se jan\u00eb probleme themelore. Dhe k\u00ebto probleme themelore nuk jan\u00eb vet\u00ebm teorike, por gjithashtu filluan t\u00eb shfaqen n\u00eb praktik\u00eb si di\u00e7ka q\u00eb mund t\u00eb shpjegohet shum\u00eb thjesht\u00eb \u2014 asgj\u00eb nuk funksionon.<\/p>\n<p><\/p>\n<h2 id=\"anchormove-to-slow-diskanchormozhno-li-slivat-vse-chasti-dannyh-voedino-perednbspperemescheniem-nanbspmedlennye-diski\"><noindex><a rel=\"nofollow\" name=\"move-to-slow-disk\"><\/a><\/noindex>A \u00ebsht\u00eb e mundur t\u00eb bashkoni t\u00eb gjitha pjes\u00ebt e t\u00eb dh\u00ebnave para se t\u00eb zhvendosni n\u00eb disqe t\u00eb ngadalta?<\/h2>\n<p><\/p>\n<blockquote><p>Pyetje n\u00eb lidhje me TTL me opsionin p\u00ebr t\u00eb l\u00ebvizur n\u00eb disk t\u00eb ngadalsh\u00ebm n\u00eb kontekstin e bashkimeve. A ka ndonj\u00eb m\u00ebnyr\u00eb, p\u00ebrve\u00e7 cron, p\u00ebr t\u00eb bashkuar t\u00eb gjitha pjes\u00ebt n\u00eb nj\u00eb p\u00ebrpara se t\u00eb l\u00ebvizen n\u00eb disqet e ngadalshme?<\/p><\/blockquote>\n<p>P\u00ebrgjigja n\u00eb pyetjen n\u00ebse \u00ebsht\u00eb e mundur t\u00eb bashkohen automatikisht t\u00eb gjitha copat n\u00eb nj\u00eb para se t'i transferojm\u00eb \u2014 jo. M\u00eb duket se nuk ka nevoj\u00eb p\u00ebr k\u00ebt\u00eb. Nuk \u00ebsht\u00eb e nevojshme t\u00eb bashkohen t\u00eb gjitha pjes\u00ebt n\u00eb nj\u00eb, mjafton t\u00eb llogaritim se ato do t\u00eb transferohen automatikisht n\u00eb disqet e ngadalshme. <\/p>\n<p><\/p>\n<p>Ne kemi dy kritere p\u00ebr rregullat e transferimit. E para \u2014 n\u00eb p\u00ebrputhje me nivelin e mbushjes. N\u00ebse n\u00eb nivelin aktual t\u00eb ruajtjes ka m\u00eb pak se nj\u00eb p\u00ebrqindje t\u00eb caktuar hap\u00ebsire t\u00eb lir\u00eb, ne zgjedhim nj\u00eb cop\u00eb dhe e transferojm\u00eb at\u00eb n\u00eb ruajtje m\u00eb t\u00eb ngadalshme. M\u00eb sakt\u00ebsisht, jo m\u00eb t\u00eb ngadalt\u00eb, por at\u00eb t\u00eb ardhshme \u2014 si do ta konfiguroni.<\/p>\n<p><\/p>\n<p>Kriteri i dyt\u00eb \u00ebsht\u00eb sipas madh\u00ebsis\u00eb. Ai ka t\u00eb b\u00ebj\u00eb me transferimin e copave t\u00eb m\u00ebdha. Ju mund t\u00eb rregulloni pragun p\u00ebr hap\u00ebsir\u00ebn e lir\u00eb n\u00eb diskun e shpejt\u00eb, dhe t\u00eb dh\u00ebnat do t\u00eb transferohen automatikisht.<\/p>\n<p><\/p>\n<h2 id=\"anchorup-to-dateanchorkak-pereezzhat-nanbspnovye-versii-clickhouse-esli-net-vozmozhnosti-zaranee-proverit-sovmestimost\"><noindex><a rel=\"nofollow\" name=\"up-to-date\"><\/a><\/noindex>Si t\u00eb kalojm\u00eb n\u00eb versione t\u00eb reja t\u00eb ClickHouse, n\u00ebse nuk ka mund\u00ebsi t\u00eb kontrollohet paraprakisht p\u00ebr kompatibilitetin?<\/h2>\n<p><\/p>\n<blockquote><p>Ky tem\u00eb diskutohen rregullisht <noindex><a rel=\"nofollow\" href=\"https:\/\/teleg.run\/clickhouse_ru\">n\u00eb bised\u00ebn Telegram ClickHouse<\/a><\/noindex> duke marr\u00eb parasysh versione t\u00eb ndryshme, megjithat\u00eb. Sa i sigurt \u00ebsht\u00eb t\u00eb p\u00ebrdit\u00ebsoheni nga versi 19.11 n\u00eb 19.16 dhe, p\u00ebr shembull, nga 19.16 n\u00eb 20.3. Si ta kaloni n\u00eb versione t\u00eb reja, pa pasur mund\u00ebsin\u00eb p\u00ebr t\u00eb verifikuar m\u00eb par\u00eb p\u00ebrputhshm\u00ebrin\u00eb n\u00eb nj\u00eb ambient testimi?<\/p><\/blockquote>\n<p>K\u00ebtu jan\u00eb disa rregulla \"t\u00eb art\u00eb\". E para \u00ebsht\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/ClickHouse\/ClickHouse\/blob\/master\/CHANGELOG.md\">lexoni changelog<\/a><\/noindex>. Ai \u00ebsht\u00eb i madh, por ka pika t\u00eb ve\u00e7anta p\u00ebr ndryshime q\u00eb nuk jan\u00eb t\u00eb pajtueshme me versionet e m\u00ebparshme. Nuk duhet t'i kushtoni r\u00ebnd\u00ebsi k\u00ebtyre pikave si nj\u00eb flamur i kuq. Zakonisht k\u00ebto jan\u00eb papajtueshm\u00ebri t\u00eb vogla, q\u00eb lidhen me ndonj\u00eb funksionalitet periferik, q\u00eb ndoshta nuk e p\u00ebrdorni.<\/p>\n<p><\/p>\n<p>E dyta \u00ebsht\u00eb n\u00ebse nuk keni mund\u00ebsi t\u00eb verifikoni p\u00ebrputhshm\u00ebrin\u00eb n\u00eb nj\u00eb ambient testimi dhe d\u00ebshironi t\u00eb p\u00ebrdit\u00ebsoheni menj\u00ebher\u00eb n\u00eb prodhim, rekomandimi \u00ebsht\u00eb ky \u2014 mos e b\u00ebni k\u00ebt\u00eb. S\u00eb pari krijoni nj\u00eb ambient testimi dhe verifikoni. N\u00ebse nuk keni nj\u00eb mjedis testimi, at\u00ebher\u00eb ndoshta nuk jeni nj\u00eb kompani shum\u00eb e madhe, prandaj mund t\u00eb kopjoni nj\u00eb pjes\u00eb t\u00eb t\u00eb dh\u00ebnave n\u00eb laptopin tuaj dhe aty t\u00eb siguroheni q\u00eb gjith\u00e7ka funksionon si\u00e7 duhet. Mund t\u00eb ngriheni edhe disa replika lokalisht n\u00eb makin\u00ebn tuaj. Ose mund t\u00eb ngrini nj\u00eb version t\u00eb ri diku pran\u00eb, dhe t\u00eb ngarkoni aty nj\u00eb pjes\u00eb t\u00eb t\u00eb dh\u00ebnave \u2014 pra t\u00eb krijoni nj\u00eb ambient testimi improvizues. <\/p>\n<p><\/p>\n<p>Nj\u00eb rregull tjet\u00ebr \u00ebsht\u00eb q\u00eb t\u00eb mos p\u00ebrdit\u00ebsoheni brenda nj\u00eb jav\u00eb pas daljes s\u00eb versionit p\u00ebr shkak t\u00eb gjetjes s\u00eb defekteve n\u00eb prodhim dhe riparimeve t\u00eb shpejta q\u00eb pasojn\u00eb. Le t\u00eb analizojm\u00eb num\u00ebrimin e versioneve ClickHouse, p\u00ebr t\u00eb mos u ngat\u00ebrruar. <\/p>\n<p><\/p>\n<p>Ka versionin 20.3.4. Numri 20 tregon vitin e l\u00ebshimit \u2014 2020. Nga pik\u00ebpamja e asaj q\u00eb ndodhet brenda, kjo nuk ka ndonj\u00eb r\u00ebnd\u00ebsi, prandaj nuk do t'i kushtojm\u00eb r\u00ebnd\u00ebsi. M\u00eb tej \u2014 20.3. Numrin e dyt\u00eb \u2014 n\u00eb k\u00ebt\u00eb rast 3 \u2014 e rritim sa her\u00eb l\u00ebshojm\u00eb nj\u00eb version me ndonj\u00eb funksionalitet t\u00eb ri. N\u00ebse d\u00ebshirojm\u00eb t\u00eb shtojm\u00eb n\u00eb ClickHouse ndonj\u00eb mund\u00ebsi, ne jemi t\u00eb detyruar ta rritim k\u00ebt\u00eb num\u00ebr. K\u00ebshtu q\u00eb, n\u00eb versionin 20.4, ClickHouse do t\u00eb funksionoj\u00eb akoma m\u00eb mir\u00eb. Numri i tret\u00eb \u2014 20.3.4. K\u00ebtu 4 \u00ebsht\u00eb numri i l\u00ebshimeve patch, n\u00eb t\u00eb cilat ne nuk kemi shtuar mund\u00ebsi t\u00eb reja, por kemi korrigjuar ndonj\u00eb defekt. Dhe 4 do t\u00eb thot\u00eb se ne e kemi b\u00ebr\u00eb k\u00ebt\u00eb kat\u00ebr her\u00eb.<\/p>\n<p><\/p>\n<p>Nuk e duhet t\u00eb mendoni se kjo \u00ebsht\u00eb ndonj\u00eb gj\u00eb e tmerrshme. Zakonisht p\u00ebrdoruesi mund t\u00eb instalohet versionin m\u00eb t\u00eb fundit, dhe ai do t\u00eb funksionoj\u00eb pa ndonj\u00eb problem me disponueshm\u00ebrin\u00eb p\u00ebr nj\u00eb vit. Por imagjinoni se n\u00eb ndonj\u00eb funksion p\u00ebr p\u00ebrpunimin e bitmap-eve, i cili u shtua nga shok\u00ebt tan\u00eb kinez\u00eb, serveri d\u00ebshton gjat\u00eb kalimit t\u00eb argumenteve t\u00eb gabuara. Ne jemi p\u00ebrgjegj\u00ebs p\u00ebr ta zgjidhur k\u00ebt\u00eb. Do t\u00eb l\u00ebshojm\u00eb nj\u00eb version t\u00eb ri patch, dhe ClickHouse do t\u00eb b\u00ebhet m\u00eb stabil.<\/p>\n<p><\/p>\n<p>N\u00ebse ClickHouse po punon n\u00eb prodhim p\u00ebr ju, dhe nj\u00eb version i ri ClickHouse del me funksione shtes\u00eb - p\u00ebr shembull, 20.4.1 - n\u00eb dit\u00ebn e par\u00eb mos u nxitoni ta vendosni n\u00eb prodhim. Pse \u00ebsht\u00eb e nevojshme? N\u00ebse ende nuk e p\u00ebrdorni ClickHouse, mund ta instaloni, dhe ndoshta gjith\u00e7ka do t\u00eb shkoj\u00eb mir\u00eb. Por n\u00ebse ClickHouse tashm\u00eb funksionon me q\u00ebndrueshm\u00ebri, at\u00ebher\u00eb ndiqni patch-et dhe p\u00ebrdit\u00ebsimet - cilat probleme po zgjidhim.<\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> Dua t\u00eb shtoj pak p\u00ebr ambientet e testimit. T\u00eb gjith\u00eb kan\u00eb frik\u00eb nga ambientet e testimit dhe ndonj\u00ebher\u00eb mendojn\u00eb se, n\u00ebse keni nj\u00eb klaster shum\u00eb t\u00eb madh ClickHouse, at\u00ebher\u00eb edhe ambienti i testimit duhet t\u00eb jet\u00eb po aq i madh ose t\u00eb pakt\u00ebn dhjet\u00eb her\u00eb m\u00eb i vog\u00ebl. Kjo nuk \u00ebsht\u00eb e v\u00ebrtet\u00eb.<\/p>\n<p><\/p>\n<p>Mund t\u00eb flas nga eksperienca ime. Kam nj\u00eb projekt, dhe aty ka ClickHouse. Ambienti yn\u00eb i testimit p\u00ebr t\u00eb \u00ebsht\u00eb nj\u00eb virtuale e vog\u00ebl n\u00eb Hetzner p\u00ebr nj\u00ebzet euro, ku \u00ebsht\u00eb e gjitha e vendosur. P\u00ebr ta b\u00ebr\u00eb k\u00ebt\u00eb, ne kemi automatizim t\u00eb plot\u00eb n\u00eb Ansible, dhe prandaj n\u00eb parim nuk ka ndonj\u00eb ndryshim se ku e zhvilloni - n\u00eb serverat fizik\u00eb apo thjesht n\u00eb virtuale.<\/p>\n<p><\/p>\n<p>\u00c7far\u00eb mund t\u00eb b\u00ebhet? Do t\u00eb ishte mir\u00eb t\u00eb kishim nj\u00eb shembull n\u00eb dokumentacionin e ClickHouse, si t\u00eb vendosni nj\u00eb klaster t\u00eb vog\u00ebl - n\u00eb Docker, n\u00eb LXC, ndoshta t\u00eb krijoni nj\u00eb playbook Ansible, sepse njer\u00ebzit kan\u00eb instalime t\u00eb ndryshme. Kjo do ta thjeshtonte shum\u00eb. Kur merrni dhe vendosni nj\u00eb klaster p\u00ebr pes\u00eb minuta, \u00ebsht\u00eb shum\u00eb m\u00eb e leht\u00eb t\u00eb p\u00ebrpiqeni t\u00eb kuptoni di\u00e7ka. Kjo \u00ebsht\u00eb shum\u00eb m\u00eb komode, sepse t\u00eb vendos\u00ebsh nj\u00eb version prodhimi q\u00eb nuk e ke testuar - \u00ebsht\u00eb nj\u00eb rrug\u00eb e cila nuk t\u00eb \u00e7on askund. Ndonj\u00ebher\u00eb funksionon, ndonj\u00ebher\u00eb jo. Prandaj shpresa p\u00ebr sukses \u00ebsht\u00eb e keqe.<\/p>\n<p><\/p>\n<p><strong>Maksim Kotyakov, inxhinier i lart\u00eb pas backend n\u00eb Avito:<\/strong> Do t\u00eb shtoja di\u00e7ka rreth mjediseve testuese q\u00eb lidhen me problemet e madhe t\u00eb kompanive. Ne kemi nj\u00eb klaster t\u00eb plot\u00eb t\u00eb ClickHouse p\u00ebr pranim, i cili \u00ebsht\u00eb nj\u00eb kopje e sakt\u00eb e skemave t\u00eb t\u00eb dh\u00ebnave dhe konfigurimeve t\u00eb asaj q\u00eb kemi n\u00eb prodhim. Ky klaster \u00ebsht\u00eb zhvilluar n\u00eb kontejner\u00eb relativisht t\u00eb nd\u00ebrlikuar me burime minimale. Ne shkruajm\u00eb atje nj\u00eb p\u00ebrqindje nga t\u00eb dh\u00ebnat e prodhimit, duke pasur fatin t\u00eb kemi mund\u00ebsin\u00eb t\u00eb replikohemi n\u00eb Kafka. Aty gjith\u00e7ka \u00ebsht\u00eb e sinkronizuar dhe e skaluar \u2014 si p\u00ebr kapacitetet, ashtu edhe p\u00ebr fluksin, dhe, n\u00eb teori, duke pasur parasysh t\u00eb gjitha gj\u00ebrat e tjera, duhet t\u00eb sillet si nj\u00eb prodhim n\u00eb metrika. \u00c7do gj\u00eb potencialisht e rrezikshme fillimisht kalon n\u00eb k\u00ebt\u00eb skend dhe q\u00ebndron atje p\u00ebr disa dit\u00eb deri n\u00eb gatishm\u00ebri. Por natyrisht, ky zgjidhje \u00ebsht\u00eb e shtrenjt\u00eb, e v\u00ebshtir\u00eb dhe me shpenzime varg zero p\u00ebr mb\u00ebshtetje. <\/p>\n<p><\/p>\n<p><strong>Alexey Milovidov:<\/strong> Do t\u00eb flas p\u00ebr at\u00eb se \u00e7far\u00eb p\u00ebrfaq\u00ebson mjedisi testues i miqve tan\u00eb nga \"Yandex.Metrica\". Nj\u00eb klaster kishte mbi 600 server\u00eb, tjetri 360, dhe ka edhe nj\u00eb t\u00eb tret\u00eb dhe disa klaster\u00eb t\u00eb tjer\u00eb. Mjedisi testues p\u00ebr nj\u00eb prej tyre \u2014 \u00ebsht\u00eb thjesht dy sharda me dy replika n\u00eb \u00e7do nj\u00ebsi. Pse dy sharda? Q\u00eb t\u00eb ket\u00eb m\u00eb shum\u00eb se nj\u00eb. Dhe gjithashtu replika q\u00eb t\u00eb ekzistojn\u00eb. Thjesht nj\u00eb sasi minimale, q\u00eb mund t\u00eb lejojm\u00eb.<\/p>\n<p><\/p>\n<p>Ky mjedis testues lejon t\u00eb kontrollohet funksionaliteti i k\u00ebrkesave dhe n\u00ebse ndonj\u00eb gj\u00eb e madhe \u00ebsht\u00eb prishur. Por shpesh problemet shfaqen nga nj\u00eb natyr\u00eb krejt\u00ebsisht tjet\u00ebr, kur gjith\u00e7ka funksionon, por ka disa ndryshime t\u00eb vogla me ngarkes\u00ebn.<\/p>\n<p><\/p>\n<p>Mund t\u00eb jap nj\u00eb shembull. Vendos\u00ebm t\u00eb instalojm\u00eb nj\u00eb version t\u00eb ri t\u00eb ClickHouse. Ai \u00ebsht\u00eb vendosur n\u00eb mjedisin testues, ku jan\u00eb p\u00ebrfunduar provat automatike n\u00eb vet\u00eb \"Yandex.Metrica\", t\u00eb cilat krahasojn\u00eb t\u00eb dh\u00ebnat n\u00eb versionin e vjet\u00ebr dhe at\u00eb t\u00eb riun, duke kaluar t\u00eb gjith\u00eb procesin. Dhe natyrisht, provat e gjelbra nga CI yn\u00eb. Ndryshe nuk do t\u00eb ofronim k\u00ebt\u00eb version.<\/p>\n<p><\/p>\n<p>Gjith\u00e7ka shkonte mire. Fillojm\u00eb t\u00eb kalojm\u00eb n\u00eb prodhim. Marr nj\u00eb mesazh q\u00eb n\u00eb grafik\u00ebt ngarkesa \u00ebsht\u00eb rritur disa her\u00eb. Ne e kthejm\u00eb versionin mbrapsht. Shikoj grafikun dhe shoh: ngarkesa v\u00ebrtet ka rritur disa her\u00eb gjat\u00eb lan\u00e7imit dhe \u00ebsht\u00eb ulur p\u00ebrs\u00ebri kur u lansua. M\u00eb pas kemi filluar t\u00eb kthejm\u00eb versionin mbrapsht. Dhe ngarkesa po ashtu \u00ebsht\u00eb rritur dhe po ashtu ka r\u00ebn\u00eb p\u00ebrs\u00ebri. Pra, p\u00ebrfundimi \u00ebsht\u00eb q\u00eb ngarkesa \u00ebsht\u00eb rritur n\u00eb lidhje me lan\u00e7imin, nuk ka asgj\u00eb t\u00eb \u00e7uditshme.<\/p>\n<p><\/p>\n<p>Pastaj ishte e v\u00ebshtir\u00eb t\u00eb bindja koleg\u00ebt p\u00ebr t\u00eb instaluar versionin e ri. I thash\u00eb: \"Gjith\u00e7ka \u00ebsht\u00eb n\u00eb rregull, lan\u00e7oni. Mbani duar t\u00eb lidhura, gjith\u00e7ka do t\u00eb funksionoj\u00eb. Tani ngarkesa \u00ebsht\u00eb rritur n\u00eb grafik\u00eb, por gjith\u00e7ka \u00ebsht\u00eb n\u00eb rregull. Q\u00ebndroni t\u00eb fort\u00eb\". N\u00eb thelb, k\u00ebshtu vepruam, dhe t\u00eb gjith\u00eb \u2014 versioni \u00ebsht\u00eb lanzuar n\u00eb prodhim. Por pothuajse me \u00e7do lan\u00e7im shfaqen probleme t\u00eb ngjashme.<\/p>\n<p><\/p>\n<h2 id=\"anchorkill-queryanchorkill-query-dolzhen-ubivat-zaprosy-no-on-etogo-ne-delaet-pochemu\"><noindex><a rel=\"nofollow\" name=\"kill-query\"><\/a><\/noindex>Kill query duhet t\u00eb ndal\u00eb k\u00ebrkesat, por kjo nuk ndodh. Pse?<\/h2>\n<p><\/p>\n<blockquote><p>Nj\u00eb p\u00ebrdorues m\u00eb erdhi, nj\u00eb analist i caktuar, dhe krijoi nj\u00eb k\u00ebrkes\u00eb q\u00eb ngarkoi klasterin tim ClickHouse. Nj\u00eb nod\u00eb ose t\u00ebr\u00eb klasteri \u2014 n\u00eb var\u00ebsi t\u00eb asaj se n\u00eb cil\u00ebn dublikat\u00eb ose shard ra k\u00ebrkesa. Un\u00eb shoh se t\u00eb gjitha burimet p\u00ebr CPU n\u00eb k\u00ebt\u00eb server jan\u00eb n\u00eb ngarkes\u00eb, gjith\u00e7ka \u00ebsht\u00eb e kuqe. Megjithat\u00eb, ClickHouse po p\u00ebrgjigjet p\u00ebr k\u00ebrkesat. Dhe un\u00eb shkruaj: \"M\u00eb trego, t\u00eb lutem, list\u00ebn e proceseve, cila k\u00ebrkes\u00eb e shkaktoi k\u00ebt\u00eb \u00e7menduri\".<\/p>\n<p>Un\u00eb e gjej k\u00ebt\u00eb k\u00ebrkes\u00eb dhe i shkruaj kill. Dhe shoh se nuk ndodh asgj\u00eb. Serveri im \u00ebsht\u00eb n\u00eb ngarkes\u00eb, ClickHouse vazhdon t\u00eb m\u00eb japi disa komanda, tregon q\u00eb serveri \u00ebsht\u00eb n\u00eb jet\u00eb, dhe gjith\u00e7ka \u00ebsht\u00eb n\u00eb rregull. Por kam degradim n\u00eb t\u00eb gjitha k\u00ebrkesat e p\u00ebrdoruesve, fillon degradimi p\u00ebr shkrimin n\u00eb ClickHouse, dhe kill query im nuk funksionon. Pse? Mendova se kill query duhet t\u00eb vras\u00eb k\u00ebrkesat, dhe kjo nuk po ndodh.<\/p><\/blockquote>\n<p>Tani do t\u00eb jet\u00eb nj\u00eb p\u00ebrgjigje mjaft e \u00e7uditshme. \u00c7\u00ebshtja \u00ebsht\u00eb se kill query nuk vret k\u00ebrkesat. <\/p>\n<p><\/p>\n<p>Kill query vendos nj\u00eb flamur t\u00eb vog\u00ebl me emrin \"dua q\u00eb kjo k\u00ebrkes\u00eb t\u00eb vritet\". Nd\u00ebrsa vet\u00eb k\u00ebrkesa, kur p\u00ebrpunon \u00e7do blok, shikon k\u00ebt\u00eb flamur. N\u00ebse \u00ebsht\u00eb vendosur, k\u00ebrkesa ndalon s\u00eb punuari. K\u00ebshtu q\u00eb askush nuk e vret k\u00ebrkes\u00ebn, ajo duhet ta kontrolloj\u00eb vet\u00eb dhe t\u00eb ndalet. Dhe kjo duhet t\u00eb funksionoj\u00eb n\u00eb t\u00eb gjitha rastet kur k\u00ebrkesa \u00ebsht\u00eb n\u00eb gjendje p\u00ebrpunimi t\u00eb blloqeve t\u00eb dh\u00ebnash. Ajo do t\u00eb p\u00ebrpunoj\u00eb bllokun tjet\u00ebr t\u00eb dh\u00ebnash, do t\u00eb kontrolloj\u00eb flamurin dhe do t\u00eb ndalet.<\/p>\n<p><\/p>\n<p>Kjo nuk funksionon n\u00eb rastet kur k\u00ebrkesa \u00ebsht\u00eb bllokuar n\u00eb nj\u00eb operacion t\u00eb caktuar. Megjithat\u00eb, mendoj se ndoshta nuk \u00ebsht\u00eb rasti juaj, sepse sipas fjal\u00ebve tuaja, ajo po p\u00ebrdor shum\u00eb burime t\u00eb serverit. Ndoshta kjo nuk funksionon n\u00eb rastin e sortimit t\u00eb jasht\u00ebm dhe disa detaje t\u00eb tjera. Por n\u00eb p\u00ebrgjith\u00ebsi nuk duhet t\u00eb ndodh\u00eb, \u00ebsht\u00eb nj\u00eb bug. Dhe e vetmja gj\u00eb q\u00eb mund t\u00eb rekomandoj \u00ebsht\u00eb q\u00eb t\u00eb p\u00ebrdit\u00ebsoni ClickHouse.<\/p>\n<p><\/p>\n<h2 id=\"anchorreading-timeanchorkak-rasschitat-vremya-otveta-pri-chitayuschey-nagruzke\"><noindex><a rel=\"nofollow\" name=\"reading-time\"><\/a><\/noindex>Si t\u00eb llogarit koh\u00ebn e p\u00ebrgjigjes n\u00ebn ngarkes\u00eb leximi?<\/h2>\n<p><\/p>\n<blockquote><p>Ka nj\u00eb tabel\u00eb ku ruhen agregat\u00ebt sipas item \u2014 numra t\u00eb ndrysh\u00ebm. Numri i rreshtave \u00ebsht\u00eb af\u00ebrsisht nj\u00ebqind milion. A mund t\u00eb presim nj\u00eb koh\u00eb p\u00ebrgjigjeje t\u00eb parashikueshme n\u00ebse hedhin 1K RPS p\u00ebr 1K item'\u00eb? <\/p><\/blockquote>\n<p>Duke u bazuar n\u00eb kontekst, duket se b\u00ebhet fjal\u00eb p\u00ebr ngarkes\u00eb lexuese, sepse nuk ka asnj\u00eb problem n\u00eb shkrim \u2014 mund t\u00eb futen nj\u00eb mij\u00eb, qoft\u00eb nj\u00ebqind mij\u00eb, ndonj\u00ebher\u00eb edhe disa miliona rreshta. <\/p>\n<p><\/p>\n<p>K\u00ebrkesat lexuese mund t\u00eb jen\u00eb shum\u00eb t\u00eb ndryshme. N\u00eb select 1 ClickHouse mund t\u00eb ekzekutohen rreth dhjet\u00ebra mij\u00ebra k\u00ebrkesa n\u00eb sekond\u00eb, prandaj edhe k\u00ebrkesat p\u00ebr nj\u00eb \u00e7el\u00ebs k\u00ebrkojn\u00eb disa burime. Dhe k\u00ebto k\u00ebrkesa t\u00eb sakta do t\u00eb jen\u00eb m\u00eb t\u00eb nd\u00ebrlikuara se n\u00eb ndonj\u00eb baz\u00eb t\u00eb dh\u00ebnash key-value, sepse p\u00ebr \u00e7do lexim \u00ebsht\u00eb e nevojshme t\u00eb lexoni nj\u00eb bllok t\u00eb dh\u00ebnash sipas indeksit. Indeksi nuk adreson secil\u00ebn regjistrim, por secil\u00ebn gam\u00eb. K\u00ebshtu q\u00eb do t\u00eb duhet t\u00eb lexoni t\u00eb gjith\u00eb gam\u00ebn \u2014 kjo \u00ebsht\u00eb 8192 rreshta si parazgjedhje. Dhe do t\u00eb duhet t\u00eb \u00e7'kompresoni bllokun e dh\u00ebnash t\u00eb kompresuar nga 64 KB n\u00eb 1 MB. Zakonisht, k\u00ebto k\u00ebrkesa t\u00eb sakta zgjat\u00ebn disa milisekonda. Por ky \u00ebsht\u00eb varianti m\u00eb i thjesht\u00eb.<\/p>\n<p><\/p>\n<p>Le t\u00eb p\u00ebrpiqemi t\u00eb b\u00ebjm\u00eb nj\u00eb aritmetik\u00eb t\u00eb thjesht\u00eb. N\u00ebse shum\u00ebzojm\u00eb disa milisekonda me nj\u00eb mij\u00eb, rezultati \u00ebsht\u00eb disa sekonda. Si t\u00eb thuash, mbajtja e nj\u00eb mij\u00eb k\u00ebrkesave n\u00eb sekond\u00eb duket e pamundur, por n\u00eb t\u00eb v\u00ebrtet\u00eb \u00ebsht\u00eb e mundur, sepse kemi disa b\u00ebrthama procesori. Pra, n\u00eb principe ClickHouse mund t\u00eb mbaj\u00eb 1000 RPS ndonj\u00ebher\u00eb, por p\u00ebr k\u00ebrkesat e shkurtra, pik\u00ebrisht ato t\u00eb sakta.<\/p>\n<p><\/p>\n<p>N\u00ebse duhet t\u00eb shkall\u00ebzoni klasterin ClickHouse p\u00ebr numrin e k\u00ebrkesave t\u00eb thjeshta, un\u00eb rekomandoj m\u00ebnyr\u00ebn m\u00eb t\u00eb thjesht\u00eb \u2014 t\u00eb rritni numrin e replika dhe t\u00eb d\u00ebrgoni k\u00ebrkesat n\u00eb nj\u00eb replik\u00eb t\u00eb rast\u00ebsishme. N\u00ebse nj\u00eb replik\u00eb mban pes\u00ebqind k\u00ebrkesa n\u00eb sekond\u00eb, q\u00eb \u00ebsht\u00eb plot\u00ebsisht e realizueshme, at\u00ebher\u00eb tre replika do t\u00eb mbajn\u00eb nj\u00eb mij\u00eb e pes\u00ebqind.<\/p>\n<p><\/p>\n<p>Ndonj\u00ebher\u00eb, natyrisht, ClickHouse mund t\u00eb konfigurohet p\u00ebr numrin maksimal t\u00eb leximeve t\u00eb sakta. \u00c7far\u00eb \u00ebsht\u00eb e nevojshme p\u00ebr k\u00ebt\u00eb? E para \u00ebsht\u00eb t\u00eb ulni granularitetin e indeksit. N\u00eb k\u00ebt\u00eb rast granularet duhet t\u00eb reduktohet, jo n\u00eb nj\u00ebsi, por me llogari q\u00eb numri i regjistrimeve n\u00eb indeks do t\u00eb jet\u00eb disa miliona ose disa dhjet\u00ebra miliona n\u00eb server. N\u00ebse tabela ka dhjet\u00eb milion rreshta, at\u00ebher\u00eb mund t\u00eb caktosh 64 si granularitet.<\/p>\n<p><\/p>\n<p>Mund t\u00eb zvog\u00ebloni madh\u00ebsin\u00eb e bllokut t\u00eb kompresuar. P\u00ebr k\u00ebt\u00eb ka cil\u00ebsime. <strong>min compress block size<\/strong>, <strong>max compress block size<\/strong>. Ato mund t\u00eb reduktohen, t\u00eb rimarrin t\u00eb dh\u00ebnat, dhe at\u00ebher\u00eb k\u00ebrkesat e sakta do t\u00eb jen\u00eb m\u00eb t\u00eb shpejta. Por prap\u00eb, ClickHouse nuk \u00ebsht\u00eb nj\u00eb baz\u00eb e dh\u00ebnash key-value. Nj\u00eb num\u00ebr i madh k\u00ebrkesash t\u00eb vogla \u00ebsht\u00eb nj\u00eb antipattern ngarkese.<\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> Nj\u00eb k\u00ebshill\u00eb p\u00ebr rastin n\u00ebse aty ka zakonisht llogaritar\u00eb. Kjo \u00ebsht\u00eb nj\u00eb situat\u00eb mjaft standarde, kur n\u00eb ClickHouse ruhet nj\u00eb lloj numri. Un\u00eb kam nj\u00eb p\u00ebrdorues, ai \u00ebsht\u00eb nga nj\u00eb vend i caktuar, plus nj\u00eb fush\u00eb tjet\u00ebr, dhe \u00ebsht\u00eb e nevojshme t\u00eb rritet ndonj\u00eb gj\u00eb n\u00eb m\u00ebnyr\u00eb inkrementale. Merrni MySQL, b\u00ebni nj\u00eb \u00e7el\u00ebs unik \u2014 n\u00eb MySQL ai \u00ebsht\u00eb \u00e7el\u00ebsi i dyfisht\u00eb, nd\u00ebrsa n\u00eb PostgreSQL ai \u00ebsht\u00eb konflikt \u2014 dhe shtoni me nj\u00eb plus. Kjo do t\u00eb funksionoj\u00eb shum\u00eb m\u00eb mir\u00eb. <\/p>\n<p><\/p>\n<p>Kur keni pak t\u00eb dh\u00ebna, nuk ka kuptim t\u00eb p\u00ebrdorni ClickHouse. Ka bazohet t\u00eb zakonshme, dhe ato e menaxhojn\u00eb mir\u00eb k\u00ebt\u00eb. <\/p>\n<p><\/p>\n<h2 id=\"anchorpimp-my-clickhouseanchorchto-podtyunit-v-clickhouse-chtoby-bolshe-dannyh-bylo-vnbspkeshe\"><noindex><a rel=\"nofollow\" name=\"pimp-my-clickhouse\"><\/a><\/noindex>\u00c7far\u00eb t\u00eb p\u00ebrshtatni n\u00eb ClickHouse p\u00ebr t\u00eb pasur m\u00eb shum\u00eb t\u00eb dh\u00ebna n\u00eb memorie?<\/h2>\n<p><\/p>\n<blockquote><p>Imaginoni nj\u00eb situat\u00eb \u2014 n\u00eb servera ka 256 GB RAM, n\u00eb rutin\u00ebn e p\u00ebrditshme ClickHouse merr rreth 60\u201480 GB, n\u00eb kulm \u2014 deri n\u00eb 130. \u00c7far\u00eb mund t\u00eb p\u00ebrfshihet dhe t\u00eb optimizohet, n\u00eb m\u00ebnyr\u00eb q\u00eb t\u00eb ket\u00eb m\u00eb shum\u00eb t\u00eb dh\u00ebna n\u00eb cache dhe, p\u00ebr rrjedhoj\u00eb, t\u00eb ket\u00eb m\u00eb pak k\u00ebrkesa p\u00ebr disk?<\/p><\/blockquote>\n<p>N\u00eb p\u00ebrgjith\u00ebsi, page cache i sistemit operativ e p\u00ebrballon mir\u00eb k\u00ebt\u00eb detyr\u00eb. N\u00ebse thjesht hapni topin, shikoni atje cached ose free \u2014 aty gjithashtu shkruhet se sa \u00ebsht\u00eb e cache-uar \u2014 mund t\u00eb v\u00ebreni se gjith\u00eb memoria e lir\u00eb \u00ebsht\u00eb p\u00ebrdorur p\u00ebr cache. Dhe k\u00ebto t\u00eb dh\u00ebna gjat\u00eb leximit do t\u00eb lexohen jo nga disku, por nga memoria e operativ\u00ebs. Duke e marr\u00eb parasysh, mund t\u00eb them se cache p\u00ebrdoret efikasht, sepse po cache-ohen t\u00eb dh\u00ebnat e kompresuara.<\/p>\n<p><\/p>\n<p>Megjithat\u00eb, n\u00ebse d\u00ebshironi t\u00eb p\u00ebrshpejtoni disa k\u00ebrkesa t\u00eb thjeshta edhe m\u00eb shum\u00eb, ka mund\u00ebsin\u00eb p\u00ebr t\u00eb aktivizuar brenda ClickHouse cache n\u00eb t\u00eb dh\u00ebna t\u00eb pakompresuara. Kjo quhet <strong>uncompressed cache<\/strong>. N\u00eb skedarin e konfigurimit config.xml, vendosni uncompressed cache size n\u00eb vler\u00ebn q\u00eb ju nevojitet \u2014 un\u00eb rekomandoj t\u00eb mos jet\u00eb m\u00eb shum\u00eb se gjysma e memories s\u00eb lir\u00eb t\u00eb operativ\u00ebs, sepse e gjith\u00eb pjesa tjet\u00ebr do t\u00eb shkoj\u00eb p\u00ebr page cache. <\/p>\n<p><\/p>\n<p>P\u00ebr m\u00eb tep\u00ebr, ka dy cil\u00ebsime n\u00eb nivelin e k\u00ebrkes\u00ebs. Cil\u00ebsimi i par\u00eb - <strong>use uncompressed cache<\/strong> \u2014 aktivizon p\u00ebrdorimin e tij. Rekomandohet t\u00eb aktivizohet p\u00ebr t\u00eb gjitha k\u00ebrkesat, p\u00ebrve\u00e7 atyre t\u00eb r\u00ebnda, t\u00eb cilat mund t\u00eb lexojn\u00eb t\u00eb dh\u00ebna t\u00eb plota dhe t\u00eb pastrojn\u00eb k\u00ebt\u00eb cache. Dhe konfigurimi i dyt\u00eb \u2014 \u00ebsht\u00eb di\u00e7ka si numri maksimal i rreshtave p\u00ebr p\u00ebrdorimin e cache-it. Ai automatikisht kufizon k\u00ebrkesat e m\u00ebdha p\u00ebr t\u00eb kaluar p\u00ebrmes cache-it.<\/p>\n<p><\/p>\n<h2 id=\"anchorstorage-configurationanchorkak-mozhno-nastroit-storage_configuration-dlya-hraneniya-v-operativke\"><noindex><a rel=\"nofollow\" name=\"storage-configuration\"><\/a><\/noindex>Si mund t\u00eb konfiguroj storage_configuration p\u00ebr ruajtjen n\u00eb RAM?<\/h2>\n<p><\/p>\n<blockquote><p>N\u00eb dokumentacionin e ri t\u00eb ClickHouse kam lexuar nj\u00eb seksion lidhur <noindex><a rel=\"nofollow\" href=\"https:\/\/clickhouse.tech\/docs\/en\/single\/#table_engine-mergetree-multiple-volumes\">me ruajtjen e t\u00eb dh\u00ebnave<\/a><\/noindex>. N\u00eb p\u00ebrshkrim ka nj\u00eb shembull me SSD t\u00eb shpejt\u00eb. <\/p>\n<p>\u00cbsht\u00eb interesante se si mund t\u00eb konfigurohet e nj\u00ebjta gj\u00eb me memorie t\u00eb nxeht\u00eb volumetrike. Dhe nj\u00eb pyetje tjet\u00ebr. Si funksionon select me nj\u00eb organizim t\u00eb till\u00eb t\u00eb t\u00eb dh\u00ebnave, a do t\u00eb lexoj\u00eb t\u00eb gjith\u00eb setin ose vet\u00ebm at\u00eb q\u00eb ndodhet n\u00eb disk, dhe a kompresohen k\u00ebto t\u00eb dh\u00ebna n\u00eb memorje? Dhe si vepron seksioni prewhere n\u00eb nj\u00eb organizim t\u00eb till\u00eb t\u00eb t\u00eb dh\u00ebnave?<\/p><\/blockquote>\n<p>Ky konfigurim ndikon n\u00eb ruajtjen e copave t\u00eb t\u00eb dh\u00ebnave, dhe formati i tyre nuk ndryshon aspak.<br \/>\nLe t\u00eb shqyrtojm\u00eb m\u00eb n\u00eb detaje. <\/p>\n<p><\/p>\n<p>Mund t\u00eb konfigurohet ruajtja e t\u00eb dh\u00ebnave n\u00eb memorje. Gjith\u00e7ka q\u00eb konfigurohet p\u00ebr disk \u2014 \u00ebsht\u00eb rruga e tij. Krijoni nj\u00eb ndarjen tmpfs, e cila \u00ebsht\u00eb e ngjitur n\u00eb nj\u00eb rrug\u00eb t\u00eb caktuar n\u00eb sistemin e skedar\u00ebve. Tregoni k\u00ebt\u00eb rrug\u00eb si rrug\u00eb p\u00ebr ruajtjen e t\u00eb dh\u00ebnave p\u00ebr ndarjen m\u00eb t\u00eb nxeht\u00eb, atje fillon t\u00eb pranohet dhe t\u00eb shkruhen copat e t\u00eb dh\u00ebnave, gjith\u00e7ka \u00ebsht\u00eb n\u00eb rregull. <\/p>\n<p><\/p>\n<p>Por nuk e rekomandoj k\u00ebt\u00eb p\u00ebr shkak t\u00eb besueshm\u00ebris\u00eb s\u00eb ul\u00ebt, megjithat\u00eb, n\u00ebse keni t\u00eb pakt\u00ebn tre kopje n\u00eb qendra t\u00eb ndryshme t\u00eb t\u00eb dh\u00ebnave, at\u00ebher\u00eb \u00ebsht\u00eb e mundur. N\u00ebse ndodh ndonj\u00ebher\u00eb, t\u00eb dh\u00ebnat do t\u00eb rinovohen. Imagjinoni se serveri papritmas \u00ebsht\u00eb fikur dhe m\u00eb pas \u00ebsht\u00eb ndezur s\u00ebrish. Ndarja u ngjitur p\u00ebrs\u00ebri, por aty \u00ebsht\u00eb boshll\u00ebk. Serveri ClickHouse gjat\u00eb nisjes sheh se ka k\u00ebto copa t\u00eb mungojn\u00eb, megjithat\u00eb, sipas k\u00ebsaj, metadata kan\u00eb qen\u00eb present. Ai kontrollon se n\u00eb cilat kopje jan\u00eb, i k\u00ebrkon ato dhe i shkarkon. K\u00ebshtu q\u00eb t\u00eb dh\u00ebnat do t\u00eb rinovohen. <\/p>\n<p><\/p>\n<p>N\u00eb k\u00ebt\u00eb kuptim, ruajtja e t\u00eb dh\u00ebnave n\u00eb RAM \u00ebsht\u00eb thelb\u00ebsisht e nj\u00ebjt\u00eb me ruajtjen e tyre n\u00eb disk, sepse kur t\u00eb dh\u00ebnat shkruhen n\u00eb disk, ato fillimisht kalojn\u00eb n\u00eb page cache dhe regjistrohen fizikisht n\u00eb m\u00ebnyr\u00eb t\u00eb deleguar. Kjo varet nga varianti i montimit t\u00eb sistemit t\u00eb skedar\u00ebve. Por p\u00ebr \u00e7do rast, dua t\u00eb them se ClickHouse nuk b\u00ebn fsync gjat\u00eb insert.<\/p>\n<p><\/p>\n<p>Megjithat\u00eb, t\u00eb dh\u00ebnat n\u00eb RAM ruhen n\u00eb formatin e nj\u00ebjt\u00eb si n\u00eb disk. K\u00ebrkesa select gjithashtu zgjedh copat q\u00eb duhen lexuar, zgjidh copat e nevojshme t\u00eb intervaleve t\u00eb t\u00eb dh\u00ebnave dhe i lexon ato. Dhe prewhere funksionon absolutisht n\u00eb t\u00eb nj\u00ebjt\u00ebn m\u00ebnyr\u00eb, pavar\u00ebsisht se n\u00ebse t\u00eb dh\u00ebnat ishin n\u00eb RAM apo n\u00eb disk.<\/p>\n<p><\/p>\n<h2 id=\"anchorlow-cardinalityanchordo-kakogo-kolichestva-unikalnyh-znacheniy-effektiven-low-cardinality\"><noindex><a rel=\"nofollow\" name=\"low-cardinality\"><\/a><\/noindex>Derisa sa unik\u00eb vlera jan\u00eb efektive Low Cardinality?<\/h2>\n<p><\/p>\n<p>Low Cardinality \u00ebsht\u00eb krijuar n\u00eb nj\u00eb m\u00ebnyr\u00eb t\u00eb zgjuar. Ai krijon fjalor\u00eb t\u00eb t\u00eb dh\u00ebnave, por ato jan\u00eb lokale. S\u00eb pari, fjalor\u00ebt jan\u00eb t\u00eb ve\u00e7ant\u00eb p\u00ebr \u00e7do cop\u00eb, s\u00eb dyti, edhe brenda nj\u00eb cope ato mund t\u00eb jen\u00eb t\u00eb ndryshme p\u00ebr \u00e7do interval. Kur numri i vlerave unike arrin nj\u00eb prag \u2014 mendoi se \u00ebsht\u00eb nj\u00eb milion \u2014 fjalori thjesht vendoset, dhe krijohet nj\u00eb i ri.<\/p>\n<p><\/p>\n<p>P\u00ebrgjigja n\u00eb t\u00ebr\u00ebsi: p\u00ebr \u00e7do interval lokal \u2014 le t\u00eb themi, p\u00ebr \u00e7do dit\u00eb \u2014 deri n\u00eb nj\u00eb milion vlera unike, Low Cardinality \u00ebsht\u00eb efikas. Pas k\u00ebsaj do t\u00eb ket\u00eb nj\u00eb fallback, ku do t\u00eb p\u00ebrdoren shum\u00eb fjalor\u00eb t\u00eb ndrysh\u00ebm, jo nj\u00eb. Do t\u00eb funksionoj\u00eb af\u00ebrsisht si nj\u00eb kolon\u00eb e zakonshme e tipit string, ndoshta pak m\u00eb pak efikas, por nuk do t\u00eb ket\u00eb nj\u00eb degradim serioz t\u00eb performanc\u00ebs. <\/p>\n<p><\/p>\n<h2 id=\"anchorfulltext-searchanchorkakie-luchshie-praktiki-ponbsppolnotekstovomu-poisku-ponbsptablice-snbsppyatyu-milliardami-strok\"><noindex><a rel=\"nofollow\" name=\"fulltext-search\"><\/a><\/noindex>Cilat jan\u00eb praktikat m\u00eb t\u00eb mira p\u00ebr k\u00ebrkimin e plot\u00eb n\u00eb nj\u00eb tabel\u00eb me pes\u00eb miliard\u00eb rreshta?<\/h2>\n<p><\/p>\n<p>Ka mjete t\u00eb ndryshme p\u00ebrgjigjeje. E para \u00ebsht\u00eb t\u00eb themi se ClickHouse nuk \u00ebsht\u00eb nj\u00eb sistem p\u00ebr k\u00ebrkimin e plote t\u00eb tekstit. P\u00ebr k\u00ebt\u00eb ka sisteme t\u00eb ve\u00e7anta si, <noindex><a rel=\"nofollow\" href=\"https:\/\/www.elastic.co\/enterprise-search\">Elasticsearch<\/a><\/noindex> dhe <noindex><a rel=\"nofollow\" href=\"http:\/\/sphinxsearch.com\/\">Sphinx<\/a><\/noindex>. Megjithat\u00eb, un\u00eb gjithnj\u00eb e m\u00eb shpesh takoj njer\u00ebz q\u00eb thon\u00eb se po kalojn\u00eb nga Elasticsearch n\u00eb ClickHouse.<\/p>\n<p><\/p>\n<p>Pse po ndodh kjo? Ata e shpjegojn\u00eb k\u00ebt\u00eb duke th\u00ebn\u00eb se Elasticsearch nuk po arrin t\u00eb p\u00ebrballoj\u00eb ngarkes\u00ebn n\u00eb disa volumetrike, duke filluar me nd\u00ebrtimin e indekseve. Indekset b\u00ebhen shum\u00eb voluminoze dhe, n\u00ebse thjesht transferoni t\u00eb dh\u00ebnat n\u00eb ClickHouse, rezulton se ato ruhen disa her\u00eb m\u00eb efikase n\u00eb volum. Nd\u00ebrkoh\u00eb, k\u00ebrkesat p\u00ebr k\u00ebrkime shpesh nuk ishin p\u00ebr t\u00eb gjetur nj\u00eb fraz\u00eb n\u00eb t\u00eb gjith\u00eb volumet e t\u00eb dh\u00ebnave, duke marr\u00eb parasysh morfologjin\u00eb, por krejt\u00ebsisht t\u00eb tjera. P\u00ebr shembull, gjetja gjat\u00eb disa or\u00ebve t\u00eb fundit n\u00eb log p\u00ebr nj\u00eb n\u00ebnsekuenc\u00eb t\u00eb caktuar byte.<\/p>\n<p><\/p>\n<p>N\u00eb k\u00ebt\u00eb rast, ju krijoni nj\u00eb indeks n\u00eb ClickHouse, me dat\u00ebn dhe koh\u00ebn si fush\u00eb kryesore. Dhe p\u00ebrjashtimi m\u00eb i madh i t\u00eb dh\u00ebnave do t\u00eb jet\u00eb pik\u00ebrisht mbi intervalin e datave. Brenda intervalit t\u00eb zgjedhur t\u00eb datave, zakonisht mund t\u00eb realizoni nj\u00eb k\u00ebrkim me tekst t\u00eb plot\u00eb, madje duke p\u00ebrdorur metod\u00ebn brute force me ndihm\u00ebn e like. Operator like n\u00eb ClickHouse \u00ebsht\u00eb operatori m\u00eb efikas i like q\u00eb mund t\u00eb gjeni. N\u00ebse gjeni ndonj\u00eb m\u00eb t\u00eb mir\u00eb, m\u00eb trego. <\/p>\n<p><\/p>\n<p>Megjithat\u00eb, prap\u00eb like \u00ebsht\u00eb nj\u00eb skanim i plot\u00eb. Dhe skanimi i plot\u00eb mund t\u00eb jet\u00eb i ngadalsh\u00ebm jo vet\u00ebm p\u00ebr CPU, por edhe p\u00ebr diskun. N\u00ebse ndonj\u00ebher\u00eb keni nj\u00eb terabajt t\u00eb dh\u00ebnash n\u00eb dit\u00eb, dhe k\u00ebrkoni nj\u00eb fjal\u00eb gjat\u00eb nj\u00eb dite, at\u00ebher\u00eb do t\u00eb duhet t\u00eb skanoni nj\u00eb terabajt. Dhe kjo sigurisht \u00ebsht\u00eb n\u00eb disqet normale t\u00eb ngurta, dhe p\u00ebrfundimisht ato do t\u00eb ngarkohen aq shum\u00eb sa q\u00eb nuk do t\u00eb mund t\u00eb hyni n\u00eb k\u00ebt\u00eb server p\u00ebrmes SSH.<\/p>\n<p><\/p>\n<p>N\u00eb k\u00ebt\u00eb rast, un\u00eb jam i gatsh\u00ebm t\u00eb ofroj nj\u00eb truk t\u00eb vog\u00ebl. Ai \u00ebsht\u00eb nga kategoria eksperimentale \u2013 ndoshta do t\u00eb funksionoj\u00eb, ndoshta jo. N\u00eb ClickHouse ka indekse t\u00eb plota tekstesh n\u00eb form\u00ebn e filtreve Bloom trigram. Kolegu yn\u00eb nga kompania Arenadata tashm\u00eb i ka provuar k\u00ebto indekse, dhe shpesh ato punojn\u00eb pik\u00ebrisht si\u00e7 \u00ebsht\u00eb parashikuar.<\/p>\n<p><\/p>\n<p>P\u00ebr t\u00eb p\u00ebrdorur ato sakt\u00ebsisht, duhet t\u00eb kuptoni mir\u00eb se si punojn\u00eb: \u00e7far\u00eb p\u00ebrfaq\u00ebson nj\u00eb filtrin Bloom trigram dhe si t\u00eb zgjidhni madh\u00ebsin\u00eb e tij. Mund t\u00eb them se ato do t\u00eb ndihmojn\u00eb p\u00ebr k\u00ebrkesat p\u00ebr ndonj\u00eb fraz\u00eb t\u00eb rrall\u00eb, substrings q\u00eb ndodhin rrall\u00eb n\u00eb t\u00eb dh\u00ebna. N\u00eb k\u00ebt\u00eb rast, n\u00ebn-indekset do t\u00eb zgjidhen dhe do t\u00eb lexohen m\u00eb pak t\u00eb dh\u00ebna.<\/p>\n<p><\/p>\n<p>Koh\u00ebt e fundit, ClickHouse ka marr\u00eb funksione edhe m\u00eb t\u00eb avancuara p\u00ebr k\u00ebrkimin me tekst t\u00eb plot\u00eb. Kjo, s\u00eb pari, \u00ebsht\u00eb k\u00ebrkimi i nj\u00eb grupi substrings n\u00eb nj\u00eb kalim, p\u00ebrfshir\u00eb varianta me ndjeshm\u00ebri nga regjistri, pa ndjeshm\u00ebri nga regjistri, me mb\u00ebshtetje p\u00ebr UTF-8 ose vet\u00ebm p\u00ebr ASCII. Zgjidhni at\u00eb q\u00eb \u00ebsht\u00eb m\u00eb efektiv p\u00ebr ju. <\/p>\n<p><\/p>\n<p>Ka gjithashtu k\u00ebrkimin e disa shprehjeve t\u00eb rregullta n\u00eb nj\u00eb kalim. Nuk keni nevoj\u00eb t\u00eb shkruani X like nj\u00eb substring ose X like nj\u00eb substring tjet\u00ebr. Thjesht shkruani, dhe gjith\u00e7ka ekzekutohet sa m\u00eb efikasht t\u00eb jet\u00eb e mundur.<\/p>\n<p><\/p>\n<p>E treta \u2013 tani ka k\u00ebrkimin afatgjat\u00eb t\u00eb shprehjeve t\u00eb rregullta dhe k\u00ebrkimin afatgjat\u00eb t\u00eb substrings. N\u00ebse dikush shkruan nj\u00eb fjal\u00eb me nj\u00eb gabim n\u00eb shtyp, ajo do t\u00eb k\u00ebrkohet sipas p\u00ebrputhjes maksimale.<\/p>\n<p><\/p>\n<h2 id=\"anchorhello-and-welcomeanchorkak-luchshe-organizovat-dostup-vnbspclickhouse-dlyanbspbolshogo-kolichestva-polzovateley\"><noindex><a rel=\"nofollow\" name=\"hello-and-welcome\"><\/a><\/noindex>Si \u00ebsht\u00eb m\u00eb mir\u00eb t\u00eb organizoni qasjen n\u00eb ClickHouse p\u00ebr nj\u00eb num\u00ebr t\u00eb madh p\u00ebrdoruesish?<\/h2>\n<p><\/p>\n<blockquote><p>M\u00eb trego se si t\u00eb organizosh leht\u00ebsisht qaccessin p\u00ebr nj\u00eb num\u00ebr t\u00eb madh konsumator\u00ebsh dhe analist\u00ebsh. Si t\u00eb formosh nj\u00eb radh\u00eb, t\u00eb prioritizosh k\u00ebrkesat si max concurrent queries, dhe me cilat mjete?<\/p><\/blockquote>\n<p>N\u00ebse klasteri \u00ebsht\u00eb mjaft i madh, nj\u00eb zgjidhje e mir\u00eb do t\u00eb ishte t\u00eb ngriheshin edhe dy server\u00eb, t\u00eb cil\u00ebt do t\u00eb sh\u00ebrbenin si pika hyr\u00ebse p\u00ebr analist\u00ebt. K\u00ebshtu q\u00eb nuk do t'u jepej analist\u00ebve akses n\u00eb shard-et e caktuara t\u00eb klasterit, por thjesht do t\u00eb krijoheshin dy server\u00eb t\u00eb zbraz\u00ebt, pa t\u00eb dh\u00ebna, dhe n\u00eb to do t\u00eb konfiguroheshin t\u00eb drejtat e aksesit. N\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, konfigurimet e p\u00ebrdoruesve p\u00ebr k\u00ebrkesat e shp\u00ebrndara do t\u00eb transmetoheshin n\u00eb server\u00ebt e larg\u00ebt. Dometh\u00ebn\u00eb, ju konfiguroheni gjith\u00e7ka n\u00eb k\u00ebta dy server\u00eb, dhe konfigurimet kan\u00eb efekt n\u00eb t\u00eb gjith\u00eb klasterin.<\/p>\n<p><\/p>\n<p>N\u00eb parim, k\u00ebta server\u00eb jan\u00eb pa t\u00eb dh\u00ebna, por volumi i RAM-it n\u00eb to \u00ebsht\u00eb shum\u00eb i r\u00ebnd\u00ebsish\u00ebm p\u00ebr ekzekutimin e k\u00ebrkesave. Disku gjithashtu mund t\u00eb p\u00ebrdoret p\u00ebr t\u00eb dh\u00ebna t\u00eb p\u00ebrkohshme, n\u00ebse \u00ebsht\u00eb aktivizuar aggregimi i jasht\u00ebm ose renditja e jashtme.<\/p>\n<p><\/p>\n<p>\u00cbsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb shikoni konfigurimet q\u00eb lidhen me t\u00eb gjitha limitet e mundshme. N\u00ebse un\u00eb tani hyj n\u00eb klasterin \"Yandex.Metrika\" si analist dhe b\u00ebj nj\u00eb k\u00ebrkes\u00eb <strong>select count from hits<\/strong>, at\u00ebher\u00eb menj\u00ebher\u00eb do t\u00eb m\u00eb jepet nj\u00eb p\u00ebrjashtim se nuk mund t\u00eb ekzekutoj k\u00ebrkes\u00ebn. Numri maksimal i rreshtave q\u00eb m\u00eb lejohet t\u00eb skanoj \u00ebsht\u00eb nj\u00ebqind miliard\u00eb, nd\u00ebrsa n\u00eb t\u00ebr\u00ebsi n\u00eb klaster jan\u00eb pes\u00ebdhjet\u00eb trilion\u00eb n\u00eb nj\u00eb tav\u00ebll. Ky \u00ebsht\u00eb kufizimi i par\u00eb. <\/p>\n<p><\/p>\n<p>Supozoni se un\u00eb heq kufizimin p\u00ebr numrin e rreshtave dhe ekzekutoj k\u00ebrkes\u00ebn p\u00ebrs\u00ebri. At\u00ebher\u00eb do t\u00eb shoh k\u00ebt\u00eb p\u00ebrjashtim tjet\u00ebr \u2014 \u00ebsht\u00eb aktivizuar konfigurimi <strong>force index by date<\/strong>. Un\u00eb nuk mund t\u00eb ekzekutoj k\u00ebrkes\u00ebn, n\u00ebse nuk kam specifikuar nj\u00eb interval datash. Nuk duhet t\u00eb llogariteni se analist\u00ebt do ta japin at\u00eb manualisht. Nj\u00eb rast tipik \u2014 \u00ebsht\u00eb shkruar nj\u00eb interval datash, ku data e ngjarjes \u00ebsht\u00eb midis jav\u00ebs. Dhe pastaj thjesht \u00ebsht\u00eb vendosur gabim sulmi, dhe n\u00eb vend t\u00eb and \u00ebsht\u00eb dola or \u2014 or URL p\u00ebrputhje. N\u00ebse nuk ka kufizime, ai do t\u00eb filloj\u00eb t\u00eb skanoj\u00eb kolon\u00ebn URL dhe do t\u00eb shpenzoj\u00eb shum\u00eb burime.<\/p>\n<p><\/p>\n<p>P\u00ebr m\u00eb tep\u00ebr, n\u00eb ClickHouse ka dy konfigurime t\u00eb prioriteteve. Fatkeq\u00ebsisht, ato jan\u00eb shum\u00eb primitive. Nj\u00ebra quhet thjesht <strong>prioriteti<\/strong>. N\u00ebse prioriteti \u2260 0, dhe ekzekutohen k\u00ebrkesat me nj\u00eb prioritet t\u00eb caktuar, por n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb ekzekutohet nj\u00eb k\u00ebrkes\u00eb me prioritet m\u00eb t\u00eb ul\u00ebt, q\u00eb do t\u00eb thot\u00eb prioritet m\u00eb t\u00eb lart\u00eb, at\u00ebher\u00eb k\u00ebrkesa me vler\u00eb m\u00eb t\u00eb lart\u00eb t\u00eb prioritetit, q\u00eb p\u00ebrfaq\u00ebson nj\u00eb prioritet m\u00eb t\u00eb ul\u00ebt, thjesht do t\u00eb pezullohet dhe nuk do t\u00eb funksionoj\u00eb fare gjat\u00eb atij koh\u00eb.<\/p>\n<p><\/p>\n<p>Kjo \u00ebsht\u00eb nj\u00eb konfigurim shum\u00eb i pap\u00ebrsosur, dhe nuk \u00ebsht\u00eb e p\u00ebrshtatshme p\u00ebr ato raste kur klasteri ka nj\u00eb ngarkes\u00eb t\u00eb vazhdueshme. Por n\u00ebse keni k\u00ebrkesa t\u00eb shkurtra, impulsive q\u00eb jan\u00eb t\u00eb r\u00ebnd\u00ebsishme, dhe kryesisht klasteri \u00ebsht\u00eb i boshatisur, nj\u00eb konfigurim i till\u00eb do t\u00eb ishte i p\u00ebrshtatsh\u00ebm.<\/p>\n<p><\/p>\n<p>Konfigurimi i ardhsh\u00ebm i prioriteteve quhet <strong>OS thread priority<\/strong>. Ajo thjesht p\u00ebrcakton p\u00ebr t\u00eb gjitha rrjedhat e kryerjes s\u00eb k\u00ebrkes\u00ebs vler\u00ebn nice p\u00ebr planifikuesin Linux. Funksionon n\u00eb m\u00ebnyr\u00eb t\u00eb mir\u00eb, por megjithat\u00eb funksionon. N\u00ebse p\u00ebrcaktoni vler\u00ebn m\u00eb t\u00eb vog\u00ebl t\u00eb mundshme nice - ajo \u00ebsht\u00eb m\u00eb e madhe n\u00eb vler\u00eb, dhe n\u00eb k\u00ebt\u00eb m\u00ebnyr\u00eb prioriteti m\u00eb i ul\u00ebt - dhe p\u00ebr k\u00ebrkesat me prioritet t\u00eb lart\u00eb p\u00ebrcaktoni -19, at\u00ebher\u00eb CPU do t\u00eb konsumoj\u00eb k\u00ebrkesat me prioritet t\u00eb ul\u00ebt rreth kat\u00ebr her\u00eb m\u00eb pak se k\u00ebrkesat me prioritet t\u00eb lart\u00eb. <\/p>\n<p><\/p>\n<p>Gjithashtu duhet t\u00eb konfigurohet koha maksimale e ekzekutimit t\u00eb k\u00ebrkes\u00ebs - le t\u00eb themi pes\u00eb minuta. Shpejt\u00ebsia minimale e ekzekutimit t\u00eb k\u00ebrkes\u00ebs - kjo \u00ebsht\u00eb m\u00eb e miri. Kjo konfigurim \u00ebsht\u00eb e njohur prej koh\u00ebsh, dhe \u00ebsht\u00eb e nevojshme, q\u00eb t\u00eb mos thjesht konfirmoj\u00eb se ClickHouse nuk ngadal\u00ebson, por p\u00ebr ta forcuar at\u00eb.<\/p>\n<p><\/p>\n<p>Imagjinoni, po konfiguroni: n\u00ebse ndonj\u00eb k\u00ebrkes\u00eb p\u00ebrpunon m\u00eb pak se nj\u00eb milion rreshta n\u00eb sekond\u00eb - k\u00ebshtu nuk duhet b\u00ebr\u00eb. Kjo q\u00ebllon reputacionin ton\u00eb t\u00eb mir\u00eb, baz\u00ebn ton\u00eb t\u00eb mir\u00eb t\u00eb t\u00eb dh\u00ebnave. Le ta ndalim thjesht k\u00ebt\u00eb. Atje n\u00eb t\u00eb v\u00ebrtet\u00eb jan\u00eb dy konfigurime. Nj\u00ebra quhet <strong>shpejt\u00ebsia minimale e ekzekutimit<\/strong> - rreshta n\u00eb sekond\u00eb, nd\u00ebrsa tjetra quhet timeout before checking min execution speed - p\u00ebrllogaritet ndryshe nga pes\u00ebmb\u00ebdhjet\u00eb sekonda. K\u00ebshtu pes\u00ebmb\u00ebdhjet\u00eb sekonda jan\u00eb t\u00eb pranueshme, dhe pastaj, n\u00ebse \u00ebsht\u00eb ngadal\u00eb, thjesht b\u00ebni nj\u00eb p\u00ebrjashtim - ndaloni k\u00ebrkes\u00ebn.<\/p>\n<p><\/p>\n<p>Gjithashtu duhet t\u00eb konfigurohen kuotat. N\u00eb ClickHouse ekziston nj\u00eb funksion i integruar i kuotave, i cili llogarit konsumimin e burimeve. Por, fatkeq\u00ebsisht, jo burimet fizike si CPU, disqet, por logjike - numri i k\u00ebrkesave t\u00eb p\u00ebrpunuara, rreshtave dhe bajt\u00ebve t\u00eb lexuar. Dhe mund t\u00eb konfigurohet, p\u00ebr shembull, maksimumi nj\u00ebqind k\u00ebrkesa brenda pes\u00eb minutash dhe nj\u00eb mij\u00eb k\u00ebrkesa n\u00eb or\u00eb.<\/p>\n<p><\/p>\n<p>Pse \u00ebsht\u00eb e r\u00ebnd\u00ebsishme kjo? Sepse disa nga k\u00ebrkesat analitike do t\u00eb kryhen manualisht direkt nga klienti ClickHouse. Dhe gjith\u00e7ka do t\u00eb shkoj\u00eb mir\u00eb. Por n\u00ebse keni analist\u00eb t\u00eb avancuar n\u00eb kompani, ata do t\u00eb shkruajn\u00eb nj\u00eb skenar, dhe n\u00eb skenar mund t\u00eb ket\u00eb nj\u00eb gabim. Dhe ky gabim do t\u00eb \u00e7oj\u00eb n\u00eb p\u00ebrfundimin e k\u00ebrkes\u00ebs n\u00eb nj\u00eb cik\u00ebl t\u00eb pafund. K\u00ebt\u00eb duhet ta mbrosh.<\/p>\n<p><\/p>\n<h2 id=\"anchorsmorgasbordanchormozhno-li-otdat-rezultaty-odnogo-zaprosa-desyati-klientam\"><noindex><a rel=\"nofollow\" name=\"smorgasbord\"><\/a><\/noindex>A \u00ebsht\u00eb e mundur t\u00eb jepen rezultatet e nj\u00eb k\u00ebrkese n\u00eb dhjet\u00eb klient\u00eb?<\/h2>\n<p><\/p>\n<blockquote><p>Kemi disa p\u00ebrdorues q\u00eb duan t\u00eb vijn\u00eb me k\u00ebrkesa shum\u00eb t\u00eb m\u00ebdha n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb. K\u00ebrkesa \u00ebsht\u00eb e madhe, p\u00ebrfundon n\u00eb m\u00ebnyr\u00eb t\u00eb shpejt\u00eb n\u00eb p\u00ebrgjith\u00ebsi, por p\u00ebr shkak se ka shum\u00eb k\u00ebrkesa t\u00eb tilla n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, b\u00ebhet shum\u00eb e dhimbshme. A mund t\u00eb ekzekutohet e nj\u00ebjta k\u00ebrkes\u00eb, e cila ka ardhur dhjet\u00eb her\u00eb radhazi, nj\u00eb her\u00eb, dhe rezultati i saj t'i jepet dhjet\u00eb klient\u00ebve?<\/p><\/blockquote>\n<p>Problemi \u00ebsht\u00eb se nuk kemi rezultatesh nga cache ose nga cache t\u00eb t\u00eb dh\u00ebnave nd\u00ebrmjet\u00ebse. Ekziston cache i faqes s\u00eb sistemit operativ, i cili do t\u00eb lejoj\u00eb t\u00eb mos lexoni t\u00eb dh\u00ebnat p\u00ebrs\u00ebri nga disku, por, fatkeq\u00ebsisht, t\u00eb dh\u00ebnat p\u00ebrs\u00ebri do t\u00eb shkojn\u00eb p\u00ebrmes procesit t\u00eb dekodimit, deserializimit dhe p\u00ebrpunimit p\u00ebrs\u00ebri. <\/p>\n<p><\/p>\n<p>Do t\u00eb ishte e d\u00ebshirueshme t\u00eb shmangeshim ndonj\u00eb m\u00ebnyr\u00eb, ndoshta duke cache-uar t\u00eb dh\u00ebnat nd\u00ebrmjet\u00ebse, ose duke krijuar nj\u00eb radh\u00eb p\u00ebr k\u00ebrkesa t\u00eb ngjashme dhe duke shtuar cache p\u00ebr rezultatet. Aktualisht, kemi n\u00eb zhvillim nj\u00eb pull request, i cili shton cache p\u00ebr k\u00ebrkesat, por vet\u00ebm p\u00ebr n\u00ebnk\u00ebrkesat n\u00eb seksionin in dhe join - \u00e7ka do t\u00eb thot\u00eb se zgjidhja nuk \u00ebsht\u00eb e plot\u00eb.<\/p>\n<p><\/p>\n<p>Megjithat\u00eb, gjithashtu na ndodh nj\u00eb situat\u00eb e till\u00eb. Shembulli kanonik \u00ebsht\u00eb k\u00ebrkesat me paginim. Ka nj\u00eb raport q\u00eb ka disa faqe, dhe b\u00ebhet nj\u00eb k\u00ebrkes\u00eb limit 10. Pastaj e nj\u00ebjta gj\u00eb, por limit 10,10. M\u00eb pas, faqja tjet\u00ebr. Dhe k\u00ebrkohet t\u00eb dihet, pse ne llogarisim gjithmon\u00eb k\u00ebt\u00eb? Por tani nuk ka zgjidhje dhe nuk mund ta shmangim.<\/p>\n<p><\/p>\n<p>Ka nj\u00eb zgjidhje alternative, e cila vendoset si nj\u00eb sidet n\u00eb an\u00eb t\u00eb ClickHouse - <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/Vertamedia\/chproxy\">ClickHouse Proxy<\/a><\/noindex>.<\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> N\u00eb ClickHouse Proxy ka nj\u00eb limiter t\u00eb brendsh\u00ebm t\u00eb shkall\u00ebs dhe nj\u00eb cache t\u00eb integruar p\u00ebr rezultatet. Aty jan\u00eb b\u00ebr\u00eb shum\u00eb cil\u00ebsime, sepse problemi \u00ebsht\u00eb trajtuar. Proxy lejon t\u00eb kufizosh k\u00ebrkesat, duke i radhitur ato dhe t\u00eb p\u00ebrcaktosh se sa koh\u00eb jeton cache i k\u00ebrkesave. N\u00ebse k\u00ebrkesat ishin v\u00ebrtet t\u00eb nj\u00ebjta, Proxy do t'i kthej\u00eb ato shum\u00eb her\u00eb, por do t'i d\u00ebrgoj\u00eb n\u00eb ClickHouse nj\u00eb her\u00eb.<\/p>\n<p><\/p>\n<p>Edhe n\u00eb Nginx ka cache n\u00eb versionin falas, dhe kjo do t\u00eb funksionoj\u00eb gjithashtu. Nginx madje ka cil\u00ebsime, q\u00eb, n\u00ebse k\u00ebrkesat arrijn\u00eb n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, do t\u00eb ngadal\u00ebsoj\u00eb t\u00eb tjerat, derisa nj\u00ebra t\u00eb p\u00ebrfundoj\u00eb. Por n\u00eb ClickHouse Proxy, cil\u00ebsimi \u00ebsht\u00eb b\u00ebr\u00eb shum\u00eb mir\u00eb. Ai \u00ebsht\u00eb b\u00ebr\u00eb pik\u00ebrisht p\u00ebr ClickHouse, p\u00ebr k\u00ebto k\u00ebrkesa, prandaj \u00ebsht\u00eb m\u00eb i p\u00ebrshtatsh\u00ebm. Dhe gjithashtu vendoset leht\u00ebsisht. <\/p>\n<p><\/p>\n<h2 id=\"anchorasynchronousanchorkak-byt-snbspasinhronnymi-operaciyami-i-materializovannymi-predstavleniyami\"><noindex><a rel=\"nofollow\" name=\"asynchronous\"><\/a><\/noindex>Si t\u00eb veprojm\u00eb me operacionet asinkrone dhe pamjet e materializuara?<\/h2>\n<p><\/p>\n<blockquote><p>Ka nj\u00eb problem, q\u00eb operacionet me motorin e z\u00ebvend\u00ebsimit jan\u00eb asinkrone - s\u00eb pari t\u00eb dh\u00ebnat shkruhen, m\u00eb pas ndodhin mbledhjet e tyre. N\u00ebse n\u00ebn tabel\u00eb ka nj\u00eb tabel\u00eb t\u00eb materializuar me disa aggragate, at\u00ebher\u00eb duplicimet do t\u00eb regjistrohen atje. Dhe n\u00ebse nuk ka ndonj\u00eb logjik\u00eb t\u00eb komplikuar, at\u00ebher\u00eb t\u00eb dh\u00ebnat do t\u00eb jen\u00eb t\u00eb dyfishuara. \u00c7far\u00eb mund t\u00eb b\u00ebjm\u00eb me k\u00ebt\u00eb?<\/p>\n<p>Ka nj\u00eb zgjidhje t\u00eb qart\u00eb - t\u00eb realizojm\u00eb nj\u00eb trigger p\u00ebr nj\u00eb klas\u00eb t\u00eb caktuar matviews gjat\u00eb operacionit asinkron t\u00eb mbledhjes. A ka ndonj\u00eb \u00abplumb argjendi\u00bb, plane p\u00ebr realizimin e funksionaliteteve t\u00eb tilla?<\/p><\/blockquote>\n<p>Duhet t\u00eb kuptojm\u00eb si funksionon deduplication. Ajo, p\u00ebr t\u00eb cil\u00ebn do t\u00eb flas tani, nuk i p\u00ebrket pyetjes, por p\u00ebr \u00e7do rast \u00ebsht\u00eb mir\u00eb t\u00eb mendojm\u00eb p\u00ebr t\u00eb.<\/p>\n<p><\/p>\n<p>Kur futemi n\u00eb tabel\u00ebn e replikueshme, ka deduplication t\u00eb z\u00ebvendosura t\u00eb blloqeve t\u00eb plota t\u00eb futur. N\u00ebse keni futur p\u00ebrs\u00ebri t\u00eb nj\u00ebjtin bllok, i cili p\u00ebrmban numrin e nj\u00ebjt\u00eb t\u00eb rreshtave t\u00eb nj\u00ebjt\u00eb n\u00eb t\u00eb nj\u00ebjtin rend, at\u00ebher\u00eb t\u00eb dh\u00ebnat do t\u00eb dedupikohen. Do t\u00eb merrni \"Ok\" n\u00eb p\u00ebrgjigje p\u00ebr insert, por n\u00eb fakt do t\u00eb regjistrohet nj\u00eb paket\u00eb e t\u00eb dh\u00ebnave dhe nuk do t\u00eb dyfishohet.<\/p>\n<p><\/p>\n<p>Kjo \u00ebsht\u00eb e nevojshme p\u00ebr qart\u00ebsi. N\u00ebse gjat\u00eb futjes keni marr\u00eb \"Ok\", at\u00ebher\u00eb t\u00eb dh\u00ebnat tuaja jan\u00eb futur. N\u00ebse keni marr\u00eb nj\u00eb gabim nga ClickHouse, at\u00ebher\u00eb ato nuk jan\u00eb futur, dhe duhet t\u00eb p\u00ebrs\u00ebrisni futjen. Por, n\u00ebse gjat\u00eb futjes u nd\u00ebrpre lidhja, at\u00ebher\u00eb nuk e dini n\u00ebse t\u00eb dh\u00ebnat jan\u00eb futur apo jo. Nj\u00eb e vetmja mund\u00ebsi \u00ebsht\u00eb t\u00eb p\u00ebrs\u00ebrisni p\u00ebrs\u00ebri futjen. N\u00ebse t\u00eb dh\u00ebnat me t\u00eb v\u00ebrtet\u00eb jan\u00eb futur dhe ju i fut\u00ebt p\u00ebrs\u00ebri, ka nj\u00eb deduplifikim t\u00eb bllokave. Kjo \u00ebsht\u00eb e nevojshme p\u00ebr t\u00eb shmangur kopjet. <\/p>\n<p><\/p>\n<p>Dhe \u00ebsht\u00eb e r\u00ebnd\u00ebsishme gjithashtu, si funksionon ajo p\u00ebr pamjet materializuese. N\u00ebse t\u00eb dh\u00ebnat jan\u00eb deduplifikuar gjat\u00eb futjes n\u00eb tabel\u00ebn kryesore, at\u00ebher\u00eb ato nuk do t\u00eb shkojn\u00eb as n\u00eb pamjen materializuese.<\/p>\n<p><\/p>\n<p>Tani p\u00ebr sa i p\u00ebrket pyetjes. Keni nj\u00eb situat\u00eb m\u00eb t\u00eb komplikuar, sepse po regjistroni kopje t\u00eb rreshtave t\u00eb ve\u00e7ant\u00eb. Dometh\u00ebn\u00eb, nuk \u00ebsht\u00eb e gjith\u00eb ngarkesa q\u00eb \u00ebsht\u00eb kopjuar, por sakt\u00ebsisht rreshtat specifik\u00eb, dhe ato po shkojn\u00eb n\u00eb sfond. N\u00eb t\u00eb v\u00ebrtet\u00eb, t\u00eb dh\u00ebnat do t\u00eb shkojn\u00eb n\u00eb tabel\u00ebn kryesore, nd\u00ebrsa n\u00eb pamjen materializuese do t\u00eb shkojn\u00eb ato t\u00eb pa shkrira, dhe gjat\u00eb merdheve nuk do t\u00eb ndodhin asgj\u00eb me pamjet materializuese. Sepse pamja materializuese \u00ebsht\u00eb asgj\u00eb m\u00eb shum\u00eb se nj\u00eb trigger p\u00ebr insert. N\u00eb operacione t\u00eb tjera, nuk ndodh asgj\u00eb shtes\u00eb me t\u00eb.<\/p>\n<p><\/p>\n<p>Dhe k\u00ebtu nuk mund t\u00eb sjell ndonj\u00eb lajm t\u00eb mir\u00eb. Duhet vet\u00ebm t\u00eb k\u00ebrkojm\u00eb nj\u00eb zgjidhje specifike p\u00ebr k\u00ebt\u00eb rast. P\u00ebr shembull, a mund t\u00eb b\u00ebhet po ashtu nj\u00eb z\u00ebvend\u00ebsim n\u00eb pamjen materializuese, dhe ndoshta m\u00ebnyra e deduplifikimit do t\u00eb funksionoj\u00eb nj\u00eblloj. Por ndoshta jo gjithmon\u00eb. N\u00ebse ajo \u00ebsht\u00eb agreguese, at\u00ebher\u00eb nuk do t\u00eb funksionoj\u00eb. <\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> Edhe ne kemi pasur shpresat tona p\u00ebr ndihm\u00eb n\u00eb nj\u00eb koh\u00eb t\u00eb caktuar. Ishte nj\u00eb problem, q\u00eb kishte shfaqje reklamash, dhe kishte disa t\u00eb dh\u00ebna, q\u00eb ne mund t\u00eb tregojm\u00eb n\u00eb koh\u00eb reale \u2014 ato jan\u00eb thjesht shfaqje. Ato rrall\u00eb kopjohen, por, n\u00ebse ndodh, ne gjithmon\u00eb do t\u2019i shkrijm\u00eb m\u00eb von\u00eb. Dhe kishte gj\u00ebra q\u00eb nuk mund t\u00eb kopjoheshin \u2014 klikimet dhe gjith\u00eb kjo histori. Por gjithashtu do t\u00eb d\u00ebshironim t'i tregojm\u00eb ato praktikisht menj\u00ebher\u00eb.<\/p>\n<p><\/p>\n<p>Si jan\u00eb b\u00ebr\u00eb pamjet materializuese? Ka pasur pamje, ku shkruhet drejtp\u00ebrdrejt \u2014 shkruhet n\u00eb t\u00eb dh\u00ebnat e pap\u00ebrpunuara, dhe shkruhet n\u00eb pamje. Atje n\u00eb nj\u00eb moment t\u00eb caktuar t\u00eb dh\u00ebnat nuk ishin shum\u00eb t\u00eb sakta, ato duplikoheshin etj. Dhe ekziston nj\u00eb pjes\u00eb e dyt\u00eb e tabel\u00ebs, ku ato duken gjithashtu si pamjet materializuese, dometh\u00ebn\u00eb sipas struktur\u00ebs ato jan\u00eb plot\u00ebsisht t\u00eb nj\u00ebjta. Nj\u00eb her\u00eb n\u00eb nj\u00eb far\u00eb kohe ne rrethin\u00eb t\u00eb dh\u00ebnat, llogarisim t\u00eb dh\u00ebnat pa kopje, shkruajm\u00eb n\u00eb ato tabela. <\/p>\n<p><\/p>\n<p>Ne kemi kaluar p\u00ebrmes API - n\u00eb ClickHouse, kjo nuk do t\u00eb funksionoj\u00eb me duar. API shqyrton: kur kam dat\u00ebn e fundit t\u00eb shtimit n\u00eb tabel\u00eb, ku t\u00eb dh\u00ebnat e sakta tashm\u00eb jan\u00eb llogaritur, dhe b\u00ebn nj\u00eb k\u00ebrkes\u00eb n\u00eb nj\u00eb tabel\u00eb dhe n\u00eb nj\u00eb tjet\u00ebr tabel\u00eb. Nga nj\u00eb k\u00ebrkes\u00eb p\u00ebrzgjedh deri n\u00eb nj\u00eb num\u00ebr t\u00eb caktuar koh\u00ebsh, dhe nga tjetra merr ato q\u00eb ende nuk jan\u00eb llogaritur. Dhe kjo funksionon, por jo p\u00ebrmes nj\u00eb ClickHouse t\u00eb vet\u00ebm.<\/p>\n<p><\/p>\n<p>N\u00ebse keni ndonj\u00eb API - p\u00ebr analist\u00eb, p\u00ebr p\u00ebrdorues - at\u00ebher\u00eb n\u00eb parim, kjo \u00ebsht\u00eb nj\u00eb mund\u00ebsi. Ju gjithmon\u00eb vazhdoni t\u00eb llogaritni, gjithmon\u00eb rillogaritni. Kjo mund t\u00eb b\u00ebhet nj\u00eb her\u00eb n\u00eb dit\u00eb apo n\u00eb ndonj\u00eb moment tjet\u00ebr. Ju zgjedhni vet\u00eb intervalin q\u00eb ju nevojitet dhe q\u00eb nuk \u00ebsht\u00eb kritik.<\/p>\n<p><\/p>\n<h2 id=\"anchordashboardanchorv-clickhouse-mnogo-logov-kak-ya-mogu-videt-vsyo-chto-proishodit-s-serverom-vnbspmomente\"><noindex><a rel=\"nofollow\" name=\"dashboard\"><\/a><\/noindex>N\u00eb ClickHouse ka shum\u00eb log-e. Si mund t\u00eb shoh gjith\u00e7ka q\u00eb ndodh me serverin n\u00eb momentin e tanish\u00ebm?<\/h2>\n<p><\/p>\n<blockquote><p>N\u00eb ClickHouse ka nj\u00eb num\u00ebr shum\u00eb t\u00eb madh log-esh t\u00eb ndryshme, dhe ky num\u00ebr po rritet. N\u00eb versionet e reja disa prej tyre madje p\u00ebrfshihen si parazgjedhje, n\u00eb versionet m\u00eb t\u00eb vjetra ata duhet t\u00eb aktivizohen gjat\u00eb p\u00ebrdit\u00ebsimit. Megjithat\u00eb, ata po b\u00ebhen gjithnj\u00eb e m\u00eb shum\u00eb. Do t\u00eb doja t\u00eb shihja n\u00eb fund se \u00e7far\u00eb po ndodh tani me serverin tim, ndoshta n\u00eb ndonj\u00eb dashboard p\u00ebrmbledh\u00ebs. <\/p>\n<p>A keni ndonj\u00eb ekip ClickHouse n\u00eb ekipin tuaj, ose n\u00eb ekipet e miqve tuaj, q\u00eb mb\u00ebshtesin nj\u00eb funksionalitet t\u00eb caktuar t\u00eb dashboard-eve t\u00eb gatshme, t\u00eb cilat do t\u00eb shfaqnin k\u00ebto log-e si nj\u00eb produkt t\u00eb gatsh\u00ebm? N\u00eb fund t\u00eb fundit, thjesht shikimi i log-eve n\u00eb ClickHouse \u00ebsht\u00eb t\u00eb shk\u00eblqyer. Por do ishte shum\u00eb m\u00eb mir\u00eb, n\u00ebse do t\u00eb ishte tashm\u00eb n\u00eb form\u00ebn e nj\u00eb dashboard-i t\u00eb gatsh\u00ebm. Do ta adhurja k\u00ebt\u00eb. <\/p><\/blockquote>\n<p>Ka dashboard-e, megjithat\u00eb ato nuk jan\u00eb standardizuar. N\u00eb kompanin\u00eb ton\u00eb, diku rreth 60 ekipe p\u00ebrdorin ClickHouse, dhe m\u00eb e \u00e7uditshme \u00ebsht\u00eb se shum\u00eb prej tyre kan\u00eb dashboard-e q\u00eb i kan\u00eb b\u00ebr\u00eb vet\u00eb dhe paksa t\u00eb ndryshme. Disa ekipe p\u00ebrdorin instalimin e brendsh\u00ebm t\u00eb 'Yandex.Cloud'. Aty ka disa raporte t\u00eb gatshme, megjithat\u00eb jo t\u00eb gjitha t\u00eb nevojshmet. T\u00eb tjer\u00ebt kan\u00eb t\u00eb tyren. <\/p>\n<p><\/p>\n<p>Koleg\u00ebt e mi nga 'Metrika' kan\u00eb dashboard-in e tyre n\u00eb Grafana, dhe un\u00eb kam t\u00eb mijin mbi klasterin e tyre. Aty shikoj gj\u00ebra si hit cache p\u00ebr cache-in e mosp\u00ebrmbushjes. Dhe \u00ebsht\u00eb edhe m\u00eb e komplikuar sepse ne p\u00ebrdorim mjete t\u00eb ndryshme. Dashboard-in tim e kam krijuar n\u00eb nj\u00eb mjet shum\u00eb t\u00eb vjet\u00ebr, q\u00eb quhet Graphite-web. Ai \u00ebsht\u00eb krejt\u00ebsisht i sh\u00ebmtuar. Dhe deri tani po e p\u00ebrdor at\u00eb, megjithat\u00eb ndoshta Grafana do t\u00eb ishte m\u00eb e leht\u00eb dhe m\u00eb e bukur. <\/p>\n<p><\/p>\n<p>Elementi baz\u00eb n\u00eb dashboard jan\u00eb t\u00eb nj\u00ebjta. K\u00ebto jan\u00eb metrikat sistemore p\u00ebr klasterin: CPU, memorie, disk, rrjet. T\u00eb tjera jan\u00eb numri i k\u00ebrkesave simultane, numri i bashkimeve simultane, numri i k\u00ebrkesave p\u00ebr sekond\u00eb, numri maksimal i cop\u00ebzimeve p\u00ebr partit\u00eb e tabelave MergeTree, lagja e replikimit, madh\u00ebsia e radh\u00ebs s\u00eb replikimit, numri i rreshtave t\u00eb futur p\u00ebr sekond\u00eb, numri i bllok\u00ebve t\u00eb futur p\u00ebr sekond\u00eb. K\u00ebto jan\u00eb t\u00eb gjitha, q\u00eb nuk vijn\u00eb nga log\u00ebt, por nga metrikat.<\/p>\n<p><\/p>\n<p><strong>Vladimir Kolobaev:<\/strong> Aleks, do t\u00eb doja t\u00eb b\u00ebnja disa korektime. Ka Grafana. Grafana ka nj\u00eb burim t\u00eb dh\u00ebnash, i cili \u00ebsht\u00eb ClickHouse. K\u00ebshtu q\u00eb mund t\u00eb b\u00ebj k\u00ebrkesa direkt n\u00eb ClickHouse nga Grafana. N\u00eb ClickHouse ka nj\u00eb tabel\u00eb me log\u00eb, e cila \u00ebsht\u00eb e nj\u00ebjt\u00eb p\u00ebr t\u00eb gjith\u00eb. Un\u00eb dua q\u00eb n\u00eb rezultat n\u00eb Grafana t\u00eb lidhem me k\u00ebt\u00eb tabel\u00eb log\u00ebsh dhe t\u00eb shoh k\u00ebrkesat q\u00eb d\u00ebrgon serveri im. Do ishte e shk\u00eblqyer t\u00eb kisha nj\u00eb dashboard t\u00eb till\u00eb.<\/p>\n<p><\/p>\n<p>Un\u00eb e kam ngritur vet\u00eb. Por kam nj\u00eb pyetje \u2014 n\u00ebse \u00ebsht\u00eb gjith\u00e7ka standardizuese, dhe Grafana p\u00ebrdoret nga t\u00eb gjith\u00eb, pse nuk ka nj\u00eb dashboard zyrtar n\u00eb \"Yandex\"?<\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> N\u00eb t\u00eb v\u00ebrtet\u00eb, burimi i t\u00eb dh\u00ebnave q\u00eb lidhet me ClickHouse, tani mb\u00ebshtetet nga Altinity. Dhe un\u00eb thjesht dua t\u00eb jap nj\u00eb drejtim, ku t\u00eb g\u00ebrmoj dhe kujt t'i b\u00ebj presion. Mund t\u00eb pyesim ata, sepse \"Yandex\" b\u00ebn ClickHouse, jo historin\u00eb p\u00ebrreth saj. Altinity \u00ebsht\u00eb kompania kryesore q\u00eb tani promovon ClickHouse. Ata nuk do ta l\u00ebn\u00eb, por do ta mb\u00ebshtesin. Sepse n\u00eb parim, p\u00ebr t\u00eb ngarkuar nj\u00eb dashboard n\u00eb sitin e Grafana, \u00ebsht\u00eb vet\u00ebm e nevojshme t\u00eb regjistrohesh dhe ta ngarkosh \u2014 nuk ka probleme t\u00eb ve\u00e7anta. <\/p>\n<p><\/p>\n<p><strong>Alexey Milovidov:<\/strong> N\u00eb vitin e fundit, ClickHouse ka shtuar shum\u00eb mund\u00ebsi p\u00ebr profilizimin e k\u00ebrkesave. Ka metrika p\u00ebr secil\u00ebn k\u00ebrkes\u00eb sipas p\u00ebrdorimit t\u00eb burimeve. Dhe s\u00eb fundmi, \u00ebsht\u00eb shtuar nj\u00eb profiler m\u00eb i ul\u00ebt q\u00eb tregon se ku \u00e7do k\u00ebrkes\u00eb kalon \u00e7do milisekond\u00eb. Por p\u00ebr t\u00eb p\u00ebrfituar nga kjo funksionalitet, duhet t\u00eb hap klientin konsol\u00eb dhe t\u00eb shkruaj k\u00ebrkes\u00ebn, t\u00eb cil\u00ebn vazhdimisht e harroj. E kam ruajtur diku dhe vazhdimisht harroj se ku. <\/p>\n<p><\/p>\n<p>Do t\u00eb doja t\u00eb kishte nj\u00eb mjet, n\u00eb t\u00eb cilin thjesht shkruhet \u2014 k\u00ebtu jan\u00eb k\u00ebrkesat e r\u00ebnda, t\u00eb grupuara sipas klasave t\u00eb k\u00ebrkesave. Klikoni mbi ndonj\u00eb, dhe do t\u00eb m\u00eb thoshin se \u00ebsht\u00eb e r\u00ebnd\u00eb p\u00ebr k\u00ebt\u00eb arsye. Aktualisht nuk ka nj\u00eb zgjidhje t\u00eb till\u00eb. Dhe \u00ebsht\u00eb v\u00ebrtet \u00e7udishme q\u00eb, kur njer\u00ebzit m\u00eb pyesin: \"A ka ndonj\u00eb dashboard t\u00eb gatsh\u00ebm p\u00ebr Grafana?\", un\u00eb them: \"Shikoni n\u00eb sitin e Grafana, atje \u00ebsht\u00eb komuniteti 'Dashboard', dhe atje ka nj\u00eb dashboard nga Dima, ka nj\u00eb dashboard nga Kostja. \u00c7far\u00eb \u00ebsht\u00eb kjo, nuk e di, vet\u00eb nuk e kam p\u00ebrdorur\".<\/p>\n<p><\/p>\n<h2 id=\"anchorzenanchorkak-vozdeystvovat-na-merdzhi-chtoby-server-ne-padal-vnbspoom\"><noindex><a rel=\"nofollow\" name=\"zen\"><\/a><\/noindex>Si t\u00eb ndikoj n\u00eb bashkimet, q\u00eb serveri t\u00eb mos bie n\u00eb OOM?<\/h2>\n<p><\/p>\n<blockquote><p>Kam nj\u00eb tav\u00eb, n\u00eb tav\u00eb ka vet\u00ebm nj\u00eb pjes\u00eb, ajo \u00ebsht\u00eb ReplacingMergeTree. Un\u00eb kam shkruar t\u00eb dh\u00ebna n\u00eb t\u00eb p\u00ebr kat\u00ebr vjet. M\u00eb duhej t\u00eb b\u00ebja nj\u00eb alter dhe t\u00eb fshija disa t\u00eb dh\u00ebna.<\/p>\n<p>E kam b\u00ebr\u00eb k\u00ebt\u00eb, dhe gjat\u00eb p\u00ebrpunimit t\u00eb k\u00ebtij k\u00ebrkese u dogj e gjith\u00eb memoria n\u00eb t\u00eb gjitha server\u00ebt e klasterit, dhe t\u00eb gjith\u00eb server\u00ebt e klasterit nj\u00ebz\u00ebri shkuan n\u00eb OOM. M\u00eb pas ata t\u00eb gjith\u00eb s\u00eb bashku u ngrit\u00ebn, filluan t\u00eb kryejn\u00eb merge t\u00eb s\u00eb nj\u00ebjt\u00ebs operacion, t\u00eb k\u00ebtij bloku t\u00eb dh\u00ebnash, dhe p\u00ebrs\u00ebri ran\u00eb n\u00eb OOM. M\u00eb pas ata u ngrit\u00ebn s\u00ebrish dhe ran\u00eb s\u00ebrish. Dhe kjo gj\u00eb nuk u ndal.<\/p>\n<p>M\u00eb von\u00eb doli se kjo n\u00eb t\u00eb v\u00ebrtet\u00eb ishte nj\u00eb gabim, t\u00eb cilin djemt\u00eb e rregulluan. Kjo \u00ebsht\u00eb shum\u00eb mir\u00eb, faleminderit shum\u00eb. Por ndjenja ka mbetur. Dhe tani, kur mendoj p\u00ebr at\u00eb se \u00e7far\u00eb duhet t\u00eb b\u00ebj nj\u00eb merge n\u00eb tav\u00ebn, m\u00eb lind pytja \u2014 pse nuk mund ta ndikoj ndonj\u00ebher\u00eb n\u00eb k\u00ebto merge? P.sh., t\u00eb kufizoj ato sipas sasis\u00eb s\u00eb memories s\u00eb nevojshme operative, ose n\u00eb p\u00ebrgjith\u00ebsi sipas numrit t\u00eb tyre, q\u00eb do t\u00eb p\u00ebrpunoj\u00eb pik\u00ebrisht k\u00ebt\u00eb tav\u00eb.<\/p>\n<p>Kam nj\u00eb tav\u00eb, e cila quhet \u00abMetrika\u00bb, t\u00eb lutem, p\u00ebrpunoma at\u00eb n\u00eb dy rrjedha. Mos e gjenero dhjet\u00eb apo pes\u00eb merge paralelisht, b\u00ebji n\u00eb dy. Mendoj se n\u00eb dy do t\u00eb kem mjaft memory, por p\u00ebr t\u00eb p\u00ebrpunuar dhjet\u00eb, ndoshta s'ka p\u00ebr t\u00eb mjaftuar. Pse mbetet frika? Sepse tav\u00eb po rritet, dhe ndonj\u00ebher\u00eb do t\u00eb p\u00ebrball\u00ebm me situat\u00ebn q\u00eb p\u00ebrfundimisht jo p\u00ebr shkak t\u00eb gabimit, por p\u00ebr shkak se t\u00eb dh\u00ebnat do t\u00eb ndryshojn\u00eb n\u00eb nj\u00eb sasi t\u00eb till\u00eb t\u00eb madhe q\u00eb thjesht nuk do t\u00eb kem memory t\u00eb mjaftueshme n\u00eb server. Dhe at\u00ebher\u00eb serveri do t\u00eb bjer\u00eb n\u00eb OOM gjat\u00eb merge. Edhe nd\u00ebrrimin mund ta anuloj, por p\u00ebr merge nuk ka mund\u00ebsi.<\/p><\/blockquote>\n<p>E dini, gjat\u00eb merge-ve serveri nuk do t\u00eb bie n\u00eb OOM, sepse gjat\u00eb merge p\u00ebrdoret nj\u00eb sasi e memories operuese vet\u00ebm p\u00ebr nj\u00eb diapazon t\u00eb vog\u00ebl t\u00eb t\u00eb dh\u00ebnave. Pra, gjith\u00e7ka do t\u00eb jet\u00eb mir\u00eb, pavar\u00ebsisht nga sasia e t\u00eb dh\u00ebnave.<\/p>\n<p><\/p>\n<p><strong>Vladimir Kolobaev:<\/strong> Mir\u00eb. K\u00ebtu momenti \u00ebsht\u00eb se pas rregullimit t\u00eb gabimit, shkarkova versionin e ri, dhe n\u00eb nj\u00eb tav\u00eb tjet\u00ebr, m\u00eb t\u00eb vog\u00ebl, ku ka shum\u00eb pjes\u00eb, b\u00ebra nj\u00eb operacion t\u00eb ngjash\u00ebm. Dhe gjat\u00eb merge n\u00eb server u dogj rreth 100 Gb mem\u00f3ria operuese. Kisha 150 t\u00eb z\u00ebna, 100 e h\u00ebngri, dhe mbeti nj\u00eb hap\u00ebsir\u00eb prej 50 Gb, k\u00ebshtu q\u00eb nuk ra n\u00eb OOM.<\/p>\n<p><\/p>\n<p>\u00c7far\u00eb m\u00eb mbron n\u00eb k\u00ebt\u00eb moment nga r\u00ebnia n\u00eb OOM, n\u00ebse ai n\u00eb t\u00eb v\u00ebrtet\u00eb konsumon rreth 100 Gb mem\u00f3ria operuese? \u00c7far\u00eb t\u00eb b\u00ebj n\u00eb situat\u00ebn n\u00ebse p\u00ebrfundimisht memoria e operimeve t\u00eb merge p\u00ebrfundon?<\/p>\n<p><\/p>\n<p><strong>Alexey Milovidov:<\/strong> Ka\u00e7kall\u00eb je nj\u00eb problem q\u00eb shpenzimi i RAM-it gjat\u00eb bashkimit nuk \u00ebsht\u00eb i kufizuar. Problemi tjet\u00ebr \u00ebsht\u00eb se, n\u00ebse ndonj\u00eb bashkim \u00ebsht\u00eb caktuar, at\u00ebher\u00eb ai duhet t\u00eb kryhet, pasi \u00ebsht\u00eb shkruar n\u00eb logun e replikimit. Logu i replikimit \u00ebsht\u00eb ajo q\u00eb nevojitet p\u00ebr t\u00eb sjell\u00eb replik\u00ebn n\u00eb nj\u00eb gjendje konsistente. N\u00ebse nuk b\u00ebhen manipulime manuale q\u00eb ta anulojn\u00eb k\u00ebt\u00eb log t\u00eb replikimit, bashkimi do t\u00eb duhet t\u00eb kryhet, n\u00eb nj\u00eb m\u00ebnyr\u00eb ose tjet\u00ebr.<\/p>\n<p><\/p>\n<p>Natyrisht, do t\u00eb ishte e dobishme t\u00eb kishe nj\u00eb kufizim n\u00eb RAM q\u00eb \"p\u00ebr \u00e7do rast\" mbron pik\u00ebrisht nga OOM. Nuk do ta ndihmoj\u00eb bashkimin t\u00eb kryhet, ai do t\u00eb filloj\u00eb p\u00ebrs\u00ebri, do t\u00eb arrij\u00eb nj\u00eb prag t\u00eb caktuar, do t\u00eb hedh\u00eb nj\u00eb p\u00ebrjashtim dhe pastaj p\u00ebrs\u00ebri do t\u00eb filloj\u00eb \u2014 nuk do t\u00eb dal\u00eb asgj\u00eb e mir\u00eb nga kjo. Por n\u00eb parim, t\u00eb vendoset ky kufizim do t\u00eb ishte e dobishme.<\/p>\n<p><\/p>\n<h2 id=\"anchorgoanchorkak-budet-proishodit-razrabotka-golang-drayvera-dlya-clickhouse\"><noindex><a rel=\"nofollow\" name=\"go\"><\/a><\/noindex>Si do t\u00eb zhvillohet driver-i i Golang p\u00ebr ClickHouse?<\/h2>\n<p><\/p>\n<blockquote><p>Driver-i i Golang, i shkruar nga Kirill Shvakov, tani duket se mb\u00ebshtetet zyrtarisht nga ekipi i ClickHouse. Ai <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/ClickHouse\/clickhouse-go\">ndodhet n\u00eb depozit\u00ebn ClickHouse<\/a><\/noindex>, tani \u00ebsht\u00eb i madh dhe i v\u00ebrtet\u00eb.<\/p>\n<p>Nj\u00eb sh\u00ebnim i vog\u00ebl. Ka nj\u00eb magazin\u00eb t\u00eb shk\u00eblqyer dhe t\u00eb dashur q\u00eb ka forma normale t\u00eb rendit t\u00eb pafund - kjo \u00ebsht\u00eb Vertica. Ata gjithashtu kan\u00eb driver-in e tyre zyrtar python, i cili mb\u00ebshtetet nga zhvilluesit e Vertica. Disa her\u00eb ka ndodhur q\u00eb versionet e magazin\u00ebs dhe versionet e driver-it jan\u00eb distancuar shum\u00eb, dhe driver-i n\u00eb nj\u00eb moment ka ndaluar s\u00eb funksionuari. Dhe nj\u00eb moment tjet\u00ebr. Mb\u00ebshtetja p\u00ebr k\u00ebt\u00eb driver zyrtar, m\u00eb duket se administrohet nga sistemi \"nip\u00ebll\" - ti shkruan atyre nj\u00eb \u00e7\u00ebshtje, dhe ajo mbetet atje p\u00ebrjet\u00ebsisht.<\/p>\n<p>Kam dy pyetje. Tani driver-i i Golang t\u00eb Kirillit \u00ebsht\u00eb m\u00ebnyra pothuajse default p\u00ebr t\u00eb komunikuar nga Golang me ClickHouse. Me p\u00ebrjashtim t\u00eb ndokujt q\u00eb komunikon p\u00ebrmes nd\u00ebrfaqes http, sepse ashtu i p\u00eblqen. Si do t\u00eb zhvillohet ky driver? A do t\u00eb sinkronizohet me ndonj\u00eb ndryshim t\u00eb r\u00ebnd\u00ebsish\u00ebm n\u00eb magazin\u00eb? Dhe cila \u00ebsht\u00eb procedura p\u00ebr shqyrtimin e \u00e7\u00ebshtjeve? <\/p><\/blockquote>\n<p><strong>Kirill Shvakov:<\/strong> E para \u2014 si \u00ebsht\u00eb e organizuar burokratikisht. Ky moment nuk \u00ebsht\u00eb diskutuar, ndaj nuk kam \u00e7far\u00eb t\u00eb p\u00ebrgjigjem.<\/p>\n<p><\/p>\n<p>P\u00ebr t\u00eb p\u00ebrgjigjur pyetjes p\u00ebr \u00e7\u00ebshtjen, nevojitet nj\u00eb histori e vog\u00ebl e driver-it. Un\u00eb kam punuar n\u00eb nj\u00eb kompani q\u00eb kishte shum\u00eb t\u00eb dh\u00ebna. Ishte nj\u00eb platform\u00eb reklamimi me nj\u00eb num\u00ebr t\u00eb madh ngjarjesh q\u00eb duhej t\u00eb ruheshin diku. Dhe n\u00eb nj\u00eb moment u shfaq ClickHouse. Ne e derdh\u00ebm atje t\u00eb dh\u00ebnat, dhe p\u00ebr disa koh\u00eb gjith\u00e7ka ishte mir\u00eb, pastaj ClickHouse ra. N\u00eb at\u00eb koh\u00eb, ne vendos\u00ebm se nuk na duhej. <\/p>\n<p><\/p>\n<p>Pasi kaloi nj\u00eb vit, ne u kthyem n\u00eb iden\u00eb e p\u00ebrdorimit t\u00eb ClickHouse, dhe na duhej nj\u00eb m\u00ebnyr\u00eb p\u00ebr t\u00eb shkruar t\u00eb dh\u00ebna atje. Hyrja ishte kjo \u2014 hardueri ishte shum\u00eb i dob\u00ebt, burimet ishin t\u00eb pakta. Por ne gjithmon\u00eb kishim punuar k\u00ebshtu, dhe prandaj shikuam drejt protokollit natyror.<\/p>\n<p><\/p>\n<p>Pasi punonim n\u00eb Go, ishte e qart\u00eb se na duhej nj\u00eb driver n\u00eb Go. Un\u00eb e zhvillova at\u00eb pothuajse me orar t\u00eb plot\u00eb \u2014 kjo ishte detyra ime e pun\u00ebs. Deri n\u00eb nj\u00ebfar\u00eb momenti e p\u00ebrfunduam, dhe n\u00eb parim askush nuk e supozonte se ndokush tjet\u00ebr do ta p\u00ebrdorte. M\u00eb pas erdhi CloudFlare me t\u00eb nj\u00ebjtin problem, dhe p\u00ebr nj\u00ebfar\u00eb kohe punuam shum\u00eb ngadal\u00eb me ta, sepse kishin t\u00eb nj\u00ebjtat detyra. Madje e b\u00ebm\u00eb k\u00ebt\u00eb si n\u00eb ClickHouse vet\u00eb, ashtu edhe n\u00eb driver. <\/p>\n<p><\/p>\n<p>N\u00eb nj\u00eb moment, thjesht ndalova s\u00eb angazhuari me ta, sepse aktiviteti im n\u00eb lidhje me ClickHouse dhe puna u ndryshua pak. Prandaj \u00e7\u00ebshtjet nuk mbyllen. Her\u00eb pas here njer\u00ebz q\u00eb kan\u00eb nevoj\u00eb p\u00ebr di\u00e7ka vet\u00eb kontribuojn\u00eb n\u00eb depo. At\u00ebher\u00eb un\u00eb shoh k\u00ebrkesat p\u00ebr p\u00ebrjashtim dhe ndonj\u00ebher\u00eb edhe p\u00ebrmir\u00ebsoj ndonj\u00eb gj\u00eb, por kjo ndodh rrall\u00eb.<\/p>\n<p><\/p>\n<p>Do t\u00eb doja t\u00eb kthehesha n\u00eb driver. Disa vjet m\u00eb par\u00eb, kur gjith\u00e7ka filloi, ClickHouse ishte gjithashtu ndryshe dhe me mund\u00ebsi t\u00eb tjera. Tani ka nj\u00eb kuptim se si ta rip\u00ebrpunosh driver-in p\u00ebr t\u00eb qen\u00eb m\u00eb i mir\u00eb. N\u00ebse kjo ndodh, versioni 2 do t\u00eb jet\u00eb n\u00eb \u00e7do rast jo i p\u00ebrputhsh\u00ebm p\u00ebr shkak t\u00eb pengesave t\u00eb grumbulluara. <\/p>\n<p><\/p>\n<p>Nuk e di si ta organizoj k\u00ebt\u00eb pun\u00eb. Vet\u00ebm kam shum\u00eb pak koh\u00eb. N\u00ebse disa njer\u00ebz do t\u00eb punonin p\u00ebr t\u00eb p\u00ebrmir\u00ebsuar driverin, do t\u00eb mundja t'i ndihmoj dhe t'u tregoj se \u00e7far\u00eb t\u00eb b\u00ebjn\u00eb. Por pik\u00ebrisht pjes\u00ebmarrja aktive e \"Yandex\" n\u00eb zhvillimin e projektit deri tani nuk \u00ebsht\u00eb diskutuar. <\/p>\n<p><\/p>\n<p><strong>Alexey Milovidov:<\/strong> N\u00eb t\u00eb v\u00ebrtet\u00eb, deri tani nuk ka asnj\u00eb burokraci lidhur me k\u00ebta drivera. E vetmja gj\u00eb \u00ebsht\u00eb se ata jan\u00eb t\u00eb organizuar n\u00eb nj\u00eb institucion zyrtar, pra ky driver \u00ebsht\u00eb pranuar si zgjidhja zyrtare e paracaktuar p\u00ebr Go. Ka disa drivera t\u00eb tjer\u00eb, por ata jan\u00eb t\u00eb ndara. <\/p>\n<p><\/p>\n<p>Nuk kemi asnj\u00eb zhvillim brenda p\u00ebr k\u00ebta drivera. Pyetja \u00ebsht\u00eb n\u00ebse do t\u00eb mund t\u00eb angazhojm\u00eb nj\u00eb person t\u00eb ve\u00e7ant\u00eb, jo p\u00ebr k\u00ebt\u00eb driver konkret, por p\u00ebr zhvillimin e t\u00eb gjith\u00eb driverave t\u00eb komunitetit, ose n\u00ebse do t\u00eb mund t\u00eb gjejm\u00eb dik\u00eb nga jasht\u00eb. <\/p>\n<p><\/p>\n<h2 id=\"anchorlazy-loadanchorvneshniy-slovar-ne-podnimaetsya-posle-perezagruzki-snbspvklyuchennoy-nastroykoy-lazy_load-chto-delat\"><noindex><a rel=\"nofollow\" name=\"lazy-load\"><\/a><\/noindex>Fjalori i jasht\u00ebm nuk ngrihet pas rinisjes me konfigurimin e r\u00ebnduar aktivizuar. \u00c7far\u00eb t\u00eb b\u00ebjm\u00eb?<\/h2>\n<p><\/p>\n<blockquote><p>Ne kemi aktivizuar konfigurimin e r\u00ebnduar, dhe pas rinisjes s\u00eb serverit, fjalori nuk ngrihet vet\u00eb. Ai ngrihet vet\u00ebm pasi p\u00ebrdoruesi t\u00eb k\u00ebrkoj\u00eb k\u00ebt\u00eb fjalor. Dhe n\u00eb k\u00ebrkes\u00ebn e par\u00eb jep nj\u00eb gabim. A mund t\u00eb ngarkohet ndonj\u00ebher\u00eb automatikisht fjalori me ndihm\u00ebn e ClickHouse, apo duhet t\u00eb kontrollojm\u00eb gjithnj\u00eb gatishm\u00ebrin\u00eb e tyre, q\u00eb p\u00ebrdoruesit t\u00eb mos marrin gabime?<\/p>\n<p>Ndoshta kemi nj\u00eb version t\u00eb vjet\u00ebr t\u00eb ClickHouse, prandaj fjalori nuk \u00ebsht\u00eb ngarkuar automatikisht. A \u00ebsht\u00eb k\u00ebshtu?<\/p><\/blockquote>\n<p>S\u00eb pari, fjalor\u00ebt mund t\u00eb ngarkohen me forc\u00eb p\u00ebrmes k\u00ebrkes\u00ebs <strong>system reload dictionaries<\/strong>. S\u00eb dyti, p\u00ebrsa i p\u00ebrket gabimit \u2014 n\u00ebse fjalori \u00ebsht\u00eb ngarkuar tashm\u00eb, k\u00ebrkesat do t\u00eb funksionojn\u00eb me t\u00eb dh\u00ebnat q\u00eb jan\u00eb ngarkuar. N\u00ebse fjalori nuk \u00ebsht\u00eb ngarkuar ende, ai do t\u00eb ngarkohet pik\u00ebrisht gjat\u00eb k\u00ebrkes\u00ebs.<\/p>\n<p><\/p>\n<p>P\u00ebr fjalor\u00ebt e r\u00ebnd\u00eb, kjo nuk \u00ebsht\u00eb shum\u00eb e p\u00ebrshtatshme. P\u00ebr shembull, duhet t\u00eb ngarkojm\u00eb nj\u00eb milion rreshta nga MySQL. Disa b\u00ebjn\u00eb nj\u00eb select t\u00eb thjesht\u00eb, por ky select do t\u00eb pres\u00eb at\u00eb nj\u00eb milion rreshta. K\u00ebtu ka dy zgjidhje. E para \u00ebsht\u00eb t\u00eb \u00e7aktivizohet flehja e r\u00ebnduar. E dyta \u00ebsht\u00eb, kur serveri ngrihet, para se t\u00eb ngarkohet, t\u00eb b\u00ebhet <strong>system reload dictionary<\/strong> ose thjesht t\u00eb ekzekutohet nj\u00eb k\u00ebrkes\u00eb q\u00eb p\u00ebrdor fjalorin. At\u00ebher\u00eb fjalori do t\u00eb ngarkohet. Ne duhet t\u00eb kontrollojm\u00eb vet\u00eb disponueshm\u00ebrin\u00eb e fjalor\u00ebve me konfigurimin e r\u00ebnduar aktivizuar, sepse ClickHouse nuk i ngre automatikisht.<\/p>\n<p><\/p>\n<p>P\u00ebr pyetjen e fundit, p\u00ebrgjigjja \u00ebsht\u00eb \u2014 ndoshta versioni \u00ebsht\u00eb i vjet\u00ebr, ndoshta duhet t\u00eb zgjidhet. <\/p>\n<p><\/p>\n<h2 id=\"anchorreload-dictionariesanchorkak-byt-snbsptem-chto-system-reload-dictionaries-ne-podgruzhaet-ni-odin-iznbspmnozhestva-slovarey-esli-hotya-by-odin-iznbspnih-padaet-snbsposhibkoy\"><noindex><a rel=\"nofollow\" name=\"reload-dictionaries\"><\/a><\/noindex>Si t\u00eb veprojm\u00eb me faktin q\u00eb system reload dictionaries nuk ngarkon asnj\u00eb nga shum\u00eb fjalor\u00ebt, n\u00ebse t\u00eb pakt\u00ebn nj\u00ebra prej tyre d\u00ebshtoi me nj\u00eb gabim?<\/h2>\n<p><\/p>\n<blockquote><p>Ka nj\u00eb pyetje tjet\u00ebr lidhur me rifreskimin e fjalor\u00ebve t\u00eb sistemit. Ne kemi dy fjalor\u00eb \u2014 nj\u00eb nuk ngarkohet, tjetri ngarkohet. Rifreskimi i sistemit t\u00eb fjalor\u00ebve, n\u00eb k\u00ebt\u00eb rast, nuk ngarkon asnj\u00eb fjalor, dhe duhet t\u00eb ngarkohet sakt\u00ebsisht i ve\u00e7anti p\u00ebr emrin e tij me ndihm\u00ebn e rifreskimit t\u00eb sistemit t\u00eb fjalorit. A \u00ebsht\u00eb kjo gjithashtu e lidhur me versionin e ClickHouse?<\/p><\/blockquote>\n<p>Dua t'ju g\u00ebzoj. Ky veprim \u00ebsht\u00eb nd\u00ebrruar. K\u00ebshtu q\u00eb, n\u00ebse p\u00ebrdit\u00ebsoni ClickHouse, edhe kjo do t\u00eb ndryshoj\u00eb. N\u00ebse nuk jeni t\u00eb k\u00ebnaqur me veprimin aktual <strong>system reload dictionaries<\/strong>, p\u00ebrmir\u00ebsohuni dhe shpresojm\u00eb se do t\u00eb ndryshoj\u00eb p\u00ebr mir\u00eb.<\/p>\n<p><\/p>\n<h2 id=\"anchorconnectionanchorest-li-sposob-konfigurirovat-rekvizity-vnbspkonfige-clickhouse-no-ne-svetit-ih-prinbsposhibkah\"><noindex><a rel=\"nofollow\" name=\"connection\"><\/a><\/noindex>A ka ndonj\u00eb m\u00ebnyr\u00eb p\u00ebr t\u00eb konfiguruar kredencialet n\u00eb konfigurimin ClickHouse, por pa i ekspozuar ato gjat\u00eb gabimeve?<\/h2>\n<p><\/p>\n<blockquote><p>Pyetjes tjet\u00ebr \u00ebsht\u00eb n\u00eb lidhje me gabimet q\u00eb lidhen me fjalorin, konkretisht me detajet e lidhjes. Ne kemi shkruar detajet e lidhjes n\u00eb konfigurimin e ClickHouse p\u00ebr fjalorin, dhe kur ndodh nj\u00eb gabim, ne i marrim ato detaje dhe fjal\u00ebkalimin n\u00eb p\u00ebrgjigje. <\/p>\n<p>Ne e zgjidh\u00ebm k\u00ebt\u00eb gabim duke i nxjerr\u00eb detajet n\u00eb konfigurimin e drejtuesit ODBC. A ka ndonj\u00eb m\u00ebnyr\u00eb p\u00ebr ta konfiguruar detajet n\u00eb konfigurimin e ClickHouse, por pa ekspozuar k\u00ebto detaje kur ndodhin gabime?<\/p><\/blockquote>\n<p>K\u00ebtu zgjidhja \u00ebsht\u00eb me t\u00eb v\u00ebrtet\u00eb \u2014 t\u00eb tregosh k\u00ebto akreditime n\u00eb odbc.ini, dhe n\u00eb vet\u00eb ClickHouse t\u00eb tregosh vet\u00ebm emrin e burimit t\u00eb t\u00eb dh\u00ebnave ODBC. P\u00ebr burimet e tjera t\u00eb fjalor\u00ebve, kjo nuk do t\u00eb ndodh\u00eb \u2014 as p\u00ebr fjalorin me MySQL, as p\u00ebr t\u00eb tjerat, nuk duhet t\u00eb shihni fjal\u00ebkalimin kur ndodh nj\u00eb mesazh gabimi. P\u00ebr ODBC do t\u00eb shikoj gjithashtu \u2014 n\u00ebse di\u00e7ka e till\u00eb ekziston, duhet ta heqim thjesht.<\/p>\n<p><\/p>\n<h2 id=\"anchorzoom-backgroundsanchorbonus-fony-dlya-zuma-snbspposidelok\"><noindex><a rel=\"nofollow\" name=\"zoom-backgrounds\"><\/a><\/noindex>Bonus: sfondet p\u00ebr Zoom nga takimet<\/h2>\n<p><\/p>\n<p>Me nj\u00eb klikim n\u00eb imazh, p\u00ebr lexuesit m\u00eb t\u00eb vendosur do t\u00eb hapen sfondet bonus nga mbledhjet. Shkruajm\u00eb zjarrin s\u00eb bashku me maskotat e teknologjive t\u00eb Avito, bisedojm\u00eb me koleg\u00ebt nga dhoma e administratorit t\u00eb sistemit ose nga klubi tradicional i kompjuter\u00ebve dhe zhvillojm\u00eb takimin ditor posht\u00eb ur\u00ebs p\u00ebrpara grafitit.<\/p>\n<p><\/p>\n<p><noindex><a rel=\"nofollow\" href=\"http:\/\/amp.gs\/KvUr\"><img decoding=\"async\" alt=\"ClickHouse p\u00ebr p\u00ebrdoruesit e avancuar n\u00eb pyetje dhe p\u00ebrgjigje\" src=\"\/wp-content\/uploads\/2020\/05\/38b2ea076283285d934913c395863eb1.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/p>\n<p>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/avito\/blog\/500678\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0412 \u0430\u043f\u0440\u0435\u043b\u0435 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b \u0410\u0432\u0438\u0442\u043e \u0441\u043e\u0431\u0438\u0440\u0430\u043b\u0438\u0441\u044c \u043d\u0430&nbsp;\u043e\u043d\u043b\u0430\u0439\u043d-\u043f\u043e\u0441\u0438\u0434\u0435\u043b\u043a\u0438 \u0441 \u0433\u043b\u0430\u0432\u043d\u044b\u043c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c ClickHouse \u0410\u043b\u0435\u043a\u0441\u0435\u0435\u043c \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432\u044b\u043c \u0438 \u041a\u0438\u0440\u0438\u043b\u043b\u043e\u043c \u0428\u0432\u0430\u043a\u043e\u0432\u044b\u043c, Golang-\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c \u0438\u0437&nbsp;\u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Integros. \u041e\u0431\u0441\u0443\u0436\u0434\u0430\u043b\u0438, \u043a\u0430\u043a \u043c\u044b \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u0441\u0438\u0441\u0442\u0435\u043c\u0443 \u0443\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0431\u0430\u0437\u0430\u043c\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 \u0438 \u043a\u0430\u043a\u0438\u0435 \u0441\u043b\u043e\u0436\u043d\u043e\u0441\u0442\u0438 \u0443&nbsp;\u043d\u0430\u0441 \u0432\u043e\u0437\u043d\u0438\u043a\u0430\u044e\u0442. \u041f\u043e&nbsp;\u043c\u043e\u0442\u0438\u0432\u0430\u043c \u0432\u0441\u0442\u0440\u0435\u0447\u0438 \u043c\u044b \u0441\u043e\u0431\u0440\u0430\u043b\u0438 \u0441\u0442\u0430\u0442\u044c\u044e \u0441&nbsp;\u043e\u0442\u0432\u0435\u0442\u0430\u043c\u0438 \u044d\u043a\u0441\u043f\u0435\u0440\u0442\u043e\u0432 \u043d\u0430&nbsp;\u043d\u0430\u0448\u0438 \u0438 \u0437\u0440\u0438\u0442\u0435\u043b\u044c\u0441\u043a\u0438\u0435 \u0432\u043e\u043f\u0440\u043e\u0441\u044b \u043f\u0440\u043e&nbsp;\u0431\u044d\u043a\u0430\u043f\u044b, \u0440\u0435\u0448\u0430\u0440\u0434\u0438\u043d\u0433 \u0434\u0430\u043d\u043d\u044b\u0445, \u0432\u043d\u0435\u0448\u043d\u0438\u0435 \u0441\u043b\u043e\u0432\u0430\u0440\u0438, Golang-\u0434\u0440\u0430\u0439\u0432\u0435\u0440 \u0438 \u043e\u0431\u043d\u043e\u0432\u043b\u0435\u043d\u0438\u0435 \u0432\u0435\u0440\u0441\u0438\u0439 ClickHouse. \u041e\u043d\u0430 \u043c\u043e\u0436\u0435\u0442 \u0431\u044b\u0442\u044c [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":80776,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-80775","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.0.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0412 \u0430\u043f\u0440\u0435\u043b\u0435 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b \u0410\u0432\u0438\u0442\u043e \u0441\u043e\u0431\u0438\u0440\u0430\u043b\u0438\u0441\u044c \u043d\u0430 \u043e\u043d\u043b\u0430\u0439\u043d-\u043f\u043e\u0441\u0438\u0434\u0435\u043b\u043a\u0438 \u0441 \u0433\u043b\u0430\u0432\u043d\u044b\u043c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c ClickHouse \u0410\u043b\u0435\u043a\u0441\u0435\u0435\u043c \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432\u044b\u043c \u0438 \u041a\u0438\u0440\u0438\u043b\u043b\u043e\u043c \u0428\u0432\u0430\u043a\u043e\u0432\u044b\u043c, Golang-\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Integros.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.0.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47ClickHouse \u0434\u043b\u044f \u043f\u0440\u043e\u0434\u0432\u0438\u043d\u0443\u0442\u044b\u0445 \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u0435\u0439 \u0432 \u0432\u043e\u043f\u0440\u043e\u0441\u0430\u0445 \u0438 \u043e\u0442\u0432\u0435\u0442\u0430\u0445 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0412 \u0430\u043f\u0440\u0435\u043b\u0435 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b \u0410\u0432\u0438\u0442\u043e \u0441\u043e\u0431\u0438\u0440\u0430\u043b\u0438\u0441\u044c \u043d\u0430 \u043e\u043d\u043b\u0430\u0439\u043d-\u043f\u043e\u0441\u0438\u0434\u0435\u043b\u043a\u0438 \u0441 \u0433\u043b\u0430\u0432\u043d\u044b\u043c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c ClickHouse \u0410\u043b\u0435\u043a\u0441\u0435\u0435\u043c \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432\u044b\u043c \u0438 \u041a\u0438\u0440\u0438\u043b\u043b\u043e\u043c \u0428\u0432\u0430\u043a\u043e\u0432\u044b\u043c, Golang-\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Integros.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-05-08T11:42:47+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-05-08T11:42:47+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47ClickHouse p\u00ebr p\u00ebrdoruesit e avancuar n\u00eb pyetje dhe p\u00ebrgjigje | ProHoster","description":"N\u00eb prill, inxhinier\u00ebt e Avito ishin duke u p\u00ebrgatitur p\u00ebr takime online me zhvilluesin kryesor t\u00eb ClickHouse, Aleksej Milovidov, dhe Kirill Shvakov, zhvilluesin Golang t\u00eb kompanis\u00eb Integros.","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47ClickHouse \u0434\u043b\u044f \u043f\u0440\u043e\u0434\u0432\u0438\u043d\u0443\u0442\u044b\u0445 \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u0435\u0439 \u0432 \u0432\u043e\u043f\u0440\u043e\u0441\u0430\u0445 \u0438 \u043e\u0442\u0432\u0435\u0442\u0430\u0445 | ProHoster","og:description":"\u0412 \u0430\u043f\u0440\u0435\u043b\u0435 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b \u0410\u0432\u0438\u0442\u043e \u0441\u043e\u0431\u0438\u0440\u0430\u043b\u0438\u0441\u044c \u043d\u0430 \u043e\u043d\u043b\u0430\u0439\u043d-\u043f\u043e\u0441\u0438\u0434\u0435\u043b\u043a\u0438 \u0441 \u0433\u043b\u0430\u0432\u043d\u044b\u043c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c ClickHouse \u0410\u043b\u0435\u043a\u0441\u0435\u0435\u043c \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432\u044b\u043c \u0438 \u041a\u0438\u0440\u0438\u043b\u043b\u043e\u043c \u0428\u0432\u0430\u043a\u043e\u0432\u044b\u043c, Golang-\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Integros.","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-05-08T11:42:47+00:00","article:modified_time":"2020-05-08T11:42:47+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"80775","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 16:11:22","updated":"2022-09-28 05:48:13","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/80775","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=80775"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/80775\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/80776"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=80775"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=80775"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=80775"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}