{"id":80775,"date":"2020-05-08T13:42:47","date_gmt":"2020-05-08T11:42:47","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah"},"modified":"2020-05-08T13:42:47","modified_gmt":"2020-05-08T11:42:47","slug":"clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah","title":{"rendered":"ClickHouse p\u00ebr p\u00ebrdoruesit e avancuar n\u00eb pyetje dhe p\u00ebrgjigje","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>N\u00eb prill, inxhinier\u00ebt e Avito po planifikonin nj\u00eb takim online me krye-uit\u00ebsin e ClickHouse, Aleksej Milovidov, dhe Kirill Shvakov, zhvilluesin Golang nga kompania Integros. Diskutuan si e p\u00ebrdorim sistemin e menaxhimit t\u00eb bazave t\u00eb t\u00eb dh\u00ebnave dhe cilat v\u00ebshtir\u00ebsi kemi. <\/p>\n<p><\/p>\n<p>P\u00ebr motivet e takimit, ne grumbulluam nj\u00eb artikull me p\u00ebrgjigjet e ekspert\u00ebve n\u00eb pyetjet tona dhe ato t\u00eb shikuesve rreth kopjeve rezerv\u00eb, ri-shp\u00ebrndarjes s\u00eb t\u00eb dh\u00ebnave, fjalor\u00ebve t\u00eb jasht\u00ebm, drejtuesit Golang dhe p\u00ebrdit\u00ebsimeve t\u00eb versioneve t\u00eb ClickHouse. Ky artikull mund t\u00eb jet\u00eb i dobish\u00ebm p\u00ebr zhvilluesit q\u00eb tashm\u00eb punojn\u00eb aktivisht me DBMS t\u00eb 'Yandex' dhe jan\u00eb t\u00eb interesuar p\u00ebr t\u00eb tashmen dhe t\u00eb ardhmen e saj. Si rregull, p\u00ebrgjigjet jan\u00eb nga Aleksej Milovidov, p\u00ebrve\u00e7 n\u00ebse tjet\u00ebr gj\u00eb \u00ebsht\u00eb shkruar. <\/p>\n<p><\/p>\n<p>Kujdes, pas k\u00ebtij titulli ka shum\u00eb tekst. Shpresojm\u00eb se p\u00ebrmbajtja me pyetje do t'ju ndihmoj\u00eb p\u00ebr t'u orientuar.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"ClickHouse p\u00ebr p\u00ebrdoruesit e avancuar n\u00eb pyetje dhe p\u00ebrgjigje\" src=\"\/wp-content\/uploads\/2020\/05\/242b1d8d002fe115614435c242297fd2.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2 id=\"soderzhanie\">P\u00ebrmbajtja<\/h2>\n<p><\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"#old-data\">ClickHouse vazhdon t\u00eb p\u00ebrdit\u00ebsohet, nd\u00ebrsa t\u00eb dh\u00ebnat tona \u2013 jo. \u00c7far\u00eb t\u00eb b\u00ebjm\u00eb me k\u00ebt\u00eb?<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#backup-best-practicies\">Cilat jan\u00eb praktikat m\u00eb t\u00eb mira p\u00ebr kopjen rezerv\u00eb t\u00eb t\u00eb dh\u00ebnave nga ClickHouse n\u00eb k\u00ebto momente?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#replication\">A do t\u00eb jet\u00eb e mundur t\u00eb organizohet nj\u00eb vones\u00eb e kontrolluar e replika n\u00eb val\u00eb?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#soooo-changeable\">\u00c7far\u00eb t\u00eb b\u00ebjm\u00eb n\u00ebse struktura e tabel\u00ebs ka ndryshuar?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#resharding-best-practices\">Cilat jan\u00eb praktikat m\u00eb t\u00eb mira aktualisht n\u00eb ri-shp\u00ebrndarjen e t\u00eb dh\u00ebnave?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#clickhouse-copier\">N\u00eb ClickHouse ka nj\u00eb utilitar t\u00eb quajtur clickhouse-copier. Mund t\u00eb na tregoni p\u00ebr t\u00eb?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#resharding-tool\">Kishit nj\u00eb gj\u00eb pilot q\u00eb quhej ri-shp\u00ebrndarje. \u00c7far\u00eb ndodhi me t\u00eb?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#move-to-slow-disk\">A mund t\u00eb bashkohen t\u00eb gjitha pjes\u00ebt e t\u00eb dh\u00ebnave para se t\u00eb transferohen n\u00eb disqe t\u00eb ngadalt\u00eb?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#up-to-date\">Si t\u00eb migrohet n\u00eb versione t\u00eb reja t\u00eb ClickHouse, n\u00ebse nuk ka mund\u00ebsi ta kontrolloni paraprakisht p\u00ebrkompatibilitetin?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#kill-query\">Kill query duhet t\u00eb vras\u00eb \u0437\u0430\u043f\u0440\u043e\u0441\u044b, por ai nuk e b\u00ebn k\u00ebt\u00eb. Pse?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#reading-time\">Si t\u00eb llogaritet koha e p\u00ebrgjigjes me nj\u00eb ngarkes\u00eb leximi?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#pimp-my-clickhouse\">\u00c7far\u00eb duhet t\u00eb rregullohet n\u00eb ClickHouse q\u00eb t\u00eb ket\u00eb m\u00eb shum\u00eb t\u00eb dh\u00ebna n\u00eb cache?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#storage-configuration\">Si mund t\u00eb konfigurojm\u00eb storage_configuration p\u00ebr ruajtjen n\u00eb memorie?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#low-cardinality\">Derisa deri n\u00eb sa vlera unike \u00ebsht\u00eb efektive Low Cardinality?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#fulltext-search\">Cilat jan\u00eb praktikat m\u00eb t\u00eb mira p\u00ebr k\u00ebrkimin me tekst t\u00eb plot\u00eb n\u00eb nj\u00eb tabel\u00eb me pes\u00eb miliard\u00eb rreshta?<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#hello-and-welcome\">Si t\u00eb organizohet m\u00eb mir\u00eb aksesin n\u00eb ClickHouse p\u00ebr nj\u00eb num\u00ebr t\u00eb madh p\u00ebrdoruesish?<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#smorgasbord\">A mund t\u00eb jepen rezultatet e nj\u00eb k\u00ebrkese n\u00eb dhjet\u00eb klient\u00eb?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#asynchronous\">Si t\u00eb merret me operacionet asinkrone dhe pamjet e materializuara?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#dashboard\">N\u00eb ClickHouse ka shum\u00eb regjistrime. Si mund t\u00eb shoh t\u00eb gjitha ato q\u00eb ndodhin me serverin n\u00eb \u00e7do moment?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#zen\">Si t\u00eb ndikosh n\u00eb bashkimet, q\u00eb serveri t\u00eb mos bjer\u00eb n\u00eb OOM?<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#go\">Si do t\u00eb zhvillohet drejtuesi Golang p\u00ebr ClickHouse?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#lazy-load\">Fjalori i jasht\u00ebm nuk ngrihet pas rinisjes me cil\u00ebsimin e aktivizuar lazy_load. \u00c7far\u00eb t\u00eb b\u00ebjm\u00eb?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#reload-dictionaries\">Si t\u00eb merremi me faktin se system reload dictionaries nuk ngarkohet asnj\u00eb nga shum\u00eb fjalor\u00ebt, n\u00ebse t\u00eb pakt\u00ebn nj\u00eb prej tyre d\u00ebshton me nj\u00eb gabim?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#connection\">A \u00ebsht\u00eb nj\u00eb m\u00ebnyr\u00eb p\u00ebr t\u00eb konfiguruar akreditimet n\u00eb&nbsp;konfigurimin e ClickHouse, por pa i ekspozuar ato gjat\u00eb gabimeve?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"#zoom-backgrounds\">Bonus: sfondet p\u00ebr Zoom me&nbsp;takimet<\/a><\/noindex><\/li>\n<\/ul>\n<p><\/p>\n<blockquote><p>N\u00ebse nuk d\u00ebshironi t\u00eb lexoni tekstin, mund t\u00eb shikoni regjistrimin e mbledhjeve <noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=n1tm4j4W8ZQ&amp;t=8147s\">n\u00eb&nbsp;kanalin ton\u00eb n\u00eb YouTube<\/a><\/noindex>. Koha e kodit - n\u00eb&nbsp;komentin e par\u00eb n\u00ebn&nbsp;video.<\/p><\/blockquote>\n<p><\/p>\n<h2 id=\"anchorold-dataanchorclickhouse-postoyanno-obnovlyaetsya-a-nashi-dannyenbsp-net-chto-snbspetim-delat\"><noindex><a rel=\"nofollow\" name=\"old-data\"><\/a><\/noindex>ClickHouse vazhdon t\u00eb p\u00ebrmir\u00ebsohet, nd\u00ebrsa t\u00eb dh\u00ebnat tona&nbsp;\u2014 jo. \u00c7far\u00eb t\u00eb b\u00ebjm\u00eb me k\u00ebt\u00eb?<\/h2>\n<p><\/p>\n<blockquote><p>ClickHouse vazhdon t\u00eb p\u00ebrmir\u00ebsohet, nd\u00ebrsa t\u00eb dh\u00ebnat tona, t\u00eb cilat jan\u00eb p\u00ebrpunuar me optimize final, nuk p\u00ebrdit\u00ebsohen dhe mbahen n\u00eb&nbsp;kopje rezerv\u00eb. <\/p>\n<p>Supozoni se na ndodhi ndonj\u00eb problem dhe t\u00eb dh\u00ebnat u humb\u00ebn. Vendos\u00ebm t\u00eb rikuperohemi dhe doli se segmentet e vjetra, t\u00eb cilat jan\u00eb ruajtur n\u00eb&nbsp;serverat e kopjave rezerv\u00eb, jan\u00eb shum\u00eb t\u00eb ndryshme nga&nbsp;versioni aktual i ClickHouse. \u00c7far\u00eb t\u00eb b\u00ebjm\u00eb n\u00eb nj\u00eb situat\u00eb t\u00eb till\u00eb, dhe a \u00ebsht\u00eb kjo e mundur?<\/p><\/blockquote>\n<p>Situata, n\u00eb t\u00eb cil\u00ebn rikuperoni t\u00eb dh\u00ebnat nga&nbsp;kopja rezerv\u00eb n\u00eb nj\u00eb format t\u00eb vjet\u00ebr dhe ato nuk lidhen n\u00eb versionin e ri, \u00ebsht\u00eb e pamundur. Ne sigurohemi q\u00eb formati i t\u00eb dh\u00ebnave n\u00eb&nbsp;ClickHouse t\u00eb mbetet gjithmon\u00eb i p\u00ebrputhsh\u00ebm me versionet e m\u00ebparshme. Kjo \u00ebsht\u00eb shum\u00eb m\u00eb e r\u00ebnd\u00ebsishme se p\u00ebrputhshm\u00ebria e funksionalitetit, n\u00ebse sjellja e ndonj\u00eb funksioni t\u00eb p\u00ebrdorur shpesh ka ndryshuar. T\u00eb dh\u00ebnat, t\u00eb cilat ruhen n\u00eb&nbsp;diske, versioni i ri i ClickHouse gjithmon\u00eb duhet t\u00eb jet\u00eb n\u00eb gjendje t'i lexoj\u00eb. Ky \u00ebsht\u00eb nj\u00eb ligj. <\/p>\n<p><\/p>\n<h2 id=\"anchorbackup-best-practiciesanchorkakie-luchshie-praktiki-est-nanbspdannyy-moment-ponbsprezervnomu-kopirovaniyu-dannyh-iznbspclickhouse\"><noindex><a rel=\"nofollow\" name=\"backup-best-practicies\"><\/a><\/noindex>Cilat jan\u00eb praktikat m\u00eb t\u00eb mira p\u00ebr kopjen rezerv\u00eb t\u00eb t\u00eb dh\u00ebnave nga ClickHouse n\u00eb k\u00ebto momente?<\/h2>\n<p><\/p>\n<blockquote><p>Si t\u00eb b\u00ebjm\u00eb kopje rezerv\u00eb me&nbsp;merakun se kemi operacione optimize final, nj\u00eb baz\u00eb t\u00eb dh\u00ebnash t\u00eb madhe me&nbsp;terabajt\u00eb, dhe t\u00eb dh\u00ebna, t\u00eb cilat p\u00ebrdit\u00ebsohen, le t\u00eb themi, p\u00ebr&nbsp;tre dit\u00ebt e fundit, dhe m\u00eb pas me to nuk ndodhin procedura t\u00eb tjera? <\/p>\n<p>Mund t\u00eb krijojm\u00eb nj\u00eb zgjidhje ton\u00ebn dhe t\u00eb shkruajm\u00eb n\u00eb&nbsp;Bash: mbledh k\u00ebshtu dhe k\u00ebshtu k\u00ebto kopje rezerv\u00eb. Ndoshta nuk ka nevoj\u00eb p\u00ebr asnj\u00eb zgjidhje ad-hoc, dhe bi\u00e7ikleta \u00ebsht\u00eb shpikur tashm\u00eb? <\/p><\/blockquote>\n<p>Fillimisht p\u00ebr sa i p\u00ebrket praktikave m\u00eb t\u00eb mira. Koleget e mia gjithmon\u00eb rekomandojn\u00eb n\u00eb p\u00ebrgjigje t\u00eb pyetjeve rreth&nbsp;kopjeve t\u00eb kujtojn\u00eb sh\u00ebrbimin \"Yandex.Cloud\", ku kjo detyr\u00eb \u00ebsht\u00eb tashm\u00eb zgjidhur. Pra, p\u00ebrdoreni at\u00eb n\u00ebse ka nj\u00eb mund\u00ebsi. <\/p>\n<p><\/p>\n<p>Nuk ka nj\u00eb zgjidhje t\u00eb plot\u00eb, t\u00eb integruar nj\u00ebqind p\u00ebr qind n\u00eb ClickHouse, p\u00ebr kopje rezerv\u00eb. Ekzistojn\u00eb disa modele, t\u00eb cilat mund t\u00eb p\u00ebrdoren. P\u00ebr t\u00eb marr\u00eb nj\u00eb zgjidhje t\u00eb plot\u00eb, do t\u00eb duhet t\u00eb punoni pak manualisht, ose t\u00eb krijoni mb\u00ebshtjellje n\u00eb form\u00ebn e skenar\u00ebve.<\/p>\n<p><\/p>\n<p>Do t\u00eb filloj me zgjidhjet m\u00eb t\u00eb thjeshta dhe do t\u00eb p\u00ebrfundoj me ato m\u00eb komplekse, n\u00eb var\u00ebsi t\u00eb sasis\u00eb s\u00eb t\u00eb dh\u00ebnave dhe madh\u00ebsis\u00eb s\u00eb klasterit. Sa m\u00eb i madh t\u00eb jet\u00eb klasteri, aq m\u00eb e v\u00ebshtir\u00eb b\u00ebhet zgjidhja.<\/p>\n<p><\/p>\n<p>N\u00ebse tabela me t\u00eb dh\u00ebna z\u00eb vet\u00ebm disa gigabajt, prapa mund t\u00eb b\u00ebhet k\u00ebshtu: <\/p>\n<p><\/p>\n<ol>\n<li>Ruani definicionet e tabelave, pra metadata \u2014 <strong>show create table<\/strong>.<\/li>\n<li>B\u00ebni nj\u00eb dump me ndihm\u00ebn e klientit ClickHouse \u2014 <strong>select<\/strong> * <strong>from table<\/strong> n\u00eb nj\u00eb skedar. N\u00ebse e lini p\u00ebr parazgjedhje, do t\u00eb merrni nj\u00eb skedar n\u00eb formatin TabSeparated. N\u00ebse d\u00ebshironi di\u00e7ka m\u00eb efikase, mund ta b\u00ebni n\u00eb formatin Native. <\/li>\n<\/ol>\n<p><\/p>\n<p>N\u00ebse sasia e t\u00eb dh\u00ebnave \u00ebsht\u00eb m\u00eb e madhe, prapa do t\u00eb marr\u00eb m\u00eb shum\u00eb koh\u00eb dhe shum\u00eb hap\u00ebsir\u00eb. Ky quhet backup logjik, ai nuk \u00ebsht\u00eb i lidhur me formatin e t\u00eb dh\u00ebnave ClickHouse. N\u00ebse e keni, n\u00eb rastin m\u00eb ekstreme do t\u00eb mund ta merrni backup-in dhe ta ngarkoni n\u00eb MySQL p\u00ebr rikuperim. <\/p>\n<p><\/p>\n<p>P\u00ebr raste m\u00eb t\u00eb avancuara, ClickHouse ka integruar mund\u00ebsin\u00eb p\u00ebr t\u00eb krijuar snapshot partitions n\u00eb sistemin lokal t\u00eb skedar\u00ebve. Kjo mund\u00ebsi \u00ebsht\u00eb e disponueshme n\u00eb form\u00ebn e nj\u00eb k\u00ebrkese <strong>alter table freeze partition<\/strong>. Ose thjesht <strong>alter table freeze<\/strong> \u2014 kjo \u00ebsht\u00eb nj\u00eb snapshot i t\u00eb gjith\u00eb tabel\u00ebs. <\/p>\n<p><\/p>\n<p>Snapshot-i do t\u00eb krijohet konsistent p\u00ebr nj\u00eb tabel\u00eb n\u00eb nj\u00eb shard t\u00eb vet\u00ebm, pra nuk \u00ebsht\u00eb e mundur t\u00eb krijoni nj\u00eb snapshot konsistent t\u00eb gjith\u00eb klasterit n\u00eb k\u00ebt\u00eb m\u00ebnyr\u00eb. Por p\u00ebr shumic\u00ebn e detyrave, nj\u00eb nevoj\u00eb e till\u00eb nuk ekziston, dhe \u00ebsht\u00eb mjaft t\u00eb ekzekutoni nj\u00eb k\u00ebrkes\u00eb n\u00eb \u00e7do shard dhe t\u00eb merrni nj\u00eb snapshot konsistent. Ai krijohet n\u00eb form\u00ebn e hardlink-eve dhe prandaj nuk z\u00eb hap\u00ebsir\u00eb shtes\u00eb. Pastaj, k\u00ebt\u00eb snapshot e kopjoni n\u00eb serverin e backup-it ose n\u00eb depozit\u00ebn q\u00eb p\u00ebrdorni p\u00ebr backup-et.<\/p>\n<p><\/p>\n<p>Rikthimi i nj\u00eb backup-i t\u00eb till\u00eb \u00ebsht\u00eb mjaft i thjesht\u00eb. E para \u2014 krijoni tabela sipas p\u00ebrcaktimet ekzistuese t\u00eb tabelave. M\u00eb pas, kopjoni snapshot-et e ruajtura t\u00eb particioneve n\u00eb Directory-Detached p\u00ebr ato tabela dhe ekzekutoni k\u00ebrkes\u00ebn <strong>attach partition<\/strong>. Ky zgjidhje \u00ebsht\u00eb e p\u00ebrshtatshme p\u00ebr sasi t\u00eb m\u00ebdha t\u00eb t\u00eb dh\u00ebnave. <\/p>\n<p><\/p>\n<p>Ndonj\u00ebher\u00eb k\u00ebrkohet di\u00e7ka edhe m\u00eb t\u00eb avancuar \u2014 n\u00eb ato raste kur keni dhjet\u00ebra ose madje qindra terabajt n\u00eb \u00e7do server dhe qindra servera. K\u00ebtu ka nj\u00eb zgjidhje q\u00eb e kam par\u00eb nga koleg\u00ebt e \"Yandex.Metrica\". Nuk do ta rekomandoja k\u00ebt\u00eb p\u00ebr t\u00eb gjith\u00eb \u2014 lexoni dhe vendosni vet\u00eb n\u00ebse \u00ebsht\u00eb e p\u00ebrshtatshme apo jo. <\/p>\n<p><\/p>\n<p>Fillimisht, duhet t\u00eb krijoni disa server\u00eb me&nbsp;raftet t\u00eb m\u00ebdha t\u00eb disk\u00ebve. M\u00eb pas, n\u00eb&nbsp;k\u00ebta server\u00eb do t\u00eb ngrem\u00eb disa server\u00eb ClickHouse dhe do t'i konfigurojm\u00eb q\u00eb t\u00eb funksionojn\u00eb si nj\u00eb replik\u00eb tjet\u00ebr p\u00ebr&nbsp;shard-at e nj\u00ebjta. M\u00eb pas, do t\u00eb p\u00ebrdorim nj\u00eb sistem skedar\u00ebsh ose nj\u00eb mjet q\u00eb lejon krijimin e snapshot-eve. Ka dy mund\u00ebsi k\u00ebtu. Opcioni i par\u00eb&nbsp;\u2014 jan\u00eb snapshot-et LVM, opcioni i dyt\u00eb&nbsp;\u2014 ZFS n\u00eb&nbsp;Linux. <\/p>\n<p><\/p>\n<p>Pas k\u00ebsaj, \u00e7do dit\u00eb duhet t\u00eb krijoni nj\u00eb snapshot, i cili do t\u00eb ruhet dhe do t\u00eb konsumoj\u00eb ndonj\u00eb hap\u00ebsir\u00eb. Sigurisht, n\u00ebse t\u00eb dh\u00ebnat ndryshojn\u00eb, koha, hapsira do t\u00eb rritet. Ky snapshot mund t\u00eb nxirret n\u00eb&nbsp;\u00e7do moment dhe t\u00eb rikthehen t\u00eb dh\u00ebnat, nj\u00eb zgjidhje e \u00e7uditshme. Plus, duhet gjithashtu t\u00eb kufizoni k\u00ebto replika n\u00eb&nbsp;konfig, q\u00eb t\u00eb mos p\u00ebrpiqen t\u00eb b\u00ebhen lider\u00eb.<\/p>\n<p><\/p>\n<h2 id=\"anchorreplicationanchormozhno-li-budet-organizovat-kontroliruemoe-otstavanie-replik-vnbspvalah\"><noindex><a rel=\"nofollow\" name=\"replication\"><\/a><\/noindex>A do t\u00eb jet\u00eb e mundur t\u00eb organizohet nj\u00eb vones\u00eb e kontrolluar e replika n\u00eb val\u00eb?<\/h2>\n<p><\/p>\n<blockquote><p>K\u00ebt\u00eb vit, planifikoni t\u00eb b\u00ebni val\u00eb n\u00eb&nbsp;ClickHouse. A do t\u00eb jet\u00eb e mundur t\u00eb organizoni nj\u00eb vones\u00eb t\u00eb kontrolluar t\u00eb replikave brenda tyre? Ne do t\u00eb donim me ndihm\u00ebn e tij t\u00eb sigurohemi nga&nbsp;skenar\u00ebt negativ\u00eb me alterant dhe ndryshime t\u00eb tjera. <\/p>\n<p>A \u00ebsht\u00eb e mundur t\u00eb b\u00ebni ndonj\u00eb rollback p\u00ebr alterant? P\u00ebr shembull, n\u00eb&nbsp;val\u00ebn ekzistuese t\u00eb themi se deri n\u00eb&nbsp;at\u00eb moment t\u00eb aplikoni ndryshimet, dhe nga&nbsp;aty t\u00eb ndaloni aplikimin e ndryshimeve?<\/p>\n<p>N\u00ebse n\u00eb&nbsp;klustin ton\u00eb erdhi nj\u00eb komand\u00eb dhe e prishi at\u00eb, ne kemi nj\u00eb replik\u00eb t\u00eb kusht\u00ebzuar me vones\u00eb nj\u00eb or\u00eb, ku mund t\u00eb themi se le t\u00eb p\u00ebrdorim sakt\u00ebsisht at\u00eb n\u00eb&nbsp;momentin aktual, por nuk do t\u00eb aplikojm\u00eb nd\u00ebrrimet e fundit t\u00eb dhjet\u00eb minutave n\u00eb t\u00eb? <\/p><\/blockquote>\n<p>S\u00eb pari p\u00ebr&nbsp;vonimin e kontrolluar t\u00eb replikave. Nj\u00eb k\u00ebrkes\u00eb e till\u00eb nga&nbsp;p\u00ebrdoruesit ishte, dhe ne krijuam nj\u00eb problem n\u00eb&nbsp;GitHub me k\u00ebrkes\u00ebn: 'N\u00ebse dikujt i nevojitet, vendosni nj\u00eb p\u00eblqim, vendosni nj\u00eb zem\u00ebr'. Askush nuk e vendosi, dhe problemi u mbyll. Megjithat\u00eb, tashm\u00eb mund t\u00eb merrni nj\u00eb mund\u00ebsi t\u00eb till\u00eb, duke konfiguruar ClickHouse. Megjithat\u00eb, vet\u00ebm q\u00eb nga&nbsp;versioni&nbsp;20.3.<\/p>\n<p><\/p>\n<p>ClickHouse vazhdimisht prapa prodhon p\u00ebrzierje t\u00eb dh\u00ebnash&nbsp;\u2014 merge. Kur merge \u00ebsht\u00eb prodhuar, disa grupe t\u00eb dh\u00ebnash z\u00ebvend\u00ebsohen me nj\u00eb grup m\u00eb t\u00eb madh. N\u00eb k\u00ebt\u00eb rast, grupet e dh\u00ebnash q\u00eb ishin m\u00eb p\u00ebrpara vazhdojn\u00eb t\u00eb mbeten n\u00eb&nbsp;disk p\u00ebr nj\u00eb koh\u00eb t\u00eb caktuar.<\/p>\n<p><\/p>\n<p>S\u00eb pari, ato vazhdojn\u00eb t\u00eb ruhen deri n\u00eb momentin q\u00eb ka k\u00ebrkesa select q\u00eb i p\u00ebrdorin, p\u00ebr t\u00eb siguruar nj\u00eb funksionim pa bllokim. K\u00ebrkesat select leht\u00ebsisht lexojn\u00eb nga cop\u00ebza t\u00eb vjetra.<\/p>\n<p><\/p>\n<p>S\u00eb dyti, ka edhe nj\u00eb kufi kohor - cop\u00ebzat e vjetra t\u00eb t\u00eb dh\u00ebnave q\u00ebndrojn\u00eb n\u00eb disk p\u00ebr tet\u00eb minuta. K\u00ebto tet\u00eb minuta mund t\u00eb konfigurohen dhe t\u00eb shnd\u00ebrrohen madje n\u00eb nj\u00eb dit\u00eb. Kjo do t\u00eb kushtoj\u00eb hap\u00ebsir\u00eb n\u00eb disk: n\u00eb var\u00ebsi t\u00eb fluksit t\u00eb t\u00eb dh\u00ebnave, mund t\u00eb ndodh\u00eb q\u00eb p\u00ebr dit\u00ebn e fundit t\u00eb dh\u00ebnat jo vet\u00ebm q\u00eb do t\u00eb dyfishohen, por mund t\u00eb b\u00ebhen deri n\u00eb pes\u00eb her\u00eb m\u00eb shum\u00eb. Por, n\u00ebse ndodh ndonj\u00eb problem serioz, mund t\u00eb ndaloni serverin ClickHouse dhe t\u00eb zgjidhni gjith\u00e7ka.<\/p>\n<p><\/p>\n<p>Tani lind pyetja, si e mbron k\u00ebt\u00eb nga alterat. K\u00ebtu vlen t\u00eb shikojm\u00eb thell\u00eb, sepse n\u00eb versionet e vjetra t\u00eb ClickHouse, alteri funksiononte n\u00eb at\u00eb m\u00ebnyr\u00eb q\u00eb thjesht ndryshonte t\u00eb drejtp\u00ebrdrejt cop\u00ebzat. Ka nj\u00eb cop\u00eb t\u00eb dh\u00ebnash me disa skedar\u00eb, dhe ne b\u00ebjm\u00eb, p\u00ebr shembull, <strong>alter drop column<\/strong>. At\u00ebher\u00eb ky kolon\u00eb fiziksht fshihet nga t\u00eb gjitha cop\u00ebzat.<\/p>\n<p><\/p>\n<p>Por duke filluar nga versioni 20.3, mekanizmi i alterave \u00ebsht\u00eb ndryshuar plot\u00ebsisht, dhe tani cop\u00ebzat e t\u00eb dh\u00ebnave jan\u00eb gjithmon\u00eb imutabile. Ato nuk ndryshojn\u00eb fare - alterat tani punojn\u00eb m\u00eb ose m\u00eb pak si bashkimet. N\u00eb vend q\u00eb t\u00eb ndryshonim nj\u00eb cop\u00eb n\u00eb vend, ne krijojm\u00eb nj\u00eb t\u00eb re. N\u00eb cop\u00ebn e re, skedar\u00ebt q\u00eb nuk jan\u00eb ndryshuar b\u00ebhen hardlinks, dhe, n\u00ebse kemi fshir\u00eb ndonj\u00eb kolon\u00eb, ai do t\u00eb mungoj\u00eb thjesht n\u00eb cop\u00ebn e re. Cop\u00ebza e vjet\u00ebr do t\u00eb fshihet automatikisht pas tet\u00eb minutash, dhe k\u00ebtu mund t\u00eb rregullojm\u00eb cil\u00ebsimet e p\u00ebrmendura m\u00eb sip\u00ebr. <\/p>\n<p><\/p>\n<p>E nj\u00ebjta gj\u00eb vlen p\u00ebr alteret e tipit mutacione. Kur ju b\u00ebni <strong>alter delete<\/strong> ose <strong>alter update<\/strong>, ai nuk e ndryshon cop\u00ebn, por krijon nj\u00eb t\u00eb re. Pastaj fshin t\u00eb vjetr\u00ebn.<\/p>\n<p><\/p>\n<h2 id=\"anchorsoooo-changeableanchorkak-byt-esli-struktura-tablicy-pomenyalas\"><noindex><a rel=\"nofollow\" name=\"soooo-changeable\"><\/a><\/noindex>\u00c7far\u00eb t\u00eb b\u00ebjm\u00eb n\u00ebse struktura e tabel\u00ebs ka ndryshuar?<\/h2>\n<p><\/p>\n<blockquote><p>Si t\u00eb ngrem\u00eb nj\u00eb kopje rezerv\u00eb q\u00eb \u00ebsht\u00eb b\u00ebr\u00eb me nj\u00eb skem\u00eb t\u00eb vjet\u00ebr? Dhe nj\u00eb pyetje tjet\u00ebr p\u00ebr rastin me snapshotet dhe mjetet e sistemit t\u00eb skedar\u00ebve. A funksionon Btrfs k\u00ebtu n\u00eb vend t\u00eb ZFS n\u00eb Linux LVM?<\/p><\/blockquote>\n<p>N\u00ebse b\u00ebni <strong>attach partition<\/strong> n\u00ebse ndahen me nj\u00eb struktur\u00eb t\u00eb ndryshme, at\u00ebher\u00eb ClickHouse do t'ju thot\u00eb se kjo nuk lejohet. Zgjidhja \u00ebsht\u00eb k\u00ebshtu. E para - krijoni nj\u00eb tabel\u00eb t\u00eb p\u00ebrkohshme t\u00eb tipit MergeTree me struktur\u00ebn e vjet\u00ebr, lidhni t\u00eb dh\u00ebnat atje me ndihm\u00ebn e attach, b\u00ebni nj\u00eb k\u00ebrkes\u00eb alter. Pastaj mund t\u00eb kopjoni ose transferoni k\u00ebto t\u00eb dh\u00ebna dhe t\u00eb b\u00ebni p\u00ebrs\u00ebri attach, ose t\u00eb p\u00ebrdorni nj\u00eb k\u00ebrkes\u00eb. <strong>alter table move partition<\/strong>.<\/p>\n<p><\/p>\n<p>Tani \u00ebsht\u00eb pyetja e dyt\u00eb&nbsp;\u2014 a mund t\u00eb p\u00ebrdoret Btrfs. Fillimisht, n\u00ebse keni LVM, mjaftojn\u00eb snapshot-et e LVM, dhe sistemi i skedar\u00ebve mund t\u00eb jet\u00eb edhe ext4, kjo nuk ka r\u00ebnd\u00ebsi. Me Btrfs, gjith\u00e7ka varet nga p\u00ebrvoja juaj n\u00eb p\u00ebrdorimin e saj. Kjo \u00ebsht\u00eb nj\u00eb sistem i skedar\u00ebve i pjekur, por ndodhin disa dyshime se si do t\u00eb funksionoj\u00eb n\u00eb praktik\u00eb n\u00eb nj\u00eb skenar t\u00eb caktuar. Nuk do t\u00eb rekomandoja ta p\u00ebrdorni, n\u00ebse nuk keni Btrfs n\u00eb prodhim.<\/p>\n<p><\/p>\n<h2 id=\"anchorresharding-best-practicesanchorkakie-seychas-luchshie-praktiki-vnbspreshardinge-dannyh\"><noindex><a rel=\"nofollow\" name=\"resharding-best-practices\"><\/a><\/noindex>Cilat jan\u00eb praktikat m\u00eb t\u00eb mira aktualisht n\u00eb ri-shp\u00ebrndarjen e t\u00eb dh\u00ebnave?<\/h2>\n<p><\/p>\n<p>Pyetja p\u00ebr rip\u00ebrshtatjen e shard-eve \u00ebsht\u00eb e komplikuar dhe shum\u00ebdimensional. K\u00ebtu mund t\u00eb p\u00ebrgjigjemi menj\u00ebher\u00eb n\u00eb disa m\u00ebnyra. Mund t\u00eb shikoni nga nj\u00eb k\u00ebndv\u00ebshtrim dhe t\u00eb thoni k\u00ebshtu&nbsp;\u2014 n\u00eb ClickHouse nuk ka mund\u00ebsi t\u00eb integruar p\u00ebr rip\u00ebrshtatje t\u00eb shard-eve. Por kam frik\u00eb se kjo p\u00ebrgjigje nuk do t'i k\u00ebnaq\u00eb ask\u00ebnd. Prandaj, mund t\u00eb shikoni nga nj\u00eb k\u00ebndv\u00ebshtrim tjet\u00ebr dhe t\u00eb thoni se n\u00eb ClickHouse ka shum\u00eb m\u00ebnyra p\u00ebr rip\u00ebrshtatjen e t\u00eb dh\u00ebnave. <\/p>\n<p><\/p>\n<p>N\u00ebse mbaron hap\u00ebsira n\u00eb klaster ose ai nuk p\u00ebrballon ngarkes\u00ebn, ju shtoni server\u00eb t\u00eb rinj. Por k\u00ebta server\u00eb jan\u00eb t\u00eb zbraz\u00ebt p\u00ebr parazgjedhje, nuk ka t\u00eb dh\u00ebna mbi ta, nuk ka ngarkes\u00eb. Ju nevojitet t\u00eb shp\u00ebrndani t\u00eb dh\u00ebnat, n\u00eb m\u00ebnyr\u00eb q\u00eb ato t\u00eb b\u00ebhen m\u00eb uniform\u00eb n\u00eb klasterin e ri, t\u00eb zgjeruar.<\/p>\n<p><\/p>\n<p>M\u00ebnyra e par\u00eb p\u00ebr ta b\u00ebr\u00eb k\u00ebt\u00eb \u00ebsht\u00eb&nbsp;\u2014 t\u00eb kopjoni nj\u00eb pjes\u00eb t\u00eb particioneve n\u00eb server\u00ebt e rinj p\u00ebrmes nj\u00eb pyetje. <strong>alter table fetch partition<\/strong>P\u00ebr shembull, keni pasur particione sipas muajve, dhe merrni muajin e par\u00eb t\u00eb vitit 2017&nbsp;dhe e kopjoni n\u00eb serverin e ri, pastaj - muajin e tret\u00eb e kopjoni n\u00eb ndonj\u00eb server tjet\u00ebr t\u00eb ri. Dhe k\u00ebshtu b\u00ebni, derisa t\u00eb b\u00ebhet m\u00eb uniform.<\/p>\n<p><\/p>\n<p>Transferimi mund t\u00eb b\u00ebhet vet\u00ebm p\u00ebr ato particione q\u00eb nuk ndryshojn\u00eb kur shkruhen. P\u00ebr partit\u00eb e reja, do t\u00eb duhet t\u00eb \u00e7aktivizoni shkrimin, sepse transferimi i tyre nuk \u00ebsht\u00eb atomar. N\u00eb t\u00eb kund\u00ebrt, do t\u00eb merrni kopje t\u00eb dyfishta ose mungesa n\u00eb t\u00eb dh\u00ebna. Megjithat\u00eb, kjo m\u00ebnyr\u00eb \u00ebsht\u00eb praktike dhe funksionon mjaft efektivisht. Particionet e kompresuara kalojn\u00eb n\u00eb rrjet, dometh\u00ebn\u00eb t\u00eb dh\u00ebnat nuk kompresohen ose riprodhohen.<\/p>\n<p><\/p>\n<p>Ky ky this metode ka nj\u00eb mang\u00ebsi, dhe ajo varet nga skema e sharding-ut, n\u00ebse e keni parashikuar k\u00ebt\u00eb skem\u00eb sharding-u, cila ka qen\u00eb \u00e7el\u00ebsi juaj i sharding-ut. N\u00eb shembullin tuaj p\u00ebr rastin me metrika, \u00e7el\u00ebsi i sharding-ut \u00ebsht\u00eb hash nga rruga. Kur ju b\u00ebni nj\u00eb select n\u00eb tabel\u00ebn Distributed, ai shkon menj\u00ebher\u00eb n\u00eb t\u00eb gjitha shard-et e klasterit dhe merr t\u00eb dh\u00ebnat nga aty. <\/p>\n<p><\/p>\n<p>Kjo do t\u00eb thot\u00eb se n\u00eb thelb nuk ka r\u00ebnd\u00ebsi p\u00ebr ju se cilat t\u00eb dh\u00ebna jan\u00eb n\u00eb cilin shard. E r\u00ebnd\u00ebsishme \u00ebsht\u00eb se t\u00eb dh\u00ebnat p\u00ebr nj\u00eb rrug\u00eb ndodhen n\u00eb nj\u00eb shard, por cila pik\u00ebrisht, nuk ka r\u00ebnd\u00ebsi. N\u00eb k\u00ebt\u00eb rast, transferimi i pjes\u00ebve t\u00eb gatshme \u00ebsht\u00eb shum\u00eb i p\u00ebrshtatsh\u00ebm, sepse gjat\u00eb k\u00ebrkesave select, ju gjithashtu \u2014 p\u00ebrpara ribashkimit dhe pas, skema e vlerave nuk ka r\u00ebnd\u00ebsi \u2014 do t\u00eb merrni t\u00eb dh\u00ebna t\u00eb plota.<\/p>\n<p><\/p>\n<p>Por ka raste dhe m\u00eb t\u00eb komplikuara. N\u00ebse n\u00eb nivelin e logjik\u00ebs s\u00eb aplikacionit ju parashikoni nj\u00eb skem\u00eb speciale sharding-u, q\u00eb ky klient ndodhet n\u00eb nj\u00eb shard t\u00eb caktuar, dhe k\u00ebrkesa mund t\u00eb d\u00ebrgohet menj\u00ebher\u00eb atje, dhe jo n\u00eb tabel\u00ebn Distributed. Ose p\u00ebrdorni nj\u00eb version t\u00eb mjaftuesh\u00ebm t\u00eb ri t\u00eb ClickHouse dhe keni aktivizuar konfigurimin <strong>optimize skip unused shards<\/strong>. N\u00eb k\u00ebt\u00eb rast, gjat\u00eb k\u00ebrkes\u00ebs select, shprehja n\u00eb seksionin where do t\u00eb analizohet, dhe do t\u00eb llogaritet se n\u00eb cilat shards duhet t\u00eb shkoj\u00eb sipas skem\u00ebs s\u00eb sharding-ut. Kjo funksionon me kushtin q\u00eb t\u00eb dh\u00ebnat t\u00eb jen\u00eb vendosur n\u00eb p\u00ebrputhje me k\u00ebt\u00eb skem\u00eb sharding-u. N\u00ebse i keni transferuar ato manualisht, p\u00ebrputhja mund t\u00eb ndryshoj\u00eb.<\/p>\n<p><\/p>\n<p>Pra, kjo \u00ebsht\u00eb m\u00ebnyra e par\u00eb. Dhe un\u00eb po pres p\u00ebrgjigjen tuaj, n\u00ebse kjo m\u00ebnyr\u00eb \u00ebsht\u00eb e p\u00ebrshtatshme, apo do t\u00eb vazhdojm\u00eb m\u00eb tej.<\/p>\n<p><\/p>\n<p><strong>Vladimir Kolobayev, administrator kryesor i sistemit n\u00eb Avito<\/strong>: Alexey, ai m\u00ebnyr\u00eb q\u00eb p\u00ebrmend\u00ebt nuk \u00ebsht\u00eb shum\u00eb e p\u00ebrshtatshme, kur duhet t\u00eb shp\u00ebrndajm\u00eb ngarkes\u00ebn p\u00ebrfshir\u00eb edhe t\u00eb lexuarit. Mund t\u00eb marrim nj\u00eb pjes\u00eb, e cila \u00ebsht\u00eb mujore dhe mund t\u00eb transferojm\u00eb muajin e kaluar n\u00eb nj\u00eb nod tjet\u00ebr, por kur t\u00eb vij\u00eb k\u00ebrkesa p\u00ebr k\u00ebto t\u00eb dh\u00ebna, ne do t\u00eb ngarkojm\u00eb vet\u00ebm at\u00eb. Dhe do t\u00eb donim t\u00eb ngarkonim t\u00eb gjith\u00eb klasterin, sepse, ndryshe, p\u00ebr nj\u00eb periudh\u00eb, e gjith\u00eb ngarkesa p\u00ebr leximin do t\u00eb p\u00ebrballohet nga dy shards.<\/p>\n<p><\/p>\n<p><strong>Alexey Milovidov:<\/strong> P\u00ebrgjigjja k\u00ebtu \u00ebsht\u00eb e \u00e7uditshme \u2014 po, keq, por mund t\u00eb funksionoj\u00eb. Do ta shpjegoj si\u00e7 duhet. Duhet t\u00eb shikoni skenarin e ngarkes\u00ebs q\u00eb ndjek t\u00eb dh\u00ebnat tuaja. N\u00ebse jan\u00eb t\u00eb dh\u00ebna monitorimi, at\u00ebher\u00eb pothuajse me siguri mund t\u00eb themi se shumica d\u00ebrrmuese e k\u00ebrkesave shkojn\u00eb p\u00ebr t\u00eb dh\u00ebn\u00eb t\u00eb fresk\u00ebta. <\/p>\n<p><\/p>\n<p>Keni vendosur server\u00eb t\u00eb rinj, keni transferuar partit\u00eb e vjetra, por gjithashtu keni ndryshuar m\u00ebnyr\u00ebn se si regjistrohen t\u00eb dh\u00ebnat e reja. T\u00eb dh\u00ebnat e reja do t\u00eb shp\u00ebrndahen n\u00eb t\u00eb gjith\u00eb klasterin. K\u00ebshtu q\u00eb, q\u00eb n\u00eb pes\u00eb minuta, k\u00ebrkesat p\u00ebr pes\u00eb minutat e fundit do t\u00eb ngarkojn\u00eb barazisht klasterin, e n\u00eb nj\u00eb dit\u00eb, k\u00ebrkesat p\u00ebr dit\u00ebt e fundit do t\u00eb ngarkojn\u00eb nj\u00eblloj klasterin. Nd\u00ebrsa k\u00ebrkesat p\u00ebr muajin e kaluar, fatkeq\u00ebsisht, do t\u00eb shkojn\u00eb vet\u00ebm n\u00eb nj\u00eb pjes\u00eb t\u00eb server\u00ebve t\u00eb klasterit.<\/p>\n<p><\/p>\n<p>Por shpesh her\u00eb nuk do t\u00eb keni k\u00ebrkesa specifikisht p\u00ebr shkurtin e vitit 2019. M\u00eb shum\u00eb mund\u00ebsi \u00ebsht\u00eb q\u00eb, n\u00ebse k\u00ebrkesat po vijn\u00eb n\u00eb vitin 2019, ato do t\u00eb jen\u00eb p\u00ebr t\u00eb gjith\u00eb vitin 2019 \u2014 p\u00ebr nj\u00eb interval t\u00eb madh kohe, dhe jo p\u00ebr nj\u00eb interval t\u00eb vog\u00ebl. Dhe k\u00ebto k\u00ebrkesa gjithashtu do t\u00eb jen\u00eb n\u00eb gjendje t\u00eb ngarkojn\u00eb klasterin barazisht. Por n\u00eb p\u00ebrgjith\u00ebsi, v\u00ebrejtja juaj \u00ebsht\u00eb plot\u00ebsisht e sakt\u00eb, q\u00eb \u00ebsht\u00eb nj\u00eb zgjidhje ad hoc q\u00eb nuk shp\u00ebrndan t\u00eb dh\u00ebnat plot\u00ebsisht baraz.<\/p>\n<p><\/p>\n<p>Kam disa pika t\u00eb tjera p\u00ebr t'iu p\u00ebrgjigjur pyetjes. Nj\u00ebra prej tyre \u00ebsht\u00eb se si mund ta b\u00ebni n\u00eb fillim planin e sharding-ut n\u00eb m\u00ebnyr\u00eb q\u00eb t\u00eb ket\u00eb m\u00eb pak dhimbje nga reshardimi. Kjo nuk \u00ebsht\u00eb gjithmon\u00eb e mundur.<\/p>\n<p><\/p>\n<p>P\u00ebr shembull, ju keni t\u00eb dh\u00ebna monitorimi. T\u00eb dh\u00ebnat e monitorimit rriten p\u00ebr tri arsye. E para \u2014 akumulimi i t\u00eb dh\u00ebnave historike. E dyta \u2014 rritja e trafikut. Dhe e treta \u2014 rritja e numrit t\u00eb gj\u00ebrave q\u00eb p\u00ebrfshihen n\u00eb monitorim. Parashikohen mikrosh\u00ebrbime dhe metrika t\u00eb reja q\u00eb duhet t\u00eb ruhen. <\/p>\n<p><\/p>\n<p>Mund t\u00eb ndodh\u00eb q\u00eb rritja m\u00eb e madhe t\u00eb lidhet me arsyen e tret\u00eb \u2014 kjo \u00ebsht\u00eb rritja e p\u00ebrdorimit t\u00eb monitorimit. Dhe n\u00eb k\u00ebt\u00eb rast, vlen t\u00eb shikohet karakteri i ngarkes\u00ebs, \u00e7far\u00eb jan\u00eb k\u00ebrkesat kryesore p\u00ebr select. K\u00ebrkesat kryesore p\u00ebr select, me shum\u00eb mund\u00ebsi, do t\u00eb vijn\u00eb nga nj\u00eb n\u00ebngrup i caktuar i metrikave.<\/p>\n<p><\/p>\n<p>P\u00ebr shembull, p\u00ebrdorimi i CPU n\u00eb disa server\u00eb nga ndonj\u00eb sh\u00ebrbim. Pra, ekziston nj\u00eb n\u00ebngrup i caktuar i \u00e7el\u00ebsve, p\u00ebrmes t\u00eb cilave po e nxirrni k\u00ebt\u00eb informacion. Dhe vet\u00eb k\u00ebrkesa p\u00ebr k\u00ebto t\u00eb dh\u00ebna, me shum\u00eb mund\u00ebsi, \u00ebsht\u00eb mjaft e thjesht\u00eb dhe ekzekutohet brenda disa dhjetra milisekondash. P\u00ebrdoret p\u00ebr sh\u00ebrbime monitorimi, p\u00ebr panel\u00eb. Shpresoj se po e kuptoj drejt.<\/p>\n<p><\/p>\n<p><strong>Vladimir Kolobaev:<\/strong> Kjo \u00ebsht\u00eb&nbsp;se \u00e7far\u00eb ne shpesh i referohemi t\u00eb dh\u00ebnave historike, pasi ne n\u00eb&nbsp;koh\u00eb reale krahasojm\u00eb pozitat aktuale me ato historike. Dhe p\u00ebr&nbsp;ne \u00ebsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb kemi qasje t\u00eb shpejt\u00eb n\u00eb&nbsp;nj\u00eb volum t\u00eb madh t\u00eb dh\u00ebnash, dhe ClickHouse e realizon k\u00ebt\u00eb n\u00eb m\u00ebnyr\u00eb t\u00eb shk\u00eblqyer.<\/p>\n<p><\/p>\n<p>Ju jeni plot\u00ebsisht t\u00eb sakt\u00eb, shumica e k\u00ebrkesave p\u00ebr&nbsp;lexim e kalojm\u00eb gjat\u00eb dit\u00ebs s\u00eb kaluar, si\u00e7 b\u00ebn \u00e7do sistem monitorimi. Por edhe ngarkesa p\u00ebr t\u00eb&nbsp;dh\u00ebnat historike \u00ebsht\u00eb e konsiderueshme. Ajo vjen kryesisht nga sistemi i alarmeve, i cili \u00e7do tridhjet\u00eb sekonda k\u00ebrkon nga ClickHouse: \"M\u00eb jep t\u00eb dh\u00ebnat p\u00ebr&nbsp;gjasht\u00eb jav\u00ebt e fundit. Tani m\u00eb ndihmo t\u00eb krijoj nj\u00eb mesatare t\u00eb l\u00ebvizshme nga ato dhe le t\u00eb krahasojm\u00eb vler\u00ebn aktuale me at\u00eb historike\". <\/p>\n<p><\/p>\n<p>Do doja t\u00eb thoshja se p\u00ebr k\u00ebto k\u00ebrkesa shum\u00eb t\u00eb fresk\u00ebta, kemi nj\u00eb tavolin\u00eb t\u00eb vog\u00ebl, ku ruajm\u00eb vet\u00ebm dy dit\u00eb t\u00eb dh\u00ebnash, dhe k\u00ebrkesat kryesore shkojn\u00eb aty. N\u00eb tavolin\u00ebn e madhe t\u00eb sharduar d\u00ebrgojm\u00eb vet\u00ebm k\u00ebrkesat e m\u00ebdha historike.<\/p>\n<p><\/p>\n<p><strong>Alexey Milovidov:<\/strong> P\u00ebr fat t\u00eb keq, ky skenar nuk aplikohet mir\u00eb, por do t'ju tregoj dy skema t\u00eb k\u00ebqija dhe t\u00eb komplikuara t\u00eb shardimit, t\u00eb cilat nuk duhet t\u00eb p\u00ebrdoren, por q\u00eb p\u00ebrdoren n\u00eb&nbsp;sh\u00ebrbimin e miqve t\u00eb mi. <\/p>\n<p><\/p>\n<p>Ka nj\u00eb klas\u00ebr kryesor me&nbsp;ngjarjet e \"Yandex.Metrika\". Ngjarjet&nbsp;s\u00eb\u00a0bashku p\u00ebrfshijn\u00eb shikimet e faqeve, klikimet dhe kalimet. Shumica e k\u00ebrkesave shkojn\u00eb p\u00ebr&nbsp;nj\u00eb faqe interneti specifike. Ju hapni sh\u00ebrbimin \"Yandex.Metrika\", keni nj\u00eb faqe \u2014 avito.ru, hyni n\u00eb&nbsp;raport, dhe b\u00ebhet nj\u00eb k\u00ebrkes\u00eb p\u00ebr faqen tuaj.<\/p>\n<p><\/p>\n<p>Por ka edhe k\u00ebrkesa t\u00eb tjera&nbsp;\u2014 analitike dhe globale, t\u00eb cilat b\u00ebhen nga analist\u00ebt e brendsh\u00ebm. P\u00ebr rastet e nevojshme p\u00ebrmend se analist\u00ebt e brendsh\u00ebm b\u00ebjn\u00eb k\u00ebrkesa vet\u00ebm p\u00ebr&nbsp;sh\u00ebrbimet e \"Yandex\". Megjithat\u00eb, edhe sh\u00ebrbimet e \"Yandex\" p\u00ebrb\u00ebjn\u00eb nj\u00eb pjes\u00eb t\u00eb konsiderueshme t\u00eb t\u00eb&nbsp;dh\u00ebnave. K\u00ebto jan\u00eb k\u00ebrkesa jo p\u00ebr&nbsp;numra specifik\u00eb, por p\u00ebr nj\u00eb filtrimin m\u00eb t\u00eb gjer\u00eb.<\/p>\n<p><\/p>\n<p>Si t\u00eb organizoni t\u00eb&nbsp;dh\u00ebnat n\u00eb m\u00ebnyr\u00eb q\u00eb t\u00eb funksiononin efikasht p\u00ebr nj\u00eb num\u00ebr dhe k\u00ebrkesat globale gjithashtu? V\u00ebshtir\u00ebsia \u00ebsht\u00eb gjithashtu se numri i k\u00ebrkesave n\u00eb&nbsp;ClickHouse p\u00ebr&nbsp;klasterin \"Metrika\" \u2014 jan\u00eb disa mij\u00ebra n\u00eb sekond\u00eb. N\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, k\u00ebrkesat jo triviale, p\u00ebr shembull, disa mij\u00ebra n\u00eb sekond\u00eb nj\u00eb server ClickHouse nuk i p\u00ebrballon.<\/p>\n<p><\/p>\n<p>Madh\u00ebsia e klasterit&nbsp;\u2014 gjithsej rreth akullore server\u00ebsh. N\u00ebse i vendosim nj\u00eb tabel\u00eb t\u00eb shp\u00ebrndar\u00eb mbi k\u00ebt\u00eb klaster dhe d\u00ebrgojm\u00eb atje disa mij\u00ebra k\u00ebrkesa, do t\u00eb b\u00ebhet edhe m\u00eb keq se sa t'i d\u00ebrgojm\u00eb ato n\u00eb nj\u00eb server t\u00eb vet\u00ebm. Nga ana tjet\u00ebr, mund\u00ebsia ku t\u00eb dh\u00ebnat jan\u00eb shp\u00ebrndar\u00eb n\u00eb m\u00ebnyr\u00eb t\u00eb barabart\u00eb, dhe ne shkojm\u00eb dhe k\u00ebrkojm\u00eb nga t\u00eb gjith\u00eb server\u00ebt, e heqim menj\u00ebher\u00eb.<\/p>\n<p><\/p>\n<p>\u00cbsht\u00eb nj\u00eb opsion diametralisht i kund\u00ebrt. Imagjinoni, n\u00ebse ne do t\u00eb shardonim t\u00eb dh\u00ebnat sipas web-faqeve, dhe k\u00ebrkesa p\u00ebr nj\u00eb web-faqe do t\u00eb shkoj\u00eb n\u00eb nj\u00eb shard. Tani klasteri e p\u00ebrballon dot t\u00eb t\u00ebrheq\u00eb dhjet\u00eb mij\u00eb k\u00ebrkesa n\u00eb sekond\u00eb, por n\u00eb nj\u00eb shard, ndonj\u00eb k\u00ebrkes\u00eb do t\u00eb funksionoj\u00eb shum\u00eb ngadal\u00eb. Nuk do t\u00eb shkall\u00ebzohet m\u00eb n\u00eb p\u00ebrmasat e kapacitetit. Sidomos n\u00ebse kjo \u00ebsht\u00eb faqja avito.ru. Nuk do t\u00eb zbuloni sekretin, n\u00ebse them se Avito \u00ebsht\u00eb nj\u00eb nga faqet m\u00eb t\u00eb vizituara n\u00eb Runet. Dhe t\u00eb trajtosh at\u00eb n\u00eb nj\u00eb shard do t\u00eb ishte marr\u00ebzi.<\/p>\n<p><\/p>\n<p>Prandaj, skema e shardimit \u00ebsht\u00eb e organizuar n\u00eb nj\u00eb m\u00ebnyr\u00eb m\u00eb t\u00eb men\u00e7ur. E gjith\u00eb klasteri \u00ebsht\u00eb e ndar\u00eb n\u00eb disa klasterk\u00eb, t\u00eb cilat i quajm\u00eb nivele. Brenda secilit klasterk ka nga nj\u00eb dhjet\u00eb deri n\u00eb disa dhjet\u00ebra shardesh. Dhe gjithsej jan\u00eb tridhjet\u00eb n\u00ebnt\u00eb klasterk\u00eb. <\/p>\n<p><\/p>\n<p>Si shkall\u00ebzohet gjith\u00e7ka kjo? Numri i klasterk\u00ebve nuk ndryshon&nbsp;\u2014 ashtu si\u00e7 kishte disa vite m\u00eb par\u00eb tridhjet\u00eb n\u00ebnt\u00eb, ashtu ka mbetur. Por brenda secilit, ne gradualisht rrisim numrin e shard\u00ebve n\u00eb p\u00ebrputhje me grumbullimin e t\u00eb dh\u00ebnave. Dhe skema e shardimit n\u00eb p\u00ebrgjith\u00ebsi \u00ebsht\u00eb k\u00ebshtu&nbsp;\u2014 ndarja n\u00eb k\u00ebto klasterk\u00eb b\u00ebhet sipas web-faqeve, dhe p\u00ebr t\u00eb kuptuar se cila faqe \u00ebsht\u00eb n\u00eb cilin klaster, p\u00ebrdoret nj\u00eb databaz\u00eb e ve\u00e7ant\u00eb n\u00eb MySQL. Nj\u00eb faqe&nbsp;\u2014 n\u00eb nj\u00eb klasterk. Dhe brenda tij shardimi vazhdon sipas identifikuesve t\u00eb vizitor\u00ebve.<\/p>\n<p><\/p>\n<p>N\u00eb regjistrimin e tyre, ne i ndajm\u00eb sipas mbetjeve nga ndarja e identifikatorit t\u00eb vizitorit. Por kur shtojm\u00eb nj\u00eb shard t\u00eb ri, skema e sharding-ut ndryshon; ne vazhdojm\u00eb t\u00eb ndajm\u00eb, por sipas mbetjes nga ndarja me nj\u00eb num\u00ebr tjet\u00ebr. Kjo do t\u00eb thot\u00eb se nj\u00eb vizitor ndodhet n\u00eb disa servera, dhe nuk mund t\u00eb bazohemi n\u00eb k\u00ebt\u00eb gj\u00eb. Kjo \u00ebsht\u00eb b\u00ebr\u00eb p\u00ebr t\u00eb siguruar q\u00eb t\u00eb dh\u00ebnat t\u00eb kompresohen m\u00eb mir\u00eb. Kur b\u00ebjm\u00eb k\u00ebrkesa, ne shkojm\u00eb n\u00eb tabel\u00ebn Distributed, e cila shikon n\u00eb klaster dhe i drejtohet dhjet\u00ebra server\u00ebve. Kjo \u00ebsht\u00eb nj\u00eb skem\u00eb e \u00e7uditshme.<\/p>\n<p><\/p>\n<p>Por tregimi im do t\u00eb ishte i paplot\u00eb n\u00ebse nuk do t\u00eb thoja se ne kemi hequr dor\u00eb nga kjo skem\u00eb. N\u00eb skem\u00ebn e re, ne e kemi ndryshuar gjith\u00e7ka dhe t\u00eb gjitha t\u00eb dh\u00ebnat i kemi kopjuar me ndihm\u00ebn e clickhouse-copier.<\/p>\n<p><\/p>\n<p>N\u00eb skem\u00ebn e re, t\u00eb gjitha faqet ndahen n\u00eb dy kategori - t\u00eb m\u00ebdha dhe t\u00eb vogla. Nuk e di se si \u00ebsht\u00eb vendosur kufiri, por rezultati \u00ebsht\u00eb se faqet e m\u00ebdha regjistrohen n\u00eb nj\u00eb klaster me 120 shard-e dhe tre replika n\u00eb secilin - pra 360 servera. Dhe skema e sharding-ut \u00ebsht\u00eb e till\u00eb q\u00eb \u00e7do k\u00ebrkes\u00eb shkon menj\u00ebher\u00eb n\u00eb t\u00eb gjitha shard-et. N\u00ebse tani hapni \u00e7do faqe raporti p\u00ebr avito.ru n\u00eb \"Yandex.Metrica\", k\u00ebrkesa do t\u00eb shkoj\u00eb n\u00eb 120 servera. Ka pak faqe t\u00eb m\u00ebdha n\u00eb Runet. K\u00ebshtu, k\u00ebrkesat nuk jan\u00eb nj\u00eb mij\u00eb n\u00eb sekond\u00eb, por madje jan\u00eb m\u00eb pak se nj\u00ebqind. T\u00eb gjitha k\u00ebto p\u00ebrpunohen leht\u00ebsisht nga tabela Distributed, e cila p\u00ebrpunon secil\u00ebn prej tyre me 120 servera.<\/p>\n<p><\/p>\n<p>Klasteri i dyt\u00eb \u00ebsht\u00eb p\u00ebr faqet e vogla. K\u00ebtu, skema e sharding-ut \u00ebsht\u00eb sipas identifikatorit t\u00eb faqes, dhe \u00e7do k\u00ebrkes\u00eb shkon sakt\u00ebsisht n\u00eb nj\u00eb shard.<\/p>\n<p><\/p>\n<h2 id=\"anchorclickhouse-copieranchorv-clickhouse-est-utilita-clickhouse-copier-mozhete-pronbspneyo-rasskazat\"><noindex><a rel=\"nofollow\" name=\"clickhouse-copier\"><\/a><\/noindex>N\u00eb ClickHouse ka nj\u00eb utilitar t\u00eb quajtur clickhouse-copier. Mund t\u00eb na tregoni p\u00ebr t\u00eb?<\/h2>\n<p><\/p>\n<p>T\u00eb them menj\u00ebher\u00eb, kjo zgjidhje \u00ebsht\u00eb m\u00eb e r\u00ebnd\u00eb dhe pak m\u00eb pak produktive. Avantazhi \u00ebsht\u00eb se ajo shp\u00ebrndan t\u00eb dh\u00ebnat plot\u00ebsisht sipas skem\u00ebs q\u00eb do t\u00eb tregoni. Por disavantazhi i mjetit \u00ebsht\u00eb se ajo nuk b\u00ebn riparimin e shard-eve. Ajo kopjon t\u00eb dh\u00ebnat nga nj\u00eb skem\u00eb klasteri n\u00eb nj\u00eb skem\u00eb tjet\u00ebr klasteri.<\/p>\n<p><\/p>\n<p>Kjo do t\u00eb thot\u00eb se p\u00ebr funksionimin e saj, duhet t\u00eb keni dy klaster\u00eb. Ato mund t\u00eb ndodhen n\u00eb servera identik\u00eb, por, megjithat\u00eb, t\u00eb dh\u00ebnat nuk do t\u00eb transferohen n\u00eb m\u00ebnyr\u00eb incrementale, por do t\u00eb kopjohen. <\/p>\n<p><\/p>\n<p>P\u00ebr shembull, ishin kat\u00ebr server\u00eb, tani jan\u00eb tet\u00eb. Ju krijoni nj\u00eb tabel\u00eb t\u00eb re Distributed n\u00eb t\u00eb gjith\u00eb server\u00ebt, tabela lokale t\u00eb reja dhe nisin clickhouse-copier, duke treguar n\u00eb t\u00eb skem\u00ebn e funksionimit, se ai duhet t\u00eb lexoj\u00eb nga aty, t\u00eb pranoj\u00eb skem\u00ebn e re t\u00eb shardimit dhe t\u00eb transferoj\u00eb t\u00eb dh\u00ebnat atje. Dhe juve n\u00eb server\u00ebt e vjet\u00ebr do t'ju nevojitet nj\u00eb hap\u00ebsir\u00eb 1.5 her\u00eb m\u00eb shum\u00eb se sa keni tani, sepse t\u00eb dh\u00ebnat e vjetra duhet t\u00eb mbeten atje dhe p\u00ebrmbi to do t\u00eb mb\u00ebrrij\u00eb nj\u00eb gjysm\u00eb e k\u00ebtyre t\u00eb dh\u00ebnave t\u00eb vjetra. N\u00ebse keni menduar paraprakisht q\u00eb t\u00eb dh\u00ebnat duhet t\u00eb ribashkohen dhe ka hap\u00ebsir\u00eb, at\u00ebher\u00eb ky metod\u00eb do t\u00eb ishte e p\u00ebrshtatshme.<\/p>\n<p><\/p>\n<p>Si \u00ebsht\u00eb brenda clickhouse-copier? Ai ndan t\u00eb gjith\u00eb pun\u00ebn n\u00eb nj\u00eb grup detyrash p\u00ebr p\u00ebrpunimin e nj\u00eb partie t\u00eb nj\u00eb tabele n\u00eb nj\u00eb shard. T\u00eb gjitha k\u00ebto detyra mund t\u00eb ekzekutohen paralelisht, dhe clickhouse-copier mund t\u00eb jet\u00eb i aktivizuar n\u00eb makineri t\u00eb ndryshme n\u00eb disa \u044d\u043a\u0437\u0435\u043c\u043f\u043b\u044fra, por ajo q\u00eb b\u00ebn p\u00ebr nj\u00eb parti \u00ebsht\u00eb asgj\u00eb m\u00eb shum\u00eb se nj\u00eb insert select. T\u00eb dh\u00ebnat lexohen, shkurtohen, rip\u00ebrshkall\u00ebzohen, pastaj s\u00ebrish kompresohen, shkruhen diku, re-organizohen. Kjo \u00ebsht\u00eb nj\u00eb zgjidhje m\u00eb e r\u00ebnd\u00eb.<\/p>\n<p><\/p>\n<h2 id=\"anchorresharding-toolanchoru-vas-byla-pilotnaya-shtuka-kotoraya-nazyvalas-resharding-chto-snbspney\"><noindex><a rel=\"nofollow\" name=\"resharding-tool\"><\/a><\/noindex>Kishit nj\u00eb gj\u00eb pilot q\u00eb quhej ri-shp\u00ebrndarje. \u00c7far\u00eb ndodhi me t\u00eb?<\/h2>\n<p><\/p>\n<blockquote><p>Keni pasur nj\u00eb gj\u00eb pilotike q\u00eb n\u00eb&nbsp;2017, e cila quhej resharding. Madje ka nj\u00eb opsion n\u00eb ClickHouse. Kuptoj q\u00eb kjo nuk funksionoi. Mund t\u00eb tregoni pse ndodhi k\u00ebshtu? Duket shum\u00eb aktuale.<\/p><\/blockquote>\n<p>E gjith\u00eb problemi \u00ebsht\u00eb q\u00eb, kur duhet t\u00eb ribashkohen t\u00eb dh\u00ebnat, k\u00ebrkohet nj\u00eb sinkronizim shum\u00eb i komplikuar p\u00ebr ta b\u00ebr\u00eb k\u00ebt\u00eb n\u00eb m\u00ebnyr\u00eb atomike. Kur filluam t\u00eb shikonim se si \u00ebsht\u00eb organizuar kjo sinkronizim, u b\u00eb e qart\u00eb se ka probleme themelore. Dhe k\u00ebto probleme themelore nuk jan\u00eb vet\u00ebm teorike, por filluan t\u00eb shfaqen n\u00eb praktik\u00eb n\u00eb m\u00ebnyr\u00eb t\u00eb till\u00eb q\u00eb mund t\u00eb shpjegohet shum\u00eb thjesht\u00eb \u2014 asgj\u00eb nuk funksionon.<\/p>\n<p><\/p>\n<h2 id=\"anchormove-to-slow-diskanchormozhno-li-slivat-vse-chasti-dannyh-voedino-perednbspperemescheniem-nanbspmedlennye-diski\"><noindex><a rel=\"nofollow\" name=\"move-to-slow-disk\"><\/a><\/noindex>A mund t\u00eb bashkohen t\u00eb gjitha pjes\u00ebt e t\u00eb dh\u00ebnave para se t\u00eb transferohen n\u00eb disqe t\u00eb ngadalt\u00eb?<\/h2>\n<p><\/p>\n<blockquote><p>Nj\u00eb pyetje n\u00eb lidhje me TTL me opsionin move to slow disk n\u00eb kontekstin e merger\u00ebve. A ka ndonj\u00eb m\u00ebnyr\u00eb, p\u00ebrve\u00e7 se me cron, p\u00ebr t\u00eb bashkuar t\u00eb gjitha pjes\u00ebt n\u00eb nj\u00eb p\u00ebrpara se t'i transferoni n\u00eb disqet e ngadalta?<\/p><\/blockquote>\n<p>P\u00ebrgjigjja n\u00eb pyetjen n\u00ebse mund t\u00eb bashkohen automatikisht t\u00eb gjitha copat n\u00eb nj\u00eb para transferimit t\u00eb tyre \u2014 jo. M\u00eb duket se nuk ka nevoj\u00eb p\u00ebr k\u00ebt\u00eb. Nuk \u00ebsht\u00eb e nevojshme t\u00eb bashkohen t\u00eb gjitha pjes\u00ebt n\u00eb nj\u00eb, por thjesht t\u00eb mb\u00ebshteteni n\u00eb faktin se ato do t\u00eb transferohen automatikisht n\u00eb disqet e ngadalta. <\/p>\n<p><\/p>\n<p>Ne kemi dy kritere p\u00ebr rregullat e transferimit. E para \u2014 n\u00eb p\u00ebrputhje me mbushjen. N\u00ebse n\u00eb nivelin aktual t\u00eb magazin\u00ebs ka m\u00eb pak se nj\u00eb p\u00ebrqindje t\u00eb caktuar hap\u00ebsire t\u00eb lir\u00eb, ne zgjedhim nj\u00eb cop\u00eb dhe e transferojm\u00eb at\u00eb n\u00eb nj\u00eb ruajtje m\u00eb t\u00eb ngadalt\u00eb. M\u00eb sakt\u00eb, jo m\u00eb t\u00eb ngadalsh\u00ebm, por n\u00eb t\u00eb ardhmen \u2014 si\u00e7 e keni konfiguruar.<\/p>\n<p><\/p>\n<p>Kriteri i dyt\u00eb \u2014 sipas madh\u00ebsis\u00eb. Ky \u00ebsht\u00eb p\u00ebr transferimin e copave t\u00eb m\u00ebdha. Mund t\u00eb rregulloni pragun e hap\u00ebsir\u00ebs s\u00eb lir\u00eb n\u00eb diskun e shpejt\u00eb dhe t\u00eb dh\u00ebnat do t\u00eb transferohen automatikisht.<\/p>\n<p><\/p>\n<h2 id=\"anchorup-to-dateanchorkak-pereezzhat-nanbspnovye-versii-clickhouse-esli-net-vozmozhnosti-zaranee-proverit-sovmestimost\"><noindex><a rel=\"nofollow\" name=\"up-to-date\"><\/a><\/noindex>Si t\u00eb migrohet n\u00eb versione t\u00eb reja t\u00eb ClickHouse, n\u00ebse nuk ka mund\u00ebsi ta kontrolloni paraprakisht p\u00ebrkompatibilitetin?<\/h2>\n<p><\/p>\n<blockquote><p>Ky tem\u00eb diskutohet rregullisht <noindex><a rel=\"nofollow\" href=\"https:\/\/teleg.run\/clickhouse_ru\">n\u00eb grupin e Telegramit ClickHouse<\/a><\/noindex> duke marr\u00eb parasysh versione t\u00eb ndryshme, dhe megjithat\u00eb. Sa t\u00eb sigurt \u00ebsht\u00eb t\u00eb p\u00ebrdit\u00ebsohesh nga versioni 19.11 n\u00eb 19.16 dhe, p\u00ebr shembull, nga 19.16 n\u00eb 20.3. Si \u00ebsht\u00eb m\u00eb mir\u00eb t\u00eb kaloni n\u00eb versione t\u00eb reja, pa pasur mund\u00ebsin\u00eb t\u00eb verifikoni p\u00ebrputhshm\u00ebrin\u00eb paraprakisht n\u00eb nj\u00eb ambient testues?<\/p><\/blockquote>\n<p>K\u00ebtu jan\u00eb disa rregulla \"t\u00eb arta\". E para \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/ClickHouse\/ClickHouse\/blob\/master\/CHANGELOG.md\">lexoni changelog<\/a><\/noindex>. Ai \u00ebsht\u00eb i madh, por ka pika t\u00eb ve\u00e7anta q\u00eb lidhen me ndryshime q\u00eb nuk jan\u00eb retro-kompatibile. Nuk duhet t'i qaseni k\u00ebtyre pikave si nj\u00eb flamur t\u00eb kuq. Zakonisht, k\u00ebto jan\u00eb mos p\u00ebrputhshm\u00ebri t\u00eb vogla, q\u00eb lidhen me disa funksionalitete periferike, t\u00eb cilat, me gjas\u00eb, nuk i p\u00ebrdorni.<\/p>\n<p><\/p>\n<p>E dyta \u2014 n\u00ebse nuk ka mund\u00ebsi t\u00eb kontrolloni p\u00ebrputhshm\u00ebrin\u00eb n\u00eb nj\u00eb ambient testues, dhe d\u00ebshironi t\u00eb p\u00ebrdit\u00ebsoni menj\u00ebher\u00eb n\u00eb prodhim, rekomandimi \u00ebsht\u00eb ky \u2014 mos e b\u00ebni k\u00ebt\u00eb. S\u00eb pari krijoni nj\u00eb ambient testues dhe kontrolloni. N\u00ebse nuk keni nj\u00eb ambient provues, at\u00ebher\u00eb, me shum\u00eb mund\u00ebsi, nuk jeni nj\u00eb kompani shum\u00eb e madhe, dhe n\u00eb k\u00ebt\u00eb rast mund t\u00eb kopjoni nj\u00eb pjes\u00eb t\u00eb t\u00eb dh\u00ebnave n\u00eb laptopin tuaj dhe t\u00eb siguroheni se gjith\u00e7ka funksionon si\u00e7 duhet. Mund t\u00eb ngrem\u00eb disa replika lokal n\u00eb makin\u00ebn tuaj. Ose mund t\u00eb ngrini nj\u00eb version t\u00eb ri diku n\u00eb af\u00ebrsi dhe t\u00eb ngarkoni atje nj\u00eb pjes\u00eb t\u00eb t\u00eb dh\u00ebnave \u2014 dmth t\u00eb krijoni nj\u00eb ambient testues improvisuar. <\/p>\n<p><\/p>\n<p>Nj\u00eb rregull tjet\u00ebr \u2014 mos u p\u00ebrdit\u00ebsoni p\u00ebr nj\u00eb jav\u00eb pas l\u00ebshimit t\u00eb versionit p\u00ebr shkak t\u00eb gjetjes s\u00eb defekteve n\u00eb prodhim dhe riparimeve t\u00eb shpejta pasuese. Le t\u00eb shqyrtojm\u00eb num\u00ebrimin e versioneve t\u00eb ClickHouse, p\u00ebr t\u00eb mos u konfuzuar. <\/p>\n<p><\/p>\n<p>Ekziston versioni 20.3.4. Numri 20&nbsp;tregon vitin e l\u00ebshimit&nbsp;\u2014 2020. N\u00eb aspektin e p\u00ebrmbajtjes, kjo nuk ka ndonj\u00eb r\u00ebnd\u00ebsi, k\u00ebshtu q\u00eb nuk do t'i kushtojm\u00eb v\u00ebmendje. M\u00eb pas \u2014 20.3. Numri i dyt\u00eb&nbsp;\u2014 n\u00eb k\u00ebt\u00eb rast 3 \u2014 ne e rrisim \u00e7do her\u00eb kur l\u00ebshojm\u00eb nj\u00eb version t\u00eb ri me ndonj\u00eb funksionalitet t\u00eb ri. N\u00ebse duam t\u00eb shtojm\u00eb ndonj\u00eb mund\u00ebsi n\u00eb&nbsp;ClickHouse, ne jemi t\u00eb detyruar ta rrisim k\u00ebt\u00eb num\u00ebr. K\u00ebshtu q\u00eb, n\u00eb versionin 20.4, ClickHouse do t\u00eb punoj\u00eb akoma m\u00eb mir\u00eb. Numri i tret\u00eb&nbsp;\u2014 20.3.4. K\u00ebtu&nbsp;4 \u00ebsht\u00eb numri i rregullimeve t\u00eb patch-it, n\u00eb t\u00eb cilat ne nuk kemi shtuar mund\u00ebsi t\u00eb reja, por kemi korrigjuar disa gabime. Dhe 4&nbsp;tregon se ne e kemi b\u00ebr\u00eb k\u00ebt\u00eb kat\u00ebr her\u00eb.<\/p>\n<p><\/p>\n<p>Nuk \u00ebsht\u00eb e nevojshme t\u00eb mendoni se \u00ebsht\u00eb di\u00e7ka horribile. Zakonisht p\u00ebrdoruesi mund t\u00eb instalohet versionin m\u00eb t\u00eb ri, dhe ai do t\u00eb funksionoj\u00eb pa&nbsp;probleme t\u00eb ndonj\u00eb lloji p\u00ebr nj\u00eb vit. Por imagjinoni se n\u00eb ndonj\u00eb funksion p\u00ebr p\u00ebrpunimin e bitmap-eve, i cili u shtua nga koleg\u00ebt tan\u00eb kinez\u00eb, kur kaloni argumente t\u00eb gabuara, serveri bie. Ne jemi t\u00eb detyruar ta rregullojm\u00eb k\u00ebt\u00eb. Ne do t\u00eb l\u00ebshojm\u00eb nj\u00eb version t\u00eb ri patch dhe ClickHouse do t\u00eb b\u00ebhet m\u00eb i q\u00ebndruesh\u00ebm.<\/p>\n<p><\/p>\n<p>N\u00ebse keni ClickHouse q\u00eb punon n\u00eb prodhim, dhe del nj\u00eb version i ri ClickHouse me funksionalitete shtes\u00eb \u2014 p\u00ebr shembull, 20.4.1 \u2014 mos u nxito ta instaloni dit\u00ebn e par\u00eb n\u00eb prodhim. Pse \u00ebsht\u00eb n\u00eb fakt e nevojshme? N\u00ebse ende nuk po p\u00ebrdorni ClickHouse, mund ta instaloni, dhe shum\u00eb mund\u00ebsi \u00ebsht\u00eb q\u00eb gjith\u00e7ka do t\u00eb shkoj\u00eb mir\u00eb. Por n\u00ebse ClickHouse po punon stabilisht, at\u00ebher\u00eb mbani syt\u00eb hapur p\u00ebr patch-et dhe p\u00ebrdit\u00ebsimet&nbsp;\u2014 \u00e7far\u00eb problemeve po i rregullojm\u00eb.<\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> D\u00ebshiroj t\u00eb shtoj pak mbi ambientet testuese. T\u00eb gjith\u00eb jan\u00eb shum\u00eb t\u00eb frik\u00ebsuar nga ambientet testuese dhe ndonj\u00ebher\u00eb mendojn\u00eb se, n\u00ebse keni nj\u00eb klaster shum\u00eb t\u00eb madh ClickHouse, ambienti testues duhet t\u00eb jet\u00eb po aq ose t\u00eb pakt\u00ebn dhjet\u00eb her\u00eb m\u00eb i vog\u00ebl. Kjo nuk \u00ebsht\u00eb aspak e v\u00ebrtet\u00eb.<\/p>\n<p><\/p>\n<p>Mund t\u00eb flas nga p\u00ebrvoja ime. Kam nj\u00eb projekt, dhe atje ka ClickHouse. Ambienti jon\u00eb testues p\u00ebr k\u00ebt\u00eb projekt \u00ebsht\u00eb nj\u00eb virtualizim i vog\u00ebl n\u00eb Hetzner p\u00ebr nj\u00ebzet euro, ku \u00ebsht\u00eb instaluar gjith\u00e7ka. P\u00ebr ta b\u00ebr\u00eb k\u00ebt\u00eb, ne kemi automatizim t\u00eb plot\u00eb n\u00eb Ansible, dhe prandaj n\u00eb parim nuk ka ndonj\u00eb dallim se ku shkon \u2014 n\u00eb server\u00ebt fizik\u00eb apo thjesht t\u00eb instalohesh n\u00eb virtualizime.<\/p>\n<p><\/p>\n<p>\u00c7far\u00eb mund t\u00eb b\u00ebhet? Do t\u00eb ishte mir\u00eb t\u00eb kishte nj\u00eb shembull n\u00eb dokumentacionin e ClickHouse, se si t\u00eb vendosni nj\u00eb klaster t\u00eb vog\u00ebl n\u00eb Docker, n\u00eb LXC, ndoshta t\u00eb krijoni nj\u00eb playbook Ansible, sepse njer\u00ebzit e ndrysh\u00ebm kan\u00eb diferenca n\u00eb vendosje. Kjo do ta thjeshtonte shum\u00eb pun\u00ebn. Kur merrni dhe vendosni nj\u00eb klaster brenda pes\u00eb minutash, \u00ebsht\u00eb shum\u00eb m\u00eb e leht\u00eb t\u00eb merresh me ndonj\u00eb gj\u00eb. K\u00ebshtu \u00ebsht\u00eb shum\u00eb m\u00eb e rehatshme, sepse t\u00eb l\u00ebsh n\u00eb versionin prodhim q\u00eb nuk e ke kontrolluar \u2013 \u00ebsht\u00eb nj\u00eb rrug\u00eb q\u00eb \u00e7on askund. Ndonj\u00ebher\u00eb funksionon, ndonj\u00ebher\u00eb jo. Dhe prandaj, t\u00eb shpresosh p\u00ebr sukses \u2013 \u00ebsht\u00eb e keqe.<\/p>\n<p><\/p>\n<p><strong>Maksim Kotyakov, inxhinier i lart\u00eb backend n\u00eb Avito:<\/strong> Pak m\u00eb shum\u00eb rreth ambientit t\u00eb testeve nga seria e problemeve t\u00eb kompanive t\u00eb m\u00ebdha. Ne kemi nj\u00eb klaster t\u00eb plot\u00eb p\u00ebr miratim ClickHouse, nj\u00eb kopje t\u00eb sakt\u00eb t\u00eb skemave t\u00eb t\u00eb dh\u00ebnave dhe cil\u00ebsimeve t\u00eb asaj q\u00eb ndodhet n\u00eb prodhim. Ky klaster \u00ebsht\u00eb vendosur n\u00eb kontejner\u00eb mjaft t\u00eb vjet\u00ebr me minimumin e burimeve. Ne shkruajm\u00eb atje nj\u00eb p\u00ebrqindje t\u00eb caktuar t\u00eb t\u00eb dh\u00ebnave t\u00eb prodhimit, fal\u00eb mund\u00ebsis\u00eb p\u00ebr t\u00eb replikuar n\u00eb Kafka. Aty gjith\u00e7ka \u00ebsht\u00eb e sinkronizuar dhe e shkall\u00ebzuar \u2013 dhe n\u00eb saje t\u00eb kapacitetit dhe t\u00eb fluxit, dhe, n\u00eb teori, n\u00ebn kushte t\u00eb tjera t\u00eb barabarta duhet t\u00eb sillen si prodhimi. \u00c7do gj\u00eb q\u00eb \u00ebsht\u00eb potencialisht e rrezikshme fillimisht kalon n\u00eb k\u00ebt\u00eb stand dhe q\u00ebndron aty disa dit\u00eb p\u00ebr t'u p\u00ebrgatitur. Por natyrisht, kjo zgjidhje \u00ebsht\u00eb e shtrenjt\u00eb, e r\u00ebnd\u00eb dhe me shpenzime jo zero p\u00ebr mir\u00ebmbajtje. <\/p>\n<p><\/p>\n<p><strong>Alexey Milovidov:<\/strong> Do t\u00eb tregoj se \u00e7far\u00eb p\u00ebrb\u00ebn ambienti i testeve t\u00eb miqve tan\u00eb nga \"Yandex.Metrika\". Nj\u00eb klaster kishte m\u00eb shum\u00eb se 600 server\u00eb, tjetri 360, dhe ka edhe nj\u00eb t\u00eb tret\u00eb dhe disa klaster\u00eb. Ambienti i testeve p\u00ebr nj\u00eb prej tyre \u2013 thjesht jan\u00eb dy sharda me dy replika n\u00eb secilin. Pse dy sharda? Q\u00eb t\u00eb mos ket\u00eb vet\u00ebm nj\u00eb. Edhe replikat, q\u00eb duhet t\u00eb ekzistojn\u00eb. Thjesht nj\u00eb num\u00ebr minimal, q\u00eb mund t\u00eb lejohet.<\/p>\n<p><\/p>\n<p>Ky ambient i testeve lejon t\u00eb kontrollohet funksionaliteti i k\u00ebrkesave dhe n\u00ebse ndonj\u00eb gj\u00eb e madhe \u00ebsht\u00eb prishur. Por shpesh problemet shfaqen t\u00eb nj\u00eb natyre krejt tjet\u00ebr, kur gjith\u00e7ka funksionon, por ka disa ndryshime t\u00eb vogla me ngarkes\u00ebn.<\/p>\n<p><\/p>\n<p>Do t\u00eb jap nj\u00eb shembull. Vendos\u00ebm t\u00eb instalojm\u00eb versionin e ri t\u00eb ClickHouse. Ai \u00ebsht\u00eb publikuar n\u00eb mjedisin testues, jan\u00eb kryer testet automatizuara n\u00eb mendimin vet\u00eb \"Yandex.Metrika\", t\u00eb cilat krahasojn\u00eb t\u00eb dh\u00ebnat n\u00eb versionin e vjet\u00ebr dhe n\u00eb at\u00eb t\u00eb ri, duke kaluar t\u00ebr\u00eb linj\u00ebn e prodhimit. Dhe natyrisht, testet pozitive t\u00eb CI ton\u00eb. N\u00eb t\u00eb kund\u00ebrt, ne as nuk do ta ofronim k\u00ebt\u00eb version.<\/p>\n<p><\/p>\n<p>Gjith\u00e7ka \u00ebsht\u00eb perfekt. Fillojm\u00eb ta \u00e7ojm\u00eb n\u00eb prodhim. M\u00eb vjen nj\u00eb mesazh q\u00eb ngarkesa n\u00eb grafik\u00eb ka u rritur disa her\u00eb. Ne e kthejm\u00eb versionin mbrapa. Un\u00eb shoh n\u00eb grafik dhe shoh: ngarkesa v\u00ebrtet u rrit disa her\u00eb gjat\u00eb publikimit, dhe u zvog\u00eblua p\u00ebrs\u00ebri kur e publikuan. M\u00eb pas, ne filluam t\u00eb kthejm\u00eb versionin mbrapa. Dhe ngarkesa po ashtu u rrit dhe po ashtu ra p\u00ebrs\u00ebri. Pra, p\u00ebrfundimi \u00ebsht\u00eb se ngarkesa u rrit n\u00eb lidhje me publikimin, nuk ka asgj\u00eb t\u00eb \u00e7uditshme.<\/p>\n<p><\/p>\n<p>M\u00eb pas ishte e v\u00ebshtir\u00eb t\u00eb bindja koleg\u00ebt q\u00eb ta instalonin versionin e ri. I thash\u00eb: \"Gjith\u00e7ka \u00ebsht\u00eb n\u00eb rregull, publikoni. Mbani gishta ngjitur, gjith\u00e7ka do t\u00eb funksionoj\u00eb. Tani ngarkesa u rrit n\u00eb grafik\u00eb, por gjith\u00e7ka \u00ebsht\u00eb n\u00eb rregull. Q\u00ebndroni fort\". N\u00eb t\u00ebr\u00ebsi, ne e b\u00ebm\u00eb k\u00ebshtu, dhe t\u00eb gjith\u00eb \u2014 versioni u publikua n\u00eb prodhim. Por thuajse n\u00eb \u00e7do publikim shfaqen probleme t\u00eb ngjashme.<\/p>\n<p><\/p>\n<h2 id=\"anchorkill-queryanchorkill-query-dolzhen-ubivat-zaprosy-no-on-etogo-ne-delaet-pochemu\"><noindex><a rel=\"nofollow\" name=\"kill-query\"><\/a><\/noindex>Kill query duhet t\u00eb vras\u00eb \u0437\u0430\u043f\u0440\u043e\u0441\u044b, por ai nuk e b\u00ebn k\u00ebt\u00eb. Pse?<\/h2>\n<p><\/p>\n<blockquote><p>M\u00eb erdhi nj\u00eb p\u00ebrdorues, nj\u00eb analist i caktuar, dhe krijoi nj\u00eb k\u00ebrkes\u00eb t\u00eb caktuar, e cila ndaloi klasterin tim ClickHouse. Nj\u00eb nod\u00eb ose klaster krejt\u00ebsisht \u2014 n\u00eb varesi t\u00eb asaj se n\u00eb cil\u00ebn riplik\u00eb ose shard ra k\u00ebrkesa. Un\u00eb shoh q\u00eb t\u00eb gjith\u00eb burimet p\u00ebr CPU n\u00eb k\u00ebt\u00eb server jan\u00eb t\u00eb ngarkuar, gjith\u00e7ka \u00ebsht\u00eb e kuqe. Nd\u00ebrkoh\u00eb, vet\u00eb ClickHouse po p\u00ebrgjigjet ndaj k\u00ebrkesave. Dhe un\u00eb shkruaj: \"M\u00eb trego, t\u00eb lutem, list\u00ebn e proceseve, cila k\u00ebrkes\u00eb e krijoi k\u00ebt\u00eb \u00e7menduri\".<\/p>\n<p>Un\u00eb e gjej k\u00ebt\u00eb k\u00ebrkes\u00eb dhe shkruaj kill. Dhe shoh q\u00eb nuk ndodh asgj\u00eb. Serveri im \u00ebsht\u00eb i ngarkuar, ClickHouse vazhdon t\u00eb m\u00eb jep disa komanda, tregon q\u00eb serveri \u00ebsht\u00eb gjall\u00eb, dhe gjith\u00e7ka \u00ebsht\u00eb mir\u00eb. Por un\u00eb kam degradim n\u00eb t\u00eb gjitha k\u00ebrkesat e p\u00ebrdoruesve, fillon degradimi i shkruajtes n\u00eb ClickHouse, dhe kill query im nuk funksionon. Pse? Mendova se kill query duhet t\u00eb vras\u00eb k\u00ebrkesat, por kjo nuk ndodh.<\/p><\/blockquote>\n<p>Tani do t\u00eb ket\u00eb nj\u00eb p\u00ebrgjigje mjaft t\u00eb \u00e7uditshme. \u00c7\u00ebshtja \u00ebsht\u00eb se kill query nuk vret k\u00ebrkesat. <\/p>\n<p><\/p>\n<p>Kill query vendos nj\u00eb flamur t\u00eb vog\u00ebl me emrin \"do t\u00eb doja q\u00eb ky k\u00ebrkes\u00eb t\u00eb fitoj\u00eb fundin\". K\u00ebshtu q\u00eb k\u00ebrkesa, gjat\u00eb p\u00ebrpunimit t\u00eb \u00e7do blloku, kontrollon k\u00ebt\u00eb flamur. N\u00ebse \u00ebsht\u00eb vendosur, k\u00ebrkesa do t\u00eb ndaloj\u00eb. Pra, askush nuk e vret k\u00ebrkes\u00ebn, ajo duhet t\u00eb verifikoj\u00eb dhe ndaloj\u00eb vet\u00eb. Kjo duhet t\u00eb funksionoj\u00eb n\u00eb t\u00eb gjitha rastet kur k\u00ebrkesa \u00ebsht\u00eb n\u00eb gjendje t\u00eb p\u00ebrpunimit t\u00eb blloqeve t\u00eb t\u00eb dh\u00ebnave. Ajo do t\u00eb procesoj\u00eb bllokun e ardhsh\u00ebm t\u00eb t\u00eb dh\u00ebnave, do t\u00eb kontrolloj\u00eb flamurin dhe do t\u00eb ndaloj\u00eb.<\/p>\n<p><\/p>\n<p>Kjo nuk funksionon n\u00eb rastet kur k\u00ebrkesa \u00ebsht\u00eb bllokuar n\u00eb ndonj\u00eb operacion. Megjithat\u00eb, shum\u00eb t\u00eb mundshme \u00ebsht\u00eb q\u00eb ky t\u00eb mos jet\u00eb rasti juaj, sepse, sipas fjal\u00ebve tuaja, po p\u00ebrdor shum\u00eb burime serveri. Mund t\u00eb mos funksionoj\u00eb n\u00eb rastin e renditjes s\u00eb jashtme dhe disa detaje t\u00eb tjera. Por n\u00eb p\u00ebrgjith\u00ebsi, kjo nuk duhet t\u00eb ndodhte, \u00ebsht\u00eb nj\u00eb gabim. Dhe e vetmja gj\u00eb q\u00eb mund t\u00eb rekomandoj \u00ebsht\u00eb t\u00eb p\u00ebrdit\u00ebsoni ClickHouse.<\/p>\n<p><\/p>\n<h2 id=\"anchorreading-timeanchorkak-rasschitat-vremya-otveta-pri-chitayuschey-nagruzke\"><noindex><a rel=\"nofollow\" name=\"reading-time\"><\/a><\/noindex>Si t\u00eb llogaritet koha e p\u00ebrgjigjes n\u00ebn ngarkes\u00eb lexuese?<\/h2>\n<p><\/p>\n<blockquote><p>Ka nj\u00eb tabel\u00eb n\u00eb t\u00eb cil\u00ebn ruhen agregatet sipas item - num\u00ebr t\u00eb ndrysh\u00ebm. Numri i rreshtave \u00ebsht\u00eb p\u00ebraf\u00ebrsisht nj\u00ebqind milion. A mund t\u00eb pritet nj\u00eb koh\u00eb p\u00ebrgjigjeje e parashikueshme n\u00ebse hedhin 1K RPS p\u00ebr 1K item\u00eb? <\/p><\/blockquote>\n<p>Sipas kontekstit, po flisni p\u00ebr ngarkes\u00ebn lexuese, sepse p\u00ebr shkruar nuk ka asnj\u00eb problem - mund t\u00eb futen sa d\u00ebshiron, qoft\u00eb nj\u00eb mij\u00eb, qoft\u00eb nj\u00ebqind mij\u00eb, madje ndonj\u00ebher\u00eb edhe disa miliona rreshta. <\/p>\n<p><\/p>\n<p>K\u00ebrkesat lexuese jan\u00eb t\u00eb ndryshme. N\u00eb select 1, ClickHouse mund t\u00eb ekzekutoj\u00eb rreth dhjet\u00ebra mij\u00eb k\u00ebrkesa n\u00eb sekond\u00eb, prandaj edhe k\u00ebrkesat p\u00ebr nj\u00eb \u00e7el\u00ebs do t\u00eb k\u00ebrkojn\u00eb disa burime. Dhe k\u00ebto k\u00ebrkesa t\u00eb caktuara do t\u00eb jen\u00eb m\u00eb t\u00eb komplikuara se n\u00eb ndonj\u00eb baz\u00eb t\u00eb dh\u00ebnash me \u00e7el\u00ebs-vler\u00eb, sepse p\u00ebr \u00e7do lexim \u00ebsht\u00eb e nevojshme t\u00eb lexoni nj\u00eb bllok t\u00eb t\u00eb dh\u00ebnave sipas indeksit. Indeksi yn\u00eb adreson jo \u00e7do rekord, por \u00e7do gam\u00eb. K\u00ebshtu q\u00eb do t\u00eb duhet t\u00eb lexoni gjith\u00eb gam\u00ebn - kjo \u00ebsht\u00eb 8192 rreshta sipas parazgjedhjes. Dhe do t\u00eb duhet t\u00eb dekompresoni bllokun e comprimit prej 64 Kb deri n\u00eb 1 Mb. N\u00eb p\u00ebrgjith\u00ebsi, k\u00ebto k\u00ebrkesa t\u00eb caktuara marrin nga disa milisekonda. Por kjo \u00ebsht\u00eb opsioni m\u00eb i thjesht\u00eb.<\/p>\n<p><\/p>\n<p>Le t\u00eb provojm\u00eb t\u00eb b\u00ebjm\u00eb nj\u00eb aritmetik\u00eb t\u00eb thjesht\u00eb. N\u00ebse shumojm\u00eb disa milisekonda me nj\u00eb mij\u00eb, del disa sekonda. Sikur t\u00eb mbahej nj\u00eb mij\u00eb k\u00ebrkesa n\u00eb sekond\u00eb nuk do t\u00eb ishte e mundur, por n\u00eb t\u00eb v\u00ebrtet\u00eb \u00ebsht\u00eb, sepse kemi disa b\u00ebrthama procesori. Pra ashtu si\u00e7 \u00ebsht\u00eb, 1000 RPS ClickHouse ndonj\u00ebher\u00eb e mbajn\u00eb, por p\u00ebr k\u00ebrkesa t\u00eb shkurtra, pik\u00ebrisht t\u00eb sakta.<\/p>\n<p><\/p>\n<p>N\u00ebse duhet t\u00eb skaloni klasterin ClickHouse n\u00eb numrin e k\u00ebrkesave t\u00eb thjeshta, un\u00eb rekomandoj gj\u00ebn\u00eb m\u00eb t\u00eb thjesht\u00eb - rritni numrin e replika dhe d\u00ebrgoni k\u00ebrkesat n\u00eb nj\u00eb replik\u00eb t\u00eb rast\u00ebsishme. N\u00ebse nj\u00eb replik\u00eb mban pes\u00ebqind k\u00ebrkesa n\u00eb sekond\u00eb, q\u00eb \u00ebsht\u00eb plot\u00ebsisht reale, at\u00ebher\u00eb tre replika do t\u00eb mbajn\u00eb nj\u00eb mij\u00eb e pes\u00ebqind.<\/p>\n<p><\/p>\n<p>Ndonj\u00ebher\u00eb, natyrisht, mund t\u00eb konfigurohet edhe ClickHouse p\u00ebr numrin maksimal t\u00eb leximeve t\u00eb sakta. \u00c7far\u00eb \u00ebsht\u00eb e nevojshme p\u00ebr k\u00ebt\u00eb? E para - zvog\u00ebloni granularitetin e indeksit. Megjithat\u00eb, duhet ta zvog\u00ebloni jo deri n\u00eb nj\u00eb, por duke llogaritur se numri i regjistrimeve n\u00eb indeks do t\u00eb jet\u00eb disa miliona ose dhjet\u00ebra miliona n\u00eb server. N\u00ebse tabela ka nj\u00ebqind milion rreshta, at\u00ebher\u00eb si granularitet mund t\u00eb vendosni 64.<\/p>\n<p><\/p>\n<p>Mund t\u00eb zvog\u00ebloni madh\u00ebsin\u00eb e bllokut t\u00eb kompresuar. P\u00ebr k\u00ebt\u00eb ka cil\u00ebsime. <strong>min compress block size<\/strong>, <strong>max compress block size<\/strong>Mund t'i zvog\u00ebloni, t\u00eb rialokoni t\u00eb dh\u00ebnat, dhe at\u00ebher\u00eb k\u00ebrkesat e sakta do t\u00eb jen\u00eb m\u00eb t\u00eb shpejta. Megjithat\u00eb, ClickHouse nuk \u00ebsht\u00eb nj\u00eb baz\u00eb t\u00eb dh\u00ebnash key-value. Nj\u00eb num\u00ebr i madh i k\u00ebrkesave t\u00eb vogla \u00ebsht\u00eb nj\u00eb anti-model ngarkese.<\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> Do t'ju jap nj\u00eb k\u00ebshill\u00eb n\u00eb rast se ka p\u00ebrdorues t\u00eb zakonsh\u00ebm. Kjo \u00ebsht\u00eb nj\u00eb situat\u00eb mjaft standarde kur n\u00eb ClickHouse ruhet nj\u00eb llogarit\u00ebs. Kam nj\u00eb p\u00ebrdorues, ai vjen nga nj\u00eb vend, nj\u00eb fush\u00eb e tret\u00eb, dhe duhet t\u00eb rritet di\u00e7ka n\u00eb m\u00ebnyr\u00eb inkrementale. Merrni MySQL, b\u00ebni nj\u00eb \u00e7el\u00ebs unik - n\u00eb MySQL \u00ebsht\u00eb \u00e7el\u00ebsi i dyfisht\u00eb, nd\u00ebrsa n\u00eb PostgreSQL \u00ebsht\u00eb konflikt - dhe e shtoni me nj\u00eb plus. Kjo do t\u00eb funksionoj\u00eb shum\u00eb m\u00eb mir\u00eb. <\/p>\n<p><\/p>\n<p>Kur keni pak t\u00eb dh\u00ebna, nuk ka kuptim t\u00eb p\u00ebrdorni ClickHouse. Ka baza t\u00eb dh\u00ebnash t\u00eb zakonshme dhe ato e trajtojn\u00eb shum\u00eb mir\u00eb k\u00ebt\u00eb. <\/p>\n<p><\/p>\n<h2 id=\"anchorpimp-my-clickhouseanchorchto-podtyunit-v-clickhouse-chtoby-bolshe-dannyh-bylo-vnbspkeshe\"><noindex><a rel=\"nofollow\" name=\"pimp-my-clickhouse\"><\/a><\/noindex>\u00c7far\u00eb t\u00eb p\u00ebrmir\u00ebsojm\u00eb n\u00eb ClickHouse q\u00eb t\u00eb ket\u00eb m\u00eb shum\u00eb t\u00eb dh\u00ebna n\u00eb cache?<\/h2>\n<p><\/p>\n<blockquote><p>Le t\u00eb imagjinojm\u00eb nj\u00eb situat\u00eb - n\u00eb serverat ka 256 GB RAM, n\u00eb rutin\u00ebn e p\u00ebrditshme ClickHouse merr rreth 60-80 GB, n\u00eb pik\u00eb - deri n\u00eb 130. \u00c7far\u00eb mund t\u00eb aktivizoni dhe t\u00eb p\u00ebrshtatni, p\u00ebr t\u00eb pasur m\u00eb shum\u00eb t\u00eb dh\u00ebna n\u00eb cache dhe, p\u00ebrkat\u00ebsisht, m\u00eb pak vizita n\u00eb disk?<\/p><\/blockquote>\n<p>Si zakonisht, cache-i i faqes s\u00eb sistemit operativ e menaxhon mir\u00eb k\u00ebt\u00eb detyr\u00eb. N\u00ebse thjesht hapni topin, shikoni atje cached ose free \u2014 gjithashtu shkruhet sa \u00ebsht\u00eb e ruajtur n\u00eb cache \u2014 mund t\u00eb v\u00ebreni se e gjith\u00eb memoria e lir\u00eb \u00ebsht\u00eb p\u00ebrdorur p\u00ebr cache. Dhe k\u00ebto t\u00eb dh\u00ebna gjat\u00eb leximit do t\u00eb lexohen jo nga disku, por nga memorja e operativ\u00ebs. N\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, mund t\u00eb them se cache p\u00ebrdoret efektivisht, sepse t\u00eb dh\u00ebnat e kompresuara jan\u00eb ato q\u00eb ruhen.<\/p>\n<p><\/p>\n<p>Megjithat\u00eb, n\u00ebse d\u00ebshironi t\u00eb p\u00ebrshpejtoni disa k\u00ebrkesa t\u00eb thjeshta edhe m\u00eb shum\u00eb, ka mund\u00ebsi t\u00eb aktivizoni brenda ClickHouse cache-in p\u00ebr t\u00eb dh\u00ebnat e dekompresuara. Kjo quhet <strong>uncompressed cache<\/strong>. N\u00eb skedarin e konfigurimit config.xml vendosni madh\u00ebsin\u00eb e cache-it t\u00eb pakompresuar n\u00eb vler\u00ebn q\u00eb ju nevojitet \u2014 un\u00eb rekomandoj t\u00eb mos kaloni gjysm\u00ebn e memories s\u00eb lir\u00eb, sepse e gjith\u00eb pjesa tjet\u00ebr do t\u00eb shkoj\u00eb p\u00ebr cache-n e faqes. <\/p>\n<p><\/p>\n<p>P\u00ebr m\u00eb tep\u00ebr, ka dy konfigurime n\u00eb nivelin e k\u00ebrkes\u00ebs. Konfigurimi i par\u00eb \u2014 <strong>p\u00ebrdor cache t\u00eb pa kompresuar<\/strong> \u2014 p\u00ebrfshin p\u00ebrdorimin e tij. Rekomandohet t\u00eb aktivizohet p\u00ebr t\u00eb gjitha k\u00ebrkesat, p\u00ebrve\u00e7 atyre t\u00eb r\u00ebnda, q\u00eb mund t\u00eb lexojn\u00eb t\u00eb dh\u00ebnat e t\u00ebr\u00ebsishme dhe t\u00eb pastrojn\u00eb k\u00ebt\u00eb cache. Dhe cil\u00ebsimi i dyt\u00eb \u00ebsht\u00eb di\u00e7ka si numri maksimal i rreshtave p\u00ebr p\u00ebrdorimin e cache-it. Ai automatikisht kufizon k\u00ebrkesat e m\u00ebdha, n\u00eb m\u00ebnyr\u00eb q\u00eb ato t\u00eb kalojn\u00eb jasht\u00eb cache-it.<\/p>\n<p><\/p>\n<h2 id=\"anchorstorage-configurationanchorkak-mozhno-nastroit-storage_configuration-dlya-hraneniya-v-operativke\"><noindex><a rel=\"nofollow\" name=\"storage-configuration\"><\/a><\/noindex>Si mund t\u00eb konfigurojm\u00eb storage_configuration p\u00ebr ruajtjen n\u00eb memorie?<\/h2>\n<p><\/p>\n<blockquote><p>N\u00eb dokumentacionin e ri t\u00eb ClickHouse, kam lexuar nj\u00eb seksion lidhur <noindex><a rel=\"nofollow\" href=\"https:\/\/clickhouse.tech\/docs\/en\/single\/#table_engine-mergetree-multiple-volumes\">me storage t\u00eb dh\u00ebnash<\/a><\/noindex>. N\u00eb p\u00ebrshkrim ka nj\u00eb shembull me SSD t\u00eb shpejt\u00eb. <\/p>\n<p>\u00cbsht\u00eb interesante se si mund t\u00eb konfigurohet e nj\u00ebjta gj\u00eb me kujtes\u00ebn e nxeht\u00eb t\u00eb volumit. Dhe nj\u00eb pyetje tjet\u00ebr. Si funksionon select me k\u00ebt\u00eb organizim t\u00eb dh\u00ebnash, a do t\u00eb lexoj\u00eb gjith\u00eb setin ose vet\u00ebm at\u00eb q\u00eb ndodhet n\u00eb disk, dhe a kompresohen k\u00ebto t\u00eb dh\u00ebna n\u00eb memorie? Dhe si funksionon seksioni prewhere n\u00eb nj\u00eb organizim t\u00eb till\u00eb t\u00eb dh\u00ebnash?<\/p><\/blockquote>\n<p>Ky cil\u00ebsim ndikon n\u00eb ruajtjen e blloqeve t\u00eb t\u00eb dh\u00ebnave, dhe formati i tyre nuk ndryshon aspak.<br \/>\nLe t\u00eb shqyrtojm\u00eb m\u00eb holl\u00ebsisht. <\/p>\n<p><\/p>\n<p>Mund t\u00eb konfigurohet ruajtja e t\u00eb dh\u00ebnave n\u00eb memorien operative. Gjith\u00e7ka q\u00eb konfigurohet p\u00ebr diskun \u2014 \u00ebsht\u00eb rruga e tij. Ju krijoni nj\u00eb ndarje tmpfs, e cila \u00ebsht\u00eb e montuar ndonj\u00eb rrug\u00eb n\u00eb sistemin e skedave. Caktoni k\u00ebt\u00eb rrug\u00eb si rrug\u00eb p\u00ebr ruajtjen e t\u00eb dh\u00ebnave p\u00ebr ndarjen m\u00eb t\u00eb nxeht\u00eb, atje fillojn\u00eb t\u00eb hyjn\u00eb dhe t\u00eb shkruhen blloqet e t\u00eb dh\u00ebnave, gjith\u00e7ka \u00ebsht\u00eb mir\u00eb. <\/p>\n<p><\/p>\n<p>Poroshtrehuar, nuk e rekomandoj k\u00ebt\u00eb p\u00ebr shkak t\u00eb besueshm\u00ebris\u00eb s\u00eb ul\u00ebt, megjithat\u00eb, n\u00ebse keni t\u00eb pakt\u00ebn tre replika n\u00eb qendra t\u00eb ndryshme t\u00eb t\u00eb dh\u00ebnave, at\u00ebher\u00eb \u00ebsht\u00eb e mundur. N\u00eb rast se ndodhi ndonj\u00eb gj\u00eb, t\u00eb dh\u00ebnat do t\u00eb rinovohen. Le t\u00eb supozojm\u00eb se serveri \u00ebsht\u00eb fikur papritur dhe \u00ebsht\u00eb ndezur p\u00ebrs\u00ebri. N\u00ebnseksioni u montua p\u00ebrs\u00ebri, por aty \u00ebsht\u00eb bosh. Serveri ClickHouse, gjat\u00eb fillimit, sheh se atij i mungojn\u00eb k\u00ebto copa, megjithat\u00eb, sipas metadata-ve t\u00eb ZooKeeper, ato duhet t\u00eb jen\u00eb aty. Ai shikon se n\u00eb cilat replika ato jan\u00eb, i k\u00ebrkon ato dhe i shkarkon. N\u00eb k\u00ebt\u00eb m\u00ebnyr\u00eb, t\u00eb dh\u00ebnat do t\u00eb rinovohen. <\/p>\n<p><\/p>\n<p>N\u00eb k\u00ebt\u00eb kuptim, ruajtja e t\u00eb dh\u00ebnave n\u00eb memorje nuk ndryshon n\u00eb esenc\u00eb nga ruajtja e tyre n\u00eb disk, sepse kur t\u00eb dh\u00ebnat shkruhen n\u00eb disk, ato fillimisht kalojn\u00eb n\u00eb page cache dhe regjistrohen fizikisht n\u00eb m\u00ebnyr\u00eb t\u00eb vonuar. Kjo varet nga opsioni i montimit t\u00eb sistemit t\u00eb skedave. Por, p\u00ebr \u00e7do rast, do t\u00eb thosha se ClickHouse nuk b\u00ebn fsync gjat\u00eb insert.<\/p>\n<p><\/p>\n<p>Megjithat\u00eb, t\u00eb dh\u00ebnat n\u00eb memorje ruhen n\u00eb t\u00eb nj\u00ebjtin format si n\u00eb disk. K\u00ebrkesa select, gjithashtu, zgjedh copa q\u00eb duhet t\u00eb lexohen, zgjedh intervalet e nevojshme t\u00eb t\u00eb dh\u00ebnave n\u00eb copa dhe i lexon ato. Dhe prewhere funksionon ashtu si duhet, pa marr\u00eb parasysh n\u00ebse t\u00eb dh\u00ebnat ishin n\u00eb memorje apo n\u00eb disk.<\/p>\n<p><\/p>\n<h2 id=\"anchorlow-cardinalityanchordo-kakogo-kolichestva-unikalnyh-znacheniy-effektiven-low-cardinality\"><noindex><a rel=\"nofollow\" name=\"low-cardinality\"><\/a><\/noindex>Derisa deri n\u00eb sa vlera unike \u00ebsht\u00eb efektive Low Cardinality?<\/h2>\n<p><\/p>\n<p>Low Cardinality \u00ebsht\u00eb i dizajnuar me shum\u00eb zgjuarsi. Ai krijon dictionar\u00eb t\u00eb t\u00eb dh\u00ebnave, por ata jan\u00eb lokal\u00eb. S\u00eb pari, dictionar\u00ebt jan\u00eb t\u00eb ndrysh\u00ebm p\u00ebr \u00e7do cop\u00eb, dhe s\u00eb dyti, madje edhe brenda nj\u00eb cop\u00eb, ata mund t\u00eb jen\u00eb t\u00eb ndrysh\u00ebm p\u00ebr \u00e7do interval. Kur numri i vlerave unike arrin nj\u00eb num\u00ebr prag \u2014 mendoj, nj\u00eb milion \u2014 dictionari thjesht shtyhet, dhe krijohet nj\u00eb i ri.<\/p>\n<p><\/p>\n<p>P\u00ebrgjigjja n\u00eb p\u00ebrgjith\u00ebsi: p\u00ebr \u00e7do interval lokal \u2014 t\u00eb themi, p\u00ebr \u00e7do dit\u00eb \u2014 diku deri n\u00eb nj\u00eb milion vlera unike, Low Cardinality \u00ebsht\u00eb efikas. Pas k\u00ebsaj, do t\u00eb ket\u00eb thjesht nj\u00eb fallback, ku do t\u00eb p\u00ebrdoren shum\u00eb dictionar\u00eb t\u00eb ndrysh\u00ebm, e jo nj\u00eb. Do t\u00eb funksionoj\u00eb p\u00ebraf\u00ebrsisht si nj\u00eb kolon\u00eb e zakonshme tip string, ndoshta pak m\u00eb pak efikas, por nuk do t\u00eb ket\u00eb nj\u00eb p\u00ebrkeq\u00ebsim serioz t\u00eb performanc\u00ebs. <\/p>\n<p><\/p>\n<h2 id=\"anchorfulltext-searchanchorkakie-luchshie-praktiki-ponbsppolnotekstovomu-poisku-ponbsptablice-snbsppyatyu-milliardami-strok\"><noindex><a rel=\"nofollow\" name=\"fulltext-search\"><\/a><\/noindex>Cilat jan\u00eb praktikat m\u00eb t\u00eb mira p\u00ebr k\u00ebrkimin me tekst t\u00eb plot\u00eb n\u00eb nj\u00eb tabel\u00eb me pes\u00eb miliard\u00eb rreshta?<\/h2>\n<p><\/p>\n<p>Ka disa opsione p\u00ebr p\u00ebrgjigje. E para \u2014 thoni se ClickHouse nuk \u00ebsht\u00eb nj\u00eb sistem p\u00ebr k\u00ebrkimin me tekst t\u00eb plot\u00eb. P\u00ebr k\u00ebt\u00eb ka sisteme t\u00eb specializuara si p.sh., <noindex><a rel=\"nofollow\" href=\"https:\/\/www.elastic.co\/enterprise-search\">Elasticsearch<\/a><\/noindex> dhe <noindex><a rel=\"nofollow\" href=\"http:\/\/sphinxsearch.com\/\">Sphinx<\/a><\/noindex>. Megjithat\u00eb, un\u00eb po takohem gjithnj\u00eb e m\u00eb shpesh me njer\u00ebz q\u00eb thon\u00eb se po kalojn\u00eb nga Elasticsearch n\u00eb ClickHouse.<\/p>\n<p><\/p>\n<p>Pse ndodh kjo? Ata e shpjegojn\u00eb me faktin se Elasticsearch nuk arrin t\u00eb p\u00ebrballoj\u00eb ngarkes\u00ebn n\u00eb disa v\u00ebllime, duke filluar nga nd\u00ebrtimi i indekseve. Indekset b\u00ebhen shum\u00eb t\u00eb m\u00ebdha, dhe n\u00ebse thjesht transferoni t\u00eb dh\u00ebnat n\u00eb ClickHouse, do t\u00eb rezultoj\u00eb se ato ruhen n\u00eb disa her\u00eb m\u00eb efektivisht. N\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, k\u00ebrkesat shpesh nuk ishin p\u00ebr t\u00eb gjetur ndonj\u00eb fraz\u00eb n\u00eb t\u00eb gjith\u00eb v\u00ebllimin e t\u00eb dh\u00ebnave duke marr\u00eb parasysh morfologjin\u00eb, por di\u00e7ka krejt tjet\u00ebr. P\u00ebr shembull, p\u00ebr t\u00eb gjetur n\u00eb log-et e disa or\u00ebve t\u00eb fundit p\u00ebr ndonj\u00eb n\u00ebnsekuenc\u00eb byte.<\/p>\n<p><\/p>\n<p>N\u00eb k\u00ebt\u00eb rast, ju krijoni nj\u00eb indeks n\u00eb ClickHouse, n\u00eb t\u00eb cilin fusha e par\u00eb do t\u00eb jet\u00eb data me koh\u00ebn. Dhe ndarja m\u00eb e madhe e t\u00eb dh\u00ebnave do t\u00eb jet\u00eb pik\u00ebrisht sipas intervalit t\u00eb datave. Brenda intervalit t\u00eb zgjedhur t\u00eb datave, zakonisht \u00ebsht\u00eb e mundur t\u00eb realizoni k\u00ebrkimin me tekst t\u00eb plot\u00eb madje edhe me metod\u00ebn brute-force duke p\u00ebrdorur like. Operatori like n\u00eb ClickHouse \u00ebsht\u00eb operatori m\u00eb efektiv like q\u00eb mund t\u00eb gjeni. N\u00ebse gjeni ndonj\u00eb gj\u00eb m\u00eb t\u00eb mir\u00eb - m\u00eb tregoni. <\/p>\n<p><\/p>\n<p>Por prap\u00eb like \u00ebsht\u00eb nj\u00eb skanim i plot\u00eb. Dhe nj\u00eb skanim i plot\u00eb mund t\u00eb jet\u00eb i ngadalsh\u00ebm jo vet\u00ebm p\u00ebr CPU, por edhe p\u00ebr diskun. N\u00ebse ndonj\u00ebher\u00eb keni nj\u00eb terabajt t\u00eb dh\u00ebnash n\u00eb dit\u00eb, dhe gjat\u00eb dit\u00ebs k\u00ebrkoni ndonj\u00eb fjal\u00eb, do t\u00eb duhet t\u00eb skanoni nj\u00eb terabajt. Dhe ai sigurisht do t\u00eb jet\u00eb n\u00eb disqet standarde, dhe n\u00eb fund ata do t\u00eb jen\u00eb t\u00eb ngarkuar aq shum\u00eb sa q\u00eb nuk do t\u00eb mund t\u00eb hyni n\u00eb k\u00ebt\u00eb server p\u00ebrmes SSH.<\/p>\n<p><\/p>\n<p>N\u00eb k\u00ebt\u00eb rast, jam i gatsh\u00ebm t\u00eb ofroj nj\u00eb tjet\u00ebr truk t\u00eb vog\u00ebl. Ai \u00ebsht\u00eb nga kategoria eksperimentale - ndoshta do t\u00eb funksionoj\u00eb, ndoshta jo. N\u00eb ClickHouse, ekzistojn\u00eb indekse t\u00eb plota tekstesh n\u00eb form\u00ebn e filtrave Bloom trigram. Koleget e mi nga kompania Arenadata tashm\u00eb i kan\u00eb provuar k\u00ebto indekse, dhe shpesh ato funksionojn\u00eb pik\u00ebrisht si\u00e7 pritej.<\/p>\n<p><\/p>\n<p>P\u00ebr t\u00eb p\u00ebrdorur ato si\u00e7 duhet, duhet t\u00eb kuptoni mir\u00eb se si funksionojn\u00eb: \u00e7far\u00eb p\u00ebrfaq\u00ebson nj\u00eb filtrin Bloom trigram dhe si t\u00eb zgjidhni madh\u00ebsin\u00eb e tij. Mund t\u00eb them se do t\u00eb ndihmojn\u00eb p\u00ebr k\u00ebrkesat p\u00ebr ndonj\u00eb fraz\u00eb t\u00eb rrall\u00eb, n\u00ebnstringa q\u00eb shpesh nuk gjenden n\u00eb t\u00eb dh\u00ebna. N\u00eb k\u00ebt\u00eb rast, do t\u00eb zgjidhen n\u00ebnintervale p\u00ebrmes indekseve, dhe do t\u00eb lexohen m\u00eb pak t\u00eb dh\u00ebna.<\/p>\n<p><\/p>\n<p>S\u00eb fundmi n\u00eb ClickHouse u shfaq\u00ebn funksione m\u00eb t\u00eb avancuara p\u00ebr k\u00ebrkimin me tekst t\u00eb plot\u00eb. Kjo, s\u00eb pari, \u00ebsht\u00eb k\u00ebrkimi i nj\u00eb morie n\u00ebnstrigash n\u00eb nj\u00eb kalim, duke p\u00ebrfshir\u00eb variante me dhe pa marr\u00eb parasysh regjistrin, me mb\u00ebshtetje p\u00ebr UTF-8 ose vet\u00ebm p\u00ebr ASCII. Zgjidhni at\u00eb m\u00eb efikase q\u00eb ju nevojitet. <\/p>\n<p><\/p>\n<p>P\u00ebr m\u00eb tep\u00ebr, tani \u00ebsht\u00eb e mundur t\u00eb k\u00ebrkoni disa shprehje t\u00eb rregullta n\u00eb nj\u00eb kalim. Ju nuk keni nevoj\u00eb t\u00eb shkruani X like nj\u00eb n\u00ebnstring\u00eb ose X like nj\u00eb n\u00ebnstring\u00eb tjet\u00ebr. Shkruani menj\u00ebher\u00eb, dhe gjith\u00e7ka ekzekutohet maksimalisht efikasisht.<\/p>\n<p><\/p>\n<p>E treta \u2014 tani ka k\u00ebrkim t\u00eb af\u00ebrt p\u00ebr regex dhe k\u00ebrkim t\u00eb af\u00ebrt p\u00ebr n\u00ebnstriga. N\u00ebse dikush shkruan nj\u00eb fjal\u00eb me gabim, ajo do t\u00eb k\u00ebrkohet sipas p\u00ebrputhjes maksimale.<\/p>\n<p><\/p>\n<h2 id=\"anchorhello-and-welcomeanchorkak-luchshe-organizovat-dostup-vnbspclickhouse-dlyanbspbolshogo-kolichestva-polzovateley\"><noindex><a rel=\"nofollow\" name=\"hello-and-welcome\"><\/a><\/noindex>Si t\u00eb organizohet m\u00eb mir\u00eb aksesin n\u00eb ClickHouse p\u00ebr nj\u00eb num\u00ebr t\u00eb madh p\u00ebrdoruesish?<\/h2>\n<p><\/p>\n<blockquote><p>Tregonni se si t\u00eb organizoni m\u00eb mir\u00eb aksesin p\u00ebr nj\u00eb num\u00ebr t\u00eb madh konsumator\u00ebsh dhe analist\u00ebsh. Si t\u00eb formoni nj\u00eb radh\u00eb, t\u00eb prioritizoni k\u00ebrkesat p\u00ebr max concurrent queries, dhe \u00e7far\u00eb mjete?<\/p><\/blockquote>\n<p>N\u00ebse klusteri \u00ebsht\u00eb mjaft i madh, nj\u00eb zgjidhje e mir\u00eb do t\u00eb ishte ngritja e dy server\u00ebve t\u00eb tjer\u00eb, t\u00eb cil\u00ebt do t\u00eb sh\u00ebrbenin si pik\u00eb hyrjeje p\u00ebr analist\u00ebt. Kjo do t\u00eb thot\u00eb t\u00eb mos lejoni analist\u00ebt n\u00eb shard-\u00ebt e caktuar t\u00eb klasterit, por thjesht t\u00eb krijoni dy server\u00eb t\u00eb zbraz\u00ebt, pa t\u00eb dh\u00ebna, dhe n\u00eb to t\u00eb konfiguroni t\u00eb drejtat e aksesit. N\u00eb k\u00ebt\u00eb rast, konfigurimet e p\u00ebrdoruesve p\u00ebr k\u00ebrkesat e shp\u00ebrndara d\u00ebrgohen n\u00eb server\u00ebt e larg\u00ebt. Pra, ju konfiguroni gjith\u00e7ka n\u00eb k\u00ebta dy server\u00eb, dhe konfigurimet kan\u00eb efekt n\u00eb t\u00eb gjith\u00eb klasterin.<\/p>\n<p><\/p>\n<p>N\u00eb parim, k\u00ebta server\u00eb jan\u00eb pa t\u00eb dh\u00ebna, por sasia e RAM-it n\u00eb to \u00ebsht\u00eb mjaft e r\u00ebnd\u00ebsishme p\u00ebr ekzekutimin e k\u00ebrkesave. Disku gjithashtu mund t\u00eb p\u00ebrdoret p\u00ebr t\u00eb dh\u00ebna p\u00ebrkoh\u00ebsore, n\u00ebse \u00ebsht\u00eb aktivizuar agregimi i jasht\u00ebm ose renditja e jashtme.<\/p>\n<p><\/p>\n<p>\u00cbsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb shikoni n\u00eb konfigurimet q\u00eb lidhen me t\u00eb gjith\u00eb limitet e mundshme. N\u00ebse tani hyj n\u00eb klasterin \"Yandex.Metrica\" si analist dhe b\u00ebj nj\u00eb k\u00ebrkes\u00eb <strong>select count from hits<\/strong>, at\u00ebher\u00eb n\u00eb menj\u00ebher\u00eb do t\u00eb m\u00eb jepet nj\u00eb p\u00ebrjashtim q\u00eb nuk mund t\u00eb realizoj k\u00ebrkes\u00ebn. Numri maksimal i rreshtave q\u00eb m\u00eb lejohet t\u00eb skanojj\u00eb \u00ebsht\u00eb nj\u00ebqind miliard\u00eb, dhe gjithsej n\u00eb klaster ka pes\u00ebdhjet\u00eb trilion\u00eb n\u00eb nj\u00eb tabel\u00eb. Kjo \u00ebsht\u00eb kufizimi i par\u00eb. <\/p>\n<p><\/p>\n<p>Supozoni se e heq kufizimin mbi numrin e rreshtave dhe e ekzekutoj k\u00ebrkes\u00ebn p\u00ebrs\u00ebri. At\u00ebher\u00eb do t\u00eb shoh nj\u00eb p\u00ebrjashtim t\u00eb ardhsh\u00ebm \u2014 konfigurimi \u00ebsht\u00eb aktivizuar <strong>force index by date<\/strong>. Nuk mund ta kryej k\u00ebrkes\u00ebn n\u00ebse nuk kam caktuar nj\u00eb gam\u00eb datash. Nuk duhet t\u00eb llogarisim se analist\u00ebt do ta caktuan at\u00eb manualisht. Nj\u00eb rast tipik \u00ebsht\u00eb shkrimi i gam\u00ebs s\u00eb datave ku data e ngjarjes \u00ebsht\u00eb midis jav\u00ebs. Dhe pastaj thjesht nuk e vendos\u00ebn sakt\u00eb parenthez\u00ebn, dhe n\u00eb vend t\u00eb and \u00ebsht\u00eb b\u00ebr\u00eb or \u2014 or URL match. N\u00ebse nuk ka kufizime, do t\u00eb filloj\u00eb t\u00eb skanoj\u00eb kolon\u00ebn URL dhe do t\u00eb shpenzoj\u00eb nj\u00eb tonelat\u00eb burimesh.<\/p>\n<p><\/p>\n<p>P\u00ebr m\u00eb tep\u00ebr, n\u00eb ClickHouse ka dy cil\u00ebsime preferenciale. Fatkeq\u00ebsisht, ato jan\u00eb shum\u00eb primitiv\u00eb. Nj\u00ebra quhet thjesht <strong>priority<\/strong>. N\u00ebse prioriteti \u2260 0, dhe ekzekutohen k\u00ebrkesa me nj\u00ebfar\u00eb prioriteti, por n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb ekzekutohet nj\u00eb k\u00ebrkes\u00eb me prioritet q\u00eb ka nj\u00eb vler\u00eb m\u00eb t\u00eb ul\u00ebt, e cila do t\u00eb thot\u00eb prioritet m\u00eb t\u00eb lart\u00eb, at\u00ebher\u00eb k\u00ebrkesa me nj\u00eb vler\u00eb prioriteti m\u00eb t\u00eb lart\u00eb, q\u00eb p\u00ebrfaq\u00ebson prioritet m\u00eb t\u00eb ul\u00ebt, thjesht pezullohet dhe nuk do t\u00eb funksionoj\u00eb fare gjat\u00eb asaj kohe.<\/p>\n<p><\/p>\n<p>Ky \u00ebsht\u00eb nj\u00eb konfigurim shum\u00eb i eg\u00ebr, dhe nuk p\u00ebrshtatet p\u00ebr ato raste kur n\u00eb klaster ka nj\u00eb ngarkes\u00eb t\u00eb vazhdueshme. Por n\u00ebse keni k\u00ebrkesa t\u00eb shkurtra, impulsive t\u00eb r\u00ebnd\u00ebsishme, nd\u00ebrsa kryesisht klasteri \u00ebsht\u00eb i papun\u00eb, ky konfigurim do t\u00eb jet\u00eb i p\u00ebrshtatsh\u00ebm.<\/p>\n<p><\/p>\n<p>Konfigurimi i ardhsh\u00ebm i prioriteteve quhet <strong>prioriteti i thread-ave OS<\/strong>. Thjesht vendos n\u00eb t\u00eb gjitha rrjedhat e ekzekutimit t\u00eb k\u00ebrkes\u00ebs vler\u00ebn e nice p\u00ebr programin e planifikimit Linux. Funksionon mesatarisht, por megjithat\u00eb funksionon. N\u00ebse vendosni vler\u00ebn m\u00eb minimale t\u00eb nice \u2014 ajo \u00ebsht\u00eb m\u00eb e madhe n\u00eb madh\u00ebsi, dhe do t\u00eb thot\u00eb prioriteti m\u00eb i ul\u00ebt \u2014 dhe p\u00ebr k\u00ebrkesat me prioritet t\u00eb lart\u00eb vendosni -19, at\u00ebher\u00eb CPU do t\u00eb konsumoj\u00eb k\u00ebrkesat me prioritet t\u00eb ul\u00ebt rreth kat\u00ebr her\u00eb m\u00eb pak se ato me prioritet t\u00eb lart\u00eb. <\/p>\n<p><\/p>\n<p>Gjithashtu duhet t\u00eb konfigurohet koha maksimale e ekzekutimit t\u00eb k\u00ebrkes\u00ebs \u2014 le t\u00eb themi, pes\u00eb minuta. Shpejt\u00ebsia minimale e ekzekutimit t\u00eb k\u00ebrkes\u00ebs \u2014 kjo \u00ebsht\u00eb m\u00eb e r\u00ebnd\u00ebsishmja. Kjo konfigurim \u00ebsht\u00eb e njohur prej koh\u00ebsh, dhe k\u00ebrkohet, q\u00eb t\u00eb mos thjesht t\u00eb pohosh se ClickHouse nuk ngadal\u00ebson, por p\u00ebr ta detyruar at\u00eb.<\/p>\n<p><\/p>\n<p>Imagjinoni, po konfiguroni: n\u00ebse ndonj\u00eb k\u00ebrkes\u00eb p\u00ebrpunon m\u00eb pak se nj\u00eb milion rreshta n\u00eb sekond\u00eb \u2014 kjo nuk duhet b\u00ebr\u00eb. Kjo na turp\u00ebron emrin ton\u00eb t\u00eb mir\u00eb, baz\u00ebn ton\u00eb t\u00eb mir\u00eb t\u00eb dh\u00ebnash. Le t\u00eb ndalojm\u00eb thjesht k\u00ebt\u00eb. Atje n\u00eb t\u00eb v\u00ebrtet\u00eb ka dy cil\u00ebsime. Nj\u00ebra quhet <strong>shpejt\u00ebsia minimale e ekzekutimit<\/strong> \u2014 n\u00eb rreshta p\u00ebr sekond\u00eb, dhe e dyta quhet timeout para kontrollit t\u00eb shpejt\u00ebsis\u00eb minimale t\u00eb ekzekutimit \u2014 n\u00eb m\u00ebnyr\u00eb t\u00eb paracaktuar pes\u00ebmb\u00ebdhjet\u00eb sekonda. Pra, pes\u00ebmb\u00ebdhjet\u00eb sekonda \u00ebsht\u00eb e pranueshme, dhe pastaj, n\u00ebse \u00ebsht\u00eb e ngadalt\u00eb, mund t\u00eb thjesht t\u00eb hedh\u00ebsh nj\u00eb p\u00ebrjashtim \u2014 t\u00eb nd\u00ebrpres\u00ebsh k\u00ebrkes\u00ebn.<\/p>\n<p><\/p>\n<p>Gjithashtu duhet t\u00eb konfigurohen kuotat. N\u00eb ClickHouse ka nj\u00eb mund\u00ebsi t\u00eb nd\u00ebrtuar p\u00ebr kuota, e cila llogarit konsumimin e burimeve. Por, fatkeq\u00ebsisht, jo burimeve fizike si CPU, disqet, por ato logjike \u2014 numri i k\u00ebrkesave t\u00eb trajtuara, rreshta dhe bajt\u00eb e lexuar. Dhe mund t\u00eb konfigurohet, p\u00ebr shembull, maksimumi nj\u00ebqind k\u00ebrkesa brenda pes\u00eb minutash dhe nj\u00eb mij\u00eb k\u00ebrkesa n\u00eb or\u00eb.<\/p>\n<p><\/p>\n<p>Pse \u00ebsht\u00eb kjo e r\u00ebnd\u00ebsishme? Sepse disa nga k\u00ebrkesat e analitik\u00ebs do t\u00eb kryhen manualisht drejtp\u00ebrdrejt nga klienti ClickHouse. Dhe gjith\u00e7ka do t\u00eb shkoj\u00eb mir\u00eb. Por n\u00ebse keni analist\u00eb t\u00eb avancuar n\u00eb kompanin\u00eb tuaj, ata do t\u00eb shkruajn\u00eb nj\u00eb skript, dhe n\u00eb skript mund t\u00eb ket\u00eb nj\u00eb gabim. Dhe ky gabim do t\u00eb \u00e7oj\u00eb n\u00eb ekzekutimin e k\u00ebrkes\u00ebs n\u00eb nj\u00eb cik\u00ebl t\u00eb pafund. K\u00ebshtu q\u00eb duhet t\u00eb mbroheni nga kjo.<\/p>\n<p><\/p>\n<h2 id=\"anchorsmorgasbordanchormozhno-li-otdat-rezultaty-odnogo-zaprosa-desyati-klientam\"><noindex><a rel=\"nofollow\" name=\"smorgasbord\"><\/a><\/noindex>A mund t\u00eb jepen rezultatet e nj\u00eb k\u00ebrkese n\u00eb dhjet\u00eb klient\u00eb?<\/h2>\n<p><\/p>\n<blockquote><p>Kemi disa p\u00ebrdorues q\u00eb preferojn\u00eb t\u00eb vijn\u00eb me k\u00ebrkesa shum\u00eb t\u00eb m\u00ebdha n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb. K\u00ebrkesa \u00ebsht\u00eb e madhe, ekzekutohet n\u00eb t\u00ebr\u00ebsi shpejt, por p\u00ebr shkak se ka kaq shum\u00eb k\u00ebrkesa n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, b\u00ebhet shum\u00eb e dhimbshme. A mund t\u00eb ekzekutohet nj\u00eb k\u00ebrkes\u00eb e nj\u00ebjt\u00eb, e cila ka ardhur dhjet\u00eb her\u00eb radhazi, vet\u00ebm nj\u00eb her\u00eb, dhe rezultati i saj t'i jepet dhjet\u00eb klient\u00ebve?<\/p><\/blockquote>\n<p>Problemi \u00ebsht\u00eb se ne nuk kemi rezultate t\u00eb memorjes s\u00eb caches apo t\u00eb caches t\u00eb t\u00eb dh\u00ebnave nd\u00ebrmjet. Ka nj\u00eb page cache t\u00eb sistemit operativ, e cila do t\u00eb lejoj\u00eb t\u00eb mos lexojm\u00eb t\u00eb dh\u00ebnat nga disku p\u00ebrs\u00ebri, por, fatkeq\u00ebsisht, t\u00eb dh\u00ebnat do t\u00eb decompressohen, deserializohen dhe do t\u00eb p\u00ebrpunohen p\u00ebrs\u00ebri. <\/p>\n<p><\/p>\n<p>D\u00ebshirojm\u00eb ta shmangim k\u00ebt\u00eb n\u00eb nj\u00eb far\u00eb m\u00ebnyre, ose duke cache-uar t\u00eb dh\u00ebnat nd\u00ebrmjet, ose duke radhitur k\u00ebrkesa t\u00eb ngjashme n\u00eb nj\u00eb radh\u00eb dhe duke shtuar cache rezultatet. Aktualisht kemi n\u00eb zhvillim nj\u00eb pull request q\u00eb shton cache-imin e k\u00ebrkesave, por vet\u00ebm p\u00ebr n\u00ebn-k\u00ebrkesat n\u00eb seksionin in dhe join \u2014 pra, zgjidhja \u00ebsht\u00eb e paplot\u00eb.<\/p>\n<p><\/p>\n<p>Megjithat\u00eb, na rezulton t\u00eb kemi nj\u00eb situat\u00eb t\u00eb till\u00eb. Nj\u00eb shembull kanonik \u2014 jan\u00eb k\u00ebrkesat me paginim. Ka nj\u00eb raport, n\u00eb t\u00eb cilin ka disa faqe, dhe b\u00ebhet k\u00ebrkesa limit 10. M\u00eb pas e nj\u00ebjta gj\u00eb, por limit 10,10. Pastaj vjen edhe nj\u00eb faqe tjet\u00ebr. Dhe pyetja \u00ebsht\u00eb, pse ne e num\u00ebrojm\u00eb k\u00ebt\u00eb gjithmon\u00eb? Por tani nuk ka zgjidhje, dhe nuk mund ta shmangim.<\/p>\n<p><\/p>\n<p>Ka nj\u00eb zgjidhje alternative, e cila instalohet si nj\u00eb sidcar pran\u00eb ClickHouse \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/Vertamedia\/chproxy\">ClickHouse Proxy<\/a><\/noindex>.<\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> N\u00eb ClickHouse Proxy ka nj\u00eb limiter t\u00eb integruar dhe nj\u00eb memorie cache p\u00ebr rezultatet. Jan\u00eb b\u00ebr\u00eb shum\u00eb cil\u00ebsime, sepse \u00ebsht\u00eb zgjidhur nj\u00eb detyr\u00eb e ngjashme. Proxy lejon t\u00eb kufizoni k\u00ebrkesat, duke i radhitur ato, dhe \u00ebsht\u00eb e mundur t\u00eb configuroni sa gjat\u00eb jeton cache i k\u00ebrkesave. N\u00ebse k\u00ebrkesat jan\u00eb v\u00ebrtet t\u00eb nj\u00ebjta, Proxy i kthehet ato shum\u00eb her\u00eb, ndersa nd\u00ebrlikon n\u00eb ClickHouse vet\u00ebm nj\u00eb her\u00eb.<\/p>\n<p><\/p>\n<p>N\u00eb Nginx ka gjithashtu nj\u00eb cache n\u00eb versionin falas, dhe kjo gjithashtu do t\u00eb funksionoj\u00eb. Nginx madje ka cil\u00ebsime q\u00eb, n\u00ebse k\u00ebrkesat arrijn\u00eb n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, do t\u00eb ngadal\u00ebsoj\u00eb t\u00eb tjerat derisa nj\u00eb t\u00eb p\u00ebrfundoj\u00eb. Por pik\u00ebrisht n\u00eb ClickHouse Proxy, cil\u00ebsimi \u00ebsht\u00eb b\u00ebr\u00eb shum\u00eb m\u00eb mir\u00eb. Ai \u00ebsht\u00eb projektuar pik\u00ebrisht p\u00ebr ClickHouse, p\u00ebr k\u00ebto k\u00ebrkesa, prandaj \u00ebsht\u00eb m\u00eb i p\u00ebrshtatsh\u00ebm. Po ashtu, instalohet leht\u00eb. <\/p>\n<p><\/p>\n<h2 id=\"anchorasynchronousanchorkak-byt-snbspasinhronnymi-operaciyami-i-materializovannymi-predstavleniyami\"><noindex><a rel=\"nofollow\" name=\"asynchronous\"><\/a><\/noindex>Si t\u00eb merret me operacionet asinkrone dhe pamjet e materializuara?<\/h2>\n<p><\/p>\n<blockquote><p>Ekziston nj\u00eb problem, q\u00eb operacionet me motorin e replacement-it jan\u00eb asinkrone \u2014 fillimisht regjistrohen t\u00eb dh\u00ebnat, pastaj ndodh kompresion i tyre. N\u00ebse n\u00ebn tabel\u00ebn ekziston nj\u00eb tabel\u00eb e materializuar me disa agrega, do t\u00eb regjistrohen dyfishime n\u00eb t\u00eb. Dhe n\u00ebse nuk ka ndonj\u00eb logjik\u00eb t\u00eb nd\u00ebrlikuar, t\u00eb dh\u00ebnat do t\u00eb dyfishohen. \u00c7far\u00eb mund t\u00eb b\u00ebhet p\u00ebr k\u00ebt\u00eb?<\/p>\n<p>Ka nj\u00eb zgjidhje t\u00eb qart\u00eb \u2014 t\u00eb implementohet nj\u00eb trigger p\u00ebr nj\u00eb klas\u00eb t\u00eb caktuar matviews gjat\u00eb operacionit asinkron t\u00eb kompresionit. A ka ndonj\u00eb \u00abplumb argjendi\u00bb, plane p\u00ebr implementimin e funksionaliteteve t\u00eb k\u00ebtij lloji?<\/p><\/blockquote>\n<p>Duhet t\u00eb merremi me m\u00ebnyr\u00ebn se si funksionon deduplication. Ajo p\u00ebr t\u00eb cil\u00ebn po flas tani, nuk lidhet me \u00e7\u00ebshtjen, por p\u00ebr \u00e7do rast, \u00ebsht\u00eb e r\u00ebnd\u00ebsishme ta mbani mend.<\/p>\n<p><\/p>\n<p>Kur vendosni n\u00eb nj\u00eb tabel\u00eb t\u00eb replikimit, ka deduplikim t\u00eb blloqeve t\u00eb vendosura t\u00ebr\u00ebsisht. N\u00ebse vendosni p\u00ebrs\u00ebri t\u00eb nj\u00ebjtin bllok q\u00eb p\u00ebrmban t\u00eb nj\u00ebjt\u00ebn sasi t\u00eb t\u00eb nj\u00ebjtave rreshta n\u00eb t\u00eb nj\u00ebjtin rend, t\u00eb dh\u00ebnat deduplikohen. Do t\u00eb merrni \"Ok\" si p\u00ebrgjigje p\u00ebr insert, por n\u00eb t\u00eb v\u00ebrtet\u00eb do t\u00eb ruhet nj\u00eb grup t\u00eb dh\u00ebnash, dhe ato nuk do t\u00eb duplikohen.<\/p>\n<p><\/p>\n<p>Kjo \u00ebsht\u00eb e nevojshme p\u00ebr sakt\u00ebsi. N\u00ebse gjat\u00eb vendosjes merrni \"Ok\", at\u00ebher\u00eb t\u00eb dh\u00ebnat tuaja jan\u00eb vendosur. N\u00ebse merrni nj\u00eb gabim nga ClickHouse, at\u00ebher\u00eb ato nuk jan\u00eb vendosur dhe duhet t\u00eb p\u00ebrs\u00ebrisni vendosjen. Por n\u00ebse gjat\u00eb vendosjes \u00ebsht\u00eb prishur lidhja, at\u00ebher\u00eb nuk e dini, n\u00ebse jan\u00eb vendosur t\u00eb dh\u00ebnat apo jo. Nj\u00eb zgjidhje e vetme \u00ebsht\u00eb t\u00eb p\u00ebrs\u00ebrisni p\u00ebrs\u00ebri vendosjen. N\u00ebse t\u00eb dh\u00ebnat n\u00eb t\u00eb v\u00ebrtet\u00eb ishin vendosur dhe ju i vendosni p\u00ebrs\u00ebri, ka deduplikim t\u00eb blloqeve. Kjo \u00ebsht\u00eb e nevojshme p\u00ebr t\u00eb shmangur duplikatat. <\/p>\n<p><\/p>\n<p>Dhe ka r\u00ebnd\u00ebsi gjithashtu, si funksionon p\u00ebr paraqitjet materializuese. N\u00ebse t\u00eb dh\u00ebnat jan\u00eb deduplikuar gjat\u00eb vendosjes n\u00eb tabel\u00ebn kryesore, at\u00ebher\u00eb ato gjithashtu nuk do t\u00eb kalojn\u00eb n\u00eb paraqitjen materializuese.<\/p>\n<p><\/p>\n<p>Tani p\u00ebr sa i p\u00ebrket pyetjes. Keni nj\u00eb situat\u00eb m\u00eb t\u00eb nd\u00ebrlikuar, sepse po regjistroni duplikatat e rreshtave t\u00eb ve\u00e7ant\u00eb. Pra, nuk \u00ebsht\u00eb nj\u00eb grup i t\u00ebr\u00ebsish\u00ebm q\u00eb \u00ebsht\u00eb duplikuar, por specifikisht rreshta t\u00eb ve\u00e7ant\u00eb, dhe ato p\u00ebrfundojn\u00eb n\u00eb sfond. N\u00eb t\u00eb v\u00ebrtet\u00eb, t\u00eb dh\u00ebnat do t\u00eb p\u00ebrfundojn\u00eb n\u00eb tabel\u00ebn kryesore, dhe n\u00eb paraqitjen materializuese do t\u00eb shkojn\u00eb ato q\u00eb nuk jan\u00eb p\u00ebrfunduara, dhe gjat\u00eb bashkimeve nuk do t\u00eb ndodh\u00eb asgj\u00eb me paraqitjet materializuese. Sepse paraqitja materializuese \u00ebsht\u00eb asgj\u00eb m\u00eb shum\u00eb se nj\u00eb gjendje p\u00ebr insert. Gjat\u00eb operacioneve t\u00eb tjera, nuk ndodh m\u00eb shum\u00eb me t\u00eb.<\/p>\n<p><\/p>\n<p>Dhe k\u00ebtu nuk mund t\u00eb g\u00ebzoj. Duhet vet\u00ebm t\u00eb k\u00ebrkoni nj\u00eb zgjidhje t\u00eb sakt\u00eb p\u00ebr k\u00ebt\u00eb rast. P\u00ebr shembull, a \u00ebsht\u00eb e mundur q\u00eb n\u00eb paraqitjen materializuese ta b\u00ebni gjithashtu z\u00ebvend\u00ebsimin e saj, dhe ndoshta m\u00ebnyra e deduplikimit do t\u00eb funksionoj\u00eb po ashtu. Por fatkeq\u00ebsisht, jo gjithmon\u00eb. N\u00ebse \u00ebsht\u00eb agreguese, at\u00ebher\u00eb nuk do t\u00eb jet\u00eb e mundur. <\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> Edhe ne kemi pasur nj\u00eb histori t\u00eb ngjashme n\u00eb nd\u00ebrtimin e sistemit ton\u00eb. Kishim nj\u00eb problem, se kishte shfaqje reklamesh dhe disa t\u00eb dh\u00ebna q\u00eb mund t\u2019i shfaqnim n\u00eb koh\u00eb reale \u2014 ato ishin thjesht shfaqje. Rrall\u00eb her\u00eb ato dublokoheshin, por, n\u00ebse ndodhte kjo, ne gjithsesi i kompresonim m\u00eb pas. E kishim edhe nj\u00eb gj\u00eb q\u00eb nuk mund t\u00eb dublokohej \u2014 klikimet dhe e gjith\u00eb kjo histori. Megjithat\u00eb, ishte d\u00ebshira p\u00ebr t\u2019i shfaqur ato pothuajse menj\u00ebher\u00eb.<\/p>\n<p><\/p>\n<p>Si jan\u00eb b\u00ebr\u00eb pamjet e materializuara? Kishim pamje ku shkruhet drejtp\u00ebrdrejt \u2014 b\u00ebhet regjistrimi n\u00eb t\u00eb dh\u00ebnat e pap\u00ebrpunuara dhe shkruhet n\u00eb pamje. N\u00eb nj\u00eb moment t\u00eb caktuar, t\u00eb dh\u00ebnat nuk ishin shum\u00eb t\u00eb sakta, ato dublokoheshin dhe k\u00ebshtu me radh\u00eb. Dhe ka nj\u00eb pjes\u00eb t\u00eb dyt\u00eb t\u00eb tabel\u00ebs, ku ato duken po ashtu si pamjet e materializuara, pra struktura e tyre \u00ebsht\u00eb krejt\u00ebsisht e nj\u00ebjt\u00eb. Nj\u00eb her\u00eb n\u00eb nj\u00eb far\u00eb kohe ne ri-llogaritnim t\u00eb dh\u00ebnat, p\u00ebrfshijm\u00eb t\u00eb dh\u00ebna pa dublika dhe shkruajm\u00eb n\u00eb ato tabela. <\/p>\n<p><\/p>\n<p>Ne shkuam p\u00ebrmes API-s\u00eb \u2014 n\u00eb ClickHouse nuk do t\u00eb funksiononte t\u00eb punonit me duar. Dhe API-ja shqyrton: kur kam dat\u00ebn e fundit t\u00eb shtimit n\u00eb tabel\u00eb, ku ka t\u00eb dh\u00ebna t\u00eb sakta t\u00eb garantuara, ai b\u00ebn nj\u00eb k\u00ebrkes\u00eb n\u00eb nj\u00eb tabel\u00eb dhe n\u00eb nj\u00eb tabel\u00eb tjet\u00ebr. Nga nj\u00eb k\u00ebrkes\u00eb zgjidh deri n\u00eb nj\u00eb sasi t\u00eb caktuar kohe, dhe nga tjetra merr ato q\u00eb ende nuk jan\u00eb llogaritur. Dhe kjo funksionon, por jo me mjete t\u00eb vet\u00ebm ClickHouse.<\/p>\n<p><\/p>\n<p>N\u00ebse keni ndonj\u00eb API \u2014 p\u00ebr analist\u00ebt, p\u00ebr p\u00ebrdoruesit \u2014 n\u00eb thelb, \u00ebsht\u00eb nj\u00eb opsion. Ju gjithmon\u00eb ri-llogaritni, gjithmon\u00eb shkruani p\u00ebrs\u00ebri. Mund ta b\u00ebni k\u00ebt\u00eb nj\u00eb her\u00eb n\u00eb dit\u00eb ose n\u00eb ndonj\u00eb koh\u00eb tjet\u00ebr. Ju vet\u00eb zgjidhni intervalin q\u00eb nuk keni nevoj\u00eb dhe nuk \u00ebsht\u00eb kritik.<\/p>\n<p><\/p>\n<h2 id=\"anchordashboardanchorv-clickhouse-mnogo-logov-kak-ya-mogu-videt-vsyo-chto-proishodit-s-serverom-vnbspmomente\"><noindex><a rel=\"nofollow\" name=\"dashboard\"><\/a><\/noindex>N\u00eb ClickHouse ka shum\u00eb mund\u00ebsi regjistrimesh. Si mund t\u00eb shoh gjith\u00e7ka q\u00eb ndodh me serverin n\u00eb momentin aktual?<\/h2>\n<p><\/p>\n<blockquote><p>N\u00eb ClickHouse ka nj\u00eb num\u00ebr shum\u00eb t\u00eb madh regjistrimesh t\u00eb ndryshme, dhe ky num\u00ebr po rritet. N\u00eb versionet e reja, disa prej tyre madje jan\u00eb t\u00eb aktivizuara si parazgjedhje, n\u00eb versionet e vjetra duhet aktivizuar gjat\u00eb p\u00ebrdit\u00ebsimit. Megjithat\u00eb, ato po b\u00ebhen gjithnj\u00eb e m\u00eb shum\u00eb. Do t\u00eb d\u00ebshiroja t\u00eb shihja n\u00eb fund se \u00e7far\u00eb po ndodh aktualisht me serverin tim, ndoshta n\u00eb ndonj\u00eb panel p\u00ebrmbledh\u00ebs. <\/p>\n<p>A ju nuk keni n\u00eb ekipin tuaj ClickHouse, ose n\u00eb ekipet e miqve tuaj, ndonj\u00eb funksionalitet dashboard-esh t\u00eb gatsh\u00ebm q\u00eb do t\u00eb shfaqnin k\u00ebto log-e n\u00eb form\u00ebn e nj\u00eb produkti t\u00eb gatsh\u00ebm? N\u00eb fund t\u00eb fundit, t\u00eb shoh\u00ebsh log-et vet\u00ebm n\u00eb ClickHouse \u00ebsht\u00eb interesante. Por do t\u00eb ishte shum\u00eb e bukur n\u00ebse do t\u00eb ishte gati si nj\u00eb dashboard. Do ta vler\u00ebsoja shum\u00eb k\u00ebt\u00eb. <\/p><\/blockquote>\n<p>Ka dashboard-e, megjithat\u00eb ato nuk jan\u00eb t\u00eb standardizuara. N\u00eb kompanin\u00eb ton\u00eb rreth 60 ekipe p\u00ebrdorin ClickHouse, dhe e \u00e7uditshmja \u00ebsht\u00eb se shum\u00eb prej tyre kan\u00eb dashboard-e q\u00eb i kan\u00eb b\u00ebr\u00eb vet\u00eb, dhe disi t\u00eb ndryshme. Disa ekipe p\u00ebrdorin instalimin e brendsh\u00ebm t\u00eb 'Yandex.Cloud'. Aty ka disa raporte t\u00eb gatshme, megjithat\u00eb jo t\u00eb gjitha t\u00eb nevojshmet. T\u00eb tjer\u00ebt kan\u00eb t\u00eb tyret. <\/p>\n<p><\/p>\n<p>Koleg\u00ebt e mi nga 'Metri' kan\u00eb dashboard-in e tyre n\u00eb Grafana, nd\u00ebrsa un\u00eb kam t\u00eb mien mbi klasterin e tyre. Atje shikoj gj\u00ebra si hit cache p\u00ebr caches. Dhe madje \u00ebsht\u00eb akoma m\u00eb e komplikuar sepse ne p\u00ebrdorim mjete t\u00eb ndryshme. Dashboard-in tim e kam krijuar n\u00eb nj\u00eb mjet t\u00eb vjet\u00ebr, q\u00eb quhet Graphite-web. Ai \u00ebsht\u00eb krejt\u00ebsisht i sh\u00ebmtuar. Dhe un\u00eb ende e p\u00ebrdor at\u00eb, megjith\u00ebse Grafana ndoshta do t\u00eb ishte m\u00eb e leht\u00eb dhe m\u00eb e bukur. <\/p>\n<p><\/p>\n<p>Gj\u00ebja bazike n\u00eb dashboard-e \u00ebsht\u00eb e nj\u00ebjt\u00eb. Ato jan\u00eb metrikat sistemore p\u00ebr klasterin: CPU, memorie, disk, rrjet. T\u00eb tjera jan\u00eb numri i k\u00ebrkesave t\u00eb nj\u00ebkohshme, numri i bashkimeve t\u00eb nj\u00ebkohshme, numri i k\u00ebrkesave p\u00ebr sekond\u00eb, numri maksimal i pjes\u00ebve p\u00ebr partit\u00eb e tabelave MergeTree, vonesa e replikimit, madh\u00ebsia e radh\u00ebs s\u00eb replikimit, numri i rreshtave t\u00eb futur p\u00ebr sekond\u00eb, numri i blloqeve t\u00eb futur p\u00ebr sekond\u00eb. K\u00ebto jan\u00eb gjith\u00e7ka q\u00eb vjen nga metrikat, jo nga log-et.<\/p>\n<p><\/p>\n<p><strong>Vladimir Kolobaev:<\/strong> Aleksand\u00ebr, doja t\u00eb b\u00ebja nj\u00eb sakt\u00ebsim t\u00eb vog\u00ebl. Ka Grafana. Grafana ka nj\u00eb datasource, q\u00eb \u00ebsht\u00eb ClickHouse. Pra, un\u00eb mund t\u00eb b\u00ebj k\u00ebrkesa t\u00eb menj\u00ebhershme n\u00eb ClickHouse nga Grafana. N\u00eb ClickHouse ka nj\u00eb tabel\u00eb me log-e, e cila \u00ebsht\u00eb e nj\u00ebjt\u00eb p\u00ebr t\u00eb gjith\u00eb. Un\u00eb dua q\u00eb n\u00eb Grafana t\u00eb kthehem te kjo tabel\u00eb log-esh dhe t\u00eb shoh k\u00ebrkesat q\u00eb vendos serveri im. Do t\u00eb ishte mjaft mir\u00eb t\u00eb kisha nj\u00eb dashboard t\u00eb till\u00eb.<\/p>\n<p><\/p>\n<p>E kam nd\u00ebrtuar vet\u00eb. Por m\u00eb lind nj\u00eb pyetje \u2014 n\u00ebse gjith\u00e7ka \u00ebsht\u00eb e standardizuar, dhe Grafana p\u00ebrdoret nga t\u00eb gjith\u00eb, pse nuk ka nj\u00eb dashboard t\u00eb till\u00eb zyrtar n\u00eb 'Yandex'?<\/p>\n<p><\/p>\n<p><strong>Kirill Shvakov:<\/strong> N\u00eb t\u00eb v\u00ebrtet\u00eb, datasource q\u00eb lidhet me ClickHouse tani mb\u00ebshtetet nga Altinity. Dhe un\u00eb thjesht dua t\u00eb jap nj\u00eb drejtim se ku t\u00eb k\u00ebrkoni dhe kujt t'i drejtoheni. Mund t\u00eb pyesni ata, sepse 'Yandex' gjithsesi e zhvillon ClickHouse, e jo historin\u00eb p\u00ebrreth tij. Altinity \u00ebsht\u00eb kompanin\u00eb kryesore q\u00eb aktualisht po promovon ClickHouse. Ata nuk do ta braktisin, por do ta mb\u00ebshtesin. Sepse n\u00eb parim, p\u00ebr t\u00eb ngarkuar nj\u00eb dashboard n\u00eb sitin Grafana, duhet vet\u00ebm t\u00eb regjistrohesh dhe ta ngarkosh \u2014 nuk ka ndonj\u00eb problem t\u00eb ve\u00e7ant\u00eb. <\/p>\n<p><\/p>\n<p><strong>Alexey Milovidov:<\/strong> Gjat\u00eb vitit t\u00eb fundit, ClickHouse ka shtuar shum\u00eb mund\u00ebsi p\u00ebr profilimin e k\u00ebrkesave. Ka metrika p\u00ebr \u00e7do k\u00ebrkes\u00eb p\u00ebr p\u00ebrdorimin e burimeve. Dhe s\u00eb fundmi \u00ebsht\u00eb shtuar nj\u00eb profiler edhe m\u00eb i ul\u00ebt p\u00ebr k\u00ebrkesat, p\u00ebr t\u00eb par\u00eb se ku kalon \u00e7do milisekond\u00eb k\u00ebrkesa. Por p\u00ebr t\u00eb shfryt\u00ebzuar k\u00ebt\u00eb funksionalitet, m\u00eb duhet t\u00eb hap klientin konsol dhe t\u00eb shkruaj k\u00ebrkes\u00ebn q\u00eb vazhdimisht e harroj. E kam ruajtur diku dhe vazhdimisht e harroj se ku. <\/p>\n<p><\/p>\n<p>Do t\u00eb doja t\u00eb kishte nj\u00eb mjet, n\u00eb t\u00eb cilin thjesht shkruhet - k\u00ebto jan\u00eb k\u00ebrkesat tuaja t\u00eb r\u00ebnda, t\u00eb grupuara sipas klasave t\u00eb k\u00ebrkesave. Klikova n\u00eb ndonj\u00eb, dhe do t\u00eb m\u00eb thoshin se \u00ebsht\u00eb e r\u00ebnd\u00eb p\u00ebr k\u00ebt\u00eb arsye. Tani nuk ka nj\u00eb zgjidhje t\u00eb till\u00eb. Dhe \u00ebsht\u00eb mjaft e \u00e7uditshme q\u00eb, kur njer\u00ebzit m\u00eb pyesin: 'M\u00eb thuaj, a ka ndonj\u00eb dashboard t\u00eb gatsh\u00ebm p\u00ebr Grafana?', un\u00eb them: 'Shkoni n\u00eb sitin Grafana, atje \u00ebsht\u00eb komuniteti \u201cDashboards\u201d, dhe atje ka nj\u00eb dashboard nga Dima, ka nj\u00eb dashboard nga Kostyan. \u00c7far\u00eb \u00ebsht\u00eb kjo, nuk e di, vet\u00eb nuk e kam p\u00ebrdorur'.<\/p>\n<p><\/p>\n<h2 id=\"anchorzenanchorkak-vozdeystvovat-na-merdzhi-chtoby-server-ne-padal-vnbspoom\"><noindex><a rel=\"nofollow\" name=\"zen\"><\/a><\/noindex>Si t\u00eb ndikojm\u00eb n\u00eb merge q\u00eb serveri t\u00eb mos bjer\u00eb n\u00eb OOM?<\/h2>\n<p><\/p>\n<blockquote><p>Kam nj\u00eb tav\u00eb, n\u00eb at\u00eb tav\u00eb ka vet\u00ebm nj\u00eb particion, ajo \u00ebsht\u00eb ReplacingMergeTree. Un\u00eb shkruaj t\u00eb dh\u00ebna n\u00eb t\u00eb p\u00ebr kat\u00ebr vjet. M\u00eb duhej t\u00eb b\u00ebja nj\u00eb alter dhe t\u00eb fshija disa t\u00eb dh\u00ebna.<\/p>\n<p>E b\u00ebra k\u00ebt\u00eb, dhe gjat\u00eb p\u00ebrpunimit t\u00eb k\u00ebsaj k\u00ebrkese u h\u00ebngri gjith\u00eb memorja n\u00eb t\u00eb gjith\u00eb serverat e klasterit dhe t\u00eb gjith\u00eb serverat e klasterit shkuan bashk\u00eb n\u00eb OOM. Pastaj t\u00eb gjith\u00eb ata u ngrit\u00ebn, filluan t\u00eb kryenin merge t\u00eb k\u00ebtij operacioni t\u00eb nj\u00ebjt\u00eb, k\u00ebtij blloku t\u00eb dh\u00ebnash, dhe s\u00ebrish ran\u00eb n\u00eb OOM. Pastaj ata u ngrit\u00ebn p\u00ebrs\u00ebri dhe s\u00ebrish ran\u00eb. Dhe ky proces nuk ndalonte.<\/p>\n<p>M\u00eb von\u00eb doli se ishte n\u00eb fakt nj\u00eb gabim, t\u00eb cilin e rregulluan. Kjo \u00ebsht\u00eb shum\u00eb mir\u00eb, faleminderit shum\u00eb. Por ndjenja negative mbeti. Tani, kur mendoj p\u00ebr procesin e nj\u00eb bashkimi n\u00eb tabel\u00eb, kam nj\u00eb pyetje \u2014 p\u00ebrse nuk mund t\u00eb ndikoj n\u00eb k\u00ebto bashkime ndonj\u00eb m\u00ebnyr\u00eb? P\u00ebr shembull, ta kufizoj at\u00eb n\u00eb sasin\u00eb e memories q\u00eb k\u00ebrkohet, ose n\u00eb p\u00ebrgjith\u00ebsi n\u00eb numrin e tyre q\u00eb do t\u00eb p\u00ebrpunojn\u00eb konkretisht k\u00ebt\u00eb tabel\u00eb.<\/p>\n<p>Kam nj\u00eb tabel\u00eb q\u00eb quhet 'Metri', do ta p\u00ebrpunosh at\u00eb p\u00ebr mua, t\u00eb lutem, n\u00eb dy rrjedha. Mos prodho shum\u00eb bashkime paralelisht, b\u00ebj n\u00eb dy. Mendoj se n\u00eb dy do t\u00eb kem mjaft k\u00ebmbim memories, nd\u00ebrsa p\u00ebr t\u00eb p\u00ebrpunuar dhjet\u00eb, ndoshta nuk do t\u00eb mjaftoj\u00eb. Pse mbetet frika? Sepse tabela po rritet dhe nj\u00eb dit\u00eb do t\u00eb p\u00ebrballem me situat\u00ebn q\u00eb n\u00eb thelb tashm\u00eb jo p\u00ebr shkak t\u00eb gabimit, por p\u00ebr faktin se t\u00eb dh\u00ebnat do t\u00eb ndryshojn\u00eb n\u00eb nj\u00eb sasi t\u00eb till\u00eb t\u00eb madhe, saq\u00eb thjesht nuk do t\u00eb kem mjaft memory n\u00eb server. Dhe at\u00ebher\u00eb serveri do t\u00eb bjer\u00eb n\u00eb OOM gjat\u00eb bashkimit. Nd\u00ebrsa mutacionin mund ta anuloj, bashkimet jo.<\/p><\/blockquote>\n<p>E dini, gjat\u00eb bashkimeve serveri nuk do t\u00eb bjer\u00eb n\u00eb OOM, sepse gjat\u00eb bashkimit p\u00ebrdoret sasia e memories vet\u00ebm p\u00ebr nj\u00eb gam\u00eb t\u00eb vog\u00ebl t\u00eb t\u00eb dh\u00ebnave. Pra, gjith\u00e7ka do t\u00eb shkoj\u00eb mir\u00eb pavar\u00ebsisht nga volumi i t\u00eb dh\u00ebnave.<\/p>\n<p><\/p>\n<p><strong>Vladimir Kolobaev:<\/strong> Mir\u00eb, ka nj\u00eb moment k\u00ebtu se pas rregullimit t\u00eb gabimit, shkarkova versionin e ri dhe n\u00eb nj\u00eb tabel\u00eb tjet\u00ebr, m\u00eb t\u00eb vog\u00ebl, ku ka shum\u00eb particione, kryeva nj\u00eb operacion t\u00eb ngjash\u00ebm. Dhe gjat\u00eb bashkimit n\u00eb server u shfryt\u00ebzua rreth 100 GB memory. Kishim 150 t\u00eb z\u00ebn\u00eb, 100 mori, dhe mbeti nj\u00eb hap\u00ebsir\u00eb prej 50 GB, k\u00ebshtu q\u00eb nuk rash\u00eb n\u00eb OOM.<\/p>\n<p><\/p>\n<p>\u00c7far\u00eb m\u00eb mbron deri tani q\u00eb t\u00eb mos bjer\u00eb n\u00eb OOM, n\u00ebse ai v\u00ebrtet konsumon rreth 100 GB memory? Si t\u00eb sillem n\u00eb situat\u00ebn n\u00ebse ndonj\u00ebher\u00eb memory n\u00eb bashkime p\u00ebrfundon?<\/p>\n<p><\/p>\n<p><strong>Alexey Milovidov:<\/strong> Ka nj\u00eb problem q\u00eb shpenzimi i memorjes operative n\u00eb merdhe nuk \u00ebsht\u00eb i kufizuar. Problemi i dyt\u00eb \u00ebsht\u00eb se, n\u00ebse nj\u00eb merdhe \u00ebsht\u00eb caktuar, duhet ta ekzekutojm\u00eb at\u00eb, sepse \u00ebsht\u00eb regjistruar n\u00eb logun e replikimit. Logu i replikimit jan\u00eb ato veprime q\u00eb nevojiten p\u00ebr ta sjell\u00eb replik\u00ebn n\u00eb nj\u00eb gjendje t\u00eb p\u00ebrputhshme. N\u00ebse nuk b\u00ebhen manipulime manuale q\u00eb logu i replikimit do t'i \u00e7oj\u00eb prapa, merdhi do t\u00eb duhet, n\u00eb nj\u00eb m\u00ebnyr\u00eb ose n\u00eb nj\u00eb tjet\u00ebr, t\u00eb ekzekutohet.<\/p>\n<p><\/p>\n<p>Sigurisht, do t\u00eb ishte e dobishme t\u00eb kishte nj\u00eb kufizim p\u00ebr memorjen operative, i cili \"p\u00ebr \u00e7do rast\" mbron nga OOM. Ai nuk do ta ndihmoj\u00eb merdhin t\u00eb realizohet, ai do t\u00eb filloj\u00eb p\u00ebrs\u00ebri, do t\u00eb arrij\u00eb n\u00eb nj\u00eb prag, do t\u00eb hedh\u00eb nj\u00eb p\u00ebrjashtim, dhe pastaj p\u00ebrs\u00ebri do t\u00eb filloj\u00eb \u2014 s'ka asgj\u00eb t\u00eb mir\u00eb q\u00eb duke e b\u00ebr\u00eb k\u00ebt\u00eb. Por n\u00eb parim, do t\u00eb ishte e dobishme t\u00eb vendosnim k\u00ebt\u00eb kufizim.<\/p>\n<p><\/p>\n<h2 id=\"anchorgoanchorkak-budet-proishodit-razrabotka-golang-drayvera-dlya-clickhouse\"><noindex><a rel=\"nofollow\" name=\"go\"><\/a><\/noindex>Si do t\u00eb zhvillohet drejtuesi Golang p\u00ebr ClickHouse?<\/h2>\n<p><\/p>\n<blockquote><p>Drejtuesi Golang, i shkruar nga Kirill Shvakov, aktualisht duket se \u00ebsht\u00eb mb\u00ebshtetur zyrtarisht nga ekipi i ClickHouse. Ai <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/ClickHouse\/clickhouse-go\">n\u00eb depozit\u00ebn ClickHouse<\/a><\/noindex>, tani \u00ebsht\u00eb i madh dhe i v\u00ebrtet\u00eb.<\/p>\n<p>Nj\u00eb v\u00ebrejtje e vog\u00ebl. Ka nj\u00eb depo t\u00eb mrekullueshme dhe t\u00eb dashur p\u00ebr t\u00eb gjith\u00eb, q\u00eb quhet Vertica. Ata gjithashtu kan\u00eb nj\u00eb drejtp\u00ebrdrejt\u00ebs python, i cili mb\u00ebshtetet nga zhvilluesit e Vertica. Disa her\u00eb, ka ndodhur q\u00eb versionet e depo dhe versionet e drejtp\u00ebrdrejt\u00ebs jan\u00eb ndar\u00eb shum\u00eb, dhe drejtp\u00ebrdrejt\u00ebsi n\u00eb nj\u00eb moment ka ndaluar s\u00eb funksionuari. Nj\u00eb pik\u00eb e dyt\u00eb. Mb\u00ebshtetjes p\u00ebr k\u00ebt\u00eb drejtp\u00ebrdrejt\u00ebs zyrtare, m\u00eb duket se i \u00ebsht\u00eb dh\u00ebn\u00eb sistemit \"nip\u00ebll\" - ti shkruan nj\u00eb problem, dhe ai mbetet aty p\u00ebrjet\u00ebsisht.<\/p>\n<p>Kam dy pyetje. Tani p\u00ebr tani, drejtp\u00ebrdrejt\u00ebsi e Kirill-it n\u00eb Golang \u00ebsht\u00eb m\u00ebnyra standarde p\u00ebr t\u00eb komunikuar nga Golang me ClickHouse. P\u00ebrve\u00e7 n\u00ebse dikush komunikon p\u00ebrmes nd\u00ebrfaqes http, sepse i p\u00eblqen k\u00ebshtu. Si do t\u00eb zhvillohet kjo drejtp\u00ebrdrejt\u00ebsi? A do t\u00eb sinkronizohet me disa ndryshime fondamentale n\u00eb depot\u00eb? Dhe cili \u00ebsht\u00eb rendi i shqyrtimit t\u00eb problemeve? <\/p><\/blockquote>\n<p><strong>Kirill Shvakov:<\/strong> E para \u2014 si \u00ebsht\u00eb e organizuar n\u00eb m\u00ebnyr\u00eb burokratik. Ky moment nuk \u00ebsht\u00eb diskutuar, prandaj nuk kam \u00e7far\u00eb t\u00eb p\u00ebrgjigjem.<\/p>\n<p><\/p>\n<p>P\u00ebr t'iu p\u00ebrgjigjur pyetjes n\u00eb lidhje me issue-n, nevojitet nj\u00eb histori e vog\u00ebl e driverit. Kam punuar n\u00eb nj\u00eb kompani q\u00eb kishte shum\u00eb t\u00eb dh\u00ebna. Ishte nj\u00eb sistem reklamash me nj\u00eb sasi t\u00eb madhe ngjarjesh q\u00eb duhej t\u00eb ruheshin diku. Dhe n\u00eb nj\u00eb moment u shfaq ClickHouse. Ne i derdh\u00ebm t\u00eb dh\u00ebnat aty, dhe gjat\u00eb ndonj\u00eb kohe gjith\u00e7ka ishte mir\u00eb, pastaj ClickHouse ra. N\u00eb at\u00eb moment vendos\u00ebm se nuk na duhej. <\/p>\n<p><\/p>\n<p>P\u00ebr nj\u00eb vit u kthyem n\u00eb iden\u00eb e p\u00ebrdorimit t\u00eb ClickHouse, dhe na duhej t\u00eb gjenim nj\u00eb m\u00ebnyr\u00eb p\u00ebr t\u00eb shkruar t\u00eb dh\u00ebna aty. Kushti ishte se hardueri ishte shum\u00eb i dob\u00ebt, resurset ishin t\u00eb pakta. Por gjithmon\u00eb k\u00ebshtu kemi punuar, dhe prandaj shikuam n\u00eb drejtim t\u00eb protokollit natyror.<\/p>\n<p><\/p>\n<p>Pasi ne punonim n\u00eb Go, ishte e qart\u00eb se na duhej nj\u00eb driver n\u00eb Go. Un\u00eb e b\u00ebra at\u00eb praktikisht n\u00eb koh\u00eb t\u00eb plot\u00eb \u2014 kjo ishte detyra ime e pun\u00ebs. Deri n\u00eb nj\u00eb moment arrit\u00ebm ta p\u00ebrfundojm\u00eb, dhe n\u00eb parim askush nuk supozonte se dikush tjet\u00ebr p\u00ebrve\u00e7 nesh do ta p\u00ebrdorte at\u00eb. Pastaj erdhi CloudFlare me t\u00eb nj\u00ebjtin problem, dhe p\u00ebr nj\u00eb koh\u00eb punuam shum\u00eb mir\u00eb me ta, sepse kishin t\u00eb nj\u00ebjtat detyra. Madje e b\u00ebm\u00eb k\u00ebt\u00eb edhe n\u00eb ClickHouse vet\u00eb, dhe n\u00eb driver. <\/p>\n<p><\/p>\n<p>N\u00eb nj\u00eb moment, thjesht e ndalova pun\u00ebn me t\u00eb, sepse aktiviteti im n\u00eb lidhje me ClickHouse dhe puna ndryshuan pak. Prandaj, problemet nuk mbyllen. Her\u00eb pas here, n\u00eb depo komitojn\u00eb njer\u00ebzit q\u00eb kan\u00eb ndonj\u00eb nevoj\u00eb. At\u00ebher\u00eb un\u00eb shoh k\u00ebrkesat p\u00ebr t\u00ebrheqje dhe ndonj\u00ebher\u00eb madje b\u00ebj ndonj\u00eb ndryshim vet\u00eb, por kjo ndodh rrall\u00eb.<\/p>\n<p><\/p>\n<p>Dua t\u00eb kthehem te dr dispenseri. Para disa vitesh, kur filloi gjith\u00eb kjo pun\u00eb, ClickHouse ishte ndryshe dhe kishte mund\u00ebsi t\u00eb tjera. Tani ka nj\u00eb kuptim se si t\u00eb ri-dizajnohet dr dispenseri p\u00ebr ta b\u00ebr\u00eb m\u00eb t\u00eb mir\u00eb. N\u00ebse kjo ndodh, versioni 2 do t\u00eb jet\u00eb n\u00eb \u00e7do rast i papajtuesh\u00ebm p\u00ebr shkak t\u00eb problemeve t\u00eb akumuluara. <\/p>\n<p><\/p>\n<p>Nuk e di si ta organizoj k\u00ebt\u00eb pun\u00eb. Vet\u00eb nuk kam shum\u00eb koh\u00eb. N\u00ebse gjenden disa njer\u00ebz p\u00ebr t\u00eb p\u00ebrmir\u00ebsuar dr dispenserin, un\u00eb mund t'u ndihmoj dhe t'u tregoj se \u00e7far\u00eb t\u00eb b\u00ebjn\u00eb. Por pjes\u00ebmarrja aktive e \"Yandex\" n\u00eb zhvillimin e projektit nuk \u00ebsht\u00eb diskutuar ende. <\/p>\n<p><\/p>\n<p><strong>Alexey Milovidov:<\/strong> N\u00eb t\u00eb v\u00ebrtet\u00eb, p\u00ebr momentin nuk ka asnj\u00eb burokraci n\u00eb lidhje me k\u00ebta dr dispenser\u00eb. E vetmja gj\u00eb \u00ebsht\u00eb se ata jan\u00eb transferuar n\u00eb nj\u00eb organizat\u00eb zyrtare, pra ky dr dispenser \u00ebsht\u00eb pranuar si zgjidhja zyrtare p\u00ebr Go. Ka edhe disa dr dispenser\u00eb t\u00eb tjer\u00eb, por ata jan\u00eb t\u00eb ve\u00e7ant\u00eb. <\/p>\n<p><\/p>\n<p>Brenda nesh nuk ka asnj\u00eb zhvillim p\u00ebr k\u00ebta dr dispenser\u00eb. Pyetja \u00ebsht\u00eb n\u00ebse mund t\u00eb rekrutojm\u00eb nj\u00eb person t\u00eb ve\u00e7ant\u00eb, jo p\u00ebr k\u00ebt\u00eb dr dispenser specifik, por p\u00ebr zhvillimin e t\u00eb gjith\u00eb dr dispenser\u00ebve komunitar\u00eb, ose n\u00ebse mund t\u00eb gjejm\u00eb dik\u00eb nga jasht\u00eb. <\/p>\n<p><\/p>\n<h2 id=\"anchorlazy-loadanchorvneshniy-slovar-ne-podnimaetsya-posle-perezagruzki-snbspvklyuchennoy-nastroykoy-lazy_load-chto-delat\"><noindex><a rel=\"nofollow\" name=\"lazy-load\"><\/a><\/noindex>Fjalori i jasht\u00ebm nuk ngrihet pas rip\u00ebrshtatjes me konfigurimin e aktivizuar lazy_load. \u00c7far\u00eb mund t\u00eb b\u00ebj?<\/h2>\n<p><\/p>\n<blockquote><p>Ne kemi aktivizuar konfigurimin lazy_load, dhe pas rip\u00ebrshtatjes s\u00eb serverit, fjalori nuk ngrihet automatikisht. Ai ngrihet vet\u00ebm pasi p\u00ebrdoruesi t\u00eb ket\u00eb aksesuar k\u00ebt\u00eb fjalor. Dhe n\u00eb aksesin e par\u00eb jep nj\u00eb gabim. A \u00ebsht\u00eb ndonj\u00eb m\u00ebnyr\u00eb q\u00eb t\u00eb ngarkojm\u00eb fjalor\u00ebt automatikisht me ndihm\u00ebn e ClickHouse, apo duhet ta kontrollojm\u00eb ne gjithmon\u00eb p\u00ebrgatitjen e tyre q\u00eb p\u00ebrdoruesit t\u00eb mos kan\u00eb gabime?<\/p>\n<p>Mund t\u00eb kemi nj\u00eb version t\u00eb vjet\u00ebr t\u00eb ClickHouse, k\u00ebshtu q\u00eb fjalori nuk u ngarkua automatikisht. A \u00ebsht\u00eb e mundur?<\/p><\/blockquote>\n<p>E para, fjalor\u00ebt mund t\u00eb ngarkohen me forc\u00eb p\u00ebrmes nj\u00eb k\u00ebrkese. <strong>system reload dictionaries<\/strong>S\u00eb pari, n\u00eb lidhje me gabimin \u2014 n\u00ebse fjalori \u00ebsht\u00eb ngarkuar tashm\u00eb, at\u00ebher\u00eb k\u00ebrkesat do t\u00eb funksionojn\u00eb sipas t\u00eb dh\u00ebnave q\u00eb jan\u00eb ngarkuar. N\u00ebse fjalori ende nuk \u00ebsht\u00eb ngarkuar, ai do t\u00eb ngarkohet gjat\u00eb k\u00ebrkes\u00ebs.<\/p>\n<p><\/p>\n<p>P\u00ebr fjalor\u00ebt e r\u00ebnd\u00eb, kjo nuk \u00ebsht\u00eb shum\u00eb e p\u00ebrshtatshme. P\u00ebr shembull, duhet t\u00eb t\u00ebrheqim nj\u00eb milion rreshta nga MySQL. Disa b\u00ebjn\u00eb nj\u00eb selektim t\u00eb thjesht\u00eb, por ky selektim do t\u00eb pres\u00eb pik\u00ebrisht at\u00eb milion rreshta. Ka dy zgjidhje k\u00ebtu. E para \u2014 t\u00eb \u00e7aktivizoni lazy_load. E dyta \u2014 kur serveri ndizet, para se ta ngarkoni at\u00eb, t\u00eb kryeni <strong>rilodim sistemin e fjalorit<\/strong> ose thjesht t\u00eb kryeni nj\u00eb k\u00ebrkes\u00eb q\u00eb p\u00ebrdor fjalorin. At\u00ebher\u00eb fjalori do t\u00eb ngarkohet. Duhet t\u00eb kontrolloni vet\u00eb disponueshm\u00ebrin\u00eb e fjalor\u00ebve me cil\u00ebsimin e aktivizuar lazy_load, sepse ClickHouse nuk i t\u00ebrheq automatikisht.<\/p>\n<p><\/p>\n<p>N\u00eb pyetjen e fundit p\u00ebrgjigja \u00ebsht\u00eb \u2014 ose versioni \u00ebsht\u00eb i vjet\u00ebr, ose duhet ta debugoni. <\/p>\n<p><\/p>\n<h2 id=\"anchorreload-dictionariesanchorkak-byt-snbsptem-chto-system-reload-dictionaries-ne-podgruzhaet-ni-odin-iznbspmnozhestva-slovarey-esli-hotya-by-odin-iznbspnih-padaet-snbsposhibkoy\"><noindex><a rel=\"nofollow\" name=\"reload-dictionaries\"><\/a><\/noindex>Si t\u00eb merremi me faktin se system reload dictionaries nuk ngarkohet asnj\u00eb nga shum\u00eb fjalor\u00ebt, n\u00ebse t\u00eb pakt\u00ebn nj\u00eb prej tyre d\u00ebshton me nj\u00eb gabim?<\/h2>\n<p><\/p>\n<blockquote><p>Ka edhe nj\u00eb pyetje n\u00eb lidhje me sistemin e rifreskimit t\u00eb fjalor\u00ebve. Ne kemi dy fjalor\u00eb \u2014 nj\u00ebri nuk ngarkohet, tjetri ngarkohet. Sistemi i rifreskimit t\u00eb fjalor\u00ebve n\u00eb k\u00ebt\u00eb rast nuk ngarkon asnj\u00eb fjalor, dhe duhet t\u00eb ngarkoni n\u00eb m\u00ebnyr\u00eb t\u00eb ve\u00e7ant\u00eb at\u00eb sipas emrit t\u00eb tij me ndihm\u00ebn e sistemit p\u00ebr rifreskimin e fjalorit. A ka lidhje kjo me versionin e ClickHouse?<\/p><\/blockquote>\n<p>Dua t'ju g\u00ebzoj. Ky q\u00ebndrim ka ndryshuar. Do t\u00eb thot\u00eb, n\u00ebse e p\u00ebrdit\u00ebsoni ClickHouse, at\u00ebher\u00eb gjithashtu do t\u00eb ndryshoj\u00eb. N\u00ebse aktualja nuk ju k\u00ebnaq, <strong>system reload dictionaries<\/strong>, p\u00ebrmir\u00ebsohuni dhe le t\u00eb shpresojm\u00eb se do t\u00eb ndryshoj\u00eb n\u00eb drejtim t\u00eb mir\u00eb.<\/p>\n<p><\/p>\n<h2 id=\"anchorconnectionanchorest-li-sposob-konfigurirovat-rekvizity-vnbspkonfige-clickhouse-no-ne-svetit-ih-prinbsposhibkah\"><noindex><a rel=\"nofollow\" name=\"connection\"><\/a><\/noindex>A \u00ebsht\u00eb nj\u00eb m\u00ebnyr\u00eb p\u00ebr t\u00eb konfiguruar akreditimet n\u00eb&nbsp;konfigurimin e ClickHouse, por pa i ekspozuar ato gjat\u00eb gabimeve?<\/h2>\n<p><\/p>\n<blockquote><p>Pyetja tjet\u00ebr \u00ebsht\u00eb p\u00ebr gabimet e lidhura me fjalorin, ve\u00e7an\u00ebrisht p\u00ebr kredencialet. Ne kemi shkruar kredencialet e lidhjes n\u00eb konfigurimin e ClickHouse p\u00ebr fjalorin, dhe n\u00eb rast gabimi ne marrim k\u00ebto kredenciale dhe fjal\u00ebkalimin n\u00eb p\u00ebrgjigje. <\/p>\n<p>Ne e zgjidh\u00ebm k\u00ebt\u00eb gabim duke nxjerr\u00eb kredencialet n\u00eb konfigurimin e shoferit ODBC. A ka ndonj\u00eb m\u00ebnyr\u00eb p\u00ebr t\u00eb konfiguruar kredencialet n\u00eb konfigurimin e ClickHouse, por pa i ekspozuar k\u00ebto kredenciale n\u00eb rast gabimesh?<\/p><\/blockquote>\n<p>Zgjidhja k\u00ebtu \u00ebsht\u00eb v\u00ebrtet \u2014 t\u00eb tregoni k\u00ebto kredenciale n\u00eb odbc.ini, dhe n\u00eb vet\u00eb ClickHouse t\u00eb tregoni vet\u00ebm Emrin e Burimit t\u00eb t\u00eb Dh\u00ebnave ODBC. P\u00ebr burimet e tjera t\u00eb fjalor\u00ebve nuk do t\u00eb ket\u00eb nj\u00eblloj \u2014 as p\u00ebr fjalorin me MySQL, as p\u00ebr t\u00eb tjer\u00ebt nuk duhet t\u00eb shihni fjal\u00ebkalimin gjat\u00eb mesazheve t\u00eb gabimit. P\u00ebr ODBC do t\u00eb kontrolloj gjithashtu \u2014 n\u00ebse di\u00e7ka ekziston, duhet thjesht ta heqim.<\/p>\n<p><\/p>\n<h2 id=\"anchorzoom-backgroundsanchorbonus-fony-dlya-zuma-snbspposidelok\"><noindex><a rel=\"nofollow\" name=\"zoom-backgrounds\"><\/a><\/noindex>Bonus: sfondet p\u00ebr Zoom me&nbsp;takimet<\/h2>\n<p><\/p>\n<p>Duke mbi imazhin, lexuesit m\u00eb t\u00eb p\u00ebrkushtuar do t'u shfaqen sfondet bonus nga mbledhjet. Fikim zjarri s\u00eb bashku me maskotat e teknologjis\u00eb Avito, bisedojm\u00eb me koleg\u00ebt n\u00eb dhom\u00ebn e administrator\u00ebve t\u00eb rrjetit ose n\u00eb nj\u00eb klub kompjuterik t\u00eb vjet\u00ebr dhe zhvillojm\u00eb djelet n\u00ebn ur\u00eb me sfondin e graffiti.<\/p>\n<p><\/p>\n<p><noindex><a rel=\"nofollow\" href=\"http:\/\/amp.gs\/KvUr\"><img decoding=\"async\" alt=\"ClickHouse p\u00ebr p\u00ebrdoruesit e avancuar n\u00eb pyetje dhe p\u00ebrgjigje\" src=\"\/wp-content\/uploads\/2020\/05\/38b2ea076283285d934913c395863eb1.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/p>\n<p>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/avito\/blog\/500678\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0412 \u0430\u043f\u0440\u0435\u043b\u0435 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b \u0410\u0432\u0438\u0442\u043e \u0441\u043e\u0431\u0438\u0440\u0430\u043b\u0438\u0441\u044c \u043d\u0430&nbsp;\u043e\u043d\u043b\u0430\u0439\u043d-\u043f\u043e\u0441\u0438\u0434\u0435\u043b\u043a\u0438 \u0441 \u0433\u043b\u0430\u0432\u043d\u044b\u043c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c ClickHouse \u0410\u043b\u0435\u043a\u0441\u0435\u0435\u043c \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432\u044b\u043c \u0438 \u041a\u0438\u0440\u0438\u043b\u043b\u043e\u043c \u0428\u0432\u0430\u043a\u043e\u0432\u044b\u043c, Golang-\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c \u0438\u0437&nbsp;\u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Integros. \u041e\u0431\u0441\u0443\u0436\u0434\u0430\u043b\u0438, \u043a\u0430\u043a \u043c\u044b \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u0441\u0438\u0441\u0442\u0435\u043c\u0443 \u0443\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0431\u0430\u0437\u0430\u043c\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 \u0438 \u043a\u0430\u043a\u0438\u0435 \u0441\u043b\u043e\u0436\u043d\u043e\u0441\u0442\u0438 \u0443&nbsp;\u043d\u0430\u0441 \u0432\u043e\u0437\u043d\u0438\u043a\u0430\u044e\u0442. \u041f\u043e&nbsp;\u043c\u043e\u0442\u0438\u0432\u0430\u043c \u0432\u0441\u0442\u0440\u0435\u0447\u0438 \u043c\u044b \u0441\u043e\u0431\u0440\u0430\u043b\u0438 \u0441\u0442\u0430\u0442\u044c\u044e \u0441&nbsp;\u043e\u0442\u0432\u0435\u0442\u0430\u043c\u0438 \u044d\u043a\u0441\u043f\u0435\u0440\u0442\u043e\u0432 \u043d\u0430&nbsp;\u043d\u0430\u0448\u0438 \u0438 \u0437\u0440\u0438\u0442\u0435\u043b\u044c\u0441\u043a\u0438\u0435 \u0432\u043e\u043f\u0440\u043e\u0441\u044b \u043f\u0440\u043e&nbsp;\u0431\u044d\u043a\u0430\u043f\u044b, \u0440\u0435\u0448\u0430\u0440\u0434\u0438\u043d\u0433 \u0434\u0430\u043d\u043d\u044b\u0445, \u0432\u043d\u0435\u0448\u043d\u0438\u0435 \u0441\u043b\u043e\u0432\u0430\u0440\u0438, Golang-\u0434\u0440\u0430\u0439\u0432\u0435\u0440 \u0438 \u043e\u0431\u043d\u043e\u0432\u043b\u0435\u043d\u0438\u0435 \u0432\u0435\u0440\u0441\u0438\u0439 ClickHouse. \u041e\u043d\u0430 \u043c\u043e\u0436\u0435\u0442 \u0431\u044b\u0442\u044c [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":80776,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-80775","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0412 \u0430\u043f\u0440\u0435\u043b\u0435 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b \u0410\u0432\u0438\u0442\u043e \u0441\u043e\u0431\u0438\u0440\u0430\u043b\u0438\u0441\u044c \u043d\u0430 \u043e\u043d\u043b\u0430\u0439\u043d-\u043f\u043e\u0441\u0438\u0434\u0435\u043b\u043a\u0438 \u0441 \u0433\u043b\u0430\u0432\u043d\u044b\u043c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c ClickHouse \u0410\u043b\u0435\u043a\u0441\u0435\u0435\u043c \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432\u044b\u043c \u0438 \u041a\u0438\u0440\u0438\u043b\u043b\u043e\u043c \u0428\u0432\u0430\u043a\u043e\u0432\u044b\u043c, Golang-\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Integros.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47ClickHouse \u0434\u043b\u044f \u043f\u0440\u043e\u0434\u0432\u0438\u043d\u0443\u0442\u044b\u0445 \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u0435\u0439 \u0432 \u0432\u043e\u043f\u0440\u043e\u0441\u0430\u0445 \u0438 \u043e\u0442\u0432\u0435\u0442\u0430\u0445 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0412 \u0430\u043f\u0440\u0435\u043b\u0435 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b \u0410\u0432\u0438\u0442\u043e \u0441\u043e\u0431\u0438\u0440\u0430\u043b\u0438\u0441\u044c \u043d\u0430 \u043e\u043d\u043b\u0430\u0439\u043d-\u043f\u043e\u0441\u0438\u0434\u0435\u043b\u043a\u0438 \u0441 \u0433\u043b\u0430\u0432\u043d\u044b\u043c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c ClickHouse \u0410\u043b\u0435\u043a\u0441\u0435\u0435\u043c \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432\u044b\u043c \u0438 \u041a\u0438\u0440\u0438\u043b\u043b\u043e\u043c \u0428\u0432\u0430\u043a\u043e\u0432\u044b\u043c, Golang-\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Integros.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-05-08T11:42:47+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-05-08T11:42:47+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47 ClickHouse p\u00ebr p\u00ebrdoruesit e avancuar n\u00eb pyetje dhe p\u00ebrgjigje | ProHoster","description":"N\u00eb prill, inxhinier\u00ebt e Avito do t\u00eb mblidheshin n\u00eb nj\u00eb takim online me kryeprogramuesin e ClickHouse, Aleksej Milovidov, dhe Kirillin Shvakov, nj\u00eb programues Golang nga kompania Integros.","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47ClickHouse \u0434\u043b\u044f \u043f\u0440\u043e\u0434\u0432\u0438\u043d\u0443\u0442\u044b\u0445 \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u0435\u0439 \u0432 \u0432\u043e\u043f\u0440\u043e\u0441\u0430\u0445 \u0438 \u043e\u0442\u0432\u0435\u0442\u0430\u0445 | ProHoster","og:description":"\u0412 \u0430\u043f\u0440\u0435\u043b\u0435 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b \u0410\u0432\u0438\u0442\u043e \u0441\u043e\u0431\u0438\u0440\u0430\u043b\u0438\u0441\u044c \u043d\u0430 \u043e\u043d\u043b\u0430\u0439\u043d-\u043f\u043e\u0441\u0438\u0434\u0435\u043b\u043a\u0438 \u0441 \u0433\u043b\u0430\u0432\u043d\u044b\u043c \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c ClickHouse \u0410\u043b\u0435\u043a\u0441\u0435\u0435\u043c \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432\u044b\u043c \u0438 \u041a\u0438\u0440\u0438\u043b\u043b\u043e\u043c \u0428\u0432\u0430\u043a\u043e\u0432\u044b\u043c, Golang-\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u043e\u043c \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Integros.","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/clickhouse-dlya-prodvinutyh-polzovatelej-v-voprosah-i-otvetah","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-05-08T11:42:47+00:00","article:modified_time":"2020-05-08T11:42:47+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"80775","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 16:11:22","updated":"2022-09-28 05:48:13","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/80775","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=80775"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/80775\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/80776"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=80775"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=80775"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=80775"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}