{"id":91438,"date":"2020-08-13T19:42:36","date_gmt":"2020-08-13T17:42:36","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/effektivnoe-ispolzovanie-clickhouse-aleksej-milovidov-yandeks"},"modified":"2020-08-13T19:42:36","modified_gmt":"2020-08-13T17:42:36","slug":"effektivnoe-ispolzovanie-clickhouse-aleksej-milovidov-yandeks","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/effektivnoe-ispolzovanie-clickhouse-aleksej-milovidov-yandeks","title":{"rendered":"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/8e42c1fffffcc964eacef240ea90bbc5.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Duke qen\u00eb se ClickHouse \u00ebsht\u00eb nj\u00eb sistem i specializuar, \u00ebsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb merren parasysh karakteristikat e arkitektur\u00ebs s\u00eb tij gjat\u00eb p\u00ebrdorimit. N\u00eb k\u00ebt\u00eb raport, Aleksej do t\u00eb flas\u00eb p\u00ebr shembuj tipik\u00eb t\u00eb gabimeve gjat\u00eb p\u00ebrdorimit t\u00eb ClickHouse, t\u00eb cilat mund t\u00eb \u00e7ojn\u00eb n\u00eb pun\u00eb t\u00eb paefikase. Me shembuj nga praktika do t\u00eb tregohen se si zgjedhja e nj\u00eb skeme t\u00eb caktuar t\u00eb p\u00ebrpunimit t\u00eb t\u00eb dh\u00ebnave mund t\u00eb ndryshoj\u00eb ndjesh\u00ebm performanc\u00ebn.<\/p>\n<p><noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<p>P\u00ebrsh\u00ebndetje t\u00eb gjith\u00ebve! M\u00eb quajn\u00eb Aleksej, un\u00eb merrem me ClickHouse.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/7f5fc01663be58d3d25e518a4169b8cf.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>S\u00eb pari, menj\u00ebher\u00eb dua t'ju g\u00ebzoj, nuk do t'ju tregoj sot se \u00e7far\u00eb \u00ebsht\u00eb ClickHouse. Me t\u00eb v\u00ebrtet\u00eb, m\u00eb ka lodhur. \u00c7do her\u00eb e tregoj se \u00e7far\u00eb \u00ebsht\u00eb. Dhe, ndoshta, t\u00eb gjith\u00eb tashm\u00eb e din\u00eb. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/fae6bd07d098200643d591b8e5bf51e8.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>N\u00eb vend t\u00eb k\u00ebsaj, do t\u00eb flas p\u00ebr t\u00eb gjitha grykat q\u00eb mund t\u00eb hasen, dmth se si mund t\u00eb p\u00ebrdoret gabimisht ClickHouse. N\u00eb t\u00eb v\u00ebrtet\u00eb, nuk ka p\u00ebrse t\u00eb keni frik\u00eb, sepse ne e zhvillojm\u00eb ClickHouse si nj\u00eb sistem q\u00eb \u00ebsht\u00eb i thjesht\u00eb, i p\u00ebrshtatsh\u00ebm, dhe funksionon menj\u00ebher\u00eb. E instalove dhe gjith\u00e7ka shkon mir\u00eb, pa asnj\u00eb problem. <\/p>\n<p><\/p>\n<p>Megjithat\u00eb, \u00ebsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb merret parasysh se kjo sistem \u00ebsht\u00eb specializuar dhe mund t\u00eb ndiheni leht\u00eb n\u00eb nj\u00eb skenar t\u00eb pazakont\u00eb p\u00ebrdorimi, i cili do ta nxjerr\u00eb k\u00ebt\u00eb sistem jasht\u00eb zon\u00ebs s\u00eb tij t\u00eb komoditetit.<\/p>\n<p><\/p>\n<p>Pra, cilat jan\u00eb pengesat? N\u00eb thelb, do t\u00eb flas p\u00ebr gj\u00ebra t\u00eb dukshme. T\u00eb gjith\u00eb e kuptojn\u00eb, t\u00eb gjith\u00eb e din\u00eb dhe mund t\u00eb g\u00ebzohen q\u00eb jan\u00eb kaq t\u00eb zgjuar, nd\u00ebrsa ata q\u00eb nuk kuptojn\u00eb, do t\u00eb m\u00ebsojn\u00eb di\u00e7ka t\u00eb re. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/bffacf353ff31b361460178e911f0e16.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Nj\u00eb shembull shum\u00eb i thjesht\u00eb, i cili, fatkeq\u00ebsisht, ndodh shpesh, \u00ebsht\u00eb numri i madh i insert-eve me batche t\u00eb vogla, dmth. shum\u00eb insert-e t\u00eb vogla.<\/p>\n<p><\/p>\n<p>N\u00ebse shqyrtoni se si ClickHouse ekzekuton insert-in, mund t\u00eb d\u00ebrgoni nj\u00eb rrjedh\u00eb t\u00eb dh\u00ebnash deri n\u00eb nj\u00eb terabajt n\u00eb nj\u00eb k\u00ebrkes\u00eb. Nuk \u00ebsht\u00eb problem. <\/p>\n<p><\/p>\n<p>Le t\u00eb shikojm\u00eb se cila do t\u00eb jet\u00eb tipike performanca. P\u00ebr shembull, ne kemi nj\u00eb tabel\u00eb me t\u00eb dh\u00ebnat e Yandex.Metrica. Hitet. 105 ndonj\u00eb kolon\u00eb. 700 byte n\u00eb form\u00eb t\u00eb pakompresuar. Dhe do t\u00eb insertojm\u00eb, ashtu si\u00e7 duhet, batche nga nj\u00eb milion rreshta. <\/p>\n<p><\/p>\n<p>Duke e futur n\u00eb tabel\u00ebn MergeTree, marrim gjysm\u00eb milioni rreshta n\u00eb sekond\u00eb. Shum\u00eb mir\u00eb. N\u00eb tabel\u00ebn e replikuar \u2013 do t\u00eb jet\u00eb pak m\u00eb e vog\u00ebl, rreth 400,000 rreshta n\u00eb sekond\u00eb. <\/p>\n<p><\/p>\n<p>Dhe n\u00ebse aktivizohet inserimi me kuorum, rezultati \u00ebsht\u00eb pak m\u00eb i ul\u00ebt, por prap\u00ebseprap\u00eb nj\u00eb performanc\u00eb e konsiderueshme, 250,000 n\u00eb sekond\u00eb. Inserimi me kuorum \u2013 \u00ebsht\u00eb nj\u00eb funksion i pa dokumentuar n\u00eb ClickHouse*.<\/p>\n<p><\/p>\n<p>* sipas informacionit deri n\u00eb vitin 2020, <noindex><a rel=\"nofollow\" href=\"https:\/\/clickhouse.tech\/docs\/en\/operations\/settings\/settings\/#settings-insert_quorum\">tani \u00ebsht\u00eb dokumentuar<\/a><\/noindex>.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/668498fc82a0f6e851d05e6e1ea67033.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u00c7far\u00eb ndodh n\u00ebse e b\u00ebjm\u00eb keq? N\u00ebse futim nj\u00eb rresht n\u00eb tabel\u00ebn MergeTree, arrijm\u00eb 59 rreshta n\u00eb sekond\u00eb. Kjo \u00ebsht\u00eb 10,000 her\u00eb m\u00eb e ngadalt\u00eb. N\u00eb ReplicatedMergeTree \u2013 6 rreshta n\u00eb sekond\u00eb. Dhe n\u00ebse aktivizohet kuorumi, arrijm\u00eb vet\u00ebm 2 rreshta n\u00eb sekond\u00eb. M\u00eb duket se kjo \u00ebsht\u00eb nj\u00eb situat\u00eb e papranueshme. Si mund t\u00eb ngadal\u00ebsohet kaq shum\u00eb? Edhe n\u00eb bluz\u00ebn time shkruan se ClickHouse nuk duhet t\u00eb ngadal\u00ebsoj\u00eb. Por megjithat\u00eb, ndodhin ndonj\u00ebher\u00eb. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/4c771eddc4e8453f60ef91123ae5d109.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>N\u00eb t\u00eb v\u00ebrtet\u00eb, kjo \u00ebsht\u00eb nj\u00eb mang\u00ebsi nga ana jon\u00eb. Ne mund ta kishim b\u00ebr\u00eb q\u00eb gjith\u00e7ka t\u00eb funksiononte mir\u00eb, por nuk e b\u00ebm\u00eb. Dhe nuk e b\u00ebm\u00eb sepse p\u00ebr skenarin ton\u00eb, kjo nuk ishte e nevojshme. Ne ishim ngjitur me grupe. Thjesht merrnim grupe dhe nuk kishim asnj\u00eb problem. I vendosim dhe gjith\u00e7ka funksionon mir\u00eb. Por, natyrisht, mund t\u00eb ndodhin nj\u00eb mori skenar\u00ebsh. P\u00ebr shembull, kur keni shum\u00eb server\u00eb n\u00eb t\u00eb cil\u00ebt krijohen t\u00eb dh\u00ebna. Dhe ato vendosin t\u00eb dh\u00ebna jo aq shpesh, por p\u00ebrs\u00ebri ndodhin vendosje t\u00eb shpeshta. Duhen ndonj\u00ebher\u00eb t\u00eb shmangen k\u00ebto. <\/p>\n<p><\/p>\n<p>Nga nj\u00eb pik\u00ebpamje teknike, thelbi \u00ebsht\u00eb se kur b\u00ebn nj\u00eb insert n\u00eb ClickHouse, t\u00eb dh\u00ebnat nuk shkojn\u00eb n\u00eb ndonj\u00eb memtable. Nuk kemi as nj\u00eb MergeTree me struktur\u00eb log, por thjesht nj\u00eb MergeTree, sepse nuk ka asnj\u00eb log dhe as memTable. Ne thjesht shkruajm\u00eb t\u00eb dh\u00ebnat menj\u00ebher\u00eb n\u00eb sistemin e skedar\u00ebve, t\u00eb vendosura n\u00eb kolona. Dhe n\u00ebse keni 100 kolona, do t'ju duhen m\u00eb shum\u00eb se 200 skedar\u00eb p\u00ebr t'u shkruar n\u00eb nj\u00eb drejtori t\u00eb ve\u00e7ant\u00eb. E gjith\u00eb kjo \u00ebsht\u00eb mjaft voluminoze. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/24688fe3d39d81b92f12ac1b8b21425b.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe lind pyetja: \"Si t\u00eb b\u00ebjm\u00eb drejt?\", n\u00ebse ndodh nj\u00eb situat\u00eb q\u00eb duhet si ndonj\u00ebher\u00eb t\u00eb regjistrohen t\u00eb dh\u00ebnat n\u00eb ClickHouse.<\/p>\n<p><\/p>\n<p>Metoda 1. Kjo \u00ebsht\u00eb m\u00ebnyra m\u00eb e thjesht\u00eb. P\u00ebrdorni nj\u00eb radh\u00eb t\u00eb shp\u00ebrndar\u00eb. P\u00ebr shembull, Kafka. Thjesht nxirrni t\u00eb dh\u00ebnat nga Kafka, gruponi \u00e7do sekond\u00eb. Dhe gjith\u00e7ka do t\u00eb funksionoj\u00eb normalisht, ju regjistroni, gjith\u00e7ka punon mir\u00eb. <\/p>\n<p><\/p>\n<p>Disavantazhet jan\u00eb se Kafka \u00ebsht\u00eb edhe nj\u00eb sistem i madh i shp\u00ebrndar\u00eb. E kuptoj n\u00ebse kompania juaj tashm\u00eb ka Kafka. Kjo \u00ebsht\u00eb e mir\u00eb, \u00ebsht\u00eb e p\u00ebrshtatshme. Por n\u00ebse nuk e keni, at\u00ebher\u00eb duhet t\u00eb mendoni tri her\u00eb para se t\u00eb sillni nj\u00eb sistem t\u00eb tjet\u00ebr shp\u00ebrndar\u00eb n\u00eb projektin tuaj. Prandaj, \u00ebsht\u00eb e nevojshme t\u00eb shqyrtoni alternativa. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/4f687221e8c7443abf02fb002c4eba5b.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Metoda 2. Nj\u00eb alternativ\u00eb e till\u00eb old-school dhe shum\u00eb e thjesht\u00eb. Keni nj\u00eb server q\u00eb gjeneron log-et tuaja. Ai thjesht regjistron log-et tuaja n\u00eb nj\u00eb skedar. Dhe \u00e7do sekond\u00eb, p\u00ebr shembull, e rinovojm\u00eb k\u00ebt\u00eb skedar, hapim nj\u00eb t\u00eb ri. Nj\u00eb skript i ve\u00e7ant\u00eb ose p\u00ebrmes cron-it, ose nj\u00eb daemon e merr skedarin m\u00eb t\u00eb vjet\u00ebr dhe e regjistron n\u00eb ClickHouse. N\u00ebse regjistroni log-et \u00e7do sekond\u00eb, gjith\u00e7ka do t\u00eb jet\u00eb e shk\u00eblqyer. <\/p>\n<p><\/p>\n<p>Por disavantazhi i k\u00ebsaj metode \u00ebsht\u00eb se n\u00ebse serveri, ku krijohen log-et, zhduket diku, at\u00ebher\u00eb t\u00eb dh\u00ebnat do t\u00eb zhduken gjithashtu.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/b4c28b6df87386c42b6908bdfccb6b90.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Metoda 3. Ka nj\u00eb m\u00ebnyr\u00eb tjet\u00ebr interesante, q\u00eb n\u00eb t\u00eb v\u00ebrtet\u00eb nuk p\u00ebrdor skedar\u00eb temporal. P\u00ebr shembull, keni ndonj\u00eb lloj reklame q\u00eb rrotullohet ose ndonj\u00eb daemon interesante tjet\u00ebr q\u00eb gjeneron t\u00eb dh\u00ebna. Dhe mund t\u00eb grumbulloni nj\u00eb grup t\u00eb dh\u00ebnash direkt n\u00eb memory, n\u00eb buffer. Dhe kur kalon nj\u00eb sasi e mjaftueshme kohe, e ruani k\u00ebt\u00eb buffer, krijoni nj\u00eb t\u00eb ri dhe n\u00eb nj\u00eb proces t\u00eb ve\u00e7ant\u00eb, e vendosni at\u00eb q\u00eb \u00ebsht\u00eb grumbulluar n\u00eb ClickHouse.<\/p>\n<p><\/p>\n<p>Nga ana tjet\u00ebr, t\u00eb dh\u00ebnat gjithashtu shp\u00ebrndahen kur p\u00ebrdorni kill -9. N\u00ebse serveri juaj bie, at\u00ebher\u00eb do t\u00eb humbni k\u00ebto t\u00eb dh\u00ebna. Nj\u00eb tjet\u00ebr problem \u00ebsht\u00eb se n\u00ebse nuk keni mundur ta shkruani n\u00eb baz\u00eb, t\u00eb dh\u00ebnat do t\u00eb grumbullohen n\u00eb memory. Dhe ose do t\u00eb mbaroni memory, ose thjesht do t\u00eb humbni t\u00eb dh\u00ebnat. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/dc375b22fcb26cfa111d0bdb4563aa88.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Metoda 4. Nj\u00eb tjet\u00ebr m\u00ebnyr\u00eb interesante. Keni nj\u00eb proces serveri. Ai mund t\u00eb d\u00ebrgoj\u00eb t\u00eb dh\u00ebna n\u00eb ClickHouse menj\u00ebher\u00eb, por t\u00eb b\u00ebj\u00eb k\u00ebt\u00eb n\u00eb nj\u00eb lidhje. P\u00ebr shembull, d\u00ebrgoi nj\u00eb k\u00ebrkes\u00eb http me transfer-encoding: chunked me insert. Dhe gjeneron chunks jo shum\u00eb shpesh, mund t\u00eb d\u00ebrgoni \u00e7do rresht, ndon\u00ebse do t\u00eb ket\u00eb overhead p\u00ebr framing t\u00eb k\u00ebtyre t\u00eb dh\u00ebnave. <\/p>\n<p><\/p>\n<p>Megjithat\u00eb, n\u00eb k\u00ebt\u00eb rast, t\u00eb dh\u00ebnat do t\u00eb d\u00ebrgohen n\u00eb ClickHouse menj\u00ebher\u00eb. Dhe ClickHouse do t'i buferizoj\u00eb vet\u00eb ato. <\/p>\n<p><\/p>\n<p>Por gjithashtu ndodhin probleme. Tani do t\u00eb humbni t\u00eb dh\u00ebna, p\u00ebrfshir\u00eb kur procesi juaj nd\u00ebrpritet dhe, n\u00ebse procesi ClickHouse nd\u00ebrpritet, sepse kjo do t\u00eb jet\u00eb nj\u00eb insert i pap\u00ebrfunduar. N\u00eb ClickHouse, inserts jan\u00eb atomare deri n\u00eb nj\u00eb prag t\u00eb caktuar t\u00eb numrit t\u00eb rreshtave. N\u00eb p\u00ebrgjith\u00ebsi, kjo \u00ebsht\u00eb nj\u00eb m\u00ebnyr\u00eb interesante. Mund t\u00eb p\u00ebrdoret gjithashtu.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/6a18a974b9600ade377a870390e95d3d.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Metoda 5. Ja nj\u00eb tjet\u00ebr m\u00ebnyr\u00eb interesante. Ky \u00ebsht\u00eb nj\u00eb server i zhvilluar nga community p\u00ebr t\u00eb grumbulluar t\u00eb dh\u00ebna. Un\u00eb vet\u00eb nuk kam mbajtur shikimin e tij, k\u00ebshtu q\u00eb nuk mund t\u00eb garantoj asgj\u00eb. Megjithat\u00eb, as p\u00ebr ClickHouse nuk ofrohen ndonj\u00eb garancin\u00eb. Ky \u00ebsht\u00eb gjithashtu open source, por nga ana tjet\u00ebr, mund t\u00eb keni zakonuar te nj\u00eb standard cil\u00ebsie, t\u00eb cilin ne mundohemi ta ofrojm\u00eb. P\u00ebr k\u00ebt\u00eb gj\u00eb \u2013 nuk e di, shkoni n\u00eb GitHub, shikoni kodin. Ndoshta kan\u00eb shkruar di\u00e7ka normale. <\/p>\n<p><\/p>\n<p>* n\u00eb p\u00ebrputhje me gjendjen deri n\u00eb vitin 2020, duhet gjithashtu t\u00eb shtohet n\u00eb shqyrtim <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/en\/company\/vk\/blog\/430168\/\">KittenHouse<\/a><\/noindex>.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/ae6a04af63ae4e7fa160c002fdb6d506.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Metoda 6. Nj\u00eb m\u00ebnyr\u00eb tjet\u00ebr \u00ebsht\u00eb p\u00ebrdorimi i tabelave Buffer. Avantazhi i k\u00ebsaj metode \u00ebsht\u00eb se \u00ebsht\u00eb shum\u00eb e thjesht\u00eb p\u00ebr t'u filluar. Krijoni nj\u00eb tabel\u00eb Buffer dhe e vendosni atje. <\/p>\n<p><\/p>\n<p>Nj\u00eb disavantazh \u00ebsht\u00eb se problemi nuk zgjidhet plot\u00ebsisht. N\u00ebse me insertimin e tipit MergeTree duhet t\u00eb gruponi t\u00eb dh\u00ebnat n\u00eb nj\u00eb grup \u00e7do sekond\u00eb, kur e b\u00ebni k\u00ebt\u00eb n\u00eb tabel\u00ebn buffer, duhet t\u00eb gruponi t\u00eb pakt\u00ebn deri n\u00eb disa mij\u00ebra n\u00eb sekond\u00eb. N\u00ebse ka m\u00eb shum\u00eb se 10,000 n\u00eb sekond\u00eb, do t\u00eb jet\u00eb ende problem. Dhe n\u00ebse b\u00ebni inserte n\u00eb grupe, do t\u00eb shihni se at\u00ebher\u00eb ka dhjet\u00ebra mij\u00ebra rreshta n\u00eb sekond\u00eb. Dhe kjo ndodh me t\u00eb dh\u00ebna mjaft t\u00eb r\u00ebnda. <\/p>\n<p><\/p>\n<p>Po ashtu, tabelat buffer nuk kan\u00eb log. Dhe n\u00ebse ka di\u00e7ka t\u00eb keqe me serverin tuaj, t\u00eb dh\u00ebnat do t\u00eb humbasin. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/9bb16bf5e8145cc8368b11e54c86c837.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe si nj\u00eb bonus, recently kemi pasur mund\u00ebsin\u00eb n\u00eb ClickHouse p\u00ebr t\u00eb marr\u00eb t\u00eb dh\u00ebna nga Kafka. Ka nj\u00eb motor tabelash \u2013 Kafka. Thjesht krijoni nj\u00eb. Dhe mund t\u00eb varni pamjet materializuese mbi t\u00eb. N\u00eb k\u00ebt\u00eb rast, ai do t\u00eb nxjerr\u00eb n\u00eb m\u00ebnyr\u00eb t\u00eb vetme t\u00eb dh\u00ebnat nga Kafka dhe do t'i vendos\u00eb n\u00eb tabelat q\u00eb ju nevojiten. <\/p>\n<p><\/p>\n<p>E ajo q\u00eb na g\u00ebzon m\u00eb s\u00eb shumti n\u00eb k\u00ebt\u00eb mund\u00ebsi \u00ebsht\u00eb se nuk e b\u00ebm\u00eb ne. Kjo \u00ebsht\u00eb nj\u00eb funksion i komunitetit. Dhe kur them \"funksion i komunitetit\", e them pa ndonj\u00eb p\u00ebrbuzje. Ne e lexuam kodin, b\u00ebm\u00eb rishikimin, dhe duhet t\u00eb funksionoj\u00eb normalisht. <\/p>\n<p><\/p>\n<p>* n\u00eb gjendjen e vitit 2020, u shpall mb\u00ebshtetje e ngjashme p\u00ebr <noindex><a rel=\"nofollow\" href=\"https:\/\/clickhouse.tech\/docs\/en\/engines\/table-engines\/integrations\/rabbitmq\/\">RabbitMQ<\/a><\/noindex>.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/15dc61dfd435f5f8640395b7de480fe2.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u00c7far\u00eb mund t\u00eb jet\u00eb e pak\u00ebndshme ose befasuese kur futni t\u00eb dh\u00ebna? N\u00ebse b\u00ebni nj\u00eb k\u00ebrkes\u00eb insert values dhe n\u00eb values shkruani ndonj\u00eb shprehje t\u00eb llogaritur. P\u00ebr shembull, now() \u2013 \u00ebsht\u00eb gjithashtu nj\u00eb shprehje e llogaritur. N\u00eb k\u00ebt\u00eb rast, ClickHouse detyrohet t\u00eb ekzekutoj\u00eb interpreterin e k\u00ebtyre shprehjeve p\u00ebr \u00e7do rresht, dhe performanca do t\u00eb bjer\u00eb ndjesh\u00ebm. \u00cbsht\u00eb m\u00eb mir\u00eb ta evitoni k\u00ebt\u00eb.<\/p>\n<p><\/p>\n<p>* deri tani, problemi \u00ebsht\u00eb zgjidhur plot\u00ebsisht, dhe nuk ka m\u00eb regresione t\u00eb performanc\u00ebs kur p\u00ebrdoren shprehjet n\u00eb VALUES.<\/p>\n<p><\/p>\n<p>Nj\u00eb shembull tjet\u00ebr i kur mund t\u00eb ket\u00eb disa probleme \u00ebsht\u00eb kur t\u00eb dh\u00ebnat n\u00eb nj\u00eb grup i p\u00ebrkasin shum\u00eb pjes\u00ebve. N\u00eb ClickHouse, pjes\u00ebt jan\u00eb n\u00eb p\u00ebrmasa mujor. N\u00ebse vendosni nj\u00eb grup prej nj\u00eb milioni rreshtash dhe t\u00eb dh\u00ebnat p\u00ebrfshijn\u00eb disa vite, do t\u00eb keni disa dhjet\u00ebra pjes\u00eb. Kjo \u00ebsht\u00eb ekuivalente me t\u00eb pasur grupe q\u00eb jan\u00eb disa dhjet\u00ebra her\u00eb m\u00eb t\u00eb vogla n\u00eb p\u00ebrmas\u00eb, sepse brenda, ato ndahen fillimisht sipas pjes\u00ebve.<\/p>\n<p><\/p>\n<p>* s\u00eb fundmi n\u00eb ClickHouse, n\u00eb modalitetin eksperimental, \u00ebsht\u00eb shtuar mb\u00ebshtetje p\u00ebr formatin kompakt t\u00eb copave dhe copave n\u00eb memorje me write-ahead log, q\u00eb pothuajse zgjidh plot\u00ebsisht problemin.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/b8c6cecdba53559dad31ed11713a980d.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Tani le t\u00eb shqyrtojm\u00eb llojin e dyt\u00eb t\u00eb problemit - tipizimin e t\u00eb dh\u00ebnave. <\/p>\n<p><\/p>\n<p>Tipizimi i t\u00eb dh\u00ebnave mund t\u00eb jet\u00eb i rrept\u00eb, ose mund t\u00eb jet\u00eb tekstual. Tekstual do t\u00eb thot\u00eb se thjesht e deklaroni se t\u00eb gjitha fushat jan\u00eb t\u00eb tipit tekst. Kjo \u00ebsht\u00eb e papranueshme. Nuk duhet t\u00eb b\u00ebni k\u00ebshtu. <\/p>\n<p><\/p>\n<p>Le t\u00eb kuptojm\u00eb si t\u00eb veprojm\u00eb drejt n\u00eb rastet kur duam t\u00eb themi se nj\u00eb fush\u00eb \u00ebsht\u00eb tekst dhe le t\u00eb kujdeset ClickHouse p\u00ebr k\u00ebt\u00eb, pa u shqet\u00ebsuar. Megjithat\u00eb, ende ia vlen t\u00eb b\u00ebni disa p\u00ebrpjekje. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/a0fdcf6423933293318411242fe2ede7.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>P\u00ebr shembull, ne kemi nj\u00eb IP adres\u00eb. N\u00eb nj\u00eb rast e ruajm\u00eb si nj\u00eb varg. P\u00ebr shembull, 192.168.1.1. Nd\u00ebrsa n\u00eb rastin tjet\u00ebr - kjo do t\u00eb jet\u00eb nj\u00eb num\u00ebr i tipit UInt32*. 32 bit jan\u00eb t\u00eb mjaftueshme p\u00ebr nj\u00eb adres\u00eb IPv4.<\/p>\n<p><\/p>\n<p>S\u00eb pari, si\u00e7 \u00ebsht\u00eb e \u00e7uditshme, t\u00eb dh\u00ebnat do t\u00eb kompresohen p\u00ebraf\u00ebrsisht n\u00eb t\u00eb nj\u00ebjt\u00ebn m\u00ebnyr\u00eb. Do t\u00eb ket\u00eb dallime, sigurisht, por jo kaq t\u00eb m\u00ebdha. Prandaj, nuk ka probleme t\u00eb ve\u00e7anta me hyrjen-daljen e disqeve. <\/p>\n<p><\/p>\n<p>Por ka nj\u00eb ndryshim t\u00eb madh n\u00eb koh\u00ebn e procesorit dhe n\u00eb koh\u00ebn e ekzekutimit t\u00eb k\u00ebrkes\u00ebs. <\/p>\n<p><\/p>\n<p>Le t\u00eb llogarisim numrin e IP adresave unike, n\u00ebse ato ruhen si numra. K\u00ebshtu del 137 milion rreshta n\u00eb sekond\u00eb. N\u00ebse e b\u00ebjm\u00eb t\u00eb nj\u00ebjt\u00ebn gj\u00eb si vargje, at\u00ebher\u00eb 37 milion rreshta n\u00eb sekond\u00eb. Nuk e di se pse ka ndodhur nj\u00eb p\u00ebrputhje e till\u00eb. Un\u00eb vet\u00eb kam kryer k\u00ebto k\u00ebrkesa. Megjithat\u00eb, \u00ebsht\u00eb p\u00ebraf\u00ebrsisht kat\u00ebr her\u00eb m\u00eb e ngadalt\u00eb. <\/p>\n<p><\/p>\n<p>Dhe n\u00ebse llogarisim ndryshimin n\u00eb hap\u00ebsir\u00eb n\u00eb disk, ka nj\u00eb ndryshim gjithashtu. Dhe ndryshimi \u00ebsht\u00eb diku rreth nj\u00eb t\u00eb kat\u00ebrt\u00ebs, pasi ka shum\u00eb adresa IP unike. Dhe n\u00ebse do t\u00eb kishte k\u00ebtu rreshta me nj\u00eb num\u00ebr t\u00eb vog\u00ebl vlerash t\u00eb ndryshme, ato do t\u00eb kompresoheshin pa probleme n\u00eb nj\u00eb volum p\u00ebraf\u00ebrisht t\u00eb nj\u00ebjt\u00eb. <\/p>\n<p><\/p>\n<p>Dhe diferencia kat\u00ebrfish n\u00eb koh\u00eb rrugore nuk \u00ebsht\u00eb ndonj\u00eb gj\u00eb e zakonshme. Ndoshta p\u00ebr ju nuk ka r\u00ebnd\u00ebsi, por kur shoh nj\u00eb diferenc\u00eb t\u00eb till\u00eb, n\u00eb fillim m\u00eb b\u00ebn t\u00eb ndih\u00ebm keq.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/f74dbfdab5a9e26fa044a5870a8cc00d.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Le t\u00eb shqyrtojm\u00eb raste t\u00eb ndryshme. <\/p>\n<p><\/p>\n<p>1. Nj\u00eb rast \u00ebsht\u00eb kur keni disa vlera unike. N\u00eb k\u00ebt\u00eb rast, ne p\u00ebrdorim nj\u00eb praktik\u00eb t\u00eb thjesht\u00eb, q\u00eb ndoshta e dini dhe mund ta p\u00ebrdorni p\u00ebr \u00e7do DB. Kjo ka kuptim jo vet\u00ebm p\u00ebr ClickHouse. Thjesht regjistroni identifikuesit numerik\u00eb n\u00eb baz\u00eb. Dhe mund t\u00eb konvertoni n\u00eb vargje dhe anasjelltas n\u00eb an\u00ebn e aplikacionit tuaj. <\/p>\n<p><\/p>\n<p>Ja p\u00ebr shembull, keni nj\u00eb rajon. Dhe po p\u00ebrpiqeni ta ruani at\u00eb si nj\u00eb varg. Do t\u00eb shkruhet: Mosk\u00eb dhe MO. Dhe kur shoh se aty shkruan \"Mosk\u00eb\", at\u00ebher\u00eb kjo \u00ebsht\u00eb akoma asgj\u00eb, por kur \u00ebsht\u00eb dhe MO, at\u00ebher\u00eb b\u00ebhet si\u00e7 duket shum\u00eb keq. Sa byte \u00ebsht\u00eb kjo. <\/p>\n<p><\/p>\n<p>N\u00eb vend t\u00eb k\u00ebsaj, ne thjesht regjistrojm\u00eb numrin Ulnt32 dhe 250. N\u00eb Yandex kemi 250, ndoshta ju ndryshe. P\u00ebr t'u siguruar, do t\u00eb them se n\u00eb ClickHouse ka nj\u00eb mund\u00ebsi t\u00eb nd\u00ebrtuar p\u00ebr t\u00eb punuar me baz\u00ebn gjeografike. Thjesht regjistroni nj\u00eb list\u00eb me rajonet, p\u00ebrfshir\u00eb hierarkike, dmth, atje do t\u00eb jet\u00eb dhe Mosk\u00eb, dhe MO, dhe gjith\u00e7ka q\u00eb ju nevojitet. Dhe mund t\u00eb konvertoni n\u00eb nivelin e k\u00ebrkes\u00ebs. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/cc6e871136dab8f8a35246a5af512cda.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Varianti i dyt\u00eb \u00ebsht\u00eb p\u00ebraf\u00ebrsisht i nj\u00ebjt\u00eb, por tashm\u00eb me mb\u00ebshtetje brenda ClickHouse. Ky \u00ebsht\u00eb nj\u00eb lloj t\u00eb dh\u00ebnash Enum. Thjesht brenda Enum shkruani t\u00eb gjitha vlerat q\u00eb ju nevojiten. P\u00ebr shembull, lloji i pajisjes dhe aty shkruani: desktop, mobil, tablet, televizor. N\u00eb total, 4 variante. <\/p>\n<p><\/p>\n<p>Disavantazhi \u00ebsht\u00eb se duhet t\u00eb b\u00ebni periodikisht nj\u00eb alter. Shtua vet\u00ebm nj\u00eb variant. B\u00ebni alter table. N\u00eb t\u00eb v\u00ebrtet\u00eb, alter table n\u00eb ClickHouse \u00ebsht\u00eb falas. Sidomos \u00ebsht\u00eb falas p\u00ebr Enum, sepse t\u00eb dh\u00ebnat n\u00eb disk nuk ndryshojn\u00eb. Megjithat\u00eb, alter kap bllokimin* e tabel\u00ebs dhe duhet t\u00eb pres\u00eb derisa t\u00eb p\u00ebrfundojn\u00eb t\u00eb gjitha select-et. Vet\u00ebm pas k\u00ebsaj do t\u00eb realizohet alter, pra disa p\u00ebrshtatje vazhdojn\u00eb t\u00eb jen\u00eb t\u00eb pranishme.<\/p>\n<p><\/p>\n<p>* n\u00eb versionet e reja t\u00eb ClickHouse, ALTER \u00ebsht\u00eb b\u00ebr\u00eb plot\u00ebsisht pa bllokim.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/d7284fa5be583ab063bb6de0b60f54d4.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Nj\u00eb variant tjet\u00ebr mjaft unik p\u00ebr ClickHouse \u00ebsht\u00eb lidhja e fjalor\u00ebve t\u00eb jasht\u00ebm. Mund t\u00eb shkruani numra n\u00eb ClickHouse, nd\u00ebrsa katalog\u00ebt tuaj t\u2019i mbani n\u00eb \u00e7do sistem q\u00eb ju p\u00ebrshtatet. P\u00ebr shembull, mund t\u00eb p\u00ebrdorni: MySQL, Mongo, Postgres. Mund t\u00eb krijoni edhe nj\u00eb mikrosh\u00ebrbim q\u00eb do t\u2019i dor\u00ebzoj\u00eb k\u00ebto t\u00eb dh\u00ebna p\u00ebrmes http. Dhe n\u00eb nivelin e ClickHouse shkruani nj\u00eb funksion q\u00eb do t\u2019i shnd\u00ebrroj\u00eb k\u00ebto t\u00eb dh\u00ebna nga numra n\u00eb vargje. <\/p>\n<p><\/p>\n<p>Ky \u00ebsht\u00eb nj\u00eb m\u00ebnyr\u00eb e specializuar, por shum\u00eb efektive p\u00ebr t\u00eb realizuar nj\u00eb bashkim me nj\u00eb tabel\u00eb t\u00eb jashtme. Ka dy variante. N\u00eb nj\u00eb variant, k\u00ebto t\u00eb dh\u00ebna do t\u00eb jen\u00eb plot\u00ebsisht t\u00eb ruajtura n\u00eb cache, do t\u00eb q\u00ebndrojn\u00eb plot\u00ebsisht n\u00eb memorjen e pun\u00ebs dhe do t\u00eb p\u00ebrdit\u00ebsohen me nj\u00eb rregullsi t\u00eb caktuar. N\u00eb variantin tjet\u00ebr, n\u00ebse k\u00ebto t\u00eb dh\u00ebna nuk mund t\u00eb ruhet n\u00eb memorje, mund t'i ruajm\u00eb pjes\u00ebrisht n\u00eb cache. <\/p>\n<p><\/p>\n<p>Ja nj\u00eb shembull. Ka Yandex.Direct. Aty ka nj\u00eb fushat\u00eb reklamash dhe bannera. Numri i fushatave reklamash \u00ebsht\u00eb ndoshta rreth dhjet\u00eb milion. Disa mund t\u00eb ruhet n\u00eb memorje. Nd\u00ebrsa ka miliarda bannerash, ato nuk mund t\u00eb ruhet. Dhe ne p\u00ebrdorim nj\u00eb fjalor t\u00eb ruajtsh\u00ebm n\u00eb cache nga MySQL.<\/p>\n<p><\/p>\n<p>Problemi i vet\u00ebm \u00ebsht\u00eb se fjalori i ruajtsh\u00ebm n\u00eb cache do t\u00eb funksionoj\u00eb normalisht, n\u00ebse raporti i goditjeve \u00ebsht\u00eb af\u00ebr 100%. N\u00ebse \u00ebsht\u00eb m\u00eb i vog\u00ebl, do t\u00eb duhet t\u00eb merrni \u00e7el\u00ebsat e munguar dhe t\u00eb merrni t\u00eb dh\u00ebnat nga MySQL p\u00ebr \u00e7do grup t\u00eb dh\u00ebnash. M\u00eb shum\u00eb se ClickHouse, mund t\u00eb sigurohem q\u00eb \u2013 po, nuk ngadal\u00ebson, p\u00ebr sistemet e tjera nuk do flas.<\/p>\n<p><\/p>\n<p>Nj\u00eb bonus \u00ebsht\u00eb se fjalor\u00ebt jan\u00eb nj\u00eb m\u00ebnyr\u00eb shum\u00eb e thjesht\u00eb p\u00ebr t\u00eb azhurnuar t\u00eb dh\u00ebnat n\u00eb ClickHouse me dat\u00eb t\u00eb kaluar. K\u00ebshtu, n\u00ebse keni nj\u00eb raport p\u00ebr kampanjat reklamuese, p\u00ebrdoruesi thjesht nd\u00ebrron kampanj\u00ebn reklamuese dhe t\u00eb gjitha t\u00eb dh\u00ebnat e m\u00ebparshme, n\u00eb t\u00eb gjith\u00eb raportet, gjithashtu ndryshojn\u00eb. N\u00ebse shkruani rreshta direkt n\u00eb tabel\u00eb, do t\u00eb ishte e pamundur t'i azhurnoni ato. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/a5a2919d6be7d00c0ad5875b9a8f0f31.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Nj\u00eb m\u00ebnyr\u00eb tjet\u00ebr, kur nuk e dini nga ta merrni identifikuesit p\u00ebr rreshtat tuaj, \u00ebsht\u00eb t\u00eb p\u00ebrdorni thjesht nj\u00eb hash. Variantin m\u00eb t\u00eb thjesht\u00eb \u00ebsht\u00eb t\u00eb merrni nj\u00eb hash 64-bit\u00ebsh. <\/p>\n<p><\/p>\n<p>Problemi i vet\u00ebm \u00ebsht\u00eb se, n\u00ebse hash-i \u00ebsht\u00eb 64-bit\u00ebsh, do t\u00eb ket\u00eb patjet\u00ebr kolizione. Sepse n\u00ebse ka nj\u00eb miliard rreshta, probabiliteti b\u00ebhet tashm\u00eb i ndjesh\u00ebm. <\/p>\n<p><\/p>\n<p>Dhe nuk do t\u00eb ishte shum\u00eb mir\u00eb t\u00eb hash-oni emrat e kampanjave reklamuese k\u00ebshtu. N\u00ebse kampanjat reklamuese t\u00eb kompanive t\u00eb ndryshme ngat\u00ebrrohen, do t\u00eb ket\u00eb di\u00e7ka q\u00eb nuk ka kuptim. <\/p>\n<p><\/p>\n<p>Dhe ka nj\u00eb tru\u0109 t\u00eb thjesht\u00eb. E v\u00ebrteta, p\u00ebr t\u00eb dh\u00ebna serioze nuk \u00ebsht\u00eb shum\u00eb e p\u00ebrshtatshme, por n\u00ebse \u00ebsht\u00eb di\u00e7ka jo shum\u00eb serioze, thjesht shtoni nj\u00eb identifikues klienti n\u00eb \u00e7el\u00ebsin e fjalor\u00ebve. Dhe at\u00ebher\u00eb do t\u00eb keni kolizione, por vet\u00ebm brenda nj\u00eb klienti. Ky metod\u00eb p\u00ebrdoret p\u00ebr map\u00ebn e lidhjeve n\u00eb Yandex.Metrica. Atje kemi URL, ruajm\u00eb hashes. Dhe e dim\u00eb se kolizionet, natyrisht, ekzistojn\u00eb. Por kur shfaqet faqja, probabiliteti q\u00eb n\u00eb nj\u00eb faqe t\u00eb vetme p\u00ebr nj\u00eb p\u00ebrdorues disa URL t\u00eb p\u00ebrzihen dhe q\u00eb kjo t\u00eb v\u00ebrehet \u00ebsht\u00eb aq e ul\u00ebt saq\u00eb mund t\u00eb injorohet. <\/p>\n<p><\/p>\n<p>Si nj\u00eb bonus \u2013 p\u00ebr shum\u00eb operacione \u00ebsht\u00eb e mjaftueshme t\u00eb keni vet\u00ebm hashes dhe nuk \u00ebsht\u00eb e nevojshme t\u00eb ruani vet\u00eb stringjet asnj\u00ebher\u00eb. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/abbb73aaba29d10e3b7e7650bf68e6a6.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Nj\u00eb shembull tjet\u00ebr, n\u00ebse stringjet jan\u00eb t\u00eb shkurtra, p\u00ebr shembull, domenet e faqeve. Ato mund t\u00eb ruhen ashtu si jan\u00eb. Ose, p\u00ebr shembull, gjuha e shfletuesit ru \u2013 2 byte. Sigurisht, m\u00eb vjen shum\u00eb keq p\u00ebr byte-at, por mos u shqet\u00ebsoni, 2 byte nuk jan\u00eb nj\u00eb problem. Ju lutem, ruani ashtu si jan\u00eb, mos e stresoni. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/901eaed0029da6ea59630b57ccf80c25.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Nj\u00eb rast tjet\u00ebr \u00ebsht\u00eb kur, p\u00ebrkundrazi, ka shum\u00eb rreshta, dhe n\u00eb to shum\u00eb unik\u00eb, p\u00ebr m\u00eb tep\u00ebr edhe nj\u00eb mori potencialisht t\u00eb pakufizuar. Nj\u00eb shembull tipik \u00ebsht\u00eb frazat e k\u00ebrkimit ose URL-t\u00eb. Frazat e k\u00ebrkimit, p\u00ebrfshir\u00eb ato p\u00ebr shkak t\u00eb gabimeve t\u00eb shtypura. Le t\u00eb shohim se sa fraza unike k\u00ebrkohen brenda nj\u00eb dite. Dhe rezulton se pothuajse gjysma e t\u00eb gjitha ngjarjeve jan\u00eb t\u00eb tilla. N\u00eb k\u00ebt\u00eb rast, mund t\u00eb mendoni se \u00ebsht\u00eb e nevojshme t\u00eb normalizoni t\u00eb dh\u00ebnat, t\u00eb llogaritni identifikuesit, t\u2019i grumbulloni n\u00eb nj\u00eb tav\u00ebll t\u00eb ve\u00e7ant\u00eb. Por kjo nuk \u00ebsht\u00eb e rekomandueshme. Thjesht ruani k\u00ebto rreshta ashtu si\u00e7 jan\u00eb. <\/p>\n<p><\/p>\n<p>M\u00eb mir\u00eb \u2013 mos krijoni asgj\u00eb t\u00eb re, sepse n\u00ebse i ruani ve\u00e7mas, do t\u00eb nevojitet t\u00eb b\u00ebni nj\u00eb bashkim (join). Dhe ky bashkim \u2013 n\u00eb rastin m\u00eb t\u00eb mir\u00eb \u00ebsht\u00eb nj\u00eb akses rast\u00ebsor n\u00eb memorie, n\u00ebse bie n\u00eb mend. N\u00ebse nuk mund t\u00eb futet n\u00eb memory, at\u00ebher\u00eb do t\u00eb ket\u00eb probleme. <\/p>\n<p><\/p>\n<p>N\u00ebse t\u00eb dh\u00ebnat ruhen n\u00eb vend, ato thjesht lexohen n\u00eb rendin e duhur nga sistemi i skedar\u00ebve dhe gjith\u00e7ka shkon mir\u00eb.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/15f1d632083def16ccbc939579c1cf6b.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>N\u00ebse keni URL-t\u00eb ose ndonj\u00eb rresht t\u00eb gjat\u00eb t\u00eb komplikuesh\u00ebm, ia vlen t\u00eb mendoni se mund t\u00eb llogariten ndonj\u00eb p\u00ebrmbledhje paraprakisht dhe t\u00eb regjistrohet n\u00eb nj\u00eb kolon\u00eb t\u00eb ve\u00e7ant\u00eb. <\/p>\n<p><\/p>\n<p>P\u00ebr URL-t\u00eb, p\u00ebr shembull, mund t\u00eb ruani ve\u00e7mas domainin. Dhe n\u00ebse ju nevojitet me t\u00eb v\u00ebrtet\u00eb nj\u00eb domain, thjesht p\u00ebrdorni k\u00ebt\u00eb kolumn\u00eb, nd\u00ebrsa URL-t\u00eb do t\u00eb q\u00ebndrojn\u00eb dhe ju nuk do t'i prekni ato fare. <\/p>\n<p><\/p>\n<p>Le t\u00eb shohim se \u00e7far\u00eb diferencash ka. N\u00eb ClickHouse ekziston nj\u00eb funksion i specializuar q\u00eb llogarit domainin. Ai \u00ebsht\u00eb shum\u00eb i shpejt\u00eb, ne e kemi optimizuar at\u00eb. Dhe, p\u00ebr t\u00eb qen\u00eb i sinqert\u00eb, ai madje nuk i p\u00ebrmbush RFC, por megjithat\u00eb llogarit gjith\u00e7ka q\u00eb na nevojitet. <\/p>\n<p><\/p>\n<p>N\u00eb nj\u00eb rast do t\u00eb nxjerrim thjesht URL-t\u00eb dhe do t\u00eb llogarisim domainin. K\u00ebshtu rezulton 166 milisekonda. Nd\u00ebrsa n\u00ebse marrim nj\u00eb domain t\u00eb gatsh\u00ebm, at\u00ebher\u00eb rezulton vet\u00ebm 67 milisekonda, pra gati tre her\u00eb m\u00eb shpejt. Dhe kjo \u00ebsht\u00eb m\u00eb e shpejt\u00eb jo p\u00ebr shkak se na nevojiten disa llogaritje, por p\u00ebr shkak se lexojm\u00eb m\u00eb pak t\u00eb dh\u00ebna. <\/p>\n<p><\/p>\n<p>P\u00ebrve\u00e7 k\u00ebsaj, p\u00ebr nj\u00eb k\u00ebrkes\u00eb q\u00eb \u00ebsht\u00eb m\u00eb e ngadalt\u00eb, ndodhet m\u00eb shum\u00eb shpejt\u00ebsi gigabajt\u00ebsh p\u00ebr sekond\u00eb. Sepse ajo lexon m\u00eb shum\u00eb gigabajt\u00eb. Ky \u00ebsht\u00eb informacion krejt\u00ebsisht i tep\u00ebrt. K\u00ebrkesa si\u00e7 duket punon m\u00eb shpejt, por ekzekutohet p\u00ebr nj\u00eb periudh\u00eb m\u00eb t\u00eb gjat\u00eb.<\/p>\n<p><\/p>\n<p>Por t\u00eb par\u00eb volumin e t\u00eb dh\u00ebnave n\u00eb disk, rezulton se URL-ja \u00ebsht\u00eb 126 megabajt, nd\u00ebrsa domeni \u00ebsht\u00eb vet\u00ebm 5 megabajt. Kjo \u00ebsht\u00eb 25 her\u00eb m\u00eb e vog\u00ebl. Megjithat\u00eb, k\u00ebrkesa realizohet vet\u00ebm 4 her\u00eb m\u00eb shpejt. Kjo \u00ebsht\u00eb p\u00ebr shkak se t\u00eb dh\u00ebnat jan\u00eb t\u00eb nxehta. N\u00ebse do t\u00eb ishin t\u00eb ftohta, do t\u00eb ishte me siguri 25 her\u00eb m\u00eb shpejt p\u00ebr shkak t\u00eb hyrjes dhe daljes nga disku. <\/p>\n<p><\/p>\n<p>E cila, n\u00ebse vler\u00ebsojm\u00eb sa m\u00eb i vog\u00ebl \u00ebsht\u00eb domeni krahasuar me URL-n\u00eb, rezulton se \u00ebsht\u00eb diku rreth 4 her\u00eb m\u00eb i vog\u00ebl. Por p\u00ebr arsyet e panjohura, t\u00eb dh\u00ebnat n\u00eb disk z\u00ebn\u00eb 25 her\u00eb m\u00eb pak. Pse? P\u00ebr shkak t\u00eb kompresimit. Si URL-ja ashtu edhe domeni kompresohen. Por shpesh URL-ja p\u00ebrmban shum\u00eb mbeturina. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/c93d2128a41561ed7abb25d77414bc2d.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Sigurisht, \u00ebsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb p\u00ebrdorni tipet e duhur t\u00eb t\u00eb dh\u00ebnave, t\u00eb cilat jan\u00eb t\u00eb destinuara posa\u00e7\u00ebrisht p\u00ebr vlerat e nevojshme ose q\u00eb p\u00ebrshtaten. N\u00ebse jeni n\u00eb IPv4, ruani UInt32*. N\u00ebse jeni n\u00eb IPv6, at\u00ebher\u00eb FixedString(16), sepse adresa IPv6 \u00ebsht\u00eb 128 bit, dmth ruani ndersh\u00ebm n\u00eb formatin binar. <\/p>\n<p><\/p>\n<p>\u00c7far\u00eb t\u00eb b\u00ebni n\u00ebse ndonj\u00ebher\u00eb keni adresa IPv4 dhe ndonj\u00ebher\u00eb IPv6? Po, mund t\u00eb ruani t\u00eb dyja. Nj\u00eb kolon\u00eb p\u00ebr IPv4, nj\u00eb tjet\u00ebr p\u00ebr IPv6. Sigurisht, ka mund\u00ebsin\u00eb p\u00ebr t\u00eb shfaqur IPv4 n\u00eb IPv6. Kjo do t\u00eb funksionoj\u00eb gjithashtu, por n\u00ebse ju nevojitet shpesh adresa IPv4 n\u00eb k\u00ebrkesa, at\u00ebher\u00eb do t\u00eb ishte mir\u00eb ta vendosni n\u00eb nj\u00eb kolon\u00eb t\u00eb ve\u00e7ant\u00eb. <\/p>\n<p><\/p>\n<p>* tani n\u00eb ClickHouse ka tipe t\u00eb ve\u00e7anta t\u00eb dh\u00ebnash IPv4, IPv6, t\u00eb cilat ruajn\u00eb t\u00eb dh\u00ebnat po aq efikas sa numrat, por i paraqesin po aq leht\u00ebsisht sa vargjet.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/d7d6342ebd6b8b4f3526b3cd8a06cf26.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u00cbsht\u00eb gjithashtu e r\u00ebnd\u00ebsishme t\u00eb theksohet se \u00ebsht\u00eb e nevojshme t\u00eb p\u00ebrpunohen t\u00eb dh\u00ebnat paraprakisht. P\u00ebr shembull, ju vijn\u00eb disa log\u00eb t\u00eb pap\u00ebrpunuara. Dhe ndoshta nuk \u00ebsht\u00eb e men\u00e7ur t'i fusni ato menj\u00ebher\u00eb n\u00eb ClickHouse, edhe pse shum\u00eb josh\u00ebse \u00ebsht\u00eb t\u00eb mos b\u00ebni asgj\u00eb dhe gjith\u00e7ka do t\u00eb funksionoj\u00eb. Por megjithat\u00eb, \u00ebsht\u00eb m\u00eb mir\u00eb t\u00eb kryeni ato llogaritjet q\u00eb mundeni.<\/p>\n<p><\/p>\n<p>P\u00ebr shembull, versioni i shfletuesit. N\u00eb nj\u00eb departament fqinj, q\u00eb nuk dua t\u00eb tregoj me gisht, atje versioni i shfletuesit ruhet k\u00ebshtu, dmth. si nj\u00eb varg: 12.3. Pastaj, p\u00ebr t\u00eb b\u00ebr\u00eb nj\u00eb raport, ata e ndajn\u00eb k\u00ebt\u00eb varg n\u00eb nj\u00eb masiv, dhe m\u00eb pas n\u00eb elementin e par\u00eb t\u00eb masivit. Natyrisht, gjith\u00e7ka ngadal\u00ebsohet. E pyesja, pse e b\u00ebjn\u00eb k\u00ebshtu. M\u00eb than\u00eb se nuk i p\u00eblqejn\u00eb optimizimet e parakohshme. Nd\u00ebrsa un\u00eb nuk e p\u00eblqej parakohshm\u00ebrin\u00eb e pessimizimit.<\/p>\n<p><\/p>\n<p>Prandaj, n\u00eb k\u00ebt\u00eb rast, do t\u00eb ishte m\u00eb e sakt\u00eb t\u00eb ndaheshin n\u00eb 4 kolona. Mos kini frik\u00eb, sepse kjo \u00ebsht\u00eb ClickHouse. ClickHouse \u00ebsht\u00eb nj\u00eb databaz\u00eb kolonash. Dhe sa m\u00eb shum\u00eb kolona t\u00eb vogla e t\u00eb rregullta, aq m\u00eb mir\u00eb. Do t\u00eb keni 5 BrowserVersion, b\u00ebni 5 kolona. Kjo \u00ebsht\u00eb normale. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/f9a67221bf8508933d7bd336b16f9e04.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Tani le t\u00eb shikojm\u00eb se \u00e7far\u00eb t\u00eb b\u00ebni n\u00ebse keni shum\u00eb rreshta shum\u00eb t\u00eb gjat\u00eb, shum\u00eb arrays t\u00eb gjat\u00eb. Nuk \u00ebsht\u00eb nevoja t'i ruani ato n\u00eb ClickHouse fare. N\u00eb vend t\u00eb k\u00ebsaj, mund t\u00eb ruani vet\u00ebm nj\u00eb identifikues n\u00eb ClickHouse. Dhe k\u00ebto rreshta t\u00eb gjat\u00eb i vendosni n\u00eb nj\u00eb sistem tjet\u00ebr. <\/p>\n<p><\/p>\n<p>P\u00ebr shembull, n\u00eb nj\u00eb nga sh\u00ebrbimet tona analitike, ka disa parametra t\u00eb ngjarjeve. Dhe n\u00ebse p\u00ebr ngjarjet vijn\u00eb shum\u00eb parametra, ne thjesht ruajm\u00eb t\u00eb par\u00ebt 512 q\u00eb hasim. Sepse 512 - nuk \u00ebsht\u00eb problem. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/755271e8788ba4bd638f493bf320c820.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe n\u00ebse nuk mund t\u00eb vendosni se cilat jan\u00eb llojet tuaja t\u00eb t\u00eb dh\u00ebnave, gjithashtu mund t'i shkruani t\u00eb dh\u00ebnat n\u00eb ClickHouse, por n\u00eb nj\u00eb tabel\u00eb p\u00ebrkohshme t\u00eb tipit Log, e cila \u00ebsht\u00eb speciale p\u00ebr t\u00eb dh\u00ebnat p\u00ebrkohshme. Pas k\u00ebsaj, mund t\u00eb analizoni se \u00e7far\u00eb shp\u00ebrndarje vlerash keni atje, \u00e7far\u00eb ka n\u00eb p\u00ebrgjith\u00ebsi dhe t\u00eb p\u00ebrb\u00ebni llojet e duhura.<\/p>\n<p><\/p>\n<p>* tani n\u00eb ClickHouse ka nj\u00eb tip t\u00eb dh\u00ebnash <noindex><a rel=\"nofollow\" href=\"https:\/\/clickhouse.tech\/docs\/en\/sql-reference\/data-types\/lowcardinality\/\">LowCardinality<\/a><\/noindex> i cili lejon ruajtjen efektive t\u00eb rreshtave me m\u00eb pak shpenzime pun\u00eb.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/7e9e53d86a4189783c28ce56a669198c.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Tani le t\u00eb shqyrtojm\u00eb nj\u00eb rast tjet\u00ebr interesant. Nd sometimes njer\u00ebzit kan\u00eb gj\u00ebra q\u00eb ndodhin ndryshe. Hyj dhe shoh di\u00e7ka t\u00eb till\u00eb. Dhe menj\u00ebher\u00eb mendja shkon tek nj\u00eb administrator shum\u00eb i p\u00ebrvojsh\u00ebm dhe i men\u00e7ur, q\u00eb ka nj\u00eb p\u00ebrvoj\u00eb t\u00eb madhe n\u00eb konfigurimin e MySQL versionit 3.23. <\/p>\n<p><\/p>\n<p>K\u00ebtu shohim mij\u00ebra tabela, n\u00eb secil\u00ebn prej t\u00eb cilave \u00ebsht\u00eb regjistruar mbetja nga ndarja e di\u00e7kaje t\u00eb paqart\u00eb me mij\u00eb. <\/p>\n<p><\/p>\n<p>N\u00eb parim, e respektoj p\u00ebrvoj\u00ebn e dikujt tjet\u00ebr, p\u00ebrfshir\u00eb k\u00ebtu t\u00eb kuptoj se \u00e7far\u00eb vuajtjesh mund t\u00eb ket\u00eb sjell\u00eb kjo p\u00ebrvoj\u00eb. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/3a5e21ef2076966fcc627670485c5b3a.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe arsyet jan\u00eb m\u00eb shum\u00eb se sa m\u00eb pak t\u00eb qarta. K\u00ebto jan\u00eb stereotipet e vjetra q\u00eb mund t\u00eb ken\u00eb akumuluar gjat\u00eb pun\u00ebs me sisteme t\u00eb tjera. P\u00ebr shembull, n\u00eb tabelat MyISAM s'ka \u00e7el\u00ebs primar klaster. Dhe ky m\u00ebnyr\u00eb ndarjeje t\u00eb dh\u00ebnash mund t\u00eb jet\u00eb nj\u00eb p\u00ebrpjekje e d\u00ebshp\u00ebruar p\u00ebr t\u00eb marr\u00eb t\u00eb nj\u00ebjtin funksionalitet. <\/p>\n<p><\/p>\n<p>Nj\u00eb arsye tjet\u00ebr \u00ebsht\u00eb se operacione t\u00eb tilla si alter n\u00eb tabela t\u00eb m\u00ebdha jan\u00eb t\u00eb v\u00ebshtira p\u00ebr t'u realizuar. T\u00eb gjitha do t\u00eb bllokohen. Megjithat\u00eb, n\u00eb versionet moderne t\u00eb MySQL kjo problematik\u00eb tashm\u00eb s\u2019\u00ebsht\u00eb aq serioze. <\/p>\n<p><\/p>\n<p>Ose, p\u00ebr shembull, mikroshardimi, por p\u00ebr k\u00ebt\u00eb m\u00eb von\u00eb.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/321b1d9caa8db7aca5b96e08c61d05b5.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>N\u00eb ClickHouse nuk \u00ebsht\u00eb e nevojshme t\u00eb veprohet k\u00ebshtu, sepse, s\u00eb pari, \u00e7el\u00ebsi primar \u00ebsht\u00eb klaster, dhe t\u00eb dh\u00ebnat jan\u00eb t\u00eb renditura sipas \u00e7el\u00ebsit primar. <\/p>\n<p><\/p>\n<p>Dhe ndonj\u00ebher\u00eb m\u00eb pyesin: \u00abSi ndryshon performanca e k\u00ebrkesave p\u00ebr diapazone n\u00eb ClickHouse nga madh\u00ebsia e tabel\u00ebs?\u00bb. Un\u00eb them se ajo nuk ndryshon aspak. P\u00ebr shembull, n\u00ebse keni nj\u00eb tabel\u00eb me nj\u00eb miliard rreshta dhe lexoni nj\u00eb diapazoni prej nj\u00eb milioni rreshtash, gjith\u00e7ka shkon mir\u00eb. N\u00ebse tabela ka nj\u00eb trilion rreshta dhe lexoni nj\u00eb milion rreshta, do t\u00eb jet\u00eb pothuajse e nj\u00ebjt\u00eb. <\/p>\n<p><\/p>\n<p>Dhe, p\u00ebr m\u00eb tep\u00ebr, nuk k\u00ebrkohen gj\u00ebra si manualet e ndarjeve. N\u00ebse hyni dhe shikoni se \u00e7far\u00eb ka n\u00eb sistemin e skedar\u00ebve, do t\u00eb shihni q\u00eb tabela \u00ebsht\u00eb nj\u00eb gj\u00eb mjaft e r\u00ebnd\u00ebsishme. Dhe brenda saj ka di\u00e7ka si ndarje. Kjo do t\u00eb thot\u00eb q\u00eb ClickHouse b\u00ebn gjith\u00e7ka p\u00ebr ju dhe nuk ka nevoj\u00eb t\u00eb vuani.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/652f3742ad3a44f172d0d7302abd09f6.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Alter n\u00eb ClickHouse \u00ebsht\u00eb falas, n\u00ebse b\u00ebni alter add\/drop column. <\/p>\n<p><\/p>\n<p>Dhe nuk ka kuptim t\u00eb krijoni tabela t\u00eb vogla, sepse n\u00ebse keni 10 rreshta ose 10,000 rreshta n\u00eb nj\u00eb tabel\u00eb, kjo \u00ebsht\u00eb krejt\u00ebsisht e par\u00ebnd\u00ebsishme. ClickHouse \u00ebsht\u00eb nj\u00eb sistem q\u00eb optimizon throughput, jo latency, k\u00ebshtu q\u00eb nuk ka kuptim t\u00eb p\u00ebrpunoni 10 rreshta. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/144144b4740e9c5daec80c8dadc445f9.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u00cbsht\u00eb e drejt\u00eb t\u00eb p\u00ebrdorni nj\u00eb tabel\u00eb t\u00eb madhe. Largohuni nga stereotype t\u00eb vjetra, gjith\u00e7ka do t\u00eb shkoj\u00eb mir\u00eb. <\/p>\n<p><\/p>\n<p>Si p\u00ebr bonus, n\u00eb versionin ton\u00eb m\u00eb t\u00eb fundit \u00ebsht\u00eb shtuar mund\u00ebsia p\u00ebr t\u00eb krijuar nj\u00eb \u00e7el\u00ebs partitionimi t\u00eb rast\u00ebsish\u00ebm, n\u00eb m\u00ebnyr\u00eb q\u00eb t\u00eb kryhen operacione t\u00eb ndryshme mir\u00ebmbajtjeje mbi parti t\u00eb ve\u00e7anta. <\/p>\n<p><\/p>\n<p>P\u00ebr shembull, ju nevojiten shum\u00eb tabela t\u00eb vogla, si\u00e7 ndodh kur keni nevoj\u00eb p\u00ebr t\u00eb trajtuar disa t\u00eb dh\u00ebna p\u00ebrkohshme, ju vijn\u00eb cop\u00ebza dhe duhet t\u00eb kryeni transformim mbi to para se t'i shkruani n\u00eb tabel\u00ebn p\u00ebrfundimtare. P\u00ebr k\u00ebt\u00eb rast, ekziston nj\u00eb motor i mrekulluesh\u00ebm tabelash \u2013 StripeLog. \u00cbsht\u00eb pak si TinyLog, por m\u00eb i mir\u00eb. <\/p>\n<p><\/p>\n<p>* tani n\u00eb ClickHouse ka gjithashtu <noindex><a rel=\"nofollow\" href=\"https:\/\/clickhouse.tech\/docs\/en\/sql-reference\/table-functions\/input\/\">funksionin tabelar input<\/a><\/noindex>.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/d62f0b3d29876c29c6c0ba236b200186.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Nj\u00eb tjet\u00ebr antipattern \u00ebsht\u00eb mikro-shardimi. P\u00ebr shembull, n\u00ebse ju nevojitet t\u00eb shardoni t\u00eb dh\u00ebnat dhe keni 5 server\u00eb, por nes\u00ebr do t\u00eb keni 6 server\u00eb. Dhe mendoni se si t\u00eb ristrukturoni k\u00ebto t\u00eb dh\u00ebna. Dhe n\u00eb vend q\u00eb t'i ndani n\u00eb 5 sharda, i ndani n\u00eb 1,000 sharda. Pastaj \u00e7do nj\u00eb prej k\u00ebtyre mikro-shardave e lidhni me nj\u00eb server t\u00eb ve\u00e7ant\u00eb. K\u00ebshtu do t\u00eb keni p\u00ebr shembull, 200 ClickHouse n\u00eb nj\u00eb server, me instance t\u00eb ve\u00e7anta n\u00eb porta t\u00eb ve\u00e7anta ose n\u00eb baza t\u00eb dh\u00ebnash t\u00eb ve\u00e7anta. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/3e550ef7e338b84e4394eb9fa4538769.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Por n\u00eb ClickHouse kjo nuk \u00ebsht\u00eb shum\u00eb mir\u00eb. Sepse edhe nj\u00eb instance ClickHouse p\u00ebrpiqet t\u00eb p\u00ebrdor\u00eb t\u00eb gjitha burimet e disponueshme t\u00eb serverit p\u00ebr t\u00eb p\u00ebrpunuar nj\u00eb k\u00ebrkes\u00eb. Dometh\u00ebn\u00eb, n\u00ebse keni nj\u00eb server dhe aty, p\u00ebr shembull, 56 b\u00ebrthama procesor\u00ebsh. Ju ekzekutoni nj\u00eb k\u00ebrkes\u00eb q\u00eb zgjat nj\u00eb sekond\u00eb, dhe do t\u00eb p\u00ebrdor\u00eb 56 b\u00ebrthama. N\u00ebse keni vendosur 200 ClickHouse n\u00eb nj\u00eb server, at\u00ebher\u00eb do t\u00eb keni 10,000 thirrje. N\u00eb p\u00ebrgjith\u00ebsi, gj\u00ebrat do t\u00eb shkojn\u00eb shum\u00eb keq.<\/p>\n<p><\/p>\n<p>Arsyeja tjet\u00ebr \u00ebsht\u00eb se shp\u00ebrndarja e pun\u00ebs nd\u00ebrmjet k\u00ebtyre instancave do t\u00eb jet\u00eb e pabarabart\u00eb. Disa do ta p\u00ebrfundojn\u00eb m\u00eb her\u00ebt, disa m\u00eb von\u00eb. Po t\u00eb ndodhte gjith\u00e7ka n\u00eb nj\u00eb instance, ClickHouse do t\u00eb dinte si t\u00eb shp\u00ebrndaj\u00eb t\u00eb dh\u00ebnat sakt\u00ebsisht mes rrjedhave. <\/p>\n<p><\/p>\n<p>N\u00eb nj\u00eb arsye tjet\u00ebr \u00ebsht\u00eb se do t\u00eb keni nd\u00ebrveprime nd\u00ebrmjet proceseve p\u00ebrmes TCP. T\u00eb dh\u00ebnat do t\u00eb duhet t\u00eb serializohen, deserializohen dhe do t\u00eb ket\u00eb nj\u00eb num\u00ebr t\u00eb madh mikro-shardesh. K\u00ebshtu, do t\u00eb funksionoj\u00eb thjesht n\u00eb m\u00ebnyr\u00eb joefikase.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/18cacc17d22ae1cb2977c6be74769896.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Nj\u00eb tjet\u00ebr antipattern, edhe pse \u00ebsht\u00eb e v\u00ebshtir\u00eb ta quash nj\u00eb antipattern. Kjo \u00ebsht\u00eb nj\u00eb sasi e madhe para-agregimi.<\/p>\n<p><\/p>\n<p>N\u00eb fakt, parashtrimi \u00ebsht\u00eb shum\u00eb i mir\u00eb. N\u00ebse kishit nj\u00eb miliard rreshta, e keni agreguar at\u00eb n\u00eb 1,000 rreshta, dhe tani k\u00ebrkesa e realizohet menj\u00ebher\u00eb. E gjith\u00eb kjo \u00ebsht\u00eb n\u00eb rregull. K\u00ebshtu mund t\u00eb veprohet. Dhe p\u00ebr k\u00ebt\u00eb, madje n\u00eb ClickHouse ka nj\u00eb tip t\u00eb ve\u00e7ant\u00eb tabele AggregatingMergeTree, e cila kryen agregimin inkremental teksa futen t\u00eb dh\u00ebna. <\/p>\n<p><\/p>\n<p>Por ndodhin raste kur mendoni se do t\u00eb agregoni t\u00eb dh\u00ebnat k\u00ebshtu dhe po ashtu. Dhe n\u00eb nj\u00eb departament fqinje, gjithashtu nuk dua t\u00eb p\u00ebrmend emrin, p\u00ebrdorin tabela SummingMergeTree p\u00ebr t\u00eb p\u00ebrmbledhur sipas \u00e7el\u00ebsit kryesor, dhe si \u00e7el\u00ebs kryesor p\u00ebrdorin rreth 20 kolona t\u00eb ndryshme. N\u00eb \u00e7do rast, e kam ndryshuar emrat e disa kolonave p\u00ebr konspiracion, por \u00ebsht\u00eb k\u00ebshtu m\u00eb pak m\u00eb shum\u00eb.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/54e3a02ab4bbce7c63ded595780ae1f6.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe ndodhin k\u00ebto probleme. S\u00eb pari, volumi i t\u00eb dh\u00ebnave nuk zvog\u00eblohet shum\u00eb. P\u00ebr shembull, zvog\u00eblohet tre her\u00eb. Tre her\u00eb \u2013 do t\u00eb ishte nj\u00eb \u00e7mim i mir\u00eb p\u00ebr t'i lejuar vetes mund\u00ebsi t\u00eb pakufizuara p\u00ebr analiz\u00eb, q\u00eb ndodhin n\u00ebse t\u00eb dh\u00ebnat jan\u00eb t\u00eb paagreguara. N\u00ebse t\u00eb dh\u00ebnat jan\u00eb t\u00eb agreguara, at\u00ebher\u00eb n\u00eb vend t\u00eb analiz\u00ebs merrni vet\u00ebm statistika t\u00eb mjerueshme. <\/p>\n<p><\/p>\n<p>Dhe \u00e7far\u00eb \u00ebsht\u00eb ve\u00e7an\u00ebrisht frustrues? Sepse k\u00ebta njer\u00ebz nga departamenti p\u00ebrball\u00eb vijn\u00eb dhe k\u00ebrkojn\u00eb ndonj\u00ebher\u00eb t\u00eb shtohet nj\u00eb kolon\u00eb tjet\u00ebr n\u00eb \u00e7el\u00ebsin primar. Do t\u00eb thot\u00eb, ne kemi aggreguar t\u00eb dh\u00ebnat k\u00ebshtu, dhe tani duam pak m\u00eb shum\u00eb. Por n\u00eb ClickHouse nuk ka mund\u00ebsi p\u00ebr t\u00eb modifikuar \u00e7el\u00ebsin primar. Prandaj, duhet t\u00eb shkruajm\u00eb ndonj\u00eb skript n\u00eb C++. Dhe un\u00eb nuk i p\u00eblqej skriptet, madje ndonj\u00ebher\u00eb edhe n\u00eb C++.<\/p>\n<p><\/p>\n<p>Dhe n\u00ebse e shikoni p\u00ebrse u krijua ClickHouse, t\u00eb dh\u00ebnat jo t\u00eb agreguara jan\u00eb pik\u00ebrisht skenari p\u00ebr t\u00eb cilin ai \u00ebsht\u00eb krijuar. N\u00ebse e p\u00ebrdorni ClickHouse p\u00ebr t\u00eb dh\u00ebna jo t\u00eb agreguara, at\u00ebher\u00eb po b\u00ebni gjith\u00e7ka si\u00e7 duhet. N\u00ebse po agregoni, at\u00ebher\u00eb ndonj\u00ebher\u00eb \u00ebsht\u00eb e pranueshme. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/09fafd4c8a3ea8c22e586cb7a5cf6564.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Nj\u00eb rast tjet\u00ebr interesant \u00ebsht\u00eb k\u00ebrkesat n\u00eb nj\u00eb cik\u00ebl t\u00eb pafund. Ndonj\u00ebher\u00eb, hyj n\u00eb ndonj\u00eb server production dhe shikoj atje show processlist. Dhe \u00e7do her\u00eb zbulloj se po ndodhin gj\u00ebra t\u00eb tmerrshme. <\/p>\n<p><\/p>\n<p>P\u00ebr shembull, di\u00e7ka e till\u00eb. K\u00ebtu \u00ebsht\u00eb menj\u00ebher\u00eb e qart\u00eb se t\u00eb gjitha mund t\u00eb ishin ekzekutuar n\u00eb nj\u00eb k\u00ebrkes\u00eb. Thjesht shkruani atje url in dhe list\u00ebn.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/ae03b659f178962f50e0640614517296.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Pse ka shum\u00eb k\u00ebrkesa t\u00eb tilla n\u00eb nj\u00eb cik\u00ebl t\u00eb pafund \u2013 kjo \u00ebsht\u00eb e keqe? N\u00ebse indeksi nuk p\u00ebrdoret, do t\u00eb keni shum\u00eb kalime mbi t\u00eb nj\u00ebjtat t\u00eb dh\u00ebna. Por n\u00ebse indeksi p\u00ebrdoret, p\u00ebr shembull, n\u00ebse keni nj\u00eb \u00e7el\u00ebs primar mbi ru dhe shkruani url = di\u00e7ka aty. Dhe mendoni se do t\u00eb lexoni pik\u00ebrisht nj\u00eb url nga tabela, do t\u00eb jet\u00eb gjith\u00e7ka n\u00eb rregull. Por n\u00eb fakt, jo. Sepse ClickHouse b\u00ebn gjith\u00e7ka sipas grupeve. <\/p>\n<p><\/p>\n<p>Kur i duhet t\u00eb lexoj\u00eb nj\u00eb interval t\u00eb caktuar t\u00eb t\u00eb dh\u00ebnave, ai lexon pak m\u00eb shum\u00eb, sepse indeksi n\u00eb ClickHouse \u00ebsht\u00eb i holl\u00eb. Ky indeks nuk lejon t\u00eb gjendet nj\u00eb rresht t\u00eb ve\u00e7ant\u00eb n\u00eb tabel\u00eb, vet\u00ebm nj\u00eb interval t\u00eb caktuar. Dhe t\u00eb dh\u00ebnat kompresohen n\u00eb blloqe. P\u00ebr t\u00eb lexuar nj\u00eb rresht, duhet t\u00eb merrni nj\u00eb bllok t\u00eb t\u00ebr\u00eb dhe ta \u00e7kompresoni. Dhe n\u00ebse kryeni shum\u00eb k\u00ebrkesa, do t\u00eb keni shum\u00eb p\u00ebrputhje t\u00eb tilla dhe shum\u00eb pun\u00eb do t'ju p\u00ebrfundoj\u00eb vazhdimisht.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/95eef43c5b869f8b145c61ce3c616cfd.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe si nj\u00eb bonus, mund t\u00eb theksoj se n\u00eb ClickHouse nuk duhet t\u00eb keni frik\u00eb t\u00eb d\u00ebrgoni madje edhe megabajt\u00eb dhe madje qindra megabajt\u00eb n\u00eb seksionin IN. E mbaj mend nga praktika jon\u00eb se n\u00ebse n\u00eb MySQL d\u00ebrgojm\u00eb nj\u00eb mori vlerash n\u00eb seksionin IN, p\u00ebr shembull, d\u00ebrgojm\u00eb aty 100 megabajt\u00eb ndonj\u00eb numri, at\u00ebher\u00eb MySQL konsumon 10 gigabajt\u00eb memorie dhe m\u00eb shum\u00eb me k\u00ebt\u00eb nuk ndodh asgj\u00eb, gjith\u00e7ka punon keq. <\/p>\n<p><\/p>\n<p>Dhe e dyta \u2013 \u00ebsht\u00eb se n\u00eb ClickHouse, n\u00ebse k\u00ebrkimet tuaja p\u00ebrdorin indeks, at\u00ebher\u00eb kjo \u00ebsht\u00eb gjithmon\u00eb jo m\u00eb e ngadalt\u00eb se skanimi i plot\u00eb, dmth. n\u00ebse duhet t\u00eb lexoni pothuajse t\u00ebr\u00eb tabel\u00ebn, ai do t\u00eb shkoj\u00eb n\u00eb m\u00ebnyr\u00eb t\u00eb vazhdueshme dhe do t\u00eb lexoj\u00eb t\u00ebr\u00eb tabel\u00ebn. N\u00eb p\u00ebrgjith\u00ebsi, ai do t\u00eb kuptoj\u00eb vet\u00eb.<\/p>\n<p><\/p>\n<p>Por megjithat\u00eb ka disa v\u00ebshtir\u00ebsi. P\u00ebr shembull, ajo q\u00eb IN me nj\u00eb n\u00ebnk\u00ebrkes\u00eb nuk p\u00ebrdor indeks. Por kjo \u00ebsht\u00eb problemi yn\u00eb dhe duhet ta rregullojm\u00eb. Nuk ka asgj\u00eb t\u00eb thell\u00eb k\u00ebtu. Do ta rregullojm\u00eb*. <\/p>\n<p><\/p>\n<p>Dhe nj\u00eb gj\u00eb tjet\u00ebr interesante \u2013 \u00ebsht\u00eb se n\u00ebse keni nj\u00eb k\u00ebrkes\u00eb shum\u00eb t\u00eb gjat\u00eb dhe p\u00ebrpunimi i k\u00ebrkesave \u00ebsht\u00eb i shp\u00ebrndar\u00eb, at\u00ebher\u00eb kjo k\u00ebrkes\u00eb shum\u00eb e gjat\u00eb do t\u00eb d\u00ebrgohet n\u00eb \u00e7do server pa kompresim. P\u00ebr shembull, 100 megabajt\u00eb dhe 500 server\u00eb. Dhe, p\u00ebr rrjedhoj\u00eb, do t\u00eb d\u00ebrgohen 50 gigabajt\u00eb p\u00ebrmes rrjetit. Do t\u00eb d\u00ebrgohen dhe pastaj gjith\u00e7ka do t\u00eb ekzekutohet me sukses.<\/p>\n<p><\/p>\n<p>* tashm\u00eb po p\u00ebrdoret; gjith\u00e7ka \u00ebsht\u00eb rregulluar, si\u00e7 u premtua.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/5fcde9bf04fff0645be2530daa12743f.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe \u00ebsht\u00eb nj\u00eb rast mjaft i zakonsh\u00ebm n\u00ebse k\u00ebrkesat vijn\u00eb nga API. P\u00ebr shembull, keni krijuar ndonj\u00eb sh\u00ebrbim tuajin. Dhe n\u00ebse sh\u00ebrbimi juaj \u00ebsht\u00eb i nevojsh\u00ebm p\u00ebr dik\u00eb, at\u00ebher\u00eb e keni hapur API-n\u00eb dhe pas dy dit\u00ebsh v\u00ebreni se ndodhin gj\u00ebra t\u00eb \u00e7uditshme. T\u00eb gjitha jan\u00eb t\u00eb ngarkuara dhe po vijn\u00eb disa k\u00ebrkesa t\u00eb tmerrshme, t\u00eb cilat kurr\u00eb nuk duhej t\u00eb ishin. <\/p>\n<p><\/p>\n<p>Dhe zgjidhja k\u00ebtu \u00ebsht\u00eb nj\u00eb. N\u00ebse keni hapur API-n\u00eb, do t'ju duhet ta kufizoni at\u00eb. P\u00ebr shembull, t\u00eb vendosni ndonj\u00eb kuot\u00eb. Nuk ka alternativa t\u00eb tjera normale. P\u00ebrndryshe, menj\u00ebher\u00eb do t\u00eb shkruajn\u00eb nj\u00eb skript dhe do t\u00eb ken\u00eb probleme. <\/p>\n<p><\/p>\n<p>Dhe n\u00eb ClickHouse ka nj\u00eb mund\u00ebsi speciale \u2013 kjo \u00ebsht\u00eb llogaritja e kuotave. Gjithashtu, mund t\u00eb transferoni \u00e7el\u00ebsin tuaj t\u00eb kuot\u00ebs. Kjo, p\u00ebr shembull, \u00ebsht\u00eb nj\u00eb identifikues i brendsh\u00ebm i p\u00ebrdoruesit. Dhe kuotat do t\u00eb llogariten pavar\u00ebsisht p\u00ebr secilin prej tyre. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/532438d0f116919e62f04e32ebbd3c45.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Tani, ka edhe nj\u00eb gj\u00eb interesante. Kjo \u00ebsht\u00eb replikimi me dor\u00eb. <\/p>\n<p><\/p>\n<p>Di shum\u00eb raste kur, megjith\u00ebse ClickHouse ka mb\u00ebshtetje t\u00eb integruar p\u00ebr replikimin, njer\u00ebzit e replikojn\u00eb ClickHouse me dor\u00eb.<\/p>\n<p><\/p>\n<p>Cili \u00ebsht\u00eb principi? Ju keni nj\u00eb pipeline p\u00ebr p\u00ebrpunimin e t\u00eb dh\u00ebnave. Dhe ai funksionon n\u00eb pavar\u00ebsi, p.sh., n\u00eb qendra t\u00eb ndryshme t\u00eb t\u00eb dh\u00ebnave. Ju regjistroni t\u00eb dh\u00ebna t\u00eb nj\u00ebjta n\u00eb nj\u00eb m\u00ebnyr\u00eb t\u00eb nj\u00ebjt\u00eb n\u00eb ClickHouse, si\u00e7 \u00ebsht\u00eb. Megjithat\u00eb, praktika tregon se t\u00eb dh\u00ebnat do t\u00eb shkojn\u00eb n\u00eb ndryshim p\u00ebr shkak t\u00eb ndonj\u00eb ve\u00e7orie n\u00eb kodin tuaj. Shpresoj se n\u00eb rastin tuaj ndodh k\u00ebshtu. <\/p>\n<p><\/p>\n<p>Dhe periodikisht do t'ju duhet gjithsesi t\u00eb sinkronizoni me dor\u00eb. P\u00ebr shembull, nj\u00eb her\u00eb n\u00eb muaj, administrator\u00ebt b\u00ebjn\u00eb rsync.<\/p>\n<p><\/p>\n<p>N\u00eb fakt, \u00ebsht\u00eb shum\u00eb m\u00eb e thjesht\u00eb t\u00eb p\u00ebrdorni replikimin e nd\u00ebrtuar n\u00eb ClickHouse. Por k\u00ebtu mund t\u00eb ket\u00eb disa kund\u00ebrindikacione, sepse p\u00ebr k\u00ebt\u00eb duhet t\u00eb p\u00ebrdorni ZooKeeper. Nuk do t\u00eb flas asgj\u00eb t\u00eb keqe p\u00ebr ZooKeeper, n\u00eb parim sistemi funksionon, por ndonj\u00ebher\u00eb ndodh q\u00eb njer\u00ebzit nuk e p\u00ebrdorin at\u00eb p\u00ebr shkak t\u00eb frik\u00ebs nga java, sepse ClickHouse \u00ebsht\u00eb nj\u00eb sistem i shk\u00eblqyer, i shkruar n\u00eb C++, q\u00eb mund t\u00eb p\u00ebrdoret dhe gjith\u00e7ka do t\u00eb shkoj\u00eb mir\u00eb. Nd\u00ebrsa ZooKeeper \u00ebsht\u00eb n\u00eb java. Dhe ndonj\u00ebher\u00eb nuk d\u00ebshiron as ta shoh\u00ebsh, por at\u00ebher\u00eb mund t\u00eb p\u00ebrdorni replikimin me dor\u00eb. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/3b5b68996daef6b8920f067b189f25e0.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>ClickHouse \u2013 \u00ebsht\u00eb nj\u00eb sistem praktik. Ai i merr parasysh nevojat tuaja. N\u00ebse keni replikim n\u00eb disqe manuale, mund t\u00eb krijoni nj\u00eb tabel\u00eb t\u00eb Distribuar, e cila shikon n\u00eb replikat tuaja manuale dhe b\u00ebn vet\u00eb kalimin midis tyre. Dhe ka madje nj\u00eb opsion t\u00eb ve\u00e7ant\u00eb, i cili lejon shmangien e problemeve, edhe n\u00ebse replikat tuaja ndahen sistematikisht.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/70f6630f8b14756163923a7db8dd7391.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>M\u00eb pas mund t\u00eb hasni probleme n\u00ebse p\u00ebrdorni motor\u00eb tabelash primitivi. ClickHouse \u00ebsht\u00eb nj\u00eb nd\u00ebrtues i till\u00eb, ku ka nj\u00eb mori motor\u00ebsh tabelash t\u00eb ndrysh\u00ebm. P\u00ebr t\u00eb gjitha rastet serioze, si\u00e7 shkruhet n\u00eb dokumentacion, p\u00ebrdorni tabelat e familjes MergeTree. T\u00eb gjitha t\u00eb tjerat jan\u00eb p\u00ebr raste t\u00eb ve\u00e7anta ose p\u00ebr teste.<\/p>\n<p><\/p>\n<p>N\u00eb tabel\u00ebn MergeTree nuk \u00ebsht\u00eb e domosdoshme t\u00eb keni ndonj\u00eb dat\u00eb dhe koh\u00eb. Mund t\u00eb p\u00ebrdorni gjithsesi. N\u00ebse nuk ka dat\u00eb dhe koh\u00eb, shkruani q\u00eb default \u2013 viti 2000. Kjo do t\u00eb funksionoj\u00eb dhe nuk do t\u00eb k\u00ebrkoj\u00eb burime. <\/p>\n<p><\/p>\n<p>Dhe n\u00eb versionin e ri t\u00eb serverit madje mund t\u00eb tregoni q\u00eb t\u00eb keni nj\u00eb ndarjen t\u00eb personalizuar pa \u00e7el\u00ebsin e ndarjes. Kjo do t\u00eb jet\u00eb e nj\u00ebjt\u00eb. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/6d4046d11afdd1e5962d49a1612b666a.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Nga ana tjet\u00ebr, mund t\u00eb p\u00ebrdorni moteur primitive tabelash. P\u00ebr shembull, ngarkoni t\u00eb dh\u00ebnat nj\u00eb her\u00eb dhe shikoni, rrotulloni dhe fshini. Mund t\u00eb p\u00ebrdorni Log.<\/p>\n<p><\/p>\n<p>Ose mbani sasi t\u00eb vogla p\u00ebr p\u00ebrpunim t\u00eb nd\u00ebrmjet\u00ebm - ky \u00ebsht\u00eb StripeLog ose TinyLog.<\/p>\n<p><\/p>\n<p>Memory mund t\u00eb p\u00ebrdoret, n\u00ebse keni nj\u00eb sasi t\u00eb vog\u00ebl t\u00eb dh\u00ebnash dhe thjesht t\u00eb rrotulloni di\u00e7ka n\u00eb RAM. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/8a3d4e0e3a8c26dd3a2b2f64123e2bf9.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>ClickHouse nuk i p\u00eblqen shum\u00eb t\u00eb dh\u00ebnat e tejnormizuara. <\/p>\n<p><\/p>\n<p>Ja nj\u00eb shembull tipik. Kjo \u00ebsht\u00eb nj\u00eb sasi e madhe e URL-ve. I keni futur n\u00eb nj\u00eb tabel\u00eb fqinje. M\u00eb pas vendos\u00ebt t\u00eb b\u00ebni JOIN me to, por kjo zakonisht nuk do t\u00eb funksionoj\u00eb, sepse ClickHouse mb\u00ebshtet vet\u00ebm Hash JOIN. N\u00ebse RAM-i \u00ebsht\u00eb i pamjaftuesh\u00ebm p\u00ebr sasin\u00eb e t\u00eb dh\u00ebnave q\u00eb duhen bashkuar, at\u00ebher\u00eb JOIN nuk do t\u00eb mund t\u00eb realizohet*. <\/p>\n<p><\/p>\n<p>N\u00ebse t\u00eb dh\u00ebnat jan\u00eb me kardinalitet t\u00eb madh, mos e shqet\u00ebsoni, ruani ato n\u00eb form\u00eb denormale, URL-t\u00eb direkt n\u00eb tabel\u00ebn kryesore. <\/p>\n<p><\/p>\n<p>* aktualisht n\u00eb ClickHouse ekziston edhe merge join q\u00eb funksionon kur t\u00eb dh\u00ebnat nd\u00ebrmjet\u00ebse nuk i p\u00ebrshtaten RAM-it. Por kjo \u00ebsht\u00eb e pasakt\u00eb dhe rekomandimi vazhdon t\u00eb mbetet n\u00eb fuqi.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/e8160a8db17c18d48f810092c55b094c.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dy shembuj t\u00eb tjer\u00eb, por tashm\u00eb dyshoj n\u00ebse ata jan\u00eb antipatterns apo jo. <\/p>\n<p><\/p>\n<p>ClickHouse ka nj\u00eb mang\u00ebsi t\u00eb njohur. Ai nuk ka p\u00ebrdit\u00ebsime*. N\u00eb nj\u00ebfar\u00eb m\u00ebnyre, kjo \u00ebsht\u00eb edhe e mir\u00eb. N\u00ebse keni ndonj\u00eb t\u00eb dh\u00ebn\u00eb t\u00eb r\u00ebnd\u00ebsishme, p\u00ebr shembull, llogarin\u00eb, askush s'do t\u00eb mund ta d\u00ebrgoj\u00eb at\u00eb, sepse nuk ka p\u00ebrdit\u00ebsime.<\/p>\n<p><\/p>\n<p>* mb\u00ebshtetje p\u00ebr p\u00ebrdit\u00ebsime dhe fshirje \u00ebsht\u00eb shtuar prej koh\u00ebsh n\u00eb modin grup.<\/p>\n<p><\/p>\n<p>Por ka disa m\u00ebnyra t\u00eb ve\u00e7anta, t\u00eb cilat lejojn\u00eb p\u00ebrdit\u00ebsime si p\u00ebr n\u00eb sfond. P\u00ebr shembull, tabelat e tipit ReplaceMergeTree. Ato b\u00ebjn\u00eb p\u00ebrdit\u00ebsime gjat\u00eb bashkimeve n\u00eb sfond. Mund ta forconi k\u00ebt\u00eb me optimizimin e tabel\u00ebs. Por mos e b\u00ebni k\u00ebt\u00eb shum\u00eb shpesh, sepse do t\u00eb b\u00ebhet nj\u00eb riparim i plot\u00eb i pjes\u00ebs. <\/p>\n<p><\/p>\n<p>JOIN t\u00eb shp\u00ebrndar\u00eb n\u00eb ClickHouse \u2013 gjithashtu trajtohet keq nga planifikuesi i k\u00ebrkesave. <\/p>\n<p><\/p>\n<p>Keq, por ndonj\u00ebher\u00eb \u00ebsht\u00eb n\u00eb rregull.<\/p>\n<p><\/p>\n<p>P\u00ebrdorimi i ClickHouse vet\u00ebm p\u00ebr t\u00eb lexuar t\u00eb dh\u00ebnat p\u00ebrs\u00ebri me select*.<\/p>\n<p><\/p>\n<p>Nuk do ta rekomandoja p\u00ebrdorimin e ClickHouse p\u00ebr llogaritje t\u00eb nd\u00ebrlikuara. Por nuk \u00ebsht\u00eb krejt\u00ebsisht k\u00ebshtu, sepse tashm\u00eb po largohemi nga kjo rekomandim. Dhe koh\u00ebt e fundit kemi shtuar mund\u00ebsin\u00eb e aplikimit t\u00eb modeleve t\u00eb m\u00ebsimit t\u00eb makinerive n\u00eb ClickHouse \u2013 Catboost. Dhe kjo m\u00eb shqet\u00ebson, sepse mendoj: \"Sa katastrofike. Sa cikle n\u00eb byte po na rezulton!\". M\u00eb vjen shum\u00eb keq t\u00eb harxhoj cikle p\u00ebr byte. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"P\u00ebrdorimi efikas i ClickHouse. Aleksej Milovidov (Yandex)\" src=\"\/wp-content\/uploads\/2020\/08\/f686d5d352ccb444cd06e2ae68897137.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Por mos u frik\u00ebs, vendosni ClickHouse, gjith\u00e7ka do t\u00eb shkoj\u00eb mir\u00eb. N\u00ebse keni ndonj\u00eb problem, ne kemi nj\u00eb komunitet. P\u00ebr m\u00eb tep\u00ebr, komuniteti jeni ju. N\u00ebse keni ndonj\u00eb problem, mund t\u00eb hyni n\u00eb bised\u00ebn ton\u00eb dhe shpresoj q\u00eb do t'ju ndihmojn\u00eb. <\/p>\n<p><\/p>\n<p>Pyetje<\/p>\n<p><\/p>\n<p><em>Faleminderit p\u00ebr raportin! Ku mund t\u00eb ankohesha p\u00ebr r\u00ebnien e ClickHouse?<\/em><\/p>\n<p><\/p>\n<p>Mund t\u00eb ankohem personalisht tani. <\/p>\n<p><\/p>\n<p><em>K\u00ebt\u00eb koh\u00eb kam filluar t\u00eb p\u00ebrdor ClickHouse. Menj\u00ebher\u00eb e humba nd\u00ebrfaqen cli.<\/em><\/p>\n<p><\/p>\n<p>Ju keni fat. <\/p>\n<p><\/p>\n<p><em>Pak m\u00eb von\u00eb e humba serverin me nj\u00eb select t\u00eb vog\u00ebl.<\/em><\/p>\n<p><\/p>\n<p>Keni talent. <\/p>\n<p><\/p>\n<p><em>Kam hapur nj\u00eb defekt n\u00eb GitHub, por e injoruan.<\/em> <\/p>\n<p><\/p>\n<p>Do ta shohim. <\/p>\n<p><\/p>\n<p><em>Aleksej m\u00eb mashtroi p\u00ebr t\u00eb ndar\u00eb nj\u00eb raport, duke premtuar se do t\u00eb tregonte si e ngjeshni t\u00eb dh\u00ebnat brenda.<\/em><\/p>\n<p><\/p>\n<p>Shum\u00eb thjesht\u00eb. <\/p>\n<p><\/p>\n<p><em>K\u00ebt\u00eb e kuptova dje. M\u00eb shum\u00eb p\u00ebr t\u00eb sakt\u00eb.<\/em> <\/p>\n<p><\/p>\n<p>Nuk ka asnj\u00eb mashtrim t\u00eb tmerrsh\u00ebm. Thjesht \u00ebsht\u00eb ngjeshje n\u00eb blloqe. N\u00eb parazgjedhje p\u00ebrdoret LZ4, mund t\u00eb aktivizoni ZSTD*. Blloqet jan\u00eb nga 64 kilobajt deri n\u00eb 1 megabajt.<\/p>\n<p><\/p>\n<p>* gjithashtu ka mb\u00ebshtetje p\u00ebr kodek\u00ebt e specializuar t\u00eb ngjeshjes, t\u00eb cilat mund t\u00eb p\u00ebrdoren n\u00eb zinxhir me algorithm\u00eb t\u00eb tjera.<\/p>\n<p><\/p>\n<p><em>A jan\u00eb blloqet thjesht t\u00eb dh\u00ebna t\u00eb pap\u00ebrpunuara?<\/em><\/p>\n<p><\/p>\n<p>Jo krejt\u00ebsisht t\u00eb pap\u00ebrpunuara. Aty ka Arrays. N\u00ebse keni nj\u00eb kolon\u00eb numerike, atje numrat jan\u00eb radhitur n\u00eb nj\u00eb array. <\/p>\n<p><\/p>\n<p><em>E kuptoj.<\/em> <\/p>\n<p><\/p>\n<p><em>Aleks, nj\u00eb shembull q\u00eb ishte me uniqExact mbi IP-t\u00eb, dmth ajo q\u00eb uniqExact llogaritet m\u00eb gjat\u00eb sipas vargjeve sesa sipas numrave e k\u00ebshtu me radh\u00eb. Dhe n\u00ebse ne aplikojm\u00eb nj\u00eb truk dhe do t\u00eb b\u00ebjm\u00eb casting n\u00eb momentin e leximit? Dmth, duket se keni th\u00ebn\u00eb q\u00eb n\u00eb disk nuk dallon shum\u00eb. N\u00ebse lexojm\u00eb vargjet nga disku, b\u00ebjm\u00eb casting, a do t\u00eb kemi m\u00eb shpejt agregat\u00ebt apo jo? Apo ne gjithsesi do t\u00eb fitojm\u00eb pak k\u00ebtu? M\u00eb duket se e keni testuar k\u00ebt\u00eb, por p\u00ebr nj\u00eb arsye nuk e keni treguar n\u00eb benchmark.<\/em> <\/p>\n<p><\/p>\n<p>Mendoj se do t\u00eb jet\u00eb m\u00eb ngadal\u00eb sesa pa casting. N\u00eb k\u00ebt\u00eb rast, adresa IP duhet t\u00eb parserohet nga stringu. Sigurisht, n\u00eb ClickHouse, parsing-u i adresave IP \u00ebsht\u00eb gjithashtu i optimizuar. Ne u munduam shum\u00eb, por numrat jan\u00eb shkruar n\u00eb form\u00ebn me dhjet\u00eb mij\u00eb. Shum\u00eb e pap\u00ebrshtatshme. Nga ana tjet\u00ebr, funksioni uniqExact do t\u00eb punoj\u00eb ngadal\u00eb mbi vargjet jo vet\u00ebm sepse jan\u00eb vargje, por gjithashtu sepse zgjidhet nj\u00eb specializim tjet\u00ebr i algoritmit. Vargjet thjesht trajtohen ndryshe.<\/p>\n<p><\/p>\n<p><em>Po n\u00ebse marrim nj\u00eb tip m\u00eb primitiv t\u00eb dh\u00ebnash? P\u00ebr shembull, regjistrojm\u00eb user id, i cili \u00ebsht\u00eb in, e regjistrojm\u00eb at\u00eb si nj\u00eb varg, dhe m\u00eb pas e b\u00ebjm\u00eb casting, a do t\u00eb jet\u00eb m\u00eb arg\u00ebtues apo jo?<\/em><\/p>\n<p><\/p>\n<p>Kam dyshoj. Mendoj se do t\u00eb jet\u00eb edhe m\u00eb trishtues, sepse n\u00eb fund t\u00eb fundit, analizimi i numrave \u00ebsht\u00eb nj\u00eb problem serioz. M\u00eb duket se ky koleg kishte madje nj\u00eb prezantim mbi sa e v\u00ebshtir\u00eb \u00ebsht\u00eb t\u00eb analizosh numrat n\u00eb form\u00ebn e dhjet\u00eb-mij\u00eb, ose ndoshta jo. <\/p>\n<p><\/p>\n<p><em>Aleksej, faleminderit shum\u00eb p\u00ebr prezantimin! Dhe faleminderit p\u00ebr ClickHouse! Kam nj\u00eb pyetje p\u00ebr planet. A ka ndonj\u00eb plan p\u00ebr nj\u00eb ve\u00e7ori p\u00ebr t\u00eb p\u00ebrdit\u00ebsuar fjalor\u00ebt jo plot\u00ebsisht?<\/em><\/p>\n<p><\/p>\n<p>Pra, nj\u00eb riciklim pjesor?<\/p>\n<p><\/p>\n<p><em>Po-po. Lloji q\u00eb mund t\u00eb shkruash ndonj\u00eb fush\u00eb MySQL, pra, t\u00eb p\u00ebrdit\u00ebsosh pas, q\u00eb t\u00eb ngarkohen vet\u00ebm k\u00ebta t\u00eb dh\u00ebna, n\u00ebse fjalori \u00ebsht\u00eb shum\u00eb i madh.<\/em><\/p>\n<p><\/p>\n<p>Nj\u00eb ve\u00e7ori shum\u00eb interesante. Dhe, m\u00eb duket se ndonj\u00eb njeri e ka propozuar k\u00ebt\u00eb n\u00eb bised\u00ebn ton\u00eb. Ndoshta ky ishe ai vet\u00eb. <\/p>\n<p><\/p>\n<p><em>Nuk mendoj se kam qen\u00eb un\u00eb.<\/em><\/p>\n<p><\/p>\n<p>Shum\u00eb mir\u00eb, tani duket se jan\u00eb dy k\u00ebrkesa. Dhe mund t\u00eb fillosh ngadal\u00eb ta realizosh. Por menj\u00ebher\u00eb dua t'ju paralajm\u00ebroj se kjo ve\u00e7ori \u00ebsht\u00eb mjaft e thjesht\u00eb p\u00ebr t'u realizuar. Q\u00eb do t\u00eb thot\u00eb, n\u00eb thelb duhet thjesht t\u00eb shkruajm\u00eb numrin e versionit n\u00eb tabel\u00eb dhe m\u00eb pas t\u00eb shkruajm\u00eb: versioni \u00ebsht\u00eb m\u00eb i vog\u00ebl se ky i till\u00eb. Dhe kjo do t\u00eb thot\u00eb se, shum\u00eb mund\u00ebsisht, ne do ta propozojm\u00eb k\u00ebt\u00eb p\u00ebr entuziast\u00ebt. A jeni entuziast? <\/p>\n<p><\/p>\n<p><em>Po, por, p\u00ebr fat t\u00eb keq, jo n\u00eb C++.<\/em><\/p>\n<p><\/p>\n<p>A din\u00eb koleg\u00ebt tuaj t\u00eb shkruajn\u00eb n\u00eb C++? <\/p>\n<p><\/p>\n<p><em>Do t\u00eb gjej dik\u00eb.<\/em><\/p>\n<p><\/p>\n<p>Shum\u00eb mir\u00eb.<\/p>\n<p><\/p>\n<p>* mund\u00ebsia u shtua dy muaj pas raportit \u2013 krijuesi i saj e zhvilloi at\u00eb dhe e d\u00ebrgoi vet\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/ClickHouse\/ClickHouse\/pull\/1771\">pull request<\/a><\/noindex>.<\/p>\n<p><\/p>\n<p><em>Faleminderit!<\/em><\/p>\n<p><\/p>\n<p><em>P\u00ebrsh\u00ebndetje! Faleminderit p\u00ebr raportin! Ju p\u00ebrmend\u00ebt se ClickHouse konsumon shum\u00eb mir\u00eb t\u00eb gjitha burimet q\u00eb i jan\u00eb ofruar. Dhe fol\u00ebsi fqinj me Luxoft tregoi p\u00ebr zgjidhjen e tij p\u00ebr Post\u00ebn e Rusis\u00eb. Ai tha se ata e p\u00eblqyen shum\u00eb ClickHouse, por nuk e p\u00ebrdor\u00ebn at\u00eb n\u00eb vend t\u00eb konkurrentit t\u00eb tyre kryesor pik\u00ebrisht sepse ai konsumonte gjith\u00eb procesorin. Dhe ata nuk mund\u00ebn ta inkorporonin n\u00eb arkitektur\u00ebn e tyre, n\u00eb ZooKeeper-in e tyre me docker-at. A ka ndonj\u00eb mund\u00ebsi p\u00ebr ta kufizuar ClickHouse, q\u00eb t\u00eb mos konsumoj\u00eb gjith\u00e7ka q\u00eb i b\u00ebhet e aksesueshme?<\/em><\/p>\n<p><\/p>\n<p>Po, \u00ebsht\u00eb shum\u00eb e leht\u00eb. N\u00ebse d\u00ebshironi q\u00eb t\u00eb konsumoni m\u00eb pak b\u00ebrthama, thjesht shkruani <code>set max_threads = 1<\/code>. Dhe k\u00ebshtu, do t\u00eb ekzekutoj\u00eb k\u00ebrkes\u00ebn n\u00eb nj\u00eb b\u00ebrtham\u00eb. Gjithashtu, mund t'i jepen konfigurime t\u00eb ndryshme p\u00ebrdoruesve t\u00eb ndrysh\u00ebm. Prandaj nuk ka asnj\u00eb problem. Dhe p\u00ebr koleg\u00ebt nga Luxoft, kaloni se nuk ishte mir\u00eb q\u00eb ata nuk e gjet\u00ebn k\u00ebt\u00eb konfigurim n\u00eb dokumentacion. <\/p>\n<p><\/p>\n<p><em>Alex, p\u00ebrsh\u00ebndetje! Doja t\u00eb pyesja p\u00ebr nj\u00eb \u00e7\u00ebshtje. Nuk \u00ebsht\u00eb hera e par\u00eb q\u00eb d\u00ebgjoj se shum\u00eb po fillojn\u00eb ta p\u00ebrdorin ClickHouse si nj\u00eb depo p\u00ebr log\u00ebt. N\u00eb prezantimin tuaj thoshit se nuk duhet b\u00ebr\u00eb k\u00ebshtu, pra nuk duhet t\u00eb ruajm\u00eb vargje t\u00eb gjata. Si e mendoni k\u00ebt\u00eb?<\/em><\/p>\n<p><\/p>\n<p>S\u00eb pari, log\u00ebt zakonisht nuk jan\u00eb vargje t\u00eb gjata. Sigurisht, ndodhin p\u00ebrjashtime. P.sh., ndonj\u00eb sh\u00ebrbim i shkruar n\u00eb java hedh nj\u00eb exception q\u00eb log-izohet. Dhe k\u00ebshtu n\u00eb nj\u00eb cik\u00ebl t\u00eb pafund, dhe p\u00ebrfundojm\u00eb pa hap n\u00eb hard disk. Zgjidhja \u00ebsht\u00eb shum\u00eb e thjesht\u00eb. N\u00ebse vargjet jan\u00eb shum\u00eb t\u00eb gjata, prini ato. Dhe \u00e7far\u00eb do t\u00eb thot\u00eb t\u00eb gjata? Dhjet\u00ebra kilobajt \u00ebsht\u00eb e keqe.<\/p>\n<p><\/p>\n<p>* \u0432 \u0441\u0432\u0435\u0436\u0438\u0445 \u0432\u0435\u0440\u0441\u0438\u044f\u0445 ClickHouse, \u0432\u043a\u043b\u044e\u0447\u0435\u043d\u0430 &quot;\u0430\u0434\u0430\u043f\u0442\u0438\u0432\u043d\u0430\u044f \u0433\u0440\u0430\u043d\u0443\u043b\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u043e\u0441\u0442\u044c \u0438\u043d\u0434\u0435\u043a\u0441\u0430&quot;, \u0447\u0442\u043e \u0441\u043d\u0438\u043c\u0430\u0435\u0442 \u043f\u0440\u043e\u0431\u043b\u0435\u043c\u0443 \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0434\u043b\u0438\u043d\u043d\u044b\u0445 \u0441\u0442\u0440\u043e\u043a \u043f\u043e \u0431\u043e\u043b\u044c\u0448\u0435\u0439 \u0447\u0430\u0441\u0442\u0438.<\/p>\n<p><\/p>\n<p><em>Dhe kilobajt \u2013 a \u00ebsht\u00eb e pranueshme?<\/em><\/p>\n<p><\/p>\n<p>E pranueshme. <\/p>\n<p><\/p>\n<p><em>P\u00ebrsh\u00ebndetje! Faleminderit p\u00ebr prezantimin! Un\u00eb tashm\u00eb kam pyetur p\u00ebr k\u00ebt\u00eb n\u00eb bised\u00eb, por nuk e mbaj mend n\u00ebse kam marr\u00eb p\u00ebrgjigje. A planifikohet ndonj\u00ebher\u00eb t\u00eb zgjerohet seksioni WITH, si CTE?<\/em><\/p>\n<p><\/p>\n<p>P\u00ebr momentin jo. Seksioni WITH \u00ebsht\u00eb disi jo serioz. Ai \u00ebsht\u00eb si nj\u00eb tipar i vog\u00ebl.<\/p>\n<p><\/p>\n<p><em>E kuptoj. Faleminderit!<\/em><\/p>\n<p><\/p>\n<p><em>Faleminderit p\u00ebr raportin! Shum\u00eb interesant! Nj\u00eb pyetje globale. A po planifikohet ndonj\u00eb modifikim p\u00ebr fshirjen e t\u00eb dh\u00ebnave, ndoshta n\u00eb form\u00ebn e ndonj\u00eb skeme?<\/em><\/p>\n<p><\/p>\n<p>Sigurisht. Kjo \u00ebsht\u00eb detyra jon\u00eb e par\u00eb n\u00eb radh\u00ebn ton\u00eb. Tani kemi menduar aktivisht se si ta b\u00ebjm\u00eb gjith\u00e7ka si\u00e7 duhet. Dhe \u00ebsht\u00eb koha t\u00eb fillojm\u00eb t\u00eb shtypim butonat n\u00eb tastier\u00eb*.<\/p>\n<p><\/p>\n<p>* e kemi shtypur \u00e7do buton n\u00eb tastier\u00eb dhe gjith\u00e7ka \u00ebsht\u00eb b\u00ebr\u00eb.<\/p>\n<p><\/p>\n<p><em>A do t\u00eb ndikoj\u00eb kjo ndonj\u00ebher\u00eb n\u00eb performanc\u00ebn e sistemit apo jo? Do t\u00eb jet\u00eb i nj\u00ebjt\u00eb q\u00ebndrimi i sot\u00ebm p\u00ebr inserimin?<\/em><\/p>\n<p><\/p>\n<p>Mund t\u00eb ndodh\u00eb q\u00eb vet\u00eb deletimet, vet\u00eb azhurnimet, t\u00eb jen\u00eb shum\u00eb t\u00eb r\u00ebnda, por kjo nuk do t\u00eb ndikoj\u00eb aspak n\u00eb performanc\u00ebn e seleksioneve dhe performanc\u00ebn e inserimeve.<\/p>\n<p><\/p>\n<p><em>Dhe nj\u00eb pyetje e vog\u00ebl. N\u00eb prezantim p\u00ebrmend\u00ebt \u00e7el\u00ebsin kryesor. Prandaj, ne kemi nj\u00eb ndarjen q\u00eb \u00ebsht\u00eb p\u00ebr default mujore, e sakt\u00eb? Dhe kur vendosim nj\u00eb interval datash q\u00eb p\u00ebrfshihet n\u00eb muaj, at\u00ebher\u00eb vet\u00ebm ajo ndarje do t\u00eb lexohet, e v\u00ebrtet\u00eb?<\/em><\/p>\n<p><\/p>\n<p>Po.<\/p>\n<p><\/p>\n<p><em>Nj\u00eb pyetje. N\u00ebse nuk mund t\u00eb dalim me ndonj\u00eb \u00e7el\u00ebs kryesor, a \u00ebsht\u00eb e sakt\u00eb ta b\u00ebjm\u00eb at\u00eb pik\u00ebrisht sipas fush\u00ebs 'Data', p\u00ebr t\u00eb patur nj\u00eb ristrukturim m\u00eb t\u00eb vog\u00ebl t\u00eb k\u00ebtyre t\u00eb dh\u00ebnave n\u00eb sfond, n\u00eb m\u00ebnyr\u00eb q\u00eb ato t\u00eb jen\u00eb m\u00eb t\u00eb organizuara? N\u00ebse nuk keni k\u00ebrkesa diapazoni dhe madje nuk mund t\u00eb zgjidhni asnj\u00eb \u00e7el\u00ebs kryesor, a ka kuptim t\u00eb v\u00ebm\u00eb dat\u00ebn n\u00eb \u00e7el\u00ebsin kryesor?<\/em><\/p>\n<p><\/p>\n<p>Po.<\/p>\n<p><\/p>\n<p>Ndoshta ka kuptim t\u00eb vendosni nj\u00eb fush\u00eb n\u00eb \u00e7el\u00ebsin kryesor p\u00ebr t\u00eb cil\u00ebn t\u00eb dh\u00ebnat do t\u00eb comprimohen m\u00eb mir\u00eb, n\u00ebse ato jan\u00eb t\u00eb renditura sipas asaj fushe. P\u00ebr shembull, identifikuesi i p\u00ebrdoruesit. P\u00ebrdoruesi, p\u00ebr shembull, viziton t\u00eb nj\u00ebjtin faqe. N\u00eb k\u00ebt\u00eb rast, vendosni ID-n\u00eb e p\u00ebrdoruesit dhe koh\u00ebn. K\u00ebshtu, t\u00eb dh\u00ebnat tuaja do t\u00eb comprimohen m\u00eb mir\u00eb. Sa i p\u00ebrket dat\u00ebs, n\u00ebse me t\u00eb v\u00ebrtet\u00eb nuk keni dhe kurr\u00eb nuk keni k\u00ebrkesa diapazoni p\u00ebr datat, at\u00ebher\u00eb nuk \u00ebsht\u00eb e nevojshme ta vendosni dat\u00ebn n\u00eb \u00e7el\u00ebsin kryesor. <\/p>\n<p><\/p>\n<p><em>Mir\u00eb, faleminderit shum\u00eb!<\/em><\/p>\n<p>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/514840\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0422\u0430\u043a \u043a\u0430\u043a ClickHouse \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u043e\u0439 \u0441\u0438\u0441\u0442\u0435\u043c\u043e\u0439, \u043f\u0440\u0438 \u0435\u0433\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0438 \u0432\u0430\u0436\u043d\u043e \u0443\u0447\u0438\u0442\u044b\u0432\u0430\u0442\u044c \u043e\u0441\u043e\u0431\u0435\u043d\u043d\u043e\u0441\u0442\u0438 \u0435\u0433\u043e \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u044b. \u0412 \u044d\u0442\u043e\u043c \u0434\u043e\u043a\u043b\u0430\u0434\u0435 \u0410\u043b\u0435\u043a\u0441\u0435\u0439 \u0440\u0430\u0441\u0441\u043a\u0430\u0436\u0435\u0442 \u043e \u043f\u0440\u0438\u043c\u0435\u0440\u0430\u0445 \u0442\u0438\u043f\u0438\u0447\u043d\u044b\u0445 \u043e\u0448\u0438\u0431\u043e\u043a \u043f\u0440\u0438 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0438 ClickHouse, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043c\u043e\u0433\u0443\u0442 \u043f\u0440\u0438\u0432\u0435\u0441\u0442\u0438 \u043a \u043d\u0435\u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0439 \u0440\u0430\u0431\u043e\u0442\u0435. \u041d\u0430 \u043f\u0440\u0438\u043c\u0435\u0440\u0430\u0445 \u0438\u0437 \u043f\u0440\u0430\u043a\u0442\u0438\u043a\u0438 \u0431\u0443\u0434\u0435\u0442 \u043f\u043e\u043a\u0430\u0437\u0430\u043d\u043e, \u043a\u0430\u043a \u0432\u044b\u0431\u043e\u0440 \u0442\u043e\u0439 \u0438\u043b\u0438 \u0438\u043d\u043e\u0439 \u0441\u0445\u0435\u043c\u044b \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 \u043c\u043e\u0436\u0435\u0442 \u0438\u0437\u043c\u0435\u043d\u0438\u0442\u044c \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u044c \u043d\u0430 \u043f\u043e\u0440\u044f\u0434\u043a\u0438. \u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442! \u041c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":91439,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-91438","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 4.9.10 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0422\u0430\u043a \u043a\u0430\u043a ClickHouse \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u043e\u0439 \u0441\u0438\u0441\u0442\u0435\u043c\u043e\u0439, \u043f\u0440\u0438 \u0435\u0433\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0438 \u0432\u0430\u0436\u043d\u043e \u0443\u0447\u0438\u0442\u044b\u0432\u0430\u0442\u044c \u043e\u0441\u043e\u0431\u0435\u043d\u043d\u043e\u0441\u0442\u0438 \u0435\u0433\u043e \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u044b. \u0412 \u044d\u0442\u043e\u043c \u0434\u043e\u043a\u043b\u0430\u0434\u0435 \u0410\u043b\u0435\u043a\u0441\u0435\u0439 \u0440\u0430\u0441\u0441\u043a\u0430\u0436\u0435\u0442 \u043e \u043f\u0440\u0438\u043c\u0435\u0440\u0430\u0445 \u0442\u0438\u043f\u0438\u0447\u043d\u044b\u0445 \u043e\u0448\u0438\u0431\u043e\u043a \u043f\u0440\u0438 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0438 ClickHouse, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043c\u043e\u0433\u0443\u0442 \u043f\u0440\u0438\u0432\u0435\u0441\u0442\u0438 \u043a \u043d\u0435\u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0439 \u0440\u0430\u0431\u043e\u0442\u0435. \u041d\u0430 \u043f\u0440\u0438\u043c\u0435\u0440\u0430\u0445 \u0438\u0437 \u043f\u0440\u0430\u043a\u0442\u0438\u043a\u0438 \u0431\u0443\u0434\u0435\u0442 \u043f\u043e\u043a\u0430\u0437\u0430\u043d\u043e, \u043a\u0430\u043a \u0432\u044b\u0431\u043e\u0440 \u0442\u043e\u0439 \u0438\u043b\u0438 \u0438\u043d\u043e\u0439 \u0441\u0445\u0435\u043c\u044b \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 \u043c\u043e\u0436\u0435\u0442 \u0438\u0437\u043c\u0435\u043d\u0438\u0442\u044c \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u044c \u043d\u0430 \u043f\u043e\u0440\u044f\u0434\u043a\u0438. \u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442! \u041c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/effektivnoe-ispolzovanie-clickhouse-aleksej-milovidov-yandeks\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 4.9.10\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u042d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0435 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435 ClickHouse. \u0410\u043b\u0435\u043a\u0441\u0435\u0439 \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432 (\u042f\u043d\u0434\u0435\u043a\u0441) | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0422\u0430\u043a \u043a\u0430\u043a ClickHouse \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u043e\u0439 \u0441\u0438\u0441\u0442\u0435\u043c\u043e\u0439, \u043f\u0440\u0438 \u0435\u0433\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0438 \u0432\u0430\u0436\u043d\u043e \u0443\u0447\u0438\u0442\u044b\u0432\u0430\u0442\u044c \u043e\u0441\u043e\u0431\u0435\u043d\u043d\u043e\u0441\u0442\u0438 \u0435\u0433\u043e \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u044b. \u0412 \u044d\u0442\u043e\u043c \u0434\u043e\u043a\u043b\u0430\u0434\u0435 \u0410\u043b\u0435\u043a\u0441\u0435\u0439 \u0440\u0430\u0441\u0441\u043a\u0430\u0436\u0435\u0442 \u043e \u043f\u0440\u0438\u043c\u0435\u0440\u0430\u0445 \u0442\u0438\u043f\u0438\u0447\u043d\u044b\u0445 \u043e\u0448\u0438\u0431\u043e\u043a \u043f\u0440\u0438 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0438 ClickHouse, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043c\u043e\u0433\u0443\u0442 \u043f\u0440\u0438\u0432\u0435\u0441\u0442\u0438 \u043a \u043d\u0435\u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0439 \u0440\u0430\u0431\u043e\u0442\u0435. \u041d\u0430 \u043f\u0440\u0438\u043c\u0435\u0440\u0430\u0445 \u0438\u0437 \u043f\u0440\u0430\u043a\u0442\u0438\u043a\u0438 \u0431\u0443\u0434\u0435\u0442 \u043f\u043e\u043a\u0430\u0437\u0430\u043d\u043e, \u043a\u0430\u043a \u0432\u044b\u0431\u043e\u0440 \u0442\u043e\u0439 \u0438\u043b\u0438 \u0438\u043d\u043e\u0439 \u0441\u0445\u0435\u043c\u044b \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 \u043c\u043e\u0436\u0435\u0442 \u0438\u0437\u043c\u0435\u043d\u0438\u0442\u044c \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u044c \u043d\u0430 \u043f\u043e\u0440\u044f\u0434\u043a\u0438. \u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442! \u041c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/effektivnoe-ispolzovanie-clickhouse-aleksej-milovidov-yandeks\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-08-13T17:42:36+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-08-13T17:42:36+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47P\u00ebrdorimi efektiv i ClickHouse. Aleksei Milovidov (Yandex) | ProHoster","description":"Meqen\u00ebse ClickHouse \u00ebsht\u00eb nj\u00eb sistem i specializuar, \u00ebsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb merret parasysh karakteristika e arkitektur\u00ebs s\u00eb tij gjat\u00eb p\u00ebrdorimit. N\u00eb k\u00ebt\u00eb raport, Aleksei do t\u00eb flas\u00eb p\u00ebr raste tipike t\u00eb gabimeve gjat\u00eb p\u00ebrdorimit t\u00eb ClickHouse, t\u00eb cilat mund t\u00eb \u00e7ojn\u00eb n\u00eb performanc\u00eb t\u00eb dob\u00ebt. Me shembuj nga praktika, do t\u00eb tregojm\u00eb se si p\u00ebrzgjedhja e nj\u00eb skeme t\u00eb caktuar t\u00eb trajtimit t\u00eb t\u00eb dh\u00ebnave mund t\u00eb ndryshoj\u00eb ndjesh\u00ebm performanc\u00ebn. P\u00ebrsh\u00ebndetje t\u00eb gjith\u00ebve! Un\u00eb quhem","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/effektivnoe-ispolzovanie-clickhouse-aleksej-milovidov-yandeks","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u042d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0435 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435 ClickHouse. \u0410\u043b\u0435\u043a\u0441\u0435\u0439 \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432 (\u042f\u043d\u0434\u0435\u043a\u0441) | ProHoster","og:description":"\u0422\u0430\u043a \u043a\u0430\u043a ClickHouse \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u0441\u043f\u0435\u0446\u0438\u0430\u043b\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u043e\u0439 \u0441\u0438\u0441\u0442\u0435\u043c\u043e\u0439, \u043f\u0440\u0438 \u0435\u0433\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0438 \u0432\u0430\u0436\u043d\u043e \u0443\u0447\u0438\u0442\u044b\u0432\u0430\u0442\u044c \u043e\u0441\u043e\u0431\u0435\u043d\u043d\u043e\u0441\u0442\u0438 \u0435\u0433\u043e \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u044b. \u0412 \u044d\u0442\u043e\u043c \u0434\u043e\u043a\u043b\u0430\u0434\u0435 \u0410\u043b\u0435\u043a\u0441\u0435\u0439 \u0440\u0430\u0441\u0441\u043a\u0430\u0436\u0435\u0442 \u043e \u043f\u0440\u0438\u043c\u0435\u0440\u0430\u0445 \u0442\u0438\u043f\u0438\u0447\u043d\u044b\u0445 \u043e\u0448\u0438\u0431\u043e\u043a \u043f\u0440\u0438 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0438 ClickHouse, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043c\u043e\u0433\u0443\u0442 \u043f\u0440\u0438\u0432\u0435\u0441\u0442\u0438 \u043a \u043d\u0435\u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0439 \u0440\u0430\u0431\u043e\u0442\u0435. \u041d\u0430 \u043f\u0440\u0438\u043c\u0435\u0440\u0430\u0445 \u0438\u0437 \u043f\u0440\u0430\u043a\u0442\u0438\u043a\u0438 \u0431\u0443\u0434\u0435\u0442 \u043f\u043e\u043a\u0430\u0437\u0430\u043d\u043e, \u043a\u0430\u043a \u0432\u044b\u0431\u043e\u0440 \u0442\u043e\u0439 \u0438\u043b\u0438 \u0438\u043d\u043e\u0439 \u0441\u0445\u0435\u043c\u044b \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 \u043c\u043e\u0436\u0435\u0442 \u0438\u0437\u043c\u0435\u043d\u0438\u0442\u044c \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u044c \u043d\u0430 \u043f\u043e\u0440\u044f\u0434\u043a\u0438. \u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442! \u041c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/effektivnoe-ispolzovanie-clickhouse-aleksej-milovidov-yandeks","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-08-13T17:42:36+00:00","article:modified_time":"2020-08-13T17:42:36+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"91438","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 12:27:23","updated":"2022-09-28 17:25:41"},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/91438","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=91438"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/91438\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/91439"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=91438"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=91438"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=91438"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}