{"id":89878,"date":"2020-07-27T01:42:42","date_gmt":"2020-07-26T23:42:42","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/teoriya-i-praktika-ispolzovaniya-clickhouse-v-realnyh-prilozheniyah-aleksandr-zajczev-2018g"},"modified":"2020-07-27T01:42:42","modified_gmt":"2020-07-26T23:42:42","slug":"teoriya-i-praktika-ispolzovaniya-clickhouse-v-realnyh-prilozheniyah-aleksandr-zajczev-2018g","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/teoriya-i-praktika-ispolzovaniya-clickhouse-v-realnyh-prilozheniyah-aleksandr-zajczev-2018g","title":{"rendered":"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/b46b964324a3311157d5df737526b1d1.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Megjith\u00ebse t\u00eb dh\u00ebnat tani jan\u00eb gjetur pothuajse kudo, DBMS analitik jan\u00eb ende mjaft ekzotike. Ato njihen dob\u00ebt dhe p\u00ebrdoren akoma m\u00eb keq p\u00ebr t\u00eb qen\u00eb efektive. Shum\u00eb vazhdojn\u00eb t\u00eb \"han\u00eb kaktus\" me MySQL ose PostgreSQL, q\u00eb jan\u00eb projektuar p\u00ebr skenar\u00eb t\u00eb tjer\u00eb, vuajn\u00eb me NoSQL ose paguajn\u00eb shum\u00eb p\u00ebr zgjidhje tregtare. ClickHouse ndryshon rregullat e loj\u00ebs dhe ndjesh\u00ebm ul pragun p\u00ebr t'iu qasur bot\u00ebs s\u00eb DBMS analitik.<\/p>\n<p><\/p>\n<p>Raporti nga BackEnd Conf 2018 dhe \u00ebsht\u00eb publikuar me leje nga fol\u00ebsi.<\/p>\n<p><noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><br \/>\n<center><div class=\"youtube-placeholder\" data-id=\"9MwKE30aUPs\" onclick=\"loadVideo(this)\">\r\n        <img decoding=\"async\" src=\"https:\/\/img.youtube.com\/vi\/9MwKE30aUPs\/hqdefault.jpg\" alt=\"Luaj videon\" loading=\"lazy\" width=\"480\" height=\"360\" style=\"width:100%;height:auto;\">\r\n        <div class=\"play-button\"><\/div>\r\n    <\/div><\/center><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/6c55b639d9703f460d99e0b0b1194507.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\nKush jam un\u00eb dhe pse po flas p\u00ebr ClickHouse? Un\u00eb jam drejtor i zhvillimit n\u00eb kompanin\u00eb LifeStreet, e cila p\u00ebrdor ClickHouse. P\u00ebr m\u00eb tep\u00ebr, un\u00eb jam themelues i Altinity. Kjo \u00ebsht\u00eb nj\u00eb partner i Yandex, i cili promovon ClickHouse dhe ndihmon Yandex-in ta b\u00ebj\u00eb ClickHouse m\u00eb t\u00eb suksessh\u00ebm. Gjithashtu, jam i gatsh\u00ebm t\u00eb ndaj njohuri mbi ClickHouse. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/36af6acba62271f7aa19dda2eb470870.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe akoma, un\u00eb nuk jam v\u00eblla i Peti Zaitsev-it. Shpesh m\u00eb pyesin p\u00ebr k\u00ebt\u00eb. Jo, ne nuk jemi v\u00ebllez\u00ebr.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/6ca605f97619a1373bb8a92f88d480d5.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\"T\u00eb gjith\u00eb e din\u00eb\", se ClickHouse:<\/p>\n<p><\/p>\n<ul>\n<li>Shum\u00eb i shpejt\u00eb,<\/li>\n<li>Shum\u00eb i p\u00ebrshtatsh\u00ebm, <\/li>\n<li>P\u00ebrdoret n\u00eb Yandex. <\/li>\n<\/ul>\n<p><\/p>\n<p>Pak m\u00eb pak dihet se n\u00eb cilat kompani dhe si p\u00ebrdoret ai. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/c097b003bf8ea13b8a3bb765270e99c8.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Do t'ju tregoj se p\u00ebr \u00e7far\u00eb, ku dhe si p\u00ebrdoret ClickHouse, p\u00ebrve\u00e7 Yandex-it. <\/p>\n<p><\/p>\n<p>Do t\u00eb flas p\u00ebr m\u00ebnyrat se si detyra specifike zgjidhen me ClickHouse n\u00eb kompani t\u00eb ndryshme, cilat mjete ClickHouse mund t\u00eb p\u00ebrdorni p\u00ebr detyrat tuaja, dhe si jan\u00eb p\u00ebrdorur ato n\u00eb kompani t\u00eb ndryshme.<\/p>\n<p><\/p>\n<p>Kam p\u00ebrgatitur tre shembuj q\u00eb tregojn\u00eb ClickHouse nga disa k\u00ebndv\u00ebshtrime. Mendoj se do t\u00eb jet\u00eb interesante.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/d3cab92ceccd6c7d01ca1f28b028803f.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Pyetja e par\u00eb: \"Pse \u00ebsht\u00eb i nevojsh\u00ebm ClickHouse?\". Duket si nj\u00eb pyetje e qart\u00eb, por ka m\u00eb shum\u00eb se nj\u00eb p\u00ebrgjigje. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/1a4e14cc041a3c01ad0fd2a305aeeab0.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<ul>\n<li>P\u00ebrgjigja e par\u00eb \u2013 p\u00ebr shkak t\u00eb performanc\u00ebs. ClickHouse \u00ebsht\u00eb shum\u00eb i shpejt\u00eb. Analizat n\u00eb ClickHouse gjithashtu jan\u00eb shum\u00eb t\u00eb shpejta. Ai shpesh mund t\u00eb p\u00ebrdoret atje ku di\u00e7ka tjet\u00ebr punon shum\u00eb ngadal\u00eb ose shum\u00eb keq. <\/li>\n<li>P\u00ebrgjigja e dyt\u00eb \u2013 \u00ebsht\u00eb kostoja. N\u00eb radh\u00eb t\u00eb par\u00eb, kostoja e skalimit. P\u00ebr shembull, Vertica \u00ebsht\u00eb nj\u00eb baz\u00eb t\u00eb dh\u00ebnash krejt\u00ebsisht e shk\u00eblqyer. Ajo funksionon shum\u00eb mir\u00eb, n\u00ebse keni nj\u00eb sasi t\u00eb vog\u00ebl t\u00eb terabajve t\u00eb t\u00eb dh\u00ebnave. Por kur b\u00ebhet fjal\u00eb p\u00ebr qindra terabaj, ose petabaj, kostoja e licenc\u00ebs dhe mb\u00ebshtetjes arrin nj\u00eb shum\u00eb t\u00eb konsiderueshme. Dhe kjo \u00ebsht\u00eb e shtrenjt\u00eb. Nd\u00ebrsa ClickHouse \u00ebsht\u00eb falas. <\/li>\n<li>P\u00ebrgjigjja e tret\u00eb \u00ebsht\u00eb kostoja operacionale. Kjo \u00ebsht\u00eb nj\u00eb qasje q\u00eb vjen pak ndryshe. RedShift \u00ebsht\u00eb nj\u00eb alternativ\u00eb e shk\u00eblqyer. N\u00eb RedShift, mund t\u00eb krijoni zgjidhje shum\u00eb shpejt. Ajo do t\u00eb funksionoj\u00eb mir\u00eb, por \u00e7do or\u00eb, \u00e7do dit\u00eb dhe \u00e7do muaj do t\u00eb paguani mjaft shtrenjt\u00eb p\u00ebr Amazon, sepse ky \u00ebsht\u00eb nj\u00eb sh\u00ebrbim tep\u00ebr i kushtuesh\u00ebm. Edhe Google BigQuery \u00ebsht\u00eb i till\u00eb. N\u00ebse dikush e ka p\u00ebrdorur, e di se atje mund t\u00eb ekzekutoni disa pyetje dhe t\u00eb merrni nj\u00eb fatur\u00eb q\u00eb shkon n\u00eb qindarka dollar\u00eb papritur. <\/li>\n<\/ul>\n<p><\/p>\n<p>N\u00eb ClickHouse nuk ka k\u00ebto probleme. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/3084a8bd9271f14726d73989946816c5.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Ku p\u00ebrdoret tani ClickHouse? P\u00ebrve\u00e7 Yandex, ClickHouse p\u00ebrdoret n\u00eb shum\u00eb biznese dhe kompani t\u00eb ndryshme. <\/p>\n<p><\/p>\n<ul>\n<li>N\u00eb radh\u00eb t\u00eb par\u00eb, kjo \u00ebsht\u00eb analiza e aplikacioneve web, pra \u00ebsht\u00eb nj\u00eb rast p\u00ebrdorimi q\u00eb ka ardhur nga Yandex.<\/li>\n<li>Shum\u00eb kompani t\u00eb AdTech p\u00ebrdorin ClickHouse. <\/li>\n<li>Shum\u00eb kompani q\u00eb kan\u00eb nevoj\u00eb t\u00eb analizojn\u00eb log\u00ebt operative nga burime t\u00eb ndryshme.<\/li>\n<li>Disa kompani p\u00ebrdorin ClickHouse p\u00ebr monitorimin e log\u00ebve t\u00eb siguris\u00eb. Ato i ngarkojn\u00eb n\u00eb ClickHouse, b\u00ebjn\u00eb raporte dhe marrin rezultatet e nevojshme.<\/li>\n<li>Kompani po fillojn\u00eb ta p\u00ebrdorin n\u00eb analiz\u00ebn financiare, pra gradualisht biznesi i madh po e pranon gjithashtu ClickHouse. <\/li>\n<li>CloudFlare. N\u00ebse dikush po ndjek ClickHouse, me siguri ka d\u00ebgjuar p\u00ebr emrin e k\u00ebsaj kompanie. Ky \u00ebsht\u00eb nj\u00eb nga kontribuesit e r\u00ebnd\u00ebsish\u00ebm nga komuniteti. Ata kan\u00eb nj\u00eb instalim shum\u00eb t\u00eb r\u00ebnd\u00ebsish\u00ebm t\u00eb ClickHouse. P\u00ebr shembull, ata krijuan Kafka Engine p\u00ebr ClickHouse. <\/li>\n<li>Kompani telekomunikacioni kan\u00eb filluar ta p\u00ebrdorin. Disa kompani ClickHouse e p\u00ebrdorin ose si prov\u00eb koncepti, ose tashm\u00eb n\u00eb prodhim.<\/li>\n<li>Nj\u00eb kompani e p\u00ebrdor ClickHouse p\u00ebr monitorimin e proceseve prodhuese. Ata testojn\u00eb \u00e7ipa, regjistrojn\u00eb shum\u00eb parametra, rreth 2,000 karakteristika. Dhe m\u00eb pas analizojn\u00eb \u2013 \u00ebsht\u00eb nj\u00eb partin\u00eb e mir\u00eb apo e keqe.<\/li>\n<li>Analiza e blockchain. Ekziston nj\u00eb kompani ruse e quajtur Bloxy.info. Kjo \u00ebsht\u00eb analiza e rrjetit ethereum. K\u00ebta e realizuan gjithashtu n\u00eb ClickHouse. <\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/8a0c1924a20ed2d15cd0f959e9b3e499.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Shkall\u00ebzimi nuk ka r\u00ebnd\u00ebsi. Ka shum\u00eb kompani q\u00eb p\u00ebrdorin nj\u00eb server t\u00eb vog\u00ebl. Dhe ai u lejon atyre t\u00eb zgjidhin problemet e tyre. Dhe m\u00eb shum\u00eb kompani p\u00ebrdorin klastera t\u00eb m\u00ebdhenj me shum\u00eb <a class=\"wpil_keyword_link\" href=\"https:\/\/prohoster.info\/sq\/server\/\"   title=\"server\u00ebve\" data-wpil-keyword-link=\"linked\"  data-wpil-monitor-id=\"1513\">server\u00ebve<\/a> ose dhjet\u00ebra servera. <\/p>\n<p><\/p>\n<p>Dhe n\u00ebse shikoni rekordet, at\u00ebher\u00eb:<\/p>\n<p><\/p>\n<ul>\n<li>Yandex: 500+ servera, 25 miliard\u00eb regjistrime n\u00eb dit\u00eb q\u00eb ata ruajn\u00eb aty.<\/li>\n<li>LifeStreet: 60 servera, rreth 75 miliard\u00eb regjistrime n\u00eb dit\u00eb. M\u00eb pak servera, m\u00eb shum\u00eb regjistrime se n\u00eb Yandex. <\/li>\n<li>CloudFlare: 36 server\u00eb, 200 miliard\u00eb regjistrime n\u00eb dit\u00eb ata ruajn\u00eb. Ata kan\u00eb edhe p\u00ebr s\u00eb tep\u00ebrmi pak server\u00eb dhe edhe m\u00eb shum\u00eb t\u00eb dh\u00ebna q\u00eb ruajn\u00eb. <\/li>\n<li>Bloomberg: 102 <a class=\"wpil_keyword_link\" href=\"https:\/\/prohoster.info\/sq\/server\/dts-los-angeles\/\"   title=\"serverit\" data-wpil-keyword-link=\"linked\"  data-wpil-monitor-id=\"3643\">serverit<\/a>, rreth nj\u00eb trilion regjistrimesh n\u00eb dit\u00eb. Rekordmeni p\u00ebr regjistrime.<\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/49194030b6586221cb17029889ce33da.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Gjeografikisht \u2013 \u00ebsht\u00eb gjithashtu shum\u00eb. Kjo hart\u00eb tregon heatmap-in, ku ClickHouse p\u00ebrdoret n\u00eb bot\u00eb. K\u00ebtu spikasin qart\u00eb Rusia, Kina, Amerika. Ka pak vende evropiane. Dhe mund t\u00eb ve\u00e7ojm\u00eb 4 klastera.<\/p>\n<p><\/p>\n<p>Ky \u00ebsht\u00eb nj\u00eb analiz\u00eb krahasuese, nuk ka nevoj\u00eb t\u00eb k\u00ebrkosh numra absolut\u00eb. Ky \u00ebsht\u00eb nj\u00eb analiz\u00eb e vizitor\u00ebve q\u00eb lexojn\u00eb materiale n\u00eb anglishte n\u00eb faqen e Altinity, sepse atje nuk ka material n\u00eb rusisht. Dhe Rusia, Ukraina, Bjellorusia, dmth. pjesa rusishtfol\u00ebse e komunitetit, jan\u00eb p\u00ebrdoruesit m\u00eb t\u00eb shumt\u00eb. M\u00eb pas vjen SHBA dhe Kanada. Kina po ngjitet shum\u00eb shpejt. Para gjasht\u00eb muajve nuk kishte pothuajse asgj\u00eb nga Kina, tani Kina ka kaluar Evrop\u00ebn dhe vazhdon t\u00eb rritet. Vjet\u00ebr-Evropa gjithashtu nuk mbetet prapa, nd\u00ebrsa lideri n\u00eb p\u00ebrdorimin e ClickHouse \u00ebsht\u00eb, si\u00e7 nuk \u00ebsht\u00eb p\u00ebr t'u besuar, Franca. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/3a8305c51b1bd62f3467b795402df4ed.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p><strong>Pse po tregoj gjith\u00eb k\u00ebt\u00eb? P\u00ebr t\u00eb treguar se ClickHouse po b\u00ebhet zgjidhja standarde p\u00ebr analizimin e t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe se po p\u00ebrdoret n\u00eb shum\u00eb vende.<\/strong> N\u00ebse e p\u00ebrdorni, jeni n\u00eb rrug\u00ebn e duhur. N\u00ebse ende nuk e p\u00ebrdorni, mund t\u00eb mos keni frik\u00eb se do t\u00eb mbeteni vet\u00ebm dhe askush nuk do t'ju ndihmoj\u00eb, sepse shum\u00eb njer\u00ebz tashm\u00eb po merren me k\u00ebt\u00eb.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/6aa635f3a35034d3af837c78b2c8fe60.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>K\u00ebto jan\u00eb shembuj t\u00eb p\u00ebrdorimit t\u00eb v\u00ebrtet\u00eb t\u00eb ClickHouse n\u00eb disa kompani. <\/p>\n<p><\/p>\n<ul>\n<li>Shembulli i par\u00eb \u00ebsht\u00eb nj\u00eb rrjet reklamash: migrimi nga Vertica n\u00eb ClickHouse. Dhe un\u00eb di disa kompani q\u00eb jan\u00eb transferuar nga Vertica ose jan\u00eb n\u00eb procesin e kalimit. <\/li>\n<li>Shembulli i dyt\u00eb \u00ebsht\u00eb nj\u00eb depo tranzaksionesh n\u00eb ClickHouse. Ky \u00ebsht\u00eb nj\u00eb shembull i nd\u00ebrtuar mbi antipat nd\u00ebrsa. \u00c7do gj\u00eb q\u00eb nuk duhet b\u00ebr\u00eb n\u00eb ClickHouse sipas k\u00ebshillave t\u00eb zhvilluesve, k\u00ebtu \u00ebsht\u00eb b\u00ebr\u00eb. Dhe gjithashtu \u00ebsht\u00eb b\u00ebr\u00eb aq efektivisht sa funksionon. Dhe funksionon shum\u00eb m\u00eb mir\u00eb se nj\u00eb zgjidhje tipike tranzaksionesh. <\/li>\n<li>Shembulli i tret\u00eb \u00ebsht\u00eb llogaritjet e shp\u00ebrndara n\u00eb ClickHouse. Kishte nj\u00eb pyetje n\u00eb lidhje me m\u00ebnyr\u00ebn se si mund t\u00eb integrohet ClickHouse n\u00eb ekosistemin Hadoop. Do t\u00eb tregoj nj\u00eb shembull se si nj\u00eb kompani b\u00ebri n\u00eb ClickHouse di\u00e7ka si nj\u00eb analog t\u00eb kontejnerit map reduce, duke mbajtur parasysh lokalizimin e t\u00eb dh\u00ebnave etj., p\u00ebr t\u00eb llogaritur nj\u00eb detyr\u00eb shum\u00eb jo triviale. <\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/e08ab5418500599f08d40324ccdfbcd0.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<ul>\n<li>LifeStreet \u2013 \u00ebsht\u00eb nj\u00eb kompani Ad Tech, q\u00eb ka t\u00eb gjitha teknologjit\u00eb p\u00ebrkat\u00ebse p\u00ebr nj\u00eb rrjet reklamash. <\/li>\n<li>Ajo merret me optimizimin e reklamave, ofertat programatike. <\/li>\n<li>Shum\u00eb t\u00eb dh\u00ebna: rreth 10 miliard ngjarje n\u00eb dit\u00eb. N\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, k\u00ebto ngjarje mund t\u00eb ndahen n\u00eb disa n\u00ebn-ngjarje.<\/li>\n<li>Shum\u00eb klient\u00eb p\u00ebrdorin k\u00ebto t\u00eb dh\u00ebna, dhe kjo nuk jan\u00eb vet\u00ebm njer\u00ebzit, ka shum\u00eb m\u00eb tep\u00ebr \u2013 jan\u00eb algoritme t\u00eb ndryshme q\u00eb merret me ofertat programatike. <\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/2da99b53eb25f675a2431027f86db934.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Kompania ka kaluar nj\u00eb rrug\u00eb t\u00eb gjat\u00eb dhe me sfida. E kam p\u00ebrmendur k\u00ebt\u00eb n\u00eb HighLoad. Fillimisht, LifeStreet kaloi nga MySQL (me nj\u00eb ndales\u00eb t\u00eb vog\u00ebl n\u00eb Oracle) n\u00eb Vertica. Dhe mund t\u00eb gjeni informacion mbi k\u00ebt\u00eb. <\/p>\n<p><\/p>\n<p>Gjith\u00e7ka shkoi shum\u00eb mir\u00eb, por mjaft shpejt u kuptua q\u00eb t\u00eb dh\u00ebnat po rriteshin dhe Vertica \u00ebsht\u00eb e shtrenjt\u00eb. Prandaj, u k\u00ebrkuan alternativa t\u00eb ndryshme. Disa prej tyre jan\u00eb p\u00ebrmendur k\u00ebtu. N\u00eb t\u00eb v\u00ebrtet\u00eb, ne b\u00ebm\u00eb nj\u00eb prov\u00eb koncepti ose ndonj\u00ebher\u00eb testim performanc\u00eb p\u00ebr gati t\u00eb gjitha databazat q\u00eb ishin n\u00eb dispozicion n\u00eb treg nga viti 2013 deri n\u00eb 2016 dhe q\u00eb p\u00ebrshtateshin p\u00ebr funksionalitetin. Dhe p\u00ebr disa prej tyre kam folur gjithashtu n\u00eb HighLoad.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/4bebd2f43892cbef9249ddded1b8a70a.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>I ishte nj\u00eb detyr\u00eb \u2013 t\u00eb migrohej nga Vertica fillimisht, sepse t\u00eb dh\u00ebnat po rriteshin. Ato rriteshin eksponencialisht p\u00ebr disa vite. Pastaj dol\u00ebn n\u00eb nj\u00eb nivel t\u00eb caktuar, megjithat\u00eb. Duke parashikuar k\u00ebt\u00eb rritje, k\u00ebrkesat e biznesit p\u00ebr volum t\u00eb t\u00eb dh\u00ebnave, p\u00ebr t\u00eb cilat duhet t\u00eb b\u00ebhej ndonj\u00eb analiz\u00eb, ishte e qart\u00eb se shum\u00eb shpejt do t\u00eb diskutohej p\u00ebr petabajt. Dhe p\u00ebr petabajt, pagesa tashm\u00eb \u00ebsht\u00eb shum\u00eb e shtrenjt\u00eb, prandaj po k\u00ebrkonim alternativ\u00eb ku mund t\u00eb iknim. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/f170a4307168e8560122966eab85c24e.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Ku t\u00eb iknim? Dhe p\u00ebr nj\u00eb koh\u00eb t\u00eb gjat\u00eb ishte krejt\u00ebsisht e paqart\u00eb, ku t\u00eb iknim, sepse nga nj\u00ebra an\u00eb ekzistojn\u00eb bazat e t\u00eb dh\u00ebnave komerciale, ato duket se funksionojn\u00eb mir\u00eb. Disa funksionojn\u00eb pothuajse po aq mir\u00eb sa Vertica, disa m\u00eb keq. Por t\u00eb gjitha jan\u00eb t\u00eb shtrenjta, nuk arrit\u00ebm t\u00eb gjenim asgj\u00eb m\u00eb t\u00eb lir\u00eb dhe m\u00eb t\u00eb mir\u00eb. <\/p>\n<p><\/p>\n<p>Nga ana tjet\u00ebr, ekzistojn\u00eb zgjidhje open source, t\u00eb cilat nuk jan\u00eb shum\u00eb, dmth. p\u00ebr analiz\u00eb mund t\u00eb num\u00ebrohen me gishta. Dhe ato jan\u00eb pa pages\u00eb ose t\u00eb lira, por funksionojn\u00eb ngadal\u00eb. Dhe shpesh u mungon funksionaliteti i nevojsh\u00ebm dhe i dobish\u00ebm.<\/p>\n<p><\/p>\n<p>Dhe p\u00ebr t\u00eb kombinuar ato t\u00eb mira q\u00eb ekzistojn\u00eb n\u00eb bazat e t\u00eb dh\u00ebnave komerciale dhe gjith\u00e7ka q\u00eb \u00ebsht\u00eb pa pages\u00eb n\u00eb open source, nuk kishte asgj\u00eb. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/abbefbf0d9578b554f81579b87213a50.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Nuk kishte asgj\u00eb deri sa papritur Yandex nxori, si nj\u00eb magjistar q\u00eb nxjerr nj\u00eb lepur nga nj\u00eb kapel\u00eb, ClickHouse. Dhe kjo ishte nj\u00eb zgjidhje e papritur, ende shijoje pyetjen: \"Pse?\", por megjithat\u00eb. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/c88981aace9d5925396ca5171241b7d2.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe menj\u00ebher\u00eb n\u00eb ver\u00ebn e vitit 2016 filluam t\u00eb shikojm\u00eb se \u00e7far\u00eb ishte ClickHouse. Dhe rezultoi se ndonj\u00ebher\u00eb mund t\u00eb ishte m\u00eb i shpejt\u00eb se Vertica. Ne testuam skenar\u00eb t\u00eb ndrysh\u00ebm me k\u00ebrkesa t\u00eb ndryshme. Dhe n\u00ebse k\u00ebrkesa p\u00ebrdorte vet\u00ebm nj\u00eb tabel\u00eb, pra pa ndonj\u00eb bashk\u00ebngjitje (join), ClickHouse ishte dy her\u00eb m\u00eb i shpejt\u00eb se Vertica. <\/p>\n<p><\/p>\n<p>Nuk u larova dhe shikova testet e Yandex n\u00eb dit\u00ebt e fundit. Aty ishte e nj\u00ebjta gj\u00eb: ClickHouse dy her\u00eb m\u00eb i shpejt\u00eb se Vertica, prandaj ata shpesh flasin p\u00ebr k\u00ebt\u00eb. <\/p>\n<p><\/p>\n<p>Por n\u00ebse k\u00ebrkesat p\u00ebrmbajn\u00eb bashk\u00ebngjitje (join), at\u00ebher\u00eb gjith\u00e7ka b\u00ebhet jo shum\u00eb e qart\u00eb. Dhe ClickHouse mund t\u00eb jet\u00eb dy her\u00eb m\u00eb i ngadalt\u00eb se Vertica. Por n\u00ebse pak e rregullojm\u00eb k\u00ebrkes\u00ebn dhe e rishkruajm\u00eb, at\u00ebher\u00eb jan\u00eb af\u00ebrsisht t\u00eb barabarta. Jo keq. Dhe falas. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/188bf56b72123175a18d2cb97d989a47.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe duke marr\u00eb rezultatet e testeve dhe duke e par\u00eb nga k\u00ebnde t\u00eb ndryshme, LifeStreet kaloi n\u00eb ClickHouse.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/1183b9b7681916686c52786fc95ccb45.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Kjo \u00ebsht\u00eb vitin 2016, kujtoj. Ishte si n\u00eb nj\u00eb anekdot\u00eb p\u00ebr miqt\u00eb, q\u00eb qanin dhe mbusheshin me gjilp\u00ebr\u00eb, por vazhdonin t\u00eb hanin kaktusin. Dhe p\u00ebr k\u00ebt\u00eb u p\u00ebrshkrua n\u00eb detaje, ka video p\u00ebr t\u00eb, etj. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/993052ed83dc295c3777e614c7703456.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Prandaj nuk do t\u00eb flas n\u00eb detaje rreth k\u00ebsaj, do t\u00eb ndaj vet\u00ebm rezultatet dhe disa gj\u00ebra interesante q\u00eb nuk i kam p\u00ebrmendur at\u00ebher\u00eb. <\/p>\n<p><\/p>\n<p>Rezultatet jan\u00eb:<\/p>\n<p><\/p>\n<ul>\n<li>Migrimi i suksessh\u00ebm dhe sistemi tashm\u00eb funksionon n\u00eb prodhim p\u00ebr m\u00eb shum\u00eb se nj\u00eb vit. <\/li>\n<li>Performanca dhe fleksibiliteti jan\u00eb rritur. Nga 10 miliard\u00eb regjistrime q\u00eb mund t\u00eb lejonim t\u00eb ruheshin p\u00ebr nj\u00eb dit\u00eb dhe at\u00eb p\u00ebr nj\u00eb koh\u00eb t\u00eb shkurt\u00ebr, tani LifeStreet ruan 75 miliard\u00eb regjistrime n\u00eb dit\u00eb dhe mund ta b\u00ebj\u00eb k\u00ebt\u00eb p\u00ebr 3 muaj e m\u00eb shum\u00eb. N\u00eb pikun e aktivitetit, ruhet deri n\u00eb nj\u00eb milion ngjarje n\u00eb sekond\u00eb. M\u00eb shum\u00eb se nj\u00eb milion k\u00ebrkesa SQL n\u00eb dit\u00eb arrijn\u00eb n\u00eb k\u00ebt\u00eb sistem, kryesisht nga robot\u00eb t\u00eb ndrysh\u00ebm. <\/li>\n<li>Megjith\u00ebse p\u00ebr ClickHouse po p\u00ebrdoren m\u00eb shum\u00eb server\u00eb se p\u00ebr Vertica, kursimi dhe n\u00eb hardware \u00ebsht\u00eb arritur, sepse n\u00eb Vertica jan\u00eb p\u00ebrdorur disqe SAS mjaft t\u00eb shtrenjta. Nd\u00ebrsa n\u00eb ClickHouse jan\u00eb p\u00ebrdorur SATA. Pse? Sepse n\u00eb Vertica, inserti \u00ebsht\u00eb sinkron. Dhe sinkronizimi k\u00ebrkon q\u00eb disqet t\u00eb mos ngadal\u00ebsohen shum\u00eb, si dhe rrjeti t\u00eb mos ngadal\u00ebsohet shum\u00eb, dmth, \u00ebsht\u00eb nj\u00eb operacion mjaft i shtrenjt\u00eb. Nd\u00ebrsa n\u00eb ClickHouse, inserti \u00ebsht\u00eb asinkron. P\u00ebr m\u00eb tep\u00ebr, gjith\u00e7ka mund t\u00eb shkruhet gjithmon\u00eb lokal, nuk ka shpenzime shtes\u00eb p\u00ebr k\u00ebt\u00eb, prandaj t\u00eb dh\u00ebnat n\u00eb ClickHouse mund t\u00eb futen shum\u00eb m\u00eb shpejt se n\u00eb Vertica edhe n\u00eb disqe q\u00eb nuk jan\u00eb nga m\u00eb t\u00eb shpejtit. Nd\u00ebrsa n\u00eb lexim jan\u00eb p\u00ebraf\u00ebrsisht t\u00eb nj\u00ebjta. Leximi n\u00eb SATA, n\u00ebse jan\u00eb n\u00eb RAID, at\u00ebher\u00eb kjo \u00ebsht\u00eb e gjitha mjaft e shpejt\u00eb. <\/li>\n<li>Nuk jan\u00eb t\u00eb kufizuar nga licenca, dmth. 3 petabayta t\u00eb dh\u00ebnash n\u00eb 60 server\u00eb (20 server\u00eb - \u00ebsht\u00eb nj\u00eb replik\u00eb) dhe 6 trilion artikuj n\u00eb fakte dhe agregate. Asgj\u00eb e till\u00eb nuk mund t\u00eb lejohej n\u00eb Vertica. <\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/f461cdc12917ee655655dd7eb9c32bb2.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Tani po kaloj n\u00eb gj\u00ebrat praktike n\u00eb k\u00ebt\u00eb shembull.<\/p>\n<p><\/p>\n<ul>\n<li>E para - \u00ebsht\u00eb nj\u00eb skem\u00eb efikase. Nga skema varet shum\u00eb. <\/li>\n<li>E dyta - \u00ebsht\u00eb gjenerimi i SQL-it efikas.<\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/e7dc2a217e77c8794ff6e8a754e78c85.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Nj\u00eb pyetje tipike OLAP \u00ebsht\u00eb nj\u00eb select. Disa kolona shkojn\u00eb n\u00eb group by, disa kolona shkojn\u00eb n\u00eb funksionet agregate. Ka nj\u00eb where, t\u00eb cil\u00ebn mund ta p\u00ebrshkruajm\u00eb si nj\u00eb prerje t\u00eb kubit. I gjith\u00eb group by mund t\u00eb p\u00ebrfaq\u00ebsohet si nj\u00eb projekcion. Pra, kjo quhet analiza multidimensionale e t\u00eb dh\u00ebnave. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/a03a2d7fd28905ae83183346f0c673e2.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe shpesh kjo modelizohet n\u00eb form\u00ebn e nj\u00eb skeme yll, kur ka nj\u00eb fakt qendror dhe karakteristikat e k\u00ebtij fakti p\u00ebrreth, n\u00eb rrezatimin jasht\u00eb. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/d9671af4b39cc83e1ae49e6fd75d0e02.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe nga k\u00ebndv\u00ebshtrimi i dizajnit fizik, se si shtrihet n\u00eb tabela, zakonisht b\u00ebhet nj\u00eb p\u00ebrfaq\u00ebsim i normalizuar. Mund t\u00eb b\u00ebni denormalizim, por kjo \u00ebsht\u00eb e shtrenjt\u00eb n\u00eb disk dhe jo shum\u00eb efikase n\u00eb pyetje. Prandaj, zakonisht b\u00ebhet nj\u00eb p\u00ebrfaq\u00ebsim i normalizuar, pra, tabela e faktit dhe shum\u00eb tabele dimensionale. <\/p>\n<p><\/p>\n<p>Por n\u00eb ClickHouse kjo funksionon keq. Ka dy arsye: <\/p>\n<p><\/p>\n<ul>\n<li>E para \u2013 \u00ebsht\u00eb sepse n\u00eb ClickHouse nuk ka shum\u00eb lidhje (join) t\u00eb mira, pra, lidhje (join) ka, por ato jan\u00eb t\u00eb k\u00ebqija. Ose p\u00ebr momentin jan\u00eb t\u00eb k\u00ebqija. <\/li>\n<li>E dyta \u2013 \u00ebsht\u00eb se tabelat nuk p\u00ebrdit\u00ebsohen. Zakonisht n\u00eb k\u00ebto tabela, q\u00eb rrethojn\u00eb skem\u00ebn e yllit, nevojitet t\u00eb ndryshohet di\u00e7ka. P\u00ebr shembull, emri i klientit, emri i kompanis\u00eb dhe t\u00eb tjera. Dhe kjo nuk funksionon. <\/li>\n<\/ul>\n<p><\/p>\n<p>Dhe ka dy zgjidhje p\u00ebr k\u00ebt\u00eb n\u00eb ClickHouse: <\/p>\n<p><\/p>\n<ul>\n<li>E para n\u00eb p\u00ebrdorimi i fjalor\u00ebve. Fjalor\u00ebt e Jasht\u00ebm jan\u00eb ato q\u00eb ndihmojn\u00eb p\u00ebr t\u00eb zgjidhur me 99% problemin e skem\u00ebs star, me p\u00ebrdit\u00ebsimet dhe t\u00eb tjera. <\/li>\n<li>E dyta \u2013 \u00ebsht\u00eb p\u00ebrdorimi i masivave. Masivat gjithashtu ndihmojn\u00eb p\u00ebr t\u00eb eliminuar bashk\u00ebngjitjet (join) dhe problemet me normalizimin. <\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/775972cc412651600799173407384492.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<ul>\n<li>Nuk nevojiten bashk\u00ebngjitje (join). <\/li>\n<li>T\u00eb p\u00ebrdit\u00ebsueshme. Q\u00eb nga marsi i vitit 2018, ka ardhur nj\u00eb mund\u00ebsi e pa dokumentuar (nuk do ta gjeni n\u00eb dokumentacion) p\u00ebr t\u00eb p\u00ebrdit\u00ebsuar pjes\u00ebrisht fjalor\u00ebt, dmth, ato sh\u00ebnime q\u00eb kan\u00eb ndryshuar. Praktikisht \u2013 \u00ebsht\u00eb si nj\u00eb tabel\u00eb.<\/li>\n<li>Gjithmon\u00eb n\u00eb memorie, k\u00ebshtu q\u00eb bashk\u00ebngjitjet (join) me fjalorin punojn\u00eb m\u00eb shpejt se sa do t\u00eb ishte nj\u00eb tabel\u00eb q\u00eb ndodhet n\u00eb disk dhe nuk dihet akoma n\u00ebse \u00ebsht\u00eb n\u00eb cache, ndoshta nuk \u00ebsht\u00eb. <\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/f73176a650b6d885011e3265c2af2fc3.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<ul>\n<li>Po ashtu nuk nevojiten bashk\u00ebngjitje (join).<\/li>\n<li>Kjo \u00ebsht\u00eb nj\u00eb p\u00ebrfaq\u00ebsim kompakt 1 ndaj shum\u00eb. <\/li>\n<li>Dhe sipas mendimit tim, masivat jan\u00eb b\u00ebr\u00eb p\u00ebr geek-\u00ebt. K\u00ebto jan\u00eb funksionet lambda dhe t\u00eb tjera. <\/li>\n<\/ul>\n<p><\/p>\n<p>Kjo nuk \u00ebsht\u00eb p\u00ebr fjal\u00eb t\u00eb kuqe. Kjo \u00ebsht\u00eb nj\u00eb funksionalitet shum\u00eb i fuqish\u00ebm q\u00eb lejon t\u00eb b\u00ebni shum\u00eb gj\u00ebra shum\u00eb thjesht dhe elegant. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/5d813952ca8f3da7a74c6f2a15d80a9b.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Shembuj tipik\u00eb q\u00eb ndihmojn\u00eb p\u00ebr t\u00eb zgjidhur masivat. K\u00ebta shembuj jan\u00eb t\u00eb thjesht\u00eb dhe mjaft t\u00eb qart\u00eb:<\/p>\n<p><\/p>\n<ul>\n<li>K\u00ebrkimi sipas etiketave. N\u00ebse keni atje hashtag dhe d\u00ebshironi t\u00eb gjeni disa sh\u00ebnime sipas hashtags. <\/li>\n<li>K\u00ebrkoni n\u00eb parametrat key-value. Ka disa atribute me vlera gjithashtu. <\/li>\n<li>Ruani listat e \u00e7el\u00ebsave q\u00eb duhet t'i p\u00ebrktheni n\u00eb ndonj\u00eb gj\u00eb tjet\u00ebr.<\/li>\n<\/ul>\n<p><\/p>\n<p>T\u00eb gjitha k\u00ebto detyra mund t\u00eb zgjidhen pa masa. Etiketat mund t\u00eb vendosen n\u00eb nj\u00eb varg dhe t\u00eb zgjedhen me shprehje reguli ose n\u00eb nj\u00eb tabel\u00eb t\u00eb ve\u00e7ant\u00eb, por at\u00ebher\u00eb do t'ju duhet t\u00eb b\u00ebni bashkime (join).<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/dbbf90eee098c4564206532f24156823.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>N\u00eb ClickHouse nuk ka nevoj\u00eb t\u00eb b\u00ebni asgj\u00eb, mjafton t\u00eb p\u00ebrshkruani nj\u00eb varg string p\u00ebr hashtag\u00ebt ose t\u00eb krijoni nj\u00eb struktur\u00eb t\u00eb thelluar p\u00ebr sisteme si key-value.<\/p>\n<p><\/p>\n<p>Struktura e thelluar \u2013 ndoshta nuk \u00ebsht\u00eb emri m\u00eb i mir\u00eb. Jan\u00eb dy vargje q\u00eb kan\u00eb nj\u00eb pjes\u00eb t\u00eb p\u00ebrbashk\u00ebt n\u00eb em\u00ebr dhe disa karakteristika t\u00eb lidhura. <\/p>\n<p><\/p>\n<p>Dhe t\u00eb k\u00ebrkosh p\u00ebr etiket\u00ebn \u00ebsht\u00eb shum\u00eb e thjesht\u00eb. Ekziston nj\u00eb funksion <code>ka<\/code>, q\u00eb kontrollon n\u00ebse nj\u00eb element \u00ebsht\u00eb n\u00eb varg. K\u00ebtu, gjet\u00ebm t\u00eb gjitha regjistrimet q\u00eb i p\u00ebrkasin konferenc\u00ebs ton\u00eb.<\/p>\n<p><\/p>\n<p>K\u00ebrkimi p\u00ebr subid \u00ebsht\u00eb pak m\u00eb i komplikuar. Fillimisht duhet t\u00eb gjejm\u00eb indeksin e \u00e7el\u00ebsit, pastaj t\u00eb marrim elementin me k\u00ebt\u00eb indeks dhe t\u00eb kontrollojm\u00eb n\u00ebse vlera \u00ebsht\u00eb ajo q\u00eb duam. Megjithat\u00eb, \u00ebsht\u00eb shum\u00eb e thjesht\u00eb dhe kompakte.<\/p>\n<p><\/p>\n<p>Nj\u00eb shprehje e rregullt q\u00eb do t\u00eb d\u00ebshironit t\u00eb shkruani, n\u00ebse do ta ruanit t\u00eb gjitha k\u00ebto n\u00eb nj\u00eb rresht, do t\u00eb ishte, para s\u00eb gjithash, e \u00e7uditshme. Dhe, p\u00ebr m\u00eb tep\u00ebr, do t\u00eb funksiononte shum\u00eb m\u00eb ngadal\u00eb se dy tabela. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/17664c3121b2b0d33188adbfaa653d4e.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Nj\u00eb shembull tjet\u00ebr. Keni nj\u00eb array ku ruani ID-t\u00eb. Dhe mund t'i konvertoni ato n\u00eb emra. Funksioni <code>arrayMap<\/code>. Ky \u00ebsht\u00eb nj\u00eb funksion tipik lambda. Ju d\u00ebrgoni atje shprehje lambda. Dhe ai nxjerr vler\u00ebn e emrit p\u00ebr secil\u00ebn ID nga fjalori.<\/p>\n<p><\/p>\n<p>N\u00eb m\u00ebnyr\u00eb t\u00eb ngjashme, mund t\u00eb b\u00ebni k\u00ebrkimin. Kalohet nj\u00eb funksion predikati, i cili kontrollon se \u00e7far\u00eb i p\u00ebrket elementeve. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/146917b71a35d38a550cd9accf6ab2f7.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>K\u00ebto gj\u00ebra ndihmojn\u00eb shum\u00eb n\u00eb thjeshtimin e skem\u00ebs dhe zgjidhin shum\u00eb probleme. <\/p>\n<p><\/p>\n<p>Por problemi tjet\u00ebr me t\u00eb cilin u p\u00ebrball\u00ebm, dhe q\u00eb do t\u00eb doja t\u00eb p\u00ebrmendja, jan\u00eb k\u00ebrkesat efektive. <\/p>\n<p><\/p>\n<ul>\n<li>N\u00eb ClickHouse nuk ka planifikues k\u00ebrkesash. N\u00eb t\u00ebr\u00ebsi, nuk ka. <\/li>\n<li>Megjithat\u00eb, k\u00ebrkesat komplekse ende duhet t\u00eb planifikohen. N\u00eb cilat raste? <\/li>\n<li>N\u00ebse n\u00eb k\u00ebrkes\u00eb ka disa bashkime (join), t\u00eb cilat i vendosni n\u00eb sub-selekta. Dhe rendi n\u00eb t\u00eb cilin ata ekzekutohen ka r\u00ebnd\u00ebsi. <\/li>\n<li>Dhe e dyta \u2013 n\u00ebse k\u00ebrkesa \u00ebsht\u00eb e shp\u00ebrndar\u00eb. Sepse n\u00eb nj\u00eb k\u00ebrkes\u00eb t\u00eb shp\u00ebrndar\u00eb, vet\u00ebm n\u00ebnklauzola m\u00eb e brendshme ekzekutohet n\u00eb m\u00ebnyr\u00eb t\u00eb shp\u00ebrndar\u00eb, nd\u00ebrsa e gjith\u00eb pjesa tjet\u00ebr d\u00ebrgohet n\u00eb nj\u00eb server, te i cili jeni lidhur dhe ekzekutohet atje. Prandaj, n\u00ebse keni k\u00ebrkesa t\u00eb shp\u00ebrndara me shum\u00eb bashk\u00ebngjitje (join), duhet t\u00eb zgjidhni rendin. <\/li>\n<\/ul>\n<p><\/p>\n<p>Edhe n\u00eb raste m\u00eb t\u00eb thjeshta, ndonj\u00ebher\u00eb duhet t\u00eb b\u00ebni pun\u00ebn e planifikuesit dhe t\u00eb riformuloni paksa k\u00ebrkesat. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/217d422b429e8b4bc13abbf0efdf6be1.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Ja nj\u00eb shembull. N\u00eb an\u00ebn e majt\u00eb \u00ebsht\u00eb k\u00ebrkesa q\u00eb tregon 5 vendet kryesore. Dhe ajo ekzekutohet p\u00ebr 2,5 sekonda, mendoj. Nd\u00ebrsa n\u00eb an\u00ebn e djatht\u00eb \u00ebsht\u00eb e nj\u00ebjta k\u00ebrkes\u00eb, por pak e riformuluar. N\u00eb vend q\u00eb t\u00eb grupojm\u00eb sipas vargut, ne filluam t\u00eb grupojm\u00eb sipas \u00e7el\u00ebsit (int). Dhe kjo \u00ebsht\u00eb m\u00eb e shpejt\u00eb. Pastaj ne lidhim nj\u00eb fjalor me rezultatin. N\u00eb vend t\u00eb 2,5 sekondave, k\u00ebrkesa ekzekutohet p\u00ebr 1,5 sekonda. Kjo \u00ebsht\u00eb mir\u00eb. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/f19fb7d7f0397a3b38b2a3bd179c156b.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Nj\u00eb shembull i ngjash\u00ebm me riformulimin e filtrave. K\u00ebtu \u00ebsht\u00eb nj\u00eb k\u00ebrkes\u00eb p\u00ebr Rusin\u00eb. Ajo ekzekutohet p\u00ebr 5 sekonda. N\u00ebse e riformulojm\u00eb n\u00eb k\u00ebt\u00eb m\u00ebnyr\u00eb, q\u00eb t\u00eb krahasojm\u00eb p\u00ebrs\u00ebri jo vargje, por numra me nj\u00eb grup t\u00eb atyre \u00e7el\u00ebsave q\u00eb lidhen me Rusin\u00eb, at\u00ebher\u00eb kjo do t\u00eb jet\u00eb shum\u00eb m\u00eb shpejt. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/0c1dd1f317133d1d106ffef85f337079.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Ka\u00e7 teknik\u00eb t\u00eb tilla ka. Ato lejojn\u00eb t\u00eb p\u00ebrshpejtohen ndjesh\u00ebm k\u00ebrkesat q\u00eb ndiheni se tashm\u00eb funksionojn\u00eb shpejt, ose, p\u00ebrkundrazi, q\u00eb funksionojn\u00eb ngadal\u00eb. Mund t\u00eb b\u00ebhen edhe m\u00eb shpejt.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/25b132790b03f239e38237af5e92ab63.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<ul>\n<li>Maksimumi i pun\u00ebs n\u00eb m\u00ebnyr\u00eb t\u00eb shp\u00ebrndar\u00eb. <\/li>\n<li>S\u00eb pari, renditja sipas llojeve minimale, ashtu si\u00e7 e kam b\u00ebr\u00eb me integer\u00ebt. <\/li>\n<li>N\u00ebse ka ndonj\u00eb bashkim (join), fjalor\u00eb, at\u00ebher\u00eb \u00ebsht\u00eb m\u00eb mir\u00eb t'i b\u00ebni ato n\u00eb fund t\u00eb fundit, kur t\u00eb keni t\u00eb dh\u00ebnat e grumbulluara t\u00eb pakt\u00ebn n\u00eb pjes\u00eb, at\u00ebher\u00eb operacioni i bashkimit (join) ose thirrja e fjalorit do t\u00eb ndodh\u00eb m\u00eb pak her\u00eb dhe do t\u00eb jet\u00eb m\u00eb shpejt. <\/li>\n<li>Z\u00ebvend\u00ebsimi i filtrave. <\/li>\n<\/ul>\n<p><\/p>\n<p>Ka edhe teknika t\u00eb tjera, jo vet\u00ebm ato q\u00eb un\u00eb demonstroj. T\u00eb gjitha k\u00ebto ndonj\u00ebher\u00eb lejojn\u00eb p\u00ebrshpejtimin ndjesh\u00ebm t\u00eb ekzekutimit t\u00eb k\u00ebrkesave.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/175832796bee525909d8875bd723a881.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>T\u00eb kalojm\u00eb n\u00eb shembullin tjet\u00ebr. Kompania X nga SHBA. \u00c7far\u00eb b\u00ebn ajo? <\/p>\n<p><\/p>\n<p>Ishte nj\u00eb detyr\u00eb: <\/p>\n<p><\/p>\n<ul>\n<li>Lidhja offline e transaksioneve t\u00eb reklamave. <\/li>\n<li>Modelimi i modeleve t\u00eb ndryshme t\u00eb lidhjes. <\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/0136b71a1205512d86f13e86761146f4.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>N\u00eb \u00e7far\u00eb p\u00ebrb\u00ebhet skenari?<\/p>\n<p><\/p>\n<p>Nj\u00eb vizitor i zakonsh\u00ebm hyn n\u00eb sit, p\u00ebr shembull, 20 her\u00eb n\u00eb muaj nga reklama t\u00eb ndryshme ose ndonj\u00ebher\u00eb vjen thjesht sepse e mban mend k\u00ebt\u00eb sit. Ai shikon disa produkte, i vendos n\u00eb shport\u00eb, i nxjerr nga shporta. Dhe, n\u00eb fund, di\u00e7ka blen. <\/p>\n<p><\/p>\n<p>Pyetje t\u00eb arsyeshme: \"K\u00eb duhet t\u00eb paguaj\u00eb p\u00ebr reklama n\u00ebse \u00ebsht\u00eb e nevojshme?\" dhe \"Cila reklam\u00eb e ndikoi, n\u00ebse ndikoi?\". Pra, p\u00ebrse bleu dhe si t\u00eb sigurohemi q\u00eb njer\u00ebz t\u00eb ngjash\u00ebm me k\u00ebt\u00eb person t\u00eb blejn\u00eb gjithashtu?<\/p>\n<p><\/p>\n<p>P\u00ebr t\u00eb zgjidhur k\u00ebt\u00eb detyr\u00eb, \u00ebsht\u00eb e nevojshme t\u00eb lidhen ngjarjet q\u00eb ndodhin n\u00eb uebsajt n\u00eb m\u00ebnyr\u00ebn e duhur, pra t\u00eb krijohet nj\u00eb lidhje nd\u00ebrmjet tyre. M\u00eb pas ato duhet t\u00eb d\u00ebrgohen p\u00ebr analiz\u00eb n\u00eb DWH. Dhe n\u00eb baz\u00eb t\u00eb k\u00ebsaj analize, t\u00eb nd\u00ebrtohen modele se kujt dhe cfar\u00eb reklame t'i tregojm\u00eb.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/e53ec81ce89150697d0eca4d72886638.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Transaksioni reklamues \u00ebsht\u00eb nj\u00eb grup ngjarjesh t\u00eb lidhura q\u00eb fillon nga shfaqja e nj\u00eb reklame, pastaj ndodhin di\u00e7ka, mund t\u00eb jet\u00eb nj\u00eb blerje, dhe m\u00eb pas mund t\u00eb ket\u00eb blerje t\u00eb tjera. P\u00ebr shembull, n\u00ebse \u00ebsht\u00eb nj\u00eb aplikacion mobil ose nj\u00eb loj\u00eb mobile, zakonisht instalimi i aplikacionit \u00ebsht\u00eb falas, dhe n\u00ebse ndodhin aktivitete t\u00eb tjera, mund t\u00eb k\u00ebrkohet pages\u00eb p\u00ebr to. Sa m\u00eb shum\u00eb t\u00eb shpenzoj\u00eb nj\u00eb person n\u00eb aplikacion, aq m\u00eb i vlefsh\u00ebm \u00ebsht\u00eb ai. Por p\u00ebr k\u00ebt\u00eb duhet t\u00eb lidh\u00ebsh gjith\u00e7ka. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/a2213613f1ef660409124ef8bcdd4fe3.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Ka shum\u00eb modele lidhjeje. <\/p>\n<p><\/p>\n<p>Modelet m\u00eb t\u00eb njohura jan\u00eb:<\/p>\n<p><\/p>\n<ul>\n<li>Interaksioni i fundit, ku interaksioni \u00ebsht\u00eb ose nj\u00eb klikim, ose nj\u00eb shfaqje.<\/li>\n<li>Interaksioni i par\u00eb, dometh\u00ebn\u00eb, ajo q\u00eb e solli fillimisht personin n\u00eb faqe.<\/li>\n<li>Kombinimi linear \u2013 t\u00eb gjith\u00eb n\u00eb m\u00ebnyr\u00eb t\u00eb barabart\u00eb. <\/li>\n<li>Zbehja.<\/li>\n<li>Dhe t\u00eb tjera. <\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/9523307d557e094d981da5b525ed0b59.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Si ka funksionuar gjith\u00e7ka fillimisht? Ka pasur Runtime dhe Cassandra. Cassandra u p\u00ebrdor si ruajtje transaksionesh, dmth. t\u00eb gjitha transaksionet e lidhura ruheshin atje. Kur ndodhte nj\u00eb ngjarje n\u00eb Runtime, p\u00ebr shembull, shfaqja e nj\u00eb faqeje ose di\u00e7ka tjet\u00ebr, dikush b\u00ebhej nj\u00eb k\u00ebrkes\u00eb n\u00eb Cassandra \u2013 n\u00ebse ekzistonte ky person apo jo. M\u00eb pas nxirrnin transaksionet q\u00eb i p\u00ebrkasin atij. Dhe b\u00ebhej lidhja.<\/p>\n<p><\/p>\n<p>N\u00ebse ka fat q\u00eb n\u00eb k\u00ebrkes\u00eb ka nj\u00eb ID transaksioni, \u00ebsht\u00eb e leht\u00eb. Por zakonisht nuk ka fat. Prandaj, duhej t\u00eb gjendej transaksioni i fundit ose transaksioni me klikimin e fundit, etj. <\/p>\n<p><\/p>\n<p>Dhe gjith\u00e7ka funksiononte shum\u00eb mir\u00eb derisa lidhja ishte me klikimin e fundit. Sepse, le t\u00eb themi, ka 10 milion klikime n\u00eb dit\u00eb, 300 milion n\u00eb muaj, n\u00ebse vendosni nj\u00eb dritare p\u00ebr muajin. Dhe pasi q\u00eb n\u00eb Cassandra gjith\u00e7ka duhet t\u00eb jet\u00eb n\u00eb memorie p\u00ebr t\u00eb funksionuar shpejt, sepse k\u00ebrkohet q\u00eb Runtime t\u00eb jap\u00eb p\u00ebrgjigje shpejt, nevojiteshin rreth 10-15 server\u00eb. <\/p>\n<p><\/p>\n<p>Dhe kur u d\u00ebshirua t\u00eb lidhej transaksioni me shfaqjen, menj\u00ebher\u00eb kjo nuk doli aq arg\u00ebtuese. Pse? Duke par\u00eb, \u00ebsht\u00eb e qart\u00eb se duhen ruajtur 30 her\u00eb m\u00eb shum\u00eb ngjarje. Dhe, p\u00ebr rrjedhoj\u00eb, nevojiten 30 her\u00eb m\u00eb shum\u00eb server\u00eb. Dhe rezulton se \u00ebsht\u00eb nj\u00eb num\u00ebr astronomik. T\u00eb mbash deri n\u00eb 500 server\u00eb p\u00ebr t\u00eb b\u00ebr\u00eb lidhje, nd\u00ebrsa n\u00eb Runtime ka shum\u00eb m\u00eb pak server\u00eb, kjo \u00ebsht\u00eb nj\u00eb shif\u00ebr e gabuar. Dhe filluam t\u00eb mendojm\u00eb se \u00e7far\u00eb t\u00eb b\u00ebjm\u00eb. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/7af35f1ade680232cc83a6887891bc5c.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe arrit\u00ebm te ClickHouse. Si mund ta b\u00ebjm\u00eb k\u00ebt\u00eb n\u00eb ClickHouse? N\u00eb shikim t\u00eb par\u00eb duket si nj\u00eb grup antipatish. <\/p>\n<p><\/p>\n<ul>\n<li>Transaksioni po rritet, ne po lidhim ngjarje t\u00eb reja me t\u00eb, pra \u00ebsht\u00eb mutable, dhe ClickHouse nuk punon shum\u00eb mir\u00eb me objekte mutable. <\/li>\n<li>Kur na vjen nj\u00eb vizitor, ne duhet t\u00eb nxjerrim transaksionet e tij sipas \u00e7el\u00ebsit, sipas visit ID-s\u00eb s\u00eb tij. Kjo gjithashtu \u00ebsht\u00eb nj\u00eb point query, dhe n\u00eb ClickHouse nuk b\u00ebhet k\u00ebshtu. Zakonisht n\u00eb ClickHouse ka skane t\u00eb m\u00ebdha... dhe k\u00ebtu na nevojitet t\u00eb nxjerrim disa regjistra. Po ashtu \u00ebsht\u00eb nj\u00eb antipattern. <\/li>\n<li>P\u00ebrve\u00e7 k\u00ebsaj, transaksioni ishte n\u00eb json, por nuk donin ta ri-shkruanin, k\u00ebshtu q\u00eb donin ta ruanin json-in n\u00eb nj\u00eb form\u00eb jo t\u00eb strukturuar, dhe n\u00ebse ishte e nevojshme, t\u00eb nxirrnin di\u00e7ka nga ai. Dhe kjo gjithashtu \u00ebsht\u00eb nj\u00eb antipattern. <\/li>\n<\/ul>\n<p><\/p>\n<p>Pra, nj\u00eb grup antipatternesh. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/2473129a5880ee3172b8ab7bf4afc376.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Por megjithat\u00eb, arrit\u00ebm t\u00eb krijojm\u00eb nj\u00eb sistem q\u00eb punonte shum\u00eb mir\u00eb. <\/p>\n<p><\/p>\n<p>\u00c7far\u00eb \u00ebsht\u00eb b\u00ebr\u00eb? U shfaq ClickHouse, n\u00eb t\u00eb cilin hidhen log\u00ebt, t\u00eb ndara n\u00eb regjistra. U shfaq nj\u00eb sh\u00ebrbim i atribuar, i cili merr log\u00ebt nga ClickHouse. Pas k\u00ebsaj, p\u00ebr secilin regjist\u00ebr sipas visit id, merrte transaksionet q\u00eb mund t\u00eb ishin akoma t\u00eb pa p\u00ebrpunuara dhe plus snapshots, dometh\u00ebn\u00eb transaksionet tashm\u00eb t\u00eb lidhura, ve\u00e7an\u00ebrisht rezultati i pun\u00ebs s\u00eb m\u00ebparshme. Nga ato, krijohej logjika, zgjidheshin transaksionet e duhura, bashkoheshin ngjarje t\u00eb reja. P\u00ebrs\u00ebri regjistrohej n\u00eb log. Logu shkonte p\u00ebrs\u00ebri n\u00eb ClickHouse, dometh\u00ebn\u00eb kjo \u00ebsht\u00eb nj\u00eb sistem i vazhduesh\u00ebm ciklik. Dhe p\u00ebrve\u00e7 k\u00ebsaj, shkonte n\u00eb DWH p\u00ebr ta analizuar atje. <\/p>\n<p><\/p>\n<p>N\u00eb k\u00ebt\u00eb form\u00eb, kjo nuk funksiononte shum\u00eb mir\u00eb. Dhe p\u00ebr ta b\u00ebr\u00eb m\u00eb t\u00eb leht\u00eb ClickHouse, kur kishte nj\u00eb k\u00ebrkes\u00eb sipas visit id, k\u00ebrkesat grupoheshin n\u00eb blloqe nga 1,000-2,000 visit id dhe merrnim p\u00ebr 1,000-2,000 njer\u00ebz t\u00eb gjitha transaksionet. Dhe at\u00ebher\u00eb gjith\u00e7ka filloi t\u00eb funksionoj\u00eb.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/d920ce231e6e4b5fd5b27ab28f848aed.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>N\u00ebse shikoni brenda ClickHouse, ka vet\u00ebm 3 tabela kryesore q\u00eb e sh\u00ebrbejn\u00eb k\u00ebt\u00eb. <\/p>\n<p><\/p>\n<p>Tabela e par\u00eb, ku hidhen log\u00ebt, dhe log\u00ebt hidhen praktikisht pa u p\u00ebrpunuar.<\/p>\n<p><\/p>\n<p>Tabela e dyt\u00eb. P\u00ebrmes materialized view nga k\u00ebto log-e, u nxorr\u00ebn evenete q\u00eb ende nuk ishin t\u00eb atribuuara, dmth t\u00eb pa lidhura. Dhe p\u00ebrmes materialized view nga k\u00ebto log-e u nxorr\u00ebn transaksionet p\u00ebr nd\u00ebrtimin e snapshot-it. Pra, nj\u00eb materialized view i ve\u00e7ant\u00eb nd\u00ebrtoi snapshot-in, konkretisht gjendjen e fundit t\u00eb akumuluar t\u00eb transaksionit. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/12b0adb691f483923a6c35bb6e4a5017.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>K\u00ebtu \u00ebsht\u00eb shkruar teksti n\u00eb SQL. Do t\u00eb doja t\u00eb komentoj disa gj\u00ebra t\u00eb r\u00ebnd\u00ebsishme n\u00eb t\u00eb. <\/p>\n<p><\/p>\n<p>Gj\u00ebja e par\u00eb e r\u00ebnd\u00ebsishme \u2013 \u00ebsht\u00eb mund\u00ebsia n\u00eb ClickHouse p\u00ebr t\u00eb nxjerr\u00eb kolona, fusha nga json. Pra, n\u00eb ClickHouse ka disa metoda p\u00ebr t\u00eb punuar me json. Ato jan\u00eb shum\u00eb, shum\u00eb primitive. <\/p>\n<p><\/p>\n<p>visitParamExtractInt lejon t\u00eb nxjerr\u00ebsh atribute nga json, dmth, e para q\u00eb kapet aktivizohet. Dhe n\u00eb k\u00ebt\u00eb m\u00ebnyr\u00eb mund t\u00eb nxjerr\u00ebsh transaction id ose visit id. Kjo \u00ebsht\u00eb nj\u00eb. <\/p>\n<p><\/p>\n<p>E dyta \u2013 k\u00ebtu \u00ebsht\u00eb p\u00ebrdorur nj\u00eb fush\u00eb materialized e zgjuar. \u00c7far\u00eb do t\u00eb thot\u00eb kjo? Do t\u00eb thot\u00eb se nuk mund ta p\u00ebrfshish n\u00eb tabel\u00eb, dmth nuk shtohet, por llogaritet dhe ruhet gjat\u00eb pro\u00e7esit t\u00eb shtimit. Gjat\u00eb shtimit, ClickHouse b\u00ebn pun\u00ebn p\u00ebr ju. Dhe pastaj nxirret nga json ajo q\u00eb do t'ju nevojitet m\u00eb pas.<\/p>\n<p><\/p>\n<p>N\u00eb k\u00ebt\u00eb rast, materialized view \u00ebsht\u00eb p\u00ebr rreshta t\u00eb pap\u00ebrpunuar. Dhe pik\u00ebrisht p\u00ebrdoret tabela e par\u00eb me log-e praktikisht t\u00eb pap\u00ebrpunuara. \u00c7far\u00eb b\u00ebn? S\u00eb pari, ndryshon renditjen, dmth. renditja tani b\u00ebhet sipas visit id, sepse na nevojitet t\u00eb nxjerrim shpejt transaksionin e nj\u00eb individi t\u00eb caktuar. <\/p>\n<p><\/p>\n<p>I dyti aspekt i r\u00ebnd\u00ebsish\u00ebm \u00ebsht\u00eb index_granularity. N\u00ebse keni par\u00eb MergeTree, zakonisht n\u00eb default \u00ebsht\u00eb 8,192 p\u00ebr index_granularity. \u00c7far\u00eb \u00ebsht\u00eb kjo? Ky \u00ebsht\u00eb nj\u00eb paramet\u00ebr i holl\u00ebsis\u00eb s\u00eb indeksit. N\u00eb ClickHouse indeksi \u00ebsht\u00eb i holl\u00eb, ai nuk indekson kurr\u00eb \u00e7do regjistrim. E b\u00ebn k\u00ebt\u00eb \u00e7do 8,192. Kjo \u00ebsht\u00eb mir\u00eb kur nevojiten shum\u00eb t\u00eb dh\u00ebna p\u00ebr t\u00eb llogaritur, por e keqe kur nevojiten pak, sepse ka nj\u00eb overhead t\u00eb madh. Dhe n\u00ebse e zvog\u00eblojm\u00eb index granularity, ne zvog\u00eblojm\u00eb overhead-in. Nuk mund ta zvog\u00eblojm\u00eb deri n\u00eb nj\u00eb, sepse mund t\u00eb mos ket\u00eb mjaftuesh\u00ebm memorie. Indeksi gjithmon\u00eb ruhet n\u00eb memorien. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/5fe303061977cad64e6c0333b125d595.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe snapshot-i p\u00ebrdor disa funksione interesante t\u00eb ClickHouse.<\/p>\n<p><\/p>\n<p>S\u00eb pari, ky \u00ebsht\u00eb AggregatingMergeTree. N\u00eb AggregatingMergeTree ruhet argMax, pra kjo \u00ebsht\u00eb gjendja e transaksionit q\u00eb korrespondon me timestamp-in e fundit. Transaksionet gjith\u00ebmon\u00eb krijohen t\u00eb reja p\u00ebr k\u00ebt\u00eb vizitor. Dhe n\u00eb gjendjen m\u00eb t\u00eb fundit t\u00eb k\u00ebtij transaksioni ne kemi shtuar nj\u00eb event dhe tani kemi nj\u00eb gjendje t\u00eb re. Ajo p\u00ebrs\u00ebri \u00ebsht\u00eb rifutur n\u00eb ClickHouse. Dhe p\u00ebrmes argMax n\u00eb k\u00ebt\u00eb paraqitje materializuar gjithmon\u00eb mund t\u00eb marrim gjendjen aktuale.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/d1b87301ab5206a36bf035679f79d35a.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<ul>\n<li>Lidhja \"e \u00e7montuar\" nga Runtime. <\/li>\n<li>Ruhet dhe p\u00ebrpunohen deri n\u00eb 3 miliard transaksionesh n\u00eb muaj. Kjo \u00ebsht\u00eb shum\u00eb m\u00eb shum\u00eb se n\u00eb Cassandra, pra n\u00eb nj\u00eb sistem tipik transaksionesh. <\/li>\n<li>Kluster i 2x5 serverave ClickHouse. 5 servera dhe \u00e7do server ka nj\u00eb replik\u00eb. Kjo \u00ebsht\u00eb edhe m\u00eb pak se sa ishte n\u00eb Cassandra p\u00ebr t\u00eb realizuar atributimin bazuar n\u00eb klik\u00eb, nd\u00ebrsa k\u00ebtu kemi atributimin bazuar n\u00eb impresion. Pra, n\u00eb vend q\u00eb t\u00eb rritej numri i serverave 30 her\u00eb, ata arrit\u00ebn ta ulen. <\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/583779fe9e133df519411e2256401eb3.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe shembulli i fundit \u2013 \u00ebsht\u00eb kompania financiare Y, e cila analizoi korrelacionet e ndryshimeve t\u00eb \u00e7mimeve t\u00eb aksioneve. <\/p>\n<p><\/p>\n<p>Dhe detyra ishte e till\u00eb:<\/p>\n<p><\/p>\n<ul>\n<li>Ka rreth 5,000 aksione. <\/li>\n<li>\u00c7mimet \u00e7do 100 milisekonda jan\u00eb t\u00eb njohura. <\/li>\n<li>T\u00eb dh\u00ebnat jan\u00eb grumbulluar p\u00ebr 10 vjet. Nj\u00ebherazi, p\u00ebr disa kompani m\u00eb shum\u00eb, p\u00ebr disa m\u00eb pak. <\/li>\n<li>N\u00eb total rreth 100 miliard\u00eb rreshta. <\/li>\n<\/ul>\n<p><\/p>\n<p>Dhe duhej t\u00eb llogaritej korelacioni i ndryshimeve. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/b02051e6cadec8c51e7f6e351ef903f9.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>K\u00ebtu jan\u00eb dy aksione dhe kotacionet e tyre. N\u00ebse nj\u00ebra ngjitet, dhe tjetra ngjitet, at\u00ebher\u00eb kjo \u00ebsht\u00eb korelacion pozitiv, dmth, nj\u00eb rritet dhe tjetra rritet. N\u00ebse nj\u00ebra ngjitet, si n\u00eb fund t\u00eb grafikut, nd\u00ebrsa tjetra bie, at\u00ebher\u00eb kjo \u00ebsht\u00eb korelacion negativ, dmth, kur nj\u00eb rritet, tjetra bie. <\/p>\n<p><\/p>\n<p>Duke analizuar k\u00ebto ndryshime t\u00eb nd\u00ebrsjellta mund t\u00eb b\u00ebhen parashikime n\u00eb tregun financiar. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/31329cf7523eef6a167632d281a634ce.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Por detyra \u00ebsht\u00eb e komplikuar. \u00c7far\u00eb b\u00ebhet p\u00ebr k\u00ebt\u00eb? Ne kemi 100 miliard\u00eb regjistrime, n\u00eb t\u00eb cilat ka: koha, aksioni dhe \u00e7mimi. Na nevojitet t\u00eb llogarisim fillimisht 100 miliard\u00eb her\u00eb vler\u00ebn e runningDifference nga algoritmi i \u00e7mimit. RunningDifference \u00ebsht\u00eb nj\u00eb funksion n\u00eb ClickHouse, i cili llogarit ndryshimin mes dy rreshtave nj\u00ebpasnj\u00ebsh\u00ebm. <\/p>\n<p><\/p>\n<p>Dhe pas k\u00ebsaj, duhet t\u00eb llogaritet korelacioni, p\u00ebr m\u00eb tep\u00ebr korelacioni duhet t\u00eb llogaritet p\u00ebr \u00e7do \u00e7ift. P\u00ebr 5,000 aksione, ka 12.5 milion\u00eb \u00e7ifte. Dhe kjo \u00ebsht\u00eb shum\u00eb, dmth, 12.5 her\u00eb duhen llogaritur nj\u00eb funksion t\u00eb till\u00eb korelacioni. <\/p>\n<p><\/p>\n<p>Dhe n\u00ebse dikush e harroi, at\u00ebher\u00eb \u035ex dhe \u035ey jan\u00eb pritet e matematik\u00ebs n\u00eb mostr\u00ebn. Pra, duhet jo vet\u00ebm t\u00eb llogarisni rr\u00ebnj\u00ebt dhe shumat, por gjithashtu brenda k\u00ebtyre shumave t\u00eb b\u00ebni edhe nj\u00eb her\u00eb shuma. Duhen kryer shum\u00eb llogaritje 12.5 milion her\u00eb, dhe gjithashtu duhet t\u00eb grupohen sipas or\u00ebve. Dhe or\u00ebt kemi t\u00eb shumta. Dhe duhet ta p\u00ebrfundojm\u00eb brenda 60 sekondave. Kjo \u00ebsht\u00eb nj\u00eb shaka. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/567a07f586a824c91671820fe21ec305.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Duhej t\u00eb arrinim t\u00eb b\u00ebnim di\u00e7ka, sepse gjith\u00eb kjo punonte shum\u00eb-ngadal\u00eb, para se t\u00eb vinte ClickHouse.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/3eca1af4c3f25ff4b23fce9b84e16659.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Ata u p\u00ebrpoq\u00ebn ta llogaritnin k\u00ebt\u00eb n\u00eb Hadoop, n\u00eb Spark, n\u00eb Greenplum. Dhe gjith\u00e7ka ishte shum\u00eb ngadal\u00eb ose e shtrenjt\u00eb. Pra, ishte e mundur ta llogarisnin n\u00eb nj\u00eb far\u00eb m\u00ebnyre, por m\u00eb pas ishte e shtrenjt\u00eb. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/37608c7e87ddb9a7d54854643044c369.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dhe pastaj erdhi ClickHouse dhe gjith\u00e7ka u p\u00ebrmir\u00ebsua shum\u00eb. <\/p>\n<p><\/p>\n<p>Ju kujtoj, problemi yn\u00eb \u00ebsht\u00eb me lokalitetin e t\u00eb dh\u00ebnave, prandaj korrelacionet nuk mund t\u00eb lokalizohen. Nuk mund t\u00eb grumbullojm\u00eb nj\u00eb pjes\u00eb t\u00eb t\u00eb dh\u00ebnave n\u00eb nj\u00eb server, nj\u00eb pjes\u00eb n\u00eb nj\u00eb tjet\u00ebr dhe ta llogarisim, sepse ne duhet t\u00eb kemi t\u00eb gjitha t\u00eb dh\u00ebnat kudo. <\/p>\n<p><\/p>\n<p>\u00c7far\u00eb b\u00ebn\u00eb ata? Fillimisht t\u00eb dh\u00ebnat ishin t\u00eb lokalizuara. N\u00eb secilin nga server\u00ebt ruheshin t\u00eb dh\u00ebnat p\u00ebr \u00e7mimet e nj\u00eb grupi t\u00eb caktuar t\u00eb aksioneve. Dhe ato nuk nd\u00ebrthuren. Prandaj \u00ebsht\u00eb e mundur t\u00eb llogaritet logReturn paralelisht dhe n\u00eb m\u00ebnyr\u00eb t\u00eb pavarur, gjith\u00e7ka ndodh nd\u00ebrkoh\u00eb n\u00eb m\u00ebnyr\u00eb paralele dhe t\u00eb shp\u00ebrndar\u00eb. <\/p>\n<p><\/p>\n<p>M\u00eb pas vendos\u00ebm t'i reduktonim k\u00ebto t\u00eb dh\u00ebna, pa humbur asnj\u00eb shprehshm\u00ebri. T'i reduktonim p\u00ebrmes masivave, dmth, p\u00ebr secil\u00ebn periudh\u00eb kohore t\u00eb krijonim nj\u00eb masiv t\u00eb aksioneve dhe nj\u00eb masiv t\u00eb \u00e7mimeve. K\u00ebshtu t\u00eb dh\u00ebnat z\u00ebn\u00eb shum\u00eb m\u00eb pak hap\u00ebsir\u00eb. Dhe \u00ebsht\u00eb m\u00eb e leht\u00eb t\u00eb punosh me to. K\u00ebto jan\u00eb praktika pothuajse paralele, dmth, ne llogarisim pjes\u00ebrisht n\u00eb m\u00ebnyr\u00eb paralele dhe m\u00eb pas i regjistrojm\u00eb n\u00eb server. <\/p>\n<p><\/p>\n<p>Pas k\u00ebsaj, k\u00ebto mund t\u00eb replikohen. Shkronja \"r\" tregon se k\u00ebto t\u00eb dh\u00ebna i kemi replikuar. Dometh\u00ebn\u00eb, ne kemi t\u00eb nj\u00ebjtat t\u00eb dh\u00ebna n\u00eb t\u00eb tri serverat - k\u00ebto masiva. <\/p>\n<p><\/p>\n<p>M\u00eb pas, me nj\u00eb skript t\u00eb ve\u00e7ant\u00eb nga ky grup prej 12.5 milion korrelacionesh q\u00eb duhet t\u00eb llogariten, mund t\u00eb krijohen pako. Dometh\u00ebn\u00eb, 2,500 detyra me 5,000 \u00e7ifte korrelacionesh. Dhe k\u00ebt\u00eb detyr\u00eb do ta llogarisim n\u00eb nj\u00eb server t\u00eb caktuar ClickHouse. T\u00eb gjith\u00eb t\u00eb dh\u00ebnat i ka, sepse t\u00eb dh\u00ebnat jan\u00eb t\u00eb nj\u00ebjta dhe ai mund t'i llogaris\u00eb ato n\u00eb m\u00ebnyr\u00eb t\u00eb radhitur. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/587c6d587a48f2407dd2687f5423d44a.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>P\u00ebr her\u00eb tjet\u00ebr, si duket kjo? S\u00eb pari, ne kemi t\u00eb gjitha t\u00eb dh\u00ebnat n\u00eb nj\u00eb struktur\u00eb t\u00eb till\u00eb: koha, aksionet, \u00e7mimi. Pastaj kemi llogaritur logReturn, pra t\u00eb dh\u00ebnat e s\u00eb nj\u00ebjt\u00ebs struktur\u00eb, vet\u00ebm se n\u00eb vend t\u00eb \u00e7mimit tani kemi logReturn. M\u00eb pas, i kemi transformuar, pra kemi marr\u00eb koh\u00ebn dhe groupArray p\u00ebr aksionet dhe \u00e7mimet. I kemi b\u00ebr\u00eb t\u00eb gjitha bashk\u00eb. Dhe, pas k\u00ebsaj, kemi gjeneruar nj\u00eb sasi t\u00eb madhe detyrash dhe i kemi ushqyer ClickHouse-i p\u00ebr ta llogaritur. Dhe kjo funksionon. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/434e7385aebcffcc03d1bd1a2020c8bf.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>N\u00eb prov\u00ebn e konceptit, detyra ishte nj\u00eb n\u00ebn-detyr\u00eb, pra kemi marr\u00eb m\u00eb pak t\u00eb dh\u00ebna. Dhe gjith\u00e7ka n\u00eb tre server\u00eb.<\/p>\n<p><\/p>\n<p>K\u00ebto dy etapa t\u00eb para: llogaritja e Log_return dhe rrethimi n\u00eb kodi, zuri rreth nj\u00eb or\u00eb secila. <\/p>\n<p><\/p>\n<p>Nd\u00ebrsa llogaritja e korrelacionit zuri rreth 50 or\u00eb. Por 50 or\u00eb jan\u00eb pak, sepse m\u00eb par\u00eb ky proces kishte zgjatur jav\u00eb t\u00eb t\u00ebra. Kjo ishte nj\u00eb sukses i madh. Dhe n\u00ebse e llogaritim, n\u00eb k\u00ebt\u00eb klaster gjith\u00e7ka llogaritej 70 her\u00eb n\u00eb sekond\u00eb.<\/p>\n<p><\/p>\n<p>Por m\u00eb e r\u00ebnd\u00ebsishmja, kjo sistem \u00ebsht\u00eb praktikisht pa ngushtica, pra ajo shkall\u00ebzohet praktikisht linear. Dhe ata e kan\u00eb verifikuar k\u00ebt\u00eb. E kan\u00eb shkall\u00ebzuar me sukses. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/5a595a84897e492b49dccfb0f058b1b8.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<ul>\n<li>Skema e duhur \u00ebsht\u00eb gjysma e suksesit. Dhe skema e duhur \u00ebsht\u00eb p\u00ebrdorimi i t\u00eb gjitha teknologjive t\u00eb nevojshme t\u00eb ClickHouse. <\/li>\n<li>Summing\/AggregatingMergeTrees \u2013 jan\u00eb teknologji q\u00eb lejojn\u00eb agregimin ose llogaritjen e snapshots state si nj\u00eb rast t\u00eb ve\u00e7ant\u00eb. Kjo e thjeshton ndjesh\u00ebm shum\u00eb gj\u00ebra. <\/li>\n<li>Materialized Views lejojn\u00eb t\u00eb anashkaloni kufizimin e nj\u00eb indeksi. Ndoshta nuk e kam shprehur shum\u00eb qart\u00eb, por kur ngarkonim log-et, log-et e pap\u00ebrpunuara ishin n\u00eb nj\u00eb tabel\u00eb me nj\u00eb indeks, nd\u00ebrsa log-et mbi atributet ishin n\u00eb nj\u00eb tabel\u00eb, pra t\u00eb nj\u00ebjtat t\u00eb dh\u00ebna, vet\u00ebm t\u00eb filtruar, por indeksi ishte krejt\u00ebsisht ndryshe. Duket si t\u00eb nj\u00ebjtat t\u00eb dh\u00ebna, por renditja \u00ebsht\u00eb e ndryshme. Dhe Materialized Views lejon, n\u00ebse e keni nevoj\u00eb, t\u00eb anashkaloni nj\u00eb kufizim t\u00eb till\u00eb n\u00eb ClickHouse. <\/li>\n<li>Reduktoni granularitetin e indeksit p\u00ebr k\u00ebrkesa specifike. <\/li>\n<li>Dhe shp\u00ebrndani t\u00eb dh\u00ebnat n\u00eb m\u00ebnyr\u00eb t\u00eb men\u00e7ur, k\u00ebrkoni t\u00eb lokalizoni sa m\u00eb shum\u00eb q\u00eb t\u00eb jet\u00eb e mundur t\u00eb dh\u00ebnat brenda serverit. Gjithashtu, p\u00ebrpiquni q\u00eb k\u00ebrkesat t\u00eb p\u00ebrdorin gjithashtu lokalizimin aty ku \u00ebsht\u00eb e mundur maksimalisht. <\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/c0fc1737986e3f18ed1d3c53df9f0252.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Duke p\u00ebrmbledhur k\u00ebt\u00eb fjalim t\u00eb vog\u00ebl, mund t\u00eb themi se ClickHouse tani ka z\u00ebn\u00eb qart\u00eb pozita n\u00eb territoret e bazave t\u00eb dh\u00ebnave komerciale dhe ato open source, dmth. pik\u00ebrisht p\u00ebr analiz\u00ebn. Ai \u00ebsht\u00eb integruar mrekullisht n\u00eb k\u00ebt\u00eb peizazh. E m\u00eb shum\u00eb se kaq, ai gradualisht po fillon t\u00eb z\u00ebvend\u00ebsoj\u00eb t\u00eb tjer\u00ebt, sepse kur keni ClickHouse, nuk ju nevojitet InfiniDB. Vertica, ndoshta p\u00ebr shpejt do t\u00eb b\u00ebhet e panevojshme, n\u00ebse ata realizojn\u00eb nj\u00eb mb\u00ebshtetje t\u00eb duhur p\u00ebr SQL. P\u00ebrdorni!<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Aleksand\u00ebr Zaitsev (2018)\" src=\"\/wp-content\/uploads\/2020\/07\/6e9a65d32eb6de7f9f139a30e7ab0d25.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>&#8212;<em>Faleminderit p\u00ebr referatin! Shum\u00eb interesante! A kishte ndonj\u00eb krahasim me Apache Phoenix?<\/em><\/p>\n<p><\/p>\n<p>-Jo, nuk kam d\u00ebgjuar q\u00eb dikush t\u00eb ket\u00eb b\u00ebr\u00eb krahasime. Ne dhe Yandex p\u00ebrpiqemi t\u00eb ndjekim t\u00eb gjitha krahasimet e ClickHouse me bazat e ndryshme t\u00eb dh\u00ebnash. Sepse n\u00ebse ndonj\u00ebher\u00eb di\u00e7ka del m\u00eb e shpejt\u00eb se ClickHouse, at\u00ebher\u00eb Alexey Milovidov nuk mund t\u00eb flej\u00eb n\u00eb net\u00eb dhe fillon ta p\u00ebrshpejtoj\u00eb at\u00eb menj\u00ebher\u00eb. Nuk kam d\u00ebgjuar p\u00ebr nj\u00eb krahasim t\u00eb till\u00eb. <\/p>\n<p><\/p>\n<ul>\n<li>\n<p><em>(\u0410\u043b\u0435\u043a\u0441\u0435\u0439 \u041c\u0438\u043b\u043e\u0432\u0438\u0434\u043e\u0432) Apache Phoenix \u2013 \u00ebsht\u00eb nj\u00eb motor SQL mbi Hbase. Hbase \u00ebsht\u00eb kryesisht e dizajnuar p\u00ebr skenar\u00eb pune t\u00eb llojit key-value. Atje n\u00eb \u00e7do rresht mund t\u00eb ket\u00eb nj\u00eb num\u00ebr t\u00eb rast\u00ebsish\u00ebm kolonash me emra t\u00eb rast\u00ebsish\u00ebm. Kjo mund t\u00eb thuhet p\u00ebr sisteme si Hbase, Cassandra. Dhe p\u00ebr ta, k\u00ebrkesat analitike t\u00eb r\u00ebnda nuk do t\u00eb funksionojn\u00eb si\u00e7 duhet. Ose mund t\u00eb mendoni se ato funksionojn\u00eb mir\u00eb, n\u00ebse nuk keni pasur asnj\u00eb p\u00ebrvoj\u00eb me ClickHouse.<\/em><\/p>\n<p>\n<\/li>\n<li>\n<p>Faleminderit<\/p>\n<p><\/p>\n<ul>\n<li>\n<p><em>P\u00ebrsh\u00ebndetje! Un\u00eb kam qen\u00eb duke u interesuar p\u00ebr k\u00ebt\u00eb tem\u00eb p\u00ebr nj\u00eb koh\u00eb t\u00eb gjat\u00eb, sepse un\u00eb kam nj\u00eb n\u00ebn-sistem analitik. Por kur shoh ClickHouse, ndiej se ClickHouse \u00ebsht\u00eb shum\u00eb i p\u00ebrshtatsh\u00ebm p\u00ebr analiz\u00ebn e eventeve, mutable. Dhe n\u00ebse kam nevoj\u00eb t\u00eb analizoj shum\u00eb t\u00eb dh\u00ebna biznesi me nj\u00eb mori tabelash t\u00eb m\u00ebdha, at\u00ebher\u00eb ClickHouse, aq sa kuptoj, nuk \u00ebsht\u00eb shum\u00eb i p\u00ebrshtatsh\u00ebm p\u00ebr mua? Sidomos n\u00ebse ato ndryshojn\u00eb. A \u00ebsht\u00eb kjo e sakt\u00eb apo ka shembuj q\u00eb mund ta falsifikojn\u00eb k\u00ebt\u00eb?<\/em><\/p>\n<p>\n<\/li>\n<li>\n<p>Kjo \u00ebsht\u00eb e sakt\u00eb. Dhe kjo vlen p\u00ebr shumic\u00ebn e bazave t\u00eb t\u00eb dh\u00ebnave analitike t\u00eb specializuara. Ato jan\u00eb t\u00eb dizajnuara p\u00ebr t\u00eb punuar me nj\u00ebr\u00ebn ose m\u00eb shum\u00eb tabela t\u00eb m\u00ebdha, q\u00eb jan\u00eb t\u00eb ndryshueshme, dhe shum\u00eb t\u00eb vogla q\u00eb ndryshojn\u00eb ngadal\u00eb. Dometh\u00ebn\u00eb, ClickHouse nuk \u00ebsht\u00eb si Oracle, ku mund t\u00eb vendosni gjith\u00e7ka dhe t\u00eb nd\u00ebrtoni k\u00ebrkesa shum\u00eb komplekse. P\u00ebr ta p\u00ebrdorur ClickHouse n\u00eb m\u00ebnyr\u00eb efektive, duhet t\u00eb zhvilloni skem\u00ebn n\u00eb nj\u00eb m\u00ebnyr\u00eb q\u00eb funksionon mir\u00eb n\u00eb ClickHouse. Dometh\u00ebn\u00eb, t\u00eb shmangni normalizimin e tepruar, t\u00eb p\u00ebrdorni fjalor\u00eb, dhe t\u00eb p\u00ebrpiqeni t\u00eb keni m\u00eb pak lidhje t\u00eb gjata. N\u00ebse skema \u00ebsht\u00eb zhvilluar n\u00eb k\u00ebt\u00eb m\u00ebnyr\u00eb, ato detyra t\u00eb ngjashme biznesi n\u00eb ClickHouse mund t\u00eb zgjidhen shum\u00eb m\u00eb efikashem se sa n\u00eb nj\u00eb baz\u00eb t\u00eb dh\u00ebnash relacional tradicionale. <\/p>\n<p>\n<\/li>\n<\/ul>\n<p>\n<\/li>\n<\/ul>\n<p><\/p>\n<p><em>Faleminderit p\u00ebr raportin! Kam nj\u00eb pyetje p\u00ebr rastin financiar t\u00eb fundit. Ata kishin analiz\u00ebn. Duhej t\u00eb krahasohej si shkonin lart-posht\u00eb. Dhe e kuptoj se ju nd\u00ebrtuat sistemin pik\u00ebrisht p\u00ebr k\u00ebt\u00eb analiz\u00eb? N\u00ebse nes\u00ebr, supozoni, atyre u nevojitet nj\u00eb raport tjet\u00ebr mbi k\u00ebto t\u00eb dh\u00ebna, a duhet ri-nd\u00ebrtuar skema dhe t\u00eb ngarkohen t\u00eb dh\u00ebnat p\u00ebrs\u00ebri? Dometh\u00ebn\u00eb, t\u00eb b\u00ebni ndonj\u00eb p\u00ebrpunim paraprak p\u00ebr t\u00eb marr\u00eb k\u00ebrkes\u00ebn?<\/em><\/p>\n<p><\/p>\n<p>Sigurisht, ky \u00ebsht\u00eb p\u00ebrdorimi i ClickHouse p\u00ebr nj\u00eb detyr\u00eb mjaft specifike. Ajo tradicionalisht mund t\u00eb zgjidhej brenda Hadoop. P\u00ebr Hadoop \u2013 kjo \u00ebsht\u00eb nj\u00eb detyr\u00eb ideale. Por n\u00eb Hadoop \u00ebsht\u00eb shum\u00eb e ngadalt\u00eb. Q\u00ebllimi im \u00ebsht\u00eb t\u00eb demonstroj se me ClickHouse mund t\u00eb zgjidhni detyra q\u00eb zakonisht zgjidhen me mjete krejt\u00ebsisht t\u00eb tjera, por duke e b\u00ebr\u00eb k\u00ebt\u00eb shum\u00eb m\u00eb efikase. Kjo \u00ebsht\u00eb e destinuar p\u00ebr nj\u00eb detyr\u00eb specifike. \u00cbsht\u00eb e qart\u00eb se n\u00ebse ka nj\u00eb detyr\u00eb t\u00eb ngjashme, mund ta zgjidhni n\u00eb nj\u00eb m\u00ebnyr\u00eb t\u00eb ngjashme. <\/p>\n<p><\/p>\n<p><em>E kuptoj. Ju that\u00eb se u p\u00ebrpunuan 50 or\u00eb. A \u00ebsht\u00eb kjo q\u00eb nga fillimi, kur u ngarkuan t\u00eb dh\u00ebnat apo kur mor\u00ebt rezultatet?<\/em><\/p>\n<p><\/p>\n<p>Po-po.<\/p>\n<p><\/p>\n<p><em>Mir\u00eb, faleminderit shum\u00eb.<\/em><\/p>\n<p><\/p>\n<p>Kjo \u00ebsht\u00eb n\u00eb nj\u00eb klaster me 3 servera. <\/p>\n<p><\/p>\n<p><em>P\u00ebrsh\u00ebndetje! Faleminderit p\u00ebr prezantimin! Gjith\u00e7ka \u00ebsht\u00eb shum\u00eb interesante. Un\u00eb do t\u00eb pyes pak jo p\u00ebr funksionalitetin, por p\u00ebr p\u00ebrdorimin e ClickHouse nga pik\u00ebpamja e stabilitetit. Dometh\u00ebn\u00eb, a keni hasur ndonj\u00ebher\u00eb n\u00eb ndonj\u00eb k\u00ebrkes\u00eb, keni pasur nevoj\u00eb t\u00eb riktheni? Si sillet ClickHouse n\u00eb k\u00ebto raste? A ka ndodhur q\u00eb edhe replicimi t\u00eb ket\u00eb d\u00ebshtuar? Na, p\u00ebr shembull, kemi hasur n\u00eb problemin me ClickHouse kur ai kalon kufirin e tij dhe d\u00ebshton.<\/em><\/p>\n<p><\/p>\n<p>Sigurisht, sistemet ideale nuk ekzistojn\u00eb. Edhe ClickHouse ka problemet e veta. Por a keni d\u00ebgjuar q\u00eb Yandex.Metrika ka funksionuar gjat\u00eb dhe nuk ka punuar? Ndoshta jo. Ajo ka funksionuar me besueshm\u00ebri q\u00eb nga viti 2012-2013 mbi ClickHouse. Edhe p\u00ebr p\u00ebrvoj\u00ebn time mund t\u00eb them. Kurr\u00eb nuk kemi pasur ndalime totale. Disa gj\u00ebra t\u00eb pjesshme mund t\u00eb kan\u00eb ndodhur, por ato kurr\u00eb nuk kan\u00eb qen\u00eb kaq kritike sa t\u00eb ndikojn\u00eb seriozisht n\u00eb biznes. Kurr\u00eb nuk ka ndodhur. ClickHouse \u00ebsht\u00eb mjaft i besuesh\u00ebm dhe nuk bien rast\u00ebsisht. Mund t\u00eb mos shqet\u00ebsoheni p\u00ebr k\u00ebt\u00eb. Nuk \u00ebsht\u00eb nj\u00eb gj\u00eb e pap\u00ebrfunduar. Kjo \u00ebsht\u00eb provuar nga shum\u00eb kompani. <\/p>\n<p><\/p>\n<p><em>P\u00ebrsh\u00ebndetje! That\u00eb se duhet t\u00eb mendohet mir\u00eb p\u00ebr skem\u00ebn e t\u00eb dh\u00ebnave nga fillimi. Po n\u00ebse kjo ndodhi? T\u00eb dh\u00ebnat e mia po derdhen. Kalon gjysm\u00eb viti, dhe kuptoj q\u00eb nuk mund t\u00eb jetoj k\u00ebshtu, duhet t\u00eb rishkarkoj t\u00eb dh\u00ebnat dhe t\u00eb b\u00ebj di\u00e7ka me to.<\/em> <\/p>\n<p><\/p>\n<p>Sigurisht, kjo varet nga sistemi juaj. Ka disa m\u00ebnyra p\u00ebr ta b\u00ebr\u00eb k\u00ebt\u00eb praktikisht pa ndalesa. P\u00ebr shembull, mund t\u00eb krijoni nj\u00eb Materialized View, n\u00eb t\u00eb cilin t\u00eb krijoni nj\u00eb struktur\u00eb t\u00eb ndryshme t\u00eb dh\u00ebnash, n\u00ebse ajo mund t\u00eb mapohet qart\u00eb. Pra, n\u00ebse ajo lejon mapimin p\u00ebrmes ClickHouse, dmth. nxjerrjen e disa gj\u00ebrave, ndryshimin e \u00e7el\u00ebsit primar, ndryshimin e ndarjes n\u00eb particione, at\u00ebher\u00eb mund t\u00eb krijoni nj\u00eb Materialized View. Atje do t\u00eb shkruani t\u00eb dh\u00ebnat tuaja t\u00eb vjetra, dhe t\u00eb rejat do t\u00eb shkruhen automatikisht. Pastaj, thjesht nd\u00ebrroni n\u00eb p\u00ebrdorimin e Materialized View, pas k\u00ebsaj ndryshoni shkrimin dhe fshini tabel\u00ebn e vjet\u00ebr. Kjo \u00ebsht\u00eb nj\u00eb m\u00ebnyr\u00eb p\u00ebr ta b\u00ebr\u00eb pra pa ndalesa. <\/p>\n<p><\/p>\n<p><em>Faleminderit.<\/em><\/p>\n<p>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/512304\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041d\u0435\u0441\u043c\u043e\u0442\u0440\u044f \u043d\u0430 \u0442\u043e, \u0447\u0442\u043e \u0434\u0430\u043d\u043d\u044b\u0445 \u0441\u0435\u0439\u0447\u0430\u0441 \u043c\u043d\u043e\u0433\u043e \u043f\u043e\u0447\u0442\u0438 \u0432\u0435\u0437\u0434\u0435, \u0430\u043d\u0430\u043b\u0438\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0435 \u0411\u0414 \u0432\u0441\u0435 \u0435\u0449\u0435 \u0434\u043e\u0432\u043e\u043b\u044c\u043d\u043e \u044d\u043a\u0437\u043e\u0442\u0438\u0447\u043d\u044b. \u0418\u0445 \u043f\u043b\u043e\u0445\u043e \u0437\u043d\u0430\u044e\u0442 \u0438 \u0435\u0449\u0435 \u0445\u0443\u0436\u0435 \u0443\u043c\u0435\u044e\u0442 \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c. \u041c\u043d\u043e\u0433\u0438\u0435 \u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0430\u044e\u0442 &quot;\u0435\u0441\u0442\u044c \u043a\u0430\u043a\u0442\u0443\u0441&quot; \u0441 MySQL \u0438\u043b\u0438 PostgreSQL, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0441\u043f\u0440\u043e\u0435\u043a\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u044b \u043f\u043e\u0434 \u0434\u0440\u0443\u0433\u0438\u0435 \u0441\u0446\u0435\u043d\u0430\u0440\u0438\u0438, \u043c\u0443\u0447\u0438\u0442\u044c\u0441\u044f \u0441 NoSQL \u0438\u043b\u0438 \u043f\u0435\u0440\u0435\u043f\u043b\u0430\u0447\u0438\u0432\u0430\u0442\u044c \u0437\u0430 \u043a\u043e\u043c\u043c\u0435\u0440\u0447\u0435\u0441\u043a\u0438\u0435 \u0440\u0435\u0448\u0435\u043d\u0438\u044f. ClickHouse \u043c\u0435\u043d\u044f\u0435\u0442 \u043f\u0440\u0430\u0432\u0438\u043b\u0430 \u0438\u0433\u0440\u044b \u0438 \u0437\u043d\u0430\u0447\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0441\u043d\u0438\u0436\u0430\u0435\u0442 \u043f\u043e\u0440\u043e\u0433 \u0432\u0445\u043e\u0436\u0434\u0435\u043d\u0438\u044f \u0432 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":89879,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-89878","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 4.9.10 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041d\u0435\u0441\u043c\u043e\u0442\u0440\u044f \u043d\u0430 \u0442\u043e, \u0447\u0442\u043e \u0434\u0430\u043d\u043d\u044b\u0445 \u0441\u0435\u0439\u0447\u0430\u0441 \u043c\u043d\u043e\u0433\u043e \u043f\u043e\u0447\u0442\u0438 \u0432\u0435\u0437\u0434\u0435, \u0430\u043d\u0430\u043b\u0438\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0435 \u0411\u0414 \u0432\u0441\u0435 \u0435\u0449\u0435 \u0434\u043e\u0432\u043e\u043b\u044c\u043d\u043e \u044d\u043a\u0437\u043e\u0442\u0438\u0447\u043d\u044b. \u0418\u0445 \u043f\u043b\u043e\u0445\u043e \u0437\u043d\u0430\u044e\u0442 \u0438 \u0435\u0449\u0435 \u0445\u0443\u0436\u0435 \u0443\u043c\u0435\u044e\u0442 \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c. \u041c\u043d\u043e\u0433\u0438\u0435 \u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0430\u044e\u0442 &quot;\u0435\u0441\u0442\u044c \u043a\u0430\u043a\u0442\u0443\u0441&quot; \u0441 MySQL \u0438\u043b\u0438 PostgreSQL, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0441\u043f\u0440\u043e\u0435\u043a\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u044b \u043f\u043e\u0434 \u0434\u0440\u0443\u0433\u0438\u0435 \u0441\u0446\u0435\u043d\u0430\u0440\u0438\u0438, \u043c\u0443\u0447\u0438\u0442\u044c\u0441\u044f \u0441 NoSQL \u0438\u043b\u0438 \u043f\u0435\u0440\u0435\u043f\u043b\u0430\u0447\u0438\u0432\u0430\u0442\u044c \u0437\u0430 \u043a\u043e\u043c\u043c\u0435\u0440\u0447\u0435\u0441\u043a\u0438\u0435 \u0440\u0435\u0448\u0435\u043d\u0438\u044f. ClickHouse \u043c\u0435\u043d\u044f\u0435\u0442 \u043f\u0440\u0430\u0432\u0438\u043b\u0430 \u0438\u0433\u0440\u044b \u0438 \u0437\u043d\u0430\u0447\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0441\u043d\u0438\u0436\u0430\u0435\u0442 \u043f\u043e\u0440\u043e\u0433 \u0432\u0445\u043e\u0436\u0434\u0435\u043d\u0438\u044f \u0432\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/teoriya-i-praktika-ispolzovaniya-clickhouse-v-realnyh-prilozheniyah-aleksandr-zajczev-2018g\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 4.9.10\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0422\u0435\u043e\u0440\u0438\u044f \u0438 \u043f\u0440\u0430\u043a\u0442\u0438\u043a\u0430 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f ClickHouse \u0432 \u0440\u0435\u0430\u043b\u044c\u043d\u044b\u0445 \u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u044f\u0445. \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440 \u0417\u0430\u0439\u0446\u0435\u0432 (2018\u0433) | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041d\u0435\u0441\u043c\u043e\u0442\u0440\u044f \u043d\u0430 \u0442\u043e, \u0447\u0442\u043e \u0434\u0430\u043d\u043d\u044b\u0445 \u0441\u0435\u0439\u0447\u0430\u0441 \u043c\u043d\u043e\u0433\u043e \u043f\u043e\u0447\u0442\u0438 \u0432\u0435\u0437\u0434\u0435, \u0430\u043d\u0430\u043b\u0438\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0435 \u0411\u0414 \u0432\u0441\u0435 \u0435\u0449\u0435 \u0434\u043e\u0432\u043e\u043b\u044c\u043d\u043e \u044d\u043a\u0437\u043e\u0442\u0438\u0447\u043d\u044b. \u0418\u0445 \u043f\u043b\u043e\u0445\u043e \u0437\u043d\u0430\u044e\u0442 \u0438 \u0435\u0449\u0435 \u0445\u0443\u0436\u0435 \u0443\u043c\u0435\u044e\u0442 \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c. \u041c\u043d\u043e\u0433\u0438\u0435 \u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0430\u044e\u0442 &quot;\u0435\u0441\u0442\u044c \u043a\u0430\u043a\u0442\u0443\u0441&quot; \u0441 MySQL \u0438\u043b\u0438 PostgreSQL, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0441\u043f\u0440\u043e\u0435\u043a\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u044b \u043f\u043e\u0434 \u0434\u0440\u0443\u0433\u0438\u0435 \u0441\u0446\u0435\u043d\u0430\u0440\u0438\u0438, \u043c\u0443\u0447\u0438\u0442\u044c\u0441\u044f \u0441 NoSQL \u0438\u043b\u0438 \u043f\u0435\u0440\u0435\u043f\u043b\u0430\u0447\u0438\u0432\u0430\u0442\u044c \u0437\u0430 \u043a\u043e\u043c\u043c\u0435\u0440\u0447\u0435\u0441\u043a\u0438\u0435 \u0440\u0435\u0448\u0435\u043d\u0438\u044f. ClickHouse \u043c\u0435\u043d\u044f\u0435\u0442 \u043f\u0440\u0430\u0432\u0438\u043b\u0430 \u0438\u0433\u0440\u044b \u0438 \u0437\u043d\u0430\u0447\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0441\u043d\u0438\u0436\u0430\u0435\u0442 \u043f\u043e\u0440\u043e\u0433 \u0432\u0445\u043e\u0436\u0434\u0435\u043d\u0438\u044f \u0432\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/teoriya-i-praktika-ispolzovaniya-clickhouse-v-realnyh-prilozheniyah-aleksandr-zajczev-2018g\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-07-26T23:42:42+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-07-26T23:42:42+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Teoria dhe praktika e p\u00ebrdorimit t\u00eb ClickHouse n\u00eb aplikacione reale. Alexander Zaytsev (2018) | ProHoster","description":"Megjith\u00ebse tani ka shum\u00eb t\u00eb dh\u00ebna pothuajse kudo, bazat e t\u00eb dh\u00ebnave analitike ende mbeten mjaft ekzotike. Ato njihen keq dhe akoma m\u00eb keq p\u00ebrdoren n\u00eb m\u00ebnyr\u00eb efektive. Shum\u00eb vazhdojn\u00eb t\u00eb \"han\u00eb kaktus\" me MySQL ose PostgreSQL, t\u00eb cilat jan\u00eb projektuar p\u00ebr skenar\u00eb t\u00eb ndrysh\u00ebm, t\u00eb vuajn\u00eb me NoSQL ose t\u00eb paguajn\u00eb shum\u00eb p\u00ebr zgjidhje komerciale. ClickHouse ndryshon rregullat e loj\u00ebs dhe ul ndjesh\u00ebm pragun e hyrjes n\u00eb","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/teoriya-i-praktika-ispolzovaniya-clickhouse-v-realnyh-prilozheniyah-aleksandr-zajczev-2018g","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0422\u0435\u043e\u0440\u0438\u044f \u0438 \u043f\u0440\u0430\u043a\u0442\u0438\u043a\u0430 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f ClickHouse \u0432 \u0440\u0435\u0430\u043b\u044c\u043d\u044b\u0445 \u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u044f\u0445. \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440 \u0417\u0430\u0439\u0446\u0435\u0432 (2018\u0433) | ProHoster","og:description":"\u041d\u0435\u0441\u043c\u043e\u0442\u0440\u044f \u043d\u0430 \u0442\u043e, \u0447\u0442\u043e \u0434\u0430\u043d\u043d\u044b\u0445 \u0441\u0435\u0439\u0447\u0430\u0441 \u043c\u043d\u043e\u0433\u043e \u043f\u043e\u0447\u0442\u0438 \u0432\u0435\u0437\u0434\u0435, \u0430\u043d\u0430\u043b\u0438\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0435 \u0411\u0414 \u0432\u0441\u0435 \u0435\u0449\u0435 \u0434\u043e\u0432\u043e\u043b\u044c\u043d\u043e \u044d\u043a\u0437\u043e\u0442\u0438\u0447\u043d\u044b. \u0418\u0445 \u043f\u043b\u043e\u0445\u043e \u0437\u043d\u0430\u044e\u0442 \u0438 \u0435\u0449\u0435 \u0445\u0443\u0436\u0435 \u0443\u043c\u0435\u044e\u0442 \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c. \u041c\u043d\u043e\u0433\u0438\u0435 \u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0430\u044e\u0442 &quot;\u0435\u0441\u0442\u044c \u043a\u0430\u043a\u0442\u0443\u0441&quot; \u0441 MySQL \u0438\u043b\u0438 PostgreSQL, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0441\u043f\u0440\u043e\u0435\u043a\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u044b \u043f\u043e\u0434 \u0434\u0440\u0443\u0433\u0438\u0435 \u0441\u0446\u0435\u043d\u0430\u0440\u0438\u0438, \u043c\u0443\u0447\u0438\u0442\u044c\u0441\u044f \u0441 NoSQL \u0438\u043b\u0438 \u043f\u0435\u0440\u0435\u043f\u043b\u0430\u0447\u0438\u0432\u0430\u0442\u044c \u0437\u0430 \u043a\u043e\u043c\u043c\u0435\u0440\u0447\u0435\u0441\u043a\u0438\u0435 \u0440\u0435\u0448\u0435\u043d\u0438\u044f. ClickHouse \u043c\u0435\u043d\u044f\u0435\u0442 \u043f\u0440\u0430\u0432\u0438\u043b\u0430 \u0438\u0433\u0440\u044b \u0438 \u0437\u043d\u0430\u0447\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0441\u043d\u0438\u0436\u0430\u0435\u0442 \u043f\u043e\u0440\u043e\u0433 \u0432\u0445\u043e\u0436\u0434\u0435\u043d\u0438\u044f \u0432","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/teoriya-i-praktika-ispolzovaniya-clickhouse-v-realnyh-prilozheniyah-aleksandr-zajczev-2018g","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-07-26T23:42:42+00:00","article:modified_time":"2020-07-26T23:42:42+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"89878","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 13:02:37","updated":"2026-02-22 15:29:33"},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/89878","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=89878"}],"version-history":[{"count":2,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/89878\/revisions"}],"predecessor-version":[{"id":162171,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/89878\/revisions\/162171"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/89879"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=89878"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=89878"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=89878"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}