Ithiyori kanye nomkhuba wokusebenzisa i-ClickHouse ezinhlelweni zangempela. U-Alexander Zaitsev (2018)

Naphezu kweqiniso lokuthi manje kunedatha eningi cishe yonke indawo, imininingwane yokuhlaziya isaxakile. Aziwa kahle futhi awakwazi nokuwasebenzisa ngempumelelo. Abaningi bayaqhubeka “nokudla i-cactus” nge-MySQL noma i-PostgreSQL, eklanyelwe ezinye izimo, ukulwa ne-NoSQL, noma ukukhokhela ngokweqile izixazululo zezentengiso. I-ClickHouse ishintsha umdlalo futhi yehlisa kakhulu isithiyo sokungena emhlabeni we-DBMS yokuhlaziya.

Umbiko uvela ku-BackEnd Conf 2018 futhi ushicilelwa ngemvume yesipika.

Ngingubani futhi kungani ngikhuluma ngeClickHouse? NginguMqondisi Wezokuthuthukiswa kwa-LifeStreet, esebenzisa i-ClickHouse. Futhi ngingumsunguli we-Altinity. Lona uzakwethu we-Yandex okhuthaza i-ClickHouse futhi usiza i-Yandex yenze i-ClickHouse iphumelele kakhulu. Ngikulungele futhi ukwabelana ngolwazi mayelana ne-ClickHouse.

Futhi angiyena umfowabo kaPetya Zaitsev. Ngivame ukubuzwa ngalokhu. Cha, asibona abazalwane.

"Wonke umuntu uyazi" ukuthi i-ClickHouse:

• Ngokushesha,
• Kulula kakhulu,
• Isetshenziswa ku-Yandex.

Kuncane okwaziwayo ukuthi yiziphi izinkampani nokuthi zisetshenziswa kanjani.

Ngizokutshela ukuthi kungani, kuphi futhi kanjani i-ClickHouse isetshenziswa, ngaphandle kwe-Yandex.

Ngizokutshela ukuthi izinkinga ezithile zixazululwa kanjani usebenzisa i-ClickHouse ezinkampanini ezahlukene, yiziphi amathuluzi we-ClickHouse ongawasebenzisa emisebenzini yakho, nokuthi asetshenziswe kanjani ezinkampanini ezahlukene.

Ngikhethe izibonelo ezintathu ezikhombisa i-ClickHouse ezinhlangothini ezihlukene. Ngicabanga ukuthi kuzoba mnandi.

Umbuzo wokuqala uthi: "Kungani sidinga i-ClickHouse?" Kubonakala sengathi umbuzo usobala, kodwa kunezimpendulo ezingaphezu kweyodwa kuwo.

• Impendulo yokuqala kungenxa yezizathu zokusebenza. I-ClickHouse ishesha kakhulu. Izibalo ku-ClickHouse nazo ziyashesha kakhulu. Ngokuvamile ingasetshenziswa lapho enye into isebenza kancane kakhulu noma kabi kakhulu.
• Impendulo yesibili yizindleko. Futhi okokuqala, izindleko zokukala. Isibonelo, i-Vertica iyisizindalwazi esihle kakhulu. Isebenza kahle kakhulu uma ungenawo ama-terabytes amaningi edatha. Kodwa uma sikhuluma ngamakhulu ama-terabytes noma ama-petabytes, izindleko zelayisense nokusekelwa zifinyelela inani elibalulekile. Futhi kuyabiza. Futhi i-ClickHouse imahhala.
• Impendulo yesithathu yizindleko zokusebenza. Lena indlela ehluke kancane. I-RedShift iyi-analogue enhle. Nge-RedShift ungenza isinqumo ngokushesha okukhulu. Kuzosebenza kahle, kepha ngasikhathi sinye, ihora ngalinye, nsuku zonke futhi nyanga zonke uzokhokha kakhulu e-Amazon, ngoba iyisevisi ebiza kakhulu. I-Google BigQuery nayo. Uma kukhona oke wayisebenzisa, khona-ke uyazi ukuthi ungagijima imibuzo embalwa lapho futhi kungazelelwe uthole i-invoyisi yamakhulu amadola.

I-ClickHouse ayinazo lezi zinkinga.

Isetshenziswa kuphi i-ClickHouse manje? Ngokungeziwe ku-Yandex, i-ClickHouse isetshenziswa eqenjini lamabhizinisi nezinkampani ezahlukene.

• Okokuqala, lokhu kuyizibalo zohlelo lokusebenza lwewebhu, okungukuthi lokhu kuyicala lokusetshenziswa elivela ku-Yandex.
• Izinkampani eziningi ze-AdTech zisebenzisa i-ClickHouse.
• Izinkampani eziningi ezidinga ukuhlaziya izingodo zokusebenza ezivela emithonjeni ehlukene.
• Izinkampani eziningana zisebenzisa i-ClickHouse ukuqapha izingodo zokuphepha. Bazilayisha ku-ClickHouse, benze imibiko, futhi bathole imiphumela abayidingayo.
• Izinkampani ziqala ukuyisebenzisa ekuhlaziyeni kwezezimali, okungukuthi kancane kancane amabhizinisi amakhulu nawo asondela ku-ClickHouse.
• CloudFlare. Uma kukhona olandela i-ClickHouse, kungenzeka ukuthi uzwile igama lale nkampani. Lona ngomunye wabanikeli ababalulekile abavela emphakathini. Futhi banokufaka kwe-ClickHouse okubucayi kakhulu. Isibonelo, benze i-Kafka Engine ye-ClickHouse.
• Izinkampani zezokuxhumana seziqalile ukusebenzisa. Izinkampani eziningi zisebenzisa i-ClickHouse njengobufakazi bomqondo noma kakade kukhiqizwa.
• Inkampani eyodwa isebenzisa i-ClickHouse ukuqapha izinqubo zokukhiqiza. Bahlola ama-microcircuits, babhale inqwaba yamapharamitha, kunezici ezingaba ngu-2. Bese behlaziya ukuthi iqoqo lihle noma libi.
• Izibalo zeBlockchain. Kunenkampani yaseRussia ebizwa nge-Bloxy.info. Lokhu kuhlaziywa kwenethiwekhi ye-Ethereum. Baphinde bakwenza lokhu ku-ClickHouse.

Ngaphezu kwalokho, usayizi akusho lutho. Kunezinkampani eziningi ezisebenzisa iseva eyodwa encane. Futhi uyabavumela ukuba baxazulule izinkinga zabo. Futhi izinkampani eziningi zisebenzisa amaqoqo amakhulu amaseva amaningi noma inqwaba yamaseva.

Futhi uma ubheka amarekhodi, ke:

• I-Yandex: Amaseva angu-500+, agcina amarekhodi ayizigidi eziyizinkulungwane ezingu-25 ngosuku lapho.
• I-LifeStreet: amaseva angama-60, cishe amarekhodi ayizigidi eziyizinkulungwane ezingama-75 ngosuku. Kunamaseva ambalwa namarekhodi amaningi kunaku-Yandex.
• I-CloudFlare: Amaseva angama-36, agcina amarekhodi ayizigidi eziyizinkulungwane ezingama-200 ngosuku. Anamaseva amancane kakhulu futhi agcina idatha eyengeziwe.
• I-Bloomberg: amaseva ayi-102, cishe amarekhodi ayizigidi eziyisigidi ngosuku. Isibambi serekhodi.

Ngokwendawo, lokhu nakho kuningi. Le mephu ibonisa imephu yokushisa lapho kusetshenziswa khona i-ClickHouse emhlabeni. Lapha iRussia, iChina, neMelika zigqama ngokusobala. Kunamazwe ambalwa aseYurophu. Futhi amaqoqo ama-4 angahlukaniswa.

Lokhu ukuhlaziya okuqhathanisayo, asikho isidingo sokubheka izinombolo eziphelele. Lokhu ukuhlaziya izivakashi ezifunda izinto zolimi lwesiNgisi kuwebhusayithi ye-Altinity, ngoba azikho izikhulumi zesiRashiya lapho. Futhi iRussia, i-Ukraine, iBelarus, okungukuthi ingxenye yomphakathi ekhuluma isiRashiya, ingabasebenzisi abaningi kakhulu. Bese kuza i-USA neCanada. I-China ibamba iqhaza kakhulu. Cishe yayingekho iChina lapho ezinyangeni eziyisithupha ezedlule; manje iChina isivele yadlula iYurophu futhi iyaqhubeka nokukhula. I-Old Europe nayo ayisali ngemuva, futhi umholi ekusetshenzisweni kweClickHouse, ngokuxakile, iFrance.

Kungani ngikutshela konke lokhu? Ukukhombisa ukuthi i-ClickHouse isiba yisixazululo esijwayelekile sokuhlaziya idatha enkulu futhi isivele isetshenziswa ezindaweni eziningi. Uma uyisebenzisa, ukuthrendi efanele. Uma ungayisebenzisi okwamanje, akufanele wesabe ukuthi uzosala wedwa futhi akekho ozokusiza, ngoba abaningi sebevele benza lokhu.

Lezi yizibonelo zokusetshenziswa kwangempela kweClickHouse ezinkampanini ezimbalwa.

• Isibonelo sokuqala inethiwekhi yokukhangisa: ukufuduka kusuka ku-Vertica kuya ku-ClickHouse. Futhi ngiyazazi izinkampani ezimbalwa ezisuke ku-Vertica noma eziphezu kwenqubo yokushintsha.
• Isibonelo sesibili isitoreji sokwenziwe ku-ClickHouse. Lesi yisibonelo esakhiwe kuma-antipattern. Konke okungadingi ukwenziwa ku-ClickHouse ngokwezeluleko zabathuthukisi kwenziwa lapha. Futhi ngesikhathi esifanayo kwenziwa ngempumelelo kangangokuthi iyasebenza. Futhi isebenza kangcono kakhulu kunesixazululo esijwayelekile sokuthengiselana.
• Isibonelo sesithathu sisatshalaliswa ngekhompyutha ku-ClickHouse. Kube nombuzo mayelana nokuthi i-ClickHouse ingahlanganiswa kanjani ne-Hadoop ecosystem. Ngizobonisa isibonelo sokuthi inkampani yenza kanjani into efana nesitsha sokunciphisa imephu ku-ClickHouse, iqapha ukwenziwa kwasendaweni kwedatha, njll., ukuze kubalwe umsebenzi ongewona omncane kakhulu.

• I-LifeStreet yinkampani ye-Ad Tech enabo bonke ubuchwepheshe obuhlobene nenethiwekhi yokukhangisa.
• Uzibandakanye nokuthuthukisa izikhangiso kanye nokubhida ngohlelo.
• Idatha eningi: cishe imicimbi eyizigidi eziyizinkulungwane eziyi-10 ngosuku. Ngaphezu kwalokho, kukhona imicimbi ingahlukaniswa ngeminye imicimbi emincane.
• Kunamakhasimende amaningi ale datha, futhi laba akubona abantu kuphela, abaningi abengeziwe ngama-algorithms ahlukahlukene abandakanyeka ekubhideni kohlelo.

Inkampani ihambe indlela ende futhi enameva. Futhi ngikhulume ngakho ku-HighLoad. Okokuqala, i-LifeStreet ithuthe isuka ku-MySQL (enokuma kancane e-Oracle) yaya ku-Vertica. Futhi ungathola indaba ngakho.

Futhi konke kwakukuhle kakhulu, kodwa kwasheshe kwacaca ukuthi idatha yayikhula futhi i-Vertica yayibiza. Ngakho-ke, kwafunwa ezinye izindlela ezihlukahlukene. Ezinye zazo zibalwe lapha. Futhi eqinisweni, senze ubufakazi bomqondo noma ngezinye izikhathi ukuhlolwa kokusebenza kwazo zonke izingobo zolwazi ezazitholakala emakethe kusukela ku-13 kuya ku-16 futhi cishe zifaneleka ekusebenzeni. Futhi ngikhulume ngezinye zazo ku-HighLoad.

Umsebenzi bekuwukusuka ku-Vertica kuqala, ngoba idatha yayikhula. Futhi bakhula ngokuphawulekayo iminyaka eminingana. Base beya eshalofini, kodwa noma kunjalo. Futhi ukubikezela lokhu kukhula, izidingo zebhizinisi zevolumu yedatha lapho uhlobo oluthile lokuhlaziya ludinga ukwenziwa, kwacaca ukuthi maduzane kuzoba nenkulumo yama-petabytes. Futhi kakade kubiza kakhulu ukukhokhela ama-petabytes, ngakho-ke besifuna enye indlela lapho singaya khona.

Ungaya kuphi? Futhi isikhathi eside kwakungacaci kahle ukuthi kufanele uye kuphi, ngoba ngakolunye uhlangothi kukhona imininingwane yezohwebo, kubonakala sengathi isebenza kahle. Ezinye zisebenza cishe njenge-Vertica, ezinye zimbi kakhulu. Kodwa zonke ziyabiza, akukho okushibhile noma okungcono okungatholakala.

Ngakolunye uhlangothi, kunezixazululo zomthombo ovulekile, ezingekho eziningi kakhulu, okungukuthi ukuhlaziya zingabalwa ngakwesokunene. Futhi amahhala noma ashibhile, kodwa asebenza kancane. Futhi ngokuvamile abanakho ukusebenza okudingekayo nokuwusizo.

Futhi kwakungekho lutho lokuhlanganisa izinto ezinhle ezisezinqolobaneni zezentengiselwano nazo zonke izinto zamahhala ezisemthonjeni ovulekile.

Akwenzekanga lutho kwaze kwaba yilapho u-Yandex evele ekhipha i-ClickHouse esigqokweni njengonogwaja womlingo. Futhi lesi kwakuyisinqumo esingalindelekile, abantu basabuza umbuzo: "Kungani?", Kodwa nokho.

Futhi ngaso leso sikhathi ehlobo lika-2016, saqala ukubheka ukuthi iyini i-ClickHouse. Futhi kwavela ukuthi kwesinye isikhathi kungaba ngokushesha kune-Vertica. Sihlole izimo ezahlukahlukene ezicelweni ezahlukahlukene. Futhi uma umbuzo usebenzisa ithebula elilodwa kuphela, okungukuthi ngaphandle kokujoyina, i-ClickHouse yayishesha ngokuphindwe kabili njenge-Vertica.

Angizange ngivilaphe kakhulu futhi ngibheke ezinye izivivinyo ze-Yandex ngolunye usuku. Kuyafana lapho: I-ClickHouse ishesha kabili njenge-Vertica, ngakho-ke bavame ukukhuluma ngayo.

Kodwa uma imibuzo iqukethe okuhlanganisayo, khona-ke yonke into ivela ngokungacaci kahle. Futhi i-ClickHouse ingahamba kancane ngokuphindwe kabili njenge-Vertica. Futhi uma ulungisa futhi ubhala kabusha isicelo kancane, khona-ke zizolingana. Akukubi. Futhi kumahhala.

Futhi ngemva kokuthola imiphumela yokuhlolwa, nokuyibuka ngama-engeli ahlukene, i-LifeStreet iye ku-ClickHouse.

Lona unyaka we-16, ngiyanikhumbuza. Kwaba sengathi ihlaya lamagundane akhala azijova kodwa aqhubeka nokudla umdolofiya. Futhi lokhu kwaxoxwa ngokuningiliziwe, kukhona ividiyo mayelana nalokhu, njll.

Ngakho-ke, ngeke ngikhulume ngalokhu ngokuningiliziwe, ngizokhuluma kuphela ngemiphumela kanye nezinto ezimbalwa ezithakazelisayo engingazange ngikhulume ngazo ngaleso sikhathi.

Imiphumela yilena:

• Ukufuduka okuyimpumelelo kanye nesistimu isikhiqize isikhathi esingaphezu konyaka.
• Ukukhiqiza nokuvumelana nezimo kunyukile. Kusukela kumarekhodi ayizigidi eziyizinkulungwane eziyi-10 ebesingakwazi ukuwagcina ngosuku isikhathi esifushane kuphela, i-LifeStreet manje igcina amarekhodi ayizigidi eziyizinkulungwane ezingama-75 ngosuku futhi ingakwenza lokho izinyanga ezi-3 noma ngaphezulu. Uma ubala phezulu, lokhu kugcinwa izehlakalo ezifika esigidini ngomzuzwana. Imibuzo ye-SQL engaphezu kwesigidi ngosuku ithunyelwa kule sistimu, ikakhulukazi kumarobhothi ahlukahlukene.
• Naphezu kweqiniso lokuthi i-ClickHouse yaqala ukusebenzisa amaseva amaningi kune-Vertica, ukugcinwa futhi kwenziwa ku-hardware, ngoba i-Vertica yasebenzisa amadiski e-SAS abiza kakhulu. I-ClickHouse isebenzise i-SATA. Futhi ngani? Ngoba ukufakwa kwe-Vertica kuyavumelana. Futhi ukuvumelanisa kudinga ukuthi ama-disks anganciphisi kakhulu, futhi nokuthi inethiwekhi ayinciphisi kakhulu, okungukuthi, ukusebenza okubiza kakhulu. Futhi ku-ClickHouse ukufaka asynchronous. Ngaphezu kwalokho, ungahlala ubhala yonke into endaweni, azikho izindleko ezengeziwe zalokhu, ngakho-ke idatha ingafakwa ku-ClickHouse ngokushesha kakhulu kune-Vertika, ngisho nakumadiski asheshayo. Futhi ukufunda kucishe kufane. Ukufunda ku-SATA, uma iku-RAID, konke kuyashesha ngokwanele.
• Akukhawulelwe ngelayisense, okungukuthi ama-petabytes angu-3 edatha kumaseva angu-60 (amaseva angu-20 ayikhophi eyodwa) kanye namarekhodi angama-trillion angu-6 amaqiniso namaqoqo. I-Vertica ayikwazanga ukukhokhela into efana nale.

Manje ngifinyelela ezintweni ezingokoqobo kulesi sibonelo.

• Esokuqala siwuhlelo olusebenzayo. Okuningi kuncike esikimini.
• Okwesibili ukukhiqiza i-SQL ephumelelayo.

Umbuzo ojwayelekile we-OLAP uyakhethwa. Amanye amakholomu aya eqenjini ngalinye, amanye amakholomu aya ekuhlanganiseni imisebenzi. Kukhona lapho, okungacatshangwa njengocezu lwe-cube. Iqembu lonke lingacatshangwa njengokuqagela. Futhi yingakho kubizwa ngokuthi ukuhlaziywa kwedatha ye-multivariate.

Futhi ngokuvamile lokhu kulinganiswa ngendlela yomdwebo wenkanyezi, lapho kukhona iqiniso eliphakathi nezici zaleli qiniso ezinhlangothini, eduze kwemisebe.

Futhi ngokombono wokuklama ngokomzimba, ukuthi kufanelana kanjani etafuleni, ngokuvamile benza ukumelela okujwayelekile. Ungakwazi ukwenza i-normalize, kodwa iyabiza kudiski futhi ayisebenzi kahle kakhulu emibuzweni. Ngakho-ke, ngokuvamile benza ukubuka okujwayelekile, okungukuthi ithebula leqiniso kanye namathebula amaningi, amaningi.

Kepha lokhu akusebenzi kahle ku-ClickHouse. Kunezizathu ezimbili:

• Esokuqala yingoba i-ClickHouse ayinakho ukuhlanganisa okuhle kakhulu, okungukuthi kukhona okujoyinayo, kepha kubi. Kuze kube manje babi.
• Okwesibili ukuthi amatafula awabuyekeziwe. Ngokuvamile kulezi zimpawu eziseduze nomdwebo wenkanyezi, kukhona okudinga ukushintshwa. Isibonelo, igama leklayenti, igama lenkampani, njll. Futhi akusebenzi.

Futhi kukhona indlela yokuphuma kulokhu ku-ClickHouse. ngisho ezimbili:

• Esokuqala ukusetshenziswa kwezichazamazwi. Izichazamazwi Zangaphandle yizo ezisiza u-99% ukuxazulula inkinga ngohlelo lwenkanyezi, ngezibuyekezo nokunye.
• Okwesibili ukusetshenziswa kwama-arrays. Ama-arrays asiza futhi ukuqeda ukujoyina kanye nezinkinga ngokujwayelekile.

• Asikho isidingo sokujoyina.
• Iyabuyekezwa. Kusukela ngoMashi 2018, kuvele ithuba elingekho emthethweni (ngeke ukuthole lokhu kumadokhumenti) ukuze ubuyekeze izichazamazwi kancane, okungukuthi lokho okufakiwe okushintshile. Empeleni, kufana netafula.
• Ihlala isenkumbulweni, ngakho-ke ukujoyina nesichazamazwi kusebenza ngokushesha kunokuba bekuyitafula elibekwe kudiski futhi akulona iqiniso ukuthi kunqolobane, cishe akunjalo.

• Awudingi ukujoyina futhi.
• Lokhu ukumelela oku-1 kuya kwabaningi.
• Futhi ngokubona kwami, ama-arrays enzelwe ama-geek. Lena imisebenzi ye-lambda nezinto.

Lokhu akungenxa yamagama. Lokhu ukusebenza okunamandla kakhulu okukuvumela ukuthi wenze izinto eziningi kalula nangobuhle.

Izibonelo ezijwayelekile ezisiza ukuxazulula amalungu afanayo. Lezi zibonelo zilula futhi zicacile:

• Sesha ngomaka. Uma unama-hashtag lapho futhi ufuna ukuthola okuthunyelwe nge-hashtag.
• Sesha ngamapheya enani elingukhiye. Kukhona futhi ezinye izimfanelo ezinencazelo.
• Ukugcina uhlu lokhiye obadingayo ukuze uhumushele kokuthile.

Zonke lezi zinkinga zingaxazululwa ngaphandle kwama-arrays. Omaka bangabekwa emugqeni othile futhi bakhethwe kusetshenziswa isisho esivamile, noma etafuleni elihlukile, kodwa-ke kuzodingeka wenze ukujoyina.

Kodwa ku-ClickHouse awudingi ukwenza lutho, vele uchaze uhlu lwezintambo zama-hashtag noma udale isakhiwo esisidleke sezinhlelo zenani elingukhiye.

Isakhiwo esisidleke singase singabi igama elingcono kakhulu. Lawa ama-array amabili anengxenye efanayo egameni nezinye izici ezihlobene.

Futhi kulula kakhulu ukusesha ngethegi. Kukhona umsebenzi has, ehlola ukuthi amalungu afanayo aqukethe into ethile. Wonke umuntu, sithole konke okufakiwe okuhlobene nengqungquthela yethu.

Ukusesha nge-subid kuyinkimbinkimbi kancane. Okokuqala sidinga ukuthola inkomba yokhiye, bese sithatha i-elementi nale nkomba futhi sihlole ukuthi leli nani yilokho esikudingayo. Kodwa nokho ilula kakhulu futhi ihlangene.

Isisho esivamile ongathanda ukusibhala, uma usigcine sonke emgqeni owodwa, singaba, okokuqala nje, sixakile. Futhi, okwesibili, isebenze isikhathi eside kakhulu kunezigaba ezimbili.

Esinye isibonelo. Unezinhlelo eziningi ogcina kuzo ama-ID. Futhi ungawahumushela emagameni. Umsebenzi arrayMap. Lona umsebenzi we-lambda ojwayelekile. Udlula izinkulumo ze-lambda lapho. Futhi ukhipha inani legama le-ID ngayinye kusichazamazwi.

Ungenza usesho ngendlela efanayo. Umsebenzi we-predicate uyadluliswa, ohlola ukuthi izakhi zifanelana ini.

Lezi zinto zenza kube lula ukujikeleza futhi zixazulule inqwaba yezinkinga.

Kodwa inkinga elandelayo esihlangabezane nayo futhi engingathanda ukuyisho imibuzo ephumelelayo.

• I-ClickHouse ayinaso isihleli semibuzo. Lutho neze.
• Kodwa nokho, imibuzo eyinkimbinkimbi isadinga ukuhlelwa. Kukuziphi izimo?
• Uma isicelo sinokujoyina okumbalwa, okugoqe ngokukhetha okuncanyana. Nendlela ezenziwa ngayo ibalulekile.
• Okwesibili, uma isicelo sisatshalaliswa. Ngoba embuzweni osabalalisiwe, ukukhetha okuncane okungaphakathi kuphela okwenziwa ngendlela esabalalisiwe, futhi konke okunye kuthunyelwa kuseva eyodwa oxhume kuyo futhi wayenze lapho. Ngakho-ke, uma usabalalise imibuzo ngokujoyina okuningi, udinga ukukhetha i-oda.

Futhi ngisho nasezimweni ezilula, ngezinye izikhathi udinga ukwenza umsebenzi womhleli futhi ubhale kabusha imibuzo kancane.

Nasi isibonelo. Ngakwesobunxele kunombuzo obonisa amazwe angu-5 aphezulu. Futhi igijima ngemizuzwana engu-2,5, ngicabanga. Futhi ngakwesokunene isicelo esifanayo, kodwa sibhalwe kabusha kancane. Esikhundleni sokuqoqa ngochungechunge, siqale ukuqoqa ngokhiye (int). Futhi kuyashesha. Sabe sesixhuma isichazamazwi kumphumela. Esikhundleni samasekhondi angu-2,5, isicelo sithatha imizuzwana engu-1,5. Lokhu kuhle.

Isibonelo esifanayo nezihlungi zokubhala kabusha. Nasi isicelo seRussia. Igijima imizuzwana emi-5. Uma siyibhala kabusha ngendlela yokuthi siphinde siqhathanise hhayi intambo, kodwa izinombolo ezinesethi ethile yalezo zihluthulelo ezihlobene neRussia, khona-ke izoshesha kakhulu.

Maningi amaqhinga anjalo. Futhi zikuvumela ukuthi usheshise kakhulu imibuzo ocabanga ukuthi isivele isebenza ngokushesha, noma, ngokuphambene, isebenza kancane. Zingenziwa ngokushesha nakakhulu.

• Umsebenzi omkhulu kumodi esabalalisiwe.
• Ukuhlunga ngezinhlobo ezincane, njengoba ngenzile ngama-ints.
• Uma kukhona okuhlanganisayo noma izichazamazwi, khona-ke kungcono ukukwenza ekugcineni, lapho usuvele unedatha okungenani eqoqwe ngokwengxenye, khona-ke ukusebenza kokujoyina noma ukubiza isichazamazwi kuzobizwa izikhathi ezimbalwa futhi kuzoshesha.
• Ukushintsha izihlungi.

Akhona amanye amasu, hhayi lawo engiwabonisile. Futhi zonke kwesinye isikhathi zikuvumela ukuthi usheshise kakhulu ukwenziwa kwemibuzo.

Asiqhubekele esibonelweni esilandelayo. Inkampani X evela e-USA. Wenzani?

Bekunomsebenzi:

• Ukuxhuma okungaxhunyiwe ku-inthanethi kokuthengiswayo kokukhangisa.
• Ukulingisa amamodeli ahlukene okubopha.

Siyini isimo?

Isivakashi esivamile sivakashela isayithi, isibonelo, izikhathi ezingu-20 ngenyanga kusukela ezikhangisweni ezahlukene, noma ngezinye izikhathi sivele sifike ngaphandle kwezikhangiso, ngoba sikhumbula le sayithi. Ubheka eminye imikhiqizo, ayifake kubhasikidi, ayikhiphe kubhasikidi. Futhi, ekugcineni, uthenga okuthile.

Imibuzo enengqondo: "Ubani okufanele akhokhele ukukhangisa, uma kunesidingo?" nokuthi “Yikuphi ukukhangisa, uma kukhona, okwamthonya?” Okungukuthi, kungani ethenge nokuthi angaqinisekisa kanjani ukuthi abantu abafana nalo muntu bayathenga?

Ukuze uxazulule le nkinga, udinga ukuxhuma izenzakalo ezenzeka kuwebhusayithi ngendlela efanele, okungukuthi, ngandlela-thile ukwakha uxhumano phakathi kwabo. Bese zidluliselwa ku-DWH ukuze zihlaziywe. Futhi ngokusekelwe kulokhu kuhlaziya, yakha amamodeli okuthi ubani ozobonisa lokho ukukhangisa.

Umsebenzi wokukhangisa isethi yemicimbi yomsebenzisi ehlobene eqala ngesikhangiso esiboniswayo, bese kwenzeka okuthile, bese mhlawumbe ukuthenga, bese kuba khona ukuthenga phakathi kokuthenga. Isibonelo, uma lokhu kuwuhlelo lokusebenza lweselula noma umdlalo weselula, ngokuvamile ukufaka isicelo kumahhala, kodwa uma kukhona okunye okwenziwa lapho, kungase kudinge imali. Futhi uma umuntu echitha isikhathi esiningi kuhlelo lokusebenza, kuba yigugu kakhulu. Kodwa kulokhu udinga ukuxhuma konke.

Kunamamodeli amaningi okubopha.

Okudume kakhulu yilezi:

• Ukusebenzisana Kokugcina, lapho ukusebenzelana kuwukuchofoza noma umbono.
• Ukusebenzelana Kokuqala, okungukuthi into yokuqala elethe umuntu kusayithi.
• Inhlanganisela yomugqa - ukwabelana okulinganayo kwawo wonke umuntu.
• Ukunciphisa.
• Njalo njalo.

Futhi konke kwakusebenza kanjani ekuqaleni? Kwakukhona i-Runtime noCassandra. I-Cassandra yasetshenziswa njengendawo yokugcina izinto, okungukuthi yonke imisebenzi ehlobene yayigcinwe kuyo. Futhi uma umcimbi othile wenzeka ku-Runtime, isibonelo, ukuboniswa kwekhasi noma enye into, isicelo senziwa ku-Cassandra ukuthi ukhona yini umuntu onjalo noma cha. Kwabe sekutholwa ukuthengiselana okuhlobene nakho. Futhi ukubopha kwenziwa.

Futhi uma unenhlanhla yokuthi isicelo siqukethe i-id yokwenziwe, lokhu kulula. Kodwa ngokuvamile awunayo inhlanhla. Ngakho-ke, bekudingekile ukuthola ukuthengiselana kokugcina noma ukuthengiselana ngokuchofoza kokugcina, njll.

Futhi konke kusebenze kahle kakhulu kwaze kwaba yilapho ukuxhumanisa kwaba ngokuchofoza kokugcina. Ngoba kukhona, ake sithi, ukuchofoza kwezigidi ezingu-10 ngosuku, izigidi ezingu-300 ngenyanga, uma usetha iwindi inyanga. Futhi njengoba eCassandra konke kufanele kube senkumbulweni ukuze kusebenze ngokushesha, ngoba i-Runtime iyadingeka ukuphendula ngokushesha, cishe amaseva ayi-10-15 ayedingeka.

Futhi lapho befuna ukuxhumanisa ukuthengiselana kusibonisi, ngokushesha kwavela kungemnandi kangako. Futhi ngani? Kungabonakala ukuthi imicimbi ephindwe izikhathi ezingu-30 idinga ukugcinwa. Futhi, ngokufanele, udinga amaseva aphindwe ka-30 ngaphezulu. Futhi kuvela ukuthi lolu uhlobo oluthile lwesibalo sezinkanyezi. Ukugcina amaseva afika ku-500 ukuze wenze ukuxhumanisa, naphezu kokuthi kunamaseva ambalwa kakhulu ku-Runtime, uhlobo oluthile lwesibalo esingalungile. Futhi baqala ukucabanga ukuthi benzeni.

Futhi siye eClickHouse. Ungakwenza kanjani lokhu ku-ClickHouse? Ekuboneni kokuqala, kubonakala sengathi lena isethi yama-antipatterns.

• Okwenziwayo kuyakhula, sinamathisela imicimbi eminingi kuyo, okungukuthi iyaguquleka, futhi i-ClickHouse ayisebenzi kahle kakhulu ngezinto eziguquguqukayo.
• Uma isivakashi siza kithi, sidinga ukubuyisa ukuthengiselana kwaso ngokhiye, nge-id yaso yokuvakasha. Lona futhi umbuzo wephuzu; I-ClickHouse ayikwenzi lokho. Ngokuvamile i-ClickHouse inezikena ezinkulu…, kodwa lapha sidinga ukuthola amarekhodi ambalwa. Futhi i-antipattern.
• Ukwengeza, ukuthengiselana bekuku-json, kodwa abafuni ukukubhala kabusha, ngakho-ke bafuna ukugcina i-json ingahlelekile, futhi uma kunesidingo, khipha okuthile kuyo. Futhi lokhu kuyi-antipattern.

Okusho ukuthi, isethi yama-antipatterns.

Kodwa noma kunjalo, sikwazile ukwakha uhlelo olusebenze kahle kakhulu.

Kwenziwani? I-ClickHouse ivele, lapho izingodo, ezihlukaniswe ngamarekhodi, zaphonswa. Kuvele isevisi ehambisanayo ethole izingodo ezivela ku-ClickHouse. Ngemva kwalokho, ekungeneni ngakunye nge-id yokuvakasha, ngathola ukuthengiselana obekungakenzeki kuze kube manje kanye nezifinyezo, okungukuthi okwenziwayo osekuxhunyiwe kakade, okuwumphumela womsebenzi wangaphambilini. Sengivele ngiwenzile umqondo ngazo, ngakhetha ukuthenga okulungile, futhi ngaxhuma imicimbi emisha. Ingenile futhi. Ilogi ibuyele emuva ku-ClickHouse, okungukuthi iwuhlelo oluhlala lujikeleza. Futhi ngaphandle kwalokho, ngaya kwa-DWH ukuyoyihlaziya lapho.

Akusebenzanga kahle kakhulu kuleli fomu. Futhi ukwenza kube lula ku-ClickHouse, ngenkathi kunesicelo se-id yokuvakasha, bahlanganisa lezi zicelo ngamabhulokhi ama-id okuvakasha ayi-1-000 futhi bakhipha konke ukuthengiselana kwabantu abayi-2-000. Futhi-ke konke kwasebenza.

Uma ubheka ngaphakathi kwe-ClickHouse, kunamatafula angu-3 kuphela anikeza konke lokhu.

Ithebula lokuqala okulayishwa kulo amalogi, futhi amalogi alayishwa ngaphandle kokucutshungulwa nhlobo.

Ithebula lesibili. Ngombono owenziwe izinto ezibonakalayo, izehlakalo ezingakabalulwa, okungukuthi, ezingahlobene, zikhishwe kulawa malogi. Futhi ngokubuka okwenyama, ukuthengiselana kwakhishwa kulawa malogi ukuze kwakhiwe isifinyezo. Okusho ukuthi, isifinyezo sakhiwe esinokubuka okukhethekile okwenziwe okwenyama, okuyisimo sokugcina esiqoqiwe somsebenzi.

Lapha umbhalo ubhalwe ku-SQL. Ngithanda ukuphawula ngezinto ezimbalwa ezibalulekile kulo.

Into yokuqala ebalulekile yikhono ku-ClickHouse ukukhipha amakholomu nezinkambu ku-json. Okusho ukuthi, i-ClickHouse inezindlela ezithile zokusebenza ne-json. Kakhulu, abakudala kakhulu.

visitParamExtractInt ikuvumela ukuthi ukhiphe izibaluli ku-json, i.e. ukushaya kokuqala kuyacushwa. Futhi ngale ndlela ungakhipha i-id yokwenziwe noma i-id yokuvakashela. Ngalesi sikhathi.

Okwesibili, inkambu eyenziwe ngezinto ezikhohlisayo isetshenziswa lapha. Kusho ukuthini? Lokhu kusho ukuthi awukwazi ukuyifaka etafuleni, okungukuthi ayifakiwe, ibalwa futhi igcinwe lapho ifakiwe. Uma ufaka, i-ClickHouse ikwenzela umsebenzi. Futhi ozodinga kamuva kukhishwa ku-json.

Kulokhu, ukubuka okwenziwe okwenyama okwezintambo ezingahluziwe. Futhi itafula lokuqala elinamapulangwe acishe aluhlaza liyasetshenziswa. Futhi lenzani? Okokuqala, kushintsha ukuhlunga, okungukuthi ukuhlunga manje kwenziwa nge-id yokuvakasha, ngoba sidinga ukukhipha ngokushesha ukuthengiselana kwakhe kumuntu othize.

Into yesibili ebalulekile inkomba_granularity. Uma uyibonile i-MergeTree, ngokuvamile inani elizenzakalelayo lingu-8 index_granularity. Yini? Lena inkomba sparsity ipharamitha. Ku-ClickHouse, inkomba imbalwa; ayilokothi ikhombe wonke amarekhodi. Ikwenza lokhu njalo ngo-192 8. Futhi lokhu kuhle uma udinga ukubala idatha eningi, kodwa kubi uma udinga ukubala kancane, ngoba kunokuningi okungaphezulu. Futhi uma sinciphisa inkomba granularity, ke sinciphisa phezulu. Awukwazi ukuyehlisela kokukodwa, ngoba kungenzeka ingabi khona inkumbulo eyanele. Inkomba ihlale igcinwe kumemori.

Futhi isifinyezo sisebenzisa eminye imisebenzi ethokozisayo ye-ClickHouse.

Okokuqala yi-AggregatingMergeTree. Futhi i-AggregatingMergeTree igcina i-argMax, i.e. lesi isimo somsebenzi esihambisana nesitembu sesikhathi sokugcina. Okwenziwayo okusha kuhlala kukhiqizwa kulesi sivakashi. Futhi esimweni sokugcina salokhu kuthenga, sengeze umcimbi futhi saba nesimo esisha. Iphinde yashaya i-ClickHouse. Futhi nge-argMax kulokhu kubuka okwenziwe ngokomzimba singakwazi ukuthola isimo samanje.

• Ukubophezela "akubanjwanga" kusukela ku-Runtime.
• Imisebenzi efinyelela ku-3 billion ngenyanga igcinwa futhi icutshungulwe. Lokhu ukuhleleka kobukhulu obukhulu kunase-Cassandra, okungukuthi, ohlelweni lokwenziwayo olujwayelekile.
• Iqoqo lamaseva angu-2x5 e-ClickHouse. Amaseva angu-5 futhi iseva ngayinye inomfanekiso. Lokhu kuncane nakakhulu kunalokho okwakuyikho e-Cassandra ukuze wenze isichasiso esisekelwe ngokuchofoza, kodwa lapha sinombono osuselwe. Okusho ukuthi, esikhundleni sokwandisa inani lamaseva izikhathi ezingu-30, ancishisiwe.

Futhi isibonelo sokugcina inkampani yezezimali Y, eyahlaziya ukuhlobana kwezinguquko kumanani esitoko.

Futhi umsebenzi wawulokhu:

• Kukhona cishe amasheya angu-5.
• Ama-Quotes njalo ngama-millisecond angu-100 ayaziwa.
• Idatha iqoqwe ngaphezulu kweminyaka eyi-10. Ngokusobala, kwezinye izinkampani kuningi, kwezinye kuncane.
• Cishe kunemigqa eyizigidigidi eziyi-100 isiyonke.

Futhi kwakudingeka ukubala ukuhlobana kwezinguquko.

Nazi izitoko ezimbili nezingcaphuno zazo. Uma enye ikhuphuka futhi enye ikhuphuka, khona-ke lokhu ukuhlobana okuphozithivu, okungukuthi omunye uya phezulu futhi omunye uya phezulu. Uma omunye ekhuphuka, njengasekupheleni kwegrafu, futhi enye yehla, khona-ke lokhu kuwukuhlobana okungekuhle, okungukuthi lapho omunye ekhuphuka, omunye wehla.

Ngokuhlaziya lezi zinguquko ezifanayo, umuntu angenza izibikezelo emakethe yezezimali.

Kodwa umsebenzi unzima. Kwenziwani lokhu? Sinamarekhodi ayizigidi eziyizinkulungwane eziyi-100 aqukethe: isikhathi, isitoko kanye nentengo. Kudingeka siqale sibale izikhathi eziyizigidi eziyizinkulungwane ezingu-100 kunomehluko osebenzayo kusukela ku-algorithm yentengo. I-RunningDifference ingumsebenzi ku-ClickHouse obala ngokulandelana umehluko phakathi kwemigqa emibili.

Futhi emva kwalokho sidinga ukubala ukuhlobana, futhi ukuhlobana kufanele kubalwe kubhangqa ngalinye. Ngamasheya angu-5, amapheya ayizigidi ezingu-000. Futhi lokhu kuningi, okungukuthi izikhathi ezingu-12,5 udinga ukubala lo msebenzi wokuhlanganisa.

Futhi uma kwenzeka noma ubani ukhohliwe, ͞x kanye no- ͞y bangabalingani. isampula yokulindela. Okusho ukuthi, udinga nje ukubala izimpande nezibalo, kodwa futhi nezinye izibalo ngaphakathi kwalezi zibalo. Izibalo eziningi neziningi zidinga ukwenziwa izikhathi eziyizigidi eziyi-12,5, futhi zidinga ukuqoqwa ngehora. Futhi sinamahora amaningi. Futhi kufanele ukwenze ngemizuzwana engama-60. Kuyihlaya.

Bekufanele sikwenze ngandlela thize, ngoba konke kusebenze kakhulu, kancane kakhulu ngaphambi kokuthi kufike i-ClickHouse.

Bazama ukubala lokhu eHadoop, eSpark, eGreenplum. Futhi konke lokhu kwakuhamba kancane kakhulu noma kubiza. Okungukuthi, kwakungenzeka ukubala ngandlela-thile, kodwa-ke kwakubiza.

Kwabe sekufika iClickHouse futhi yonke into yaba ngcono kakhulu.

Ake ngikukhumbuze ukuthi sinenkinga yendawo yedatha, ngakho ukuhlobana akukwazi ukwenziwa okwasendaweni. Asikwazi ukwengeza idatha kwenye iseva, enye kwenye futhi sibale; kufanele sibe nayo yonke idatha yonke indawo.

Benzeni? Ekuqaleni, idatha yenziwa yasendaweni. Iseva ngayinye igcina idatha yentengo yesethi ethile yamasheya. Futhi aziphambani. Ngakho-ke, kuyenzeka ukubala ilogiBuyisela ngokuhambisana nangokuzimele; konke lokhu kwenzeka ngokuhambisana futhi kwasatshalaliswa.

Sabe sesinquma ukunciphisa le datha ngaphandle kokulahlekelwa ukuvezwa. Nciphisa ukusebenzisa amalungu afanayo, okungukuthi isikhathi ngasinye yenza uchungechunge lwesitoko kanye nenani lezintengo. Ngakho kuthatha isikhala esincane kakhulu sedatha. Futhi zikulungele ngandlela-thile ukusebenza nazo. Lena imisebenzi ecishe ifane, okungukuthi sibala kancane ngokufana bese sibhalela iseva.

Lokhu-ke kungaphindwa. Uhlamvu “r” lusho ukuthi siphindaphinde le datha. Okusho ukuthi, sinedatha efanayo kuwo womathathu amaseva - lawa ama-array.

Bese-ke, usebenzisa iskripthi esikhethekile, ungenza amaphakheji kusuka kule sethi ye-12,5 million correlations okudingeka ibalwe. Okusho ukuthi, imisebenzi engu-2 enamapheya angu-500 okuhlobana. Futhi lo msebenzi kufanele ubalwe kuseva ethile ye-ClickHouse. Unayo yonke imininingwane ngoba idatha iyafana futhi uyakwazi ukuyibala ngokulandelana.

Nansi indlela ebukeka ngayo futhi. Okokuqala, sinayo yonke idatha esakhiweni esilandelayo: isikhathi, amasheya, intengo. Sabe sesibala i-logReturn, okungukuthi idatha yesakhiwo esifanayo, esikhundleni senani kuphela esine-logReturn. Bese zenziwa kabusha, i.e. sithole isikhathi neqembuArray ngamaphromoshini nohlu lwamanani. Kuphindwaphindwa. Futhi emva kwalokho, bakhiqiza inqwaba yemisebenzi futhi baphakela i-ClickHouse ukuze ikwazi ukuyibala. Futhi iyasebenza.

Ebufakazini bomqondo, umsebenzi bekuwumsebenzi ongaphansi, okungukuthi bathathe idatha encane. Futhi kumaseva amathathu kuphela.

Lezi zigaba ezimbili zokuqala: ukubala i-Log_return nokuyisonga ngohlelo kuthathe cishe ihora sisinye.

Futhi ukubala ukuhlobana kuthatha cishe amahora angama-50. Kodwa amahora angu-50 awanele, ngoba ngaphambili ayebasebenzela amasonto. Kube yimpumelelo enkulu. Futhi uma ubala, konke kubalwa izikhathi ezingu-70 ngomzuzwana kuleli qoqo.

Kodwa okubaluleke kakhulu ukuthi lesi simiso asinazo izithiyo, okungukuthi, silinganisa ngokulinganayo. Base beyihlola. Ikalwe ngempumelelo.

• Isikimu esifanele siyingxenye yempumelelo. Futhi uhlelo olufanele ukusebenzisa bonke ubuchwepheshe obudingekayo be-ClickHouse.
• I-Summing/AggregatingMergeTrees ubuchwepheshe obukuvumela ukuthi uhlanganise noma ubale isifinyezo sezwe njengekesi elikhethekile. Futhi lokhu kwenza izinto eziningi zibe lula.
• Ukubuka Kwezinto ezibonakalayo kukuvumela ukuthi ufinyelele umkhawulo wenkomba eyodwa. Mhlawumbe angizange ngikusho lokhu ngokucacile, kodwa lapho silayisha izingodo, izingodo eziluhlaza zazisetafuleni elinenkomba eyodwa, futhi esibalulweni izingodo zazisethebula, okungukuthi idatha efanayo, ihlungiwe kuphela, kodwa inkomba yayikhona. ngokuphelele kwabanye. Kubonakala kuyidatha efanayo, kodwa ukuhlunga okuhlukile. Futhi Ukubuka Okuphathekayo kukuvumela, uma ukudinga, ukuba udlule lo mkhawulo we-ClickHouse.
• Yehlisa imbudumbudu yenkomba yemibuzo yamaphuzu.
• Futhi usabalalise idatha ngobuhlakani, zama ukwenza idatha yendawo ngaphakathi kweseva ngangokunokwenzeka. Futhi zama ukuqinisekisa ukuthi izicelo nazo zisebenzisa ukwenziwa kwasendaweni lapho kungenzeka khona ngangokunokwenzeka.

Futhi ukufingqa le nkulumo emfushane, singasho ukuthi i-ClickHouse manje isithathe ngokuqinile indawo yazo zombili izingosi zolwazi zezentengiselwano kanye nesizindalwazi semithombo evulekile, okungukuthi ngokuqondile ukuhlaziya. Ungena kahle kulendawo. Futhi ngaphezu kwalokho, iqala kancane ukususa abanye, ngoba uma i-ClickHouse ikhona, awudingi i-InfiniDB. Okuqondile kungase kungadingeki maduzane uma kunikeza ukwesekwa okuvamile kwe-SQL. Yisebenzise!

-Siyabonga ngombiko! Inohlonze impela! Ingabe kube khona ukuqhathanisa ne-Apache Phoenix?

-Cha, angizange ngizwe muntu eqhathanisa. Thina kanye ne-Yandex sizama ukugcina umkhondo wakho konke ukuqhathanisa kwe-ClickHouse enemininingwane ehlukene. Ngoba uma kungazelelwe okuthile kuvela ngokushesha kuneClickHouse, uLesha Milovidov akakwazi ukulala ebusuku futhi uqala ukusheshisa ngokushesha. Angikaze ngizwe ngokuqhathaniswa okunjalo.

• (Alexey Milovidov) I-Apache Phoenix iyinjini ye-SQL esekelwe ku-Hbase. I-Hbase yakhelwe kakhulu isimo somsebenzi wohlobo lwenani elingukhiye. Lapho, umugqa ngamunye ungaba nenombolo engafanele yamakholomu anamagama angafanele. Lokhu kungashiwo mayelana nezinhlelo ezifana ne-Hbase ne-Cassandra. Futhi yimibuzo yokuhlaziya enzima impela engeke isebenze ngokujwayelekile kuyo. Noma ungase ucabange ukuthi zisebenza kahle uma ungakaze ube nesipiliyoni nge-ClickHouse.

• Спасибо

  • Sawubona Sengivele nginentshisekelo enkulu kulesi sihloko, ngoba nginesistimu engaphansi yokuhlaziya. Kepha uma ngibheka i-ClickHouse, ngiba nomuzwa wokuthi i-ClickHouse ifaneleka kahle ekuhlaziyweni komcimbi, eguquguqukayo. Futhi uma ngidinga ukuhlaziya idatha yebhizinisi eminingi ngenqwaba yamatafula amakhulu, khona-ke i-ClickHouse, ngokuqonda kwami, ayingifanele kakhulu? Ikakhulukazi uma beshintsha. Ingabe lokhu kulungile noma zikhona izibonelo ezingakuphikisa lokhu?

  • Lokhu kulungile. Futhi lokhu kuyiqiniso mayelana nemininingwane eminingi yokuhlaziya eyisipesheli. Zenzelwe ukuthi kunetafula elilodwa noma amaningana amakhulu aguquguqukayo, futhi amaningi amancane ashintsha kancane. Okusho ukuthi, i-ClickHouse ayifani ne-Oracle, lapho ungabeka khona yonke into futhi wakhe imibuzo eyinkimbinkimbi kakhulu. Ukuze usebenzise i-ClickHouse ngempumelelo, udinga ukwakha uhlelo ngendlela esebenza kahle ku-ClickHouse. Okungukuthi, gwema ukujwayela ngokweqile, sebenzisa izichazamazwi, zama ukwenza ukuxhumana okude okumbalwa. Futhi uma uhlelo lwakhiwe ngale ndlela, khona-ke izinkinga zebhizinisi ezifanayo zingaxazululwa ku-ClickHouse ngokuphumelelayo kakhulu kune-database yendabuko yobudlelwane.

Siyabonga ngombiko! Nginombuzo mayelana necala lezezimali lakamuva. Babenezibalo. Kwakudingeka ukuthi kuqhathaniswe ukuthi zehla zenyuka kanjani. Futhi ngiyaqonda ukuthi wakhe isistimu yalezi zibalo? Uma kusasa, ake sithi, badinga omunye umbiko ngale datha, ingabe badinga ukwakha umdwebo futhi futhi balayishe idatha? Okungukuthi, ingabe uhlobo oluthile lokucubungula ukuze uthole isicelo?

Vele, lokhu kusebenzisa i-ClickHouse ngomsebenzi othize kakhulu. Kungaxazululwa ngokwesiko ngaphakathi kwe-Hadoop. Ku-Hadoop lona umsebenzi ofanelekile. Kodwa kuHadoop kuhamba kancane. Futhi umgomo wami ukukhombisa ukuthi i-ClickHouse ingakwazi ukuxazulula izinkinga ezivame ukuxazululwa ngezindlela ezihluke ngokuphelele, kodwa ngasikhathi sinye zikwenze ngempumelelo kakhulu. Lokhu kwenzelwa umsebenzi othile. Kuyacaca ukuthi uma kunenkinga ethi ayifane, ingaxazululeka ngendlela efanayo.

Kuyabonakala. Uthe kuthathe amahora angu-50 ukucubungula. Ingabe iqala kusukela ekuqaleni, lapho ulayisha idatha noma uthola imiphumela?

Yebo Yebo.

Kulungile ngiyabonga kakhulu.

Lokhu kuqoqo leseva engu-3.

Sanibonani! Siyabonga ngombiko! Konke kuyathakazelisa kakhulu. Angibuzi okuncane mayelana nokusebenza, kodwa mayelana nokusebenzisa i-ClickHouse ngokombono wokuzinza. Okusho ukuthi, ingabe ube nezinkinga futhi kwadingeka uzibuyisele? Ngabe iClickHouse iziphatha kanjani? Futhi kuke kwenzeka yini ukuthi ikhophi yakho nayo iphahlazeke? Isibonelo, sihlangabezane nenkinga nge-ClickHouse ngenkathi isadlula umkhawulo wayo futhi yawa.

Yiqiniso, azikho izinhlelo ezifanele. Futhi i-ClickHouse nayo inezinkinga zayo. Kodwa uke wezwa nge-Yandex.Metrica engasebenzi isikhathi eside? Cishe cha. Isebenza ngokuthembekile kusukela cishe ngo-2012-2013 ku-ClickHouse. Ngingasho okufanayo ngesipiliyoni sami. Asikaze sibe nokwehluleka okuphelele. Ezinye izinto zingenzeka, kepha azikaze zibaluleke ngokwanele ukuthi zingathinta kakhulu ibhizinisi. Akukaze kwenzeke lokhu ngaphambili. I-ClickHouse ithembekile impela futhi ayiphahlazeki ngokungahleliwe. Awudingi ukukhathazeka ngakho. Akuyona into eluhlaza. Lokhu kufakazelwe yizinkampani eziningi.

Sawubona! Uthe udinga ukucabanga ngokushesha nge-schema yedatha. Kuthiwani uma lokhu kwenzeka? Idatha yami ingena iphuma. Kudlula izinyanga eziyisithupha, futhi ngiyaqonda ukuthi angikwazi ukuphila kanje, ngidinga ukulayisha kabusha idatha futhi ngenze okuthile ngayo.

Lokhu kuncike, kunjalo, ohlelweni lwakho. Kunezindlela ezimbalwa zokwenza lokhu cishe ngokungami. Isibonelo, ungakha Ukubuka Okubalulekile lapho ungakha khona isakhiwo sedatha esihlukile uma singadwetshwa ngokuhlukile. Okusho ukuthi, uma kuvumela ukwenza imephu kusetshenziswa i-ClickHouse, okungukuthi ukukhipha ezinye izinto, ukushintsha ukhiye oyinhloko, ukushintsha ukwahlukanisa, khona-ke ungenza Ukubuka Okubalulekile. Lapho idatha yakho endala izobhalwa kabusha, emisha izobhalwa ngokuzenzakalelayo. Bese uvele ushintshele ekusebenziseni i-Materialized View, bese ushintsha irekhodi futhi ubulale ithebula elidala. Lena indlela ngokuvamile engami.

Спасибо.

Source: www.habr.com