Ithiyori kunye nokuziqhelanisa nokusebenzisa iClickHouse kwizicelo zokwenyani. Alexander Zaitsev (2018)

Ngaphandle kwento yokuba ngoku kukho idatha eninzi phantse kuyo yonke indawo, uhlalutyo lwedatha lusaqhelekile. Ayaziwa kwaye ade abi nakakhulu ukuwasebenzisa ngokufanelekileyo. Abaninzi bayaqhubeka "bedla i-cactus" kunye ne-MySQL okanye i-PostgreSQL, eyenzelwe ezinye iimeko, ihlupheke nge-NoSQL, okanye i-overpay kwizisombululo zorhwebo. I-ClickHouse itshintsha imigaqo yomdlalo kwaye inciphisa kakhulu umda wokungena kwihlabathi le-DBMS yohlalutyo.

Ingxelo evela kwi-BackEnd Conf 2018 kwaye ipapashwe ngemvume yesithethi.

Ndingubani mna kwaye kutheni ndithetha ngeClickHouse? Ndingumlawuli wophuhliso kwi-LifeStreet, esebenzisa i-ClickHouse. Kwakhona, ndingumseki we-Altinity. Liqabane laseYandex elikhuthaza iClickHouse kwaye inceda iYandex yenza iClickHouse iphumelele ngakumbi. Kwakhona ukulungele ukwabelana ngolwazi malunga ClickHouse.

Kwaye andiyena umzalwana kaPetya Zaitsev. Ndihlala ndibuzwa ngale nto. Hayi, asingobazalwana.

"Wonke umntu uyazi" ukuba ClickHouse:

• Ngesatya esikhulu,
• Ukhululekile kakhulu
• Isetyenziswa kwiYandex.

Okuncinci kuyaziwa ukuba zeziphi iinkampani kunye nendlela esetyenziswa ngayo.

Ndiza kukuxelela ukuba kutheni, phi kwaye iClickHouse isetyenziswe njani, ngaphandle kweYandex.

Ndiza kukuxelela indlela imisebenzi ethile esonjululwe ngayo ngoncedo lweClickHouse kwiinkampani ezahlukeneyo, zeziphi izixhobo zeClickHouse ongazisebenzisa kwimisebenzi yakho, kunye nendlela ezisetyenziswe ngayo kwiinkampani ezahlukeneyo.

Ndithathe imizekelo emithathu ebonisa i-ClickHouse kwii-engile ezahlukeneyo. Ndicinga ukuba iya kuba nomdla.

Umbuzo wokuqala ngulo: "Kutheni sifuna iClickHouse?". Kubonakala ngathi ngumbuzo ocacileyo, kodwa kukho iimpendulo ezingaphezu kwenye.

• Impendulo yokuqala yeyokusebenza. ClickHouse ikhawuleza kakhulu. I-Analytics kwiClickHouse nayo ikhawuleza kakhulu. Inokusetyenziswa rhoqo apho enye into icotha kakhulu okanye imbi kakhulu.
• Impendulo yesibini yindleko. Kwaye okokuqala, iindleko zokunyusa. Umzekelo, i-Vertica yidathabheyisi enkulu ngokupheleleyo. Isebenza kakuhle kakhulu ukuba awunazo ii-terabytes ezininzi zedatha. Kodwa xa kufikwa kumakhulu eeterabytes okanye iipetabytes, ixabiso lelayisenisi kunye nenkxaso iya kwisixa esibalulekileyo. Kwaye kuyabiza. Kwaye iClickHouse isimahla.
• Impendulo yesithathu yindleko yokusebenza. Le yindlela eyahlukileyo kancinane. I-RedShift yi-analog enkulu. KwiRedShift, unokwenza isigqibo ngokukhawuleza. Iza kusebenza kakuhle, kodwa kwangaxeshanye, iyure nganye, yonke imihla, kwaye nyanga zonke, uya kuhlawula iAmazon kakhulu, kuba le yinkonzo ebiza kakhulu. I-Google BigQuery nayo. Ukuba umntu usebenzile, ngoko uyazi ukuba ungaqhuba izicelo ezininzi kwaye ufumane ibhilikhi yekhulu leedola ngequbuliso.

I-ClickHouse ayinayo ezi ngxaki.

Isetyenziswa phi ngoku iClickHouse? Ukongeza kwiYandex, iClickHouse isetyenziswe kwiqela lamashishini ahlukeneyo kunye neenkampani.

• Okokuqala, oku kuhlalutyo lwesicelo sewebhu, oko kukuthi le yimeko yokusetyenziswa evela kwiYandex.
• Iinkampani ezininzi ze-AdTech zisebenzisa i-ClickHouse.
• Iinkampani ezininzi ezifuna ukuhlalutya iilog zentengiselwano ezivela kwimithombo eyahlukeneyo.
• Iinkampani ezininzi zisebenzisa i-ClickHouse ukujonga iinkuni zokhuseleko. Bazilayisha kwiClickHouse, benze iingxelo, kwaye bafumane iziphumo abazidingayo.
• Iinkampani ziqala ukuyisebenzisa kuhlalutyo lwemali, oko kukuthi ngokuthe ngcembe amashishini amakhulu nawo asondela kwiClickHouse.
• cloudflare. Ukuba umntu ulandela i-ClickHouse, ngoko mhlawumbi uvile igama lale nkampani. Lo ngomnye wabanegalelo elibalulekileyo kuluntu. Kwaye banofakelo olubi kakhulu lweClickHouse. Ngokomzekelo, benza i-Kafka Engine yeClickHouse.
• Iinkampani zonxibelelwano zaqala ukusebenzisa. Iinkampani ezininzi zisebenzisa i-ClickHouse njengobungqina kwingcamango okanye sele ikwimveliso.
• Enye inkampani isebenzisa iClickHouse ukujonga iinkqubo zokuvelisa. Bavavanya i-microcircuits, babhale iqela leeparamitha, kukho malunga neempawu ezingama-2. Kwaye ke bahlalutya ukuba umdlalo ulungile okanye umbi.
• Uhlalutyo lweBlockchain. Kukho inkampani yaseRashiya efana neBloxy.info. Olu luhlalutyo lwenethiwekhi ye-ethereum. Bakwenza oku nakwiClickHouse.

Kwaye ubungakanani abunamsebenzi. Kukho iinkampani ezininzi ezisebenzisa iseva enye encinci. Yaye uyabavumela ukuba bacombulule iingxaki zabo. Kwaye iinkampani ezininzi zisebenzisa amaqela amakhulu amaseva amaninzi okanye iiseva ezininzi.

Kwaye ukuba ujonga iirekhodi, ngoko:

• Yandex: Iiseva ze-500 +, zigcina iirekhodi ze-25 yezigidigidi ngosuku apho.
• LifeStreet: 60 abancedisi, malunga 75 billion iirekhodi ngosuku. Kukho iiseva ezimbalwa, iirekhodi ezininzi kuneYandex.
• I-CloudFlare: iiseva ezingama-36, zigcina iirekhodi ezingama-200 eebhiliyoni ngosuku. Baneseva ezimbalwa kwaye bagcina idatha eninzi.
• I-Bloomberg: iiseva ezili-102, malunga namangenelo etriliyoni ngosuku. Umbambi werekhodi.

Ngokwejografi, oku nako kuninzi. Le mephu ilapha ibonisa imephu yobushushu apho iClickHouse isetyenziswa khona kwihlabathi. IRashiya, iChina, iMelika zigqame ngokucacileyo apha. Kukho amazwe ambalwa aseYurophu. Kwaye kukho ama-4 amaqela.

Olu lucazululo oluthelekisayo, akukho mfuneko yokujonga amanani apheleleyo. Olu luhlalutyo lweendwendwe ezifunda izixhobo zolwimi lwesiNgesi kwiwebhusayithi ye-Altinity, kuba akukho zithetha isiRashiya apho. Kwaye iRashiya, i-Ukraine, iBelarus, oko kukuthi indawo ethetha isiRashiya yoluntu, ngaba baninzi abasebenzisi abaninzi. Emva koko kuza i-US neCanada. I-China inomdla kakhulu. Phantse kwakungekho China apho kwiinyanga ezintandathu ezidlulileyo, ngoku iChina sele iyigqithile iYurophu kwaye iyaqhubeka nokukhula. IYurophu yakudala nayo ayikude ngasemva, kwaye inkokeli ekusetyenzisweni kweClickHouse, ngokungaqhelekanga, iFransi.

Kutheni ndiyithetha yonke lento? Ukubonisa ukuba iClickHouse iba sisisombululo esiqhelekileyo sokuhlalutya idatha enkulu kwaye sele isetyenziswe kwiindawo ezininzi. Ukuba uyayisebenzisa, ukwindlela elungileyo. Ukuba awuyisebenzisi okwangoku, ngoko awukwazi ukwesaba ukuba uya kuba yedwa kwaye akukho mntu uya kukunceda, kuba abaninzi sele besenza oku.

Le yimizekelo yokusetyenziswa kwangempela kweClickHouse kwiinkampani ezininzi.

• Umzekelo wokuqala yinethiwekhi yentengiso: ukufuduka ukusuka kwiVertica ukuya kwiClickHouse. Kwaye ndiyazi iinkampani ezimbalwa eziye zatshintsha ukusuka kwi-Vertica okanye zikwinkqubo yokutshintsha.
• Umzekelo wesibini kukugcinwa kwentengiselwano kwiClickHouse. Lo ngumzekelo owakhiwe kwii-antipatterns. Yonke into engafanelekanga ukuba yenziwe kwiClickHouse kwiingcebiso zabaphuhlisi yenziwa apha. Kwaye yenziwe ngokufanelekileyo kangangokuba iyasebenza. Kwaye isebenza ngcono kakhulu kunesisombululo esiqhelekileyo sentengiselwano.
• Umzekelo wesithathu usasazwa kwikhompyuter kwiClickHouse. Kwakukho umbuzo malunga nendlela i-ClickHouse inokudibaniswa ngayo kwi-ecosystem ye-Hadoop. Ndiza kubonisa umzekelo wendlela inkampani eyenza ngayo into efana nemephu yokunciphisa i-container kwi-ClickHouse, ukugcina umkhondo wedatha yendawo, njl., Ukubala umsebenzi ongewona umncinci kakhulu.

• I-LifeStreet yinkampani ye-Ad Tech enabo bonke ubuchwepheshe obuza kunye nenethiwekhi yentengiso.
• Uzibandakanye nokwandisa intengiso, ukubhida ngokwenkqubo.
• Uninzi lwedatha: malunga ne-10 yeebhiliyoni zeziganeko ngosuku. Kwangaxeshanye, iziganeko apho zinokwahlulwa zibe ziziganeko ezingaphantsi eziliqela.
• Baninzi abathengi bale datha, kwaye aba ayingobantu kuphela, ngakumbi - ezi ziialgorithms ezahlukeneyo ezibandakanyekayo kwibhidi yenkqubo.

Inkampani ifike indlela ende kwaye inameva. Kwaye ndathetha ngayo kwi-HighLoad. Okokuqala, i-LifeStreet yasuka kwi-MySQL (ngokumisa okufutshane e-Oracle) ukuya kwi-Vertica. Kwaye ungafumana ibali malunga nayo.

Kwaye yonke into yayilungile kakhulu, kodwa ngokukhawuleza kwacaca ukuba idatha iyakhula kwaye iVertica iyabiza. Ngoko ke, kuye kwafunwa ezinye iindlela ezahlukahlukeneyo. Ezinye zazo zidweliswe apha. Kwaye eneneni, senze ubungqina bengcamango okanye ngamanye amaxesha uvavanyo lokusebenza malunga nazo zonke iinkcukacha ezigciniweyo ezifumaneka kwimarike ukusuka kwi-13 ukuya kwi-16th unyaka kwaye zifanelekile ngokumalunga nokusebenza. Kwaye ndathetha ngezinye zazo kwi-HighLoad.

Umsebenzi yayikukufuduka kwi-Vertica kwindawo yokuqala, kuba idatha yanda. Kwaye baye bakhula ngokuphawulekayo ukutyhubela iminyaka. Emva koko baya eshelufini, kodwa nangona kunjalo. Kwaye ukubikezela oku kukhula, iimfuno zeshishini kwinani ledatha apho uhlobo oluthile lohlalutyo olufunekayo lwenziwe, kwacaca ukuba iipetabytes ziza kuxutyushwa ngokukhawuleza. Kwaye ukuhlawula iipetabytes sele kubiza kakhulu, ngoko besikhangela enye indawo apho siya khona.

Masiye phi? Kwaye ixesha elide kwakungacacanga kwaphela apho kuya khona, kuba kwelinye icala kukho i-database yezorhwebo, zibonakala zisebenza kakuhle. Abanye basebenza phantse kunye neVertica, abanye bambi ngakumbi. Kodwa zonke ziyabiza, akukho nto inexabiso eliphantsi kwaye ingcono ayifumanekanga.

Ngakolunye uhlangothi, kukho izisombululo zomthombo ovulekileyo, ezingekho zininzi kakhulu, oko kukuthi uhlalutyo, zingabalwa kwiminwe. Kwaye zisimahla okanye zitshiphu, kodwa zicotha. Kwaye bahlala beswele umsebenzi oyimfuneko kunye noluncedo.

Kwaye kwakungekho nto yokudibanisa okulungileyo kwi-database yorhwebo kunye nayo yonke into ekhululekile ekhoyo kumthombo ovulekileyo.

Kwakungekho nto de, ngokungalindelekanga, i-Yandex yakhupha i-ClickHouse, njengomlingo ovela kwisigqoko, njengomvundla. Kwaye kwaba sisigqibo esingalindelekanga, basabuza umbuzo othi: "Kutheni?", Kodwa nangona kunjalo.

Kwaye kwangoko ehlobo lika-2016, saqala ukujonga ukuba yintoni na iClickHouse. Kwaye kwavela ukuba ngamanye amaxesha inokukhawuleza kuneVertica. Sivavanye iimeko ezahlukeneyo kwizicelo ezahlukeneyo. Kwaye ukuba umbuzo usebenzise itheyibhile enye kuphela, oko kukuthi, ngaphandle kokudibanisa (dibanisa), ngoko iClickHouse ikhawuleze kabini njengeVertica.

Andizange ndivila kwaye ndajonga iimvavanyo zeYandex ngolunye usuku. Iyafana nalapho: ClickHouse ikhawuleza kabini njengeVertica, ke bahlala bethetha ngayo.

Kodwa ukuba kukho ukudibanisa kwimibuzo, ke yonke into ijika ingabikho ngokungathandabuzekiyo. Kwaye iClickHouse inokucotha kabini njengeVertica. Kwaye ukuba usilungisa kancinane isicelo kwaye usibhale kwakhona, ngoko baphantse balingane. Akukubanga. Kwaye simahla.

Kwaye emva kokufumana iziphumo zovavanyo, kwaye ujonge kuyo kwii-engile ezahlukeneyo, i-LifeStreet iye kwi-ClickHouse.

Lo ngunyaka we-16, ndiyanikhumbuza. Kwaba ngathi yintlekisa ngeempuku ezikhalayo zaza zaziqweba, kodwa zaqhubeka zisitya i-cactus. Kwaye oku kwachazwa ngokweenkcukacha, kukho ividiyo malunga nale nto, njl.

Ngoko ke, andiyi kuthetha ngokubanzi ngayo, ndiza kuthetha kuphela ngeziphumo kunye nezinto ezimbalwa ezinomdla endingazange ndithethe ngazo ngoko.

Iziphumo zezi:

• Ukufuduka okuphumelelayo kunye nangaphezulu konyaka inkqubo sele isebenza kwimveliso.
• Ukuvelisa kunye nokuguquguquka kuye kwanda. Kwiirekhodi ezili-10 zeebhiliyoni esinokukwazi ukuzigcina ngosuku kwaye emva koko ixesha elifutshane, i-LifeStreet ngoku igcina iirekhodi ze-75 yezigidigidi ngosuku kwaye inokukwenza oku kwiinyanga ezi-3 okanye ngaphezulu. Ukuba ubala encotsheni, ke le nto ifikelela kwisigidi seziganeko ngomzuzwana. Imibuzo engaphezu kwesigidi ye-SQL ngosuku ifika kule nkqubo, ubukhulu becala ivela kwiirobhothi ezahlukeneyo.
• Ngaphandle kwento yokuba iiseva ezininzi zisetyenziselwe iClickHouse kuneVertica, nazo zigcinwe kwihardware, kuba iidiski ze-SAS ezibizayo zazisetyenziswa kwiVertica. ClickHouse isetyenziswe SATA. Kwaye kutheni? Kuba kwi-Vertica faka i-synchronous. Kwaye ungqamaniso lufuna ukuba iidiski zingacothi kakhulu, kwaye inethiwekhi ayicothi kakhulu, oko kukuthi, umsebenzi obiza kakhulu. Kwaye kwiClickHouse faka ayisynchronous. Ngaphezu koko, unokuhlala ubhala yonke into kwindawo, akukho zindleko ezongezelelweyo zale nto, ngoko ke idatha ingafakwa kwi-ClickHouse ngokukhawuleza kakhulu kune-Vertika, nakwii-drives ezicothayo. Kwaye ukufunda kuyafana. Ukufunda kwi-SATA, ukuba bakwi-RAID, konke oku kukhawuleza ngokwaneleyo.
• Ayikhawulelwanga yilayisenisi, oko kukuthi i-3 petabytes yedatha kwiiseva ezingama-60 (iiseva ezingama-20 ziyi-replica enye) kunye neerekhodi ze-6 trillion kwiinyani kunye nokuhlanganiswa. Akukho nto inje inokufumaneka eVertica.

Ngoku ndiphendukela kwizinto eziphathekayo kulo mzekelo.

• Eyokuqala yinkqubo esebenzayo. Okuninzi kuxhomekeke kwischema.
• Okwesibini kukuvelisa iSQL esebenzayo.

Umbuzo oqhelekileyo we-OLAP kukukhetha. Eminye imihlathi iya kwiqela nge, eminye yemiqolo iya kuhlanganisa imisebenzi. Kukho apho, enokuthi imelwe njengeqhekeza letyhubhu. Iqela xa lilonke linokucingwa njengentelekelelo. Yiyo loo nto ibizwa ngokuba yi-multivariate data analysis.

Kwaye kaninzi oku kumodareyithwa ngendlela yeskimu seenkwenkwezi, xa kukho inyani esembindini kunye neempawu zale nyaniso ecaleni, ecaleni kwemitha.

Kwaye ngokumalunga noyilo lomzimba, ukuba lungena njani etafileni, ngokuqhelekileyo benza umboniso oqhelekileyo. Uyakwazi ukwenza i-denormalize, kodwa iyabiza kwidiski kwaye ayisebenzi kakuhle kwimibuzo. Ke ngoko, bakholisa ukwenza umelo oluqhelekileyo, o.t. itafile yenyani kunye neetafile ezininzi, ezininzi zomda.

Kodwa ayisebenzi kakuhle kwiClickHouse. Kukho izizathu ezibini:

• Eyokuqala kungenxa yokuba i-ClickHouse ayinamakhonkco amahle kakhulu, okt kukho ukujoyina, kodwa kubi. Ngelixa embi.
• Okwesibini kukuba iitafile azihlaziywa. Ngokuqhelekileyo kula macwecwe, ajikeleze i-star-circuit, kukho into efuna ukutshintshwa. Umzekelo, igama lomthengi, igama lenkampani, njl. Kwaye ayisebenzi.

Kwaye kukho indlela yokuphuma kule ClickHouse. nokuba zimbini:

• Eyokuqala kukusetyenziswa kwezichazi-magama. Izichazi-magama zangaphandle yinto enceda i-99% ukusombulula ingxaki nge-schema yeenkwenkwezi, kunye nohlaziyo kunye nokunye.
• Okwesibini kukusetyenziswa kwee-arrays. Ii-arrays zikwanceda ukulahla ukudibanisa kunye neengxaki ngokuqhelanisa.

• Akudingeki ukujoyina.
• Iyaphuculwa. Ukususela ngo-Matshi 2018, ithuba elingabhalwanga liye lavela (awuyi kufumana oku kumaxwebhu) ukuhlaziya izichazi-magama ngokuyinxenye, oko kukuthi loo mangeno atshintshileyo. Enyanisweni, ifana netafile.
• Ihlala ikwinkumbulo, ngoko ukudibanisa nesichazi-magama sisebenza ngokukhawuleza kunokuba ibiyitafile ekwidiski kwaye ayikabi yinyani yokuba ikwindawo efihlakeleyo, kusenokwenzeka ukuba akunjalo.

• Awudingi kudibaniswa.
• Olu luphawu oluhlangeneyo lwe-1 ukuya kwabaninzi.
• Kwaye ngokoluvo lwam, uluhlu lwenzelwe iigeeks. Le yimisebenzi ye-lambda njalo njalo.

Oku akusiyo eyamagama abomvu. Lo ngumsebenzi onamandla kakhulu okuvumela ukuba wenze izinto ezininzi ngendlela elula kakhulu nenhle.

Imizekelo eqhelekileyo enceda ukusombulula uluhlu. Le mizekelo ilula kwaye icace ngokwaneleyo:

• Phendla ngeethegi. Ukuba unee-hashtag apho kwaye ufuna ukufumana ezinye izithuba nge-hashtag.
• Phendla ngokwexabiso eliphambili. Kukwakho ezinye iimpawu ezinexabiso.
• Ukugcina uluhlu lwamaqhosha ekufuneka uwaguqulele kwenye into.

Yonke le misebenzi inokusombulula ngaphandle koluhlu. Iithegi zingafakwa kumgca othile kwaye zikhethwe ngokuchazwa okuqhelekileyo okanye kwitafile eyahlukileyo, kodwa ke kufuneka wenze ukujoyina.

Kwaye kwiClickHouse, awudingi ukwenza nantoni na, kwanele ukuchaza uluhlu lweentambo zee-hashtag okanye ukwenza isakhiwo esifakwe kwi-key-value systems.

Isakhiwo esifakwe kwindlwane isenokungabi lelona gama lilungileyo. Ezi zintlu ezimbini ezinendima efanayo kwigama kunye neempawu ezithile ezinxulumeneyo.

Kwaye kulula kakhulu ukukhangela ngethegi. Yiba nomdlalo has, ejonga ukuba uluhlu luqulathe isiqalelo. Wonke umntu, ufumene onke amangenelo anxulumene nenkomfa yethu.

Ukukhangela nge-subid kunzima ngakumbi. Kufuneka siqale sifumane isalathiso sesitshixo, kwaye emva koko sithathe isiqalelo esinesalathiso kwaye sijonge ukuba eli xabiso yilento siyifunayo. Nangona kunjalo, ilula kakhulu kwaye idibene.

Intetho eqhelekileyo ongathanda ukuyibhala ukuba uyigcine yonke kumgca omnye, iya kuba, okokuqala, ibhideke. Kwaye, okwesibini, isebenze ixesha elide kuneendlela ezimbini.

Omnye umzekelo. Unoluhlu apho ugcina khona i-ID. Kwaye unokuziguqulela kumagama. Umsebenzi arrayMap. Lo ngumsebenzi we lambda eqhelekileyo. Udlula lambda intetho apho. Kwaye ukhupha ixabiso legama le-ID nganye kwisichazi-magama.

Ukukhangela kunokwenziwa ngendlela efanayo. Umsebenzi wesivisa uyagqithiswa ojonga ukuba iziqalelo zidibana ngantoni.

Ezi zinto zilula kakhulu isekethe kwaye zisombulule iqela leengxaki.

Kodwa ingxaki elandelayo esijongene nayo, nendingathanda ukuyikhankanya, yimibuzo esebenzayo.

• ClickHouse ayinaso isicwangcisi sombuzo. Akunjalo.
• Nangona kunjalo, imibuzo enzima isafuna ukucwangciswa. Kweziphi iimeko?
• Ukuba kukho izidibaniselwano ezininzi kumbuzo, uzisonge ngokukhetha abancinci. Kunye nolandelelwano ababulawa ngalo.
• Kwaye okwesibini - ukuba isicelo sisasazwa. Kuba kumbuzo osasaziweyo, kuphela ukhetho lwangaphakathi olusezantsi luphunyeziweyo, kwaye yonke enye into idluliselwe kumncedisi omnye oqhagamshele kuyo kwaye uqhutywe apho. Ke ngoko, ukuba usasaze imibuzo ngokudibanisa okuninzi (joyina), kufuneka ukhethe iodolo.

Kwaye nakwiimeko ezilula, ngamanye amaxesha kuyafuneka ukuba wenze umsebenzi womcwangcisi kwaye uphinde ubhale imibuzo encinci.

Nanku umzekelo. Kwicala lasekhohlo kukho umbuzo obonisa amazwe ama-5 aphezulu. Kwaye kuthatha imizuzwana eyi-2,5, ngokombono wam. Kwaye kwicala lasekunene, umbuzo ofanayo, kodwa ubhalwe kancinci. Endaweni yokudibanisa ngomtya, saqala ukuhlanganisana ngesitshixo (int). Kwaye ikhawuleza. Kwaye emva koko siqhagamshele isichazi-magama kwisiphumo. Esikhundleni semizuzwana eyi-2,5, isicelo sithatha imizuzwana eyi-1,5. Oku Kulungile.

Umzekelo ofanayo nezihluzo zokubhala kwakhona. Nasi isicelo seRashiya. Ibaleka imizuzwana emi-5. Ukuba siyibhala kwakhona ngendlela yokuba siphinde sithelekise kwakhona kungekhona umtya, kodwa amanani anesethi yezo zitshixo ezihambelana neRashiya, ngoko kuya kukhawuleza kakhulu.

Maninzi amaqhinga anjalo. Kwaye zikuvumela ukuba ukhawuleze ngokukhawuleza imibuzo ocinga ukuba sele ibaleka ngokukhawuleza, okanye, ngokuchaseneyo, ibaleka kancinci. Zingenziwa ngokukhawuleza.

• Ubuninzi bomsebenzi kwimodi yokusabalalisa.
• Ukuhlela ngeentlobo ezincinci, njengoko ndenze nge-ints.
• Ukuba kukho naziphi na ukujoyina (joyina), izichazi-magama, ngoko ke kungcono ukuzenza njengendlela yokugqibela, xa sele unayo idatha ubuncinane ngokuyinxenye, ngoko umsebenzi wokujoyina okanye umnxeba wesichazi-magama uya kubizwa ngokuba ngamaxesha amancinci kwaye kuya kukhawuleza. .
• Ukutshintsha izihluzi.

Kukho ezinye iindlela, kwaye hayi nje ezo ndibonise. Kwaye bonke ngamanye amaxesha banokukhawulezisa ngokukhawuleza ukwenziwa kwemibuzo.

Masiqhubele phambili kumzekelo olandelayo. Inkampani X evela e-USA. Ingaba wenzantoni?

Kwakukho umsebenzi:

• Udibaniso olungaxhunyiwe kwi-intanethi yeentengiselwano zentengiso.
• Ukulinganisa iimodeli ezahlukeneyo zokubopha.

Ithini imeko?

Isivakashi esiqhelekileyo siza kwisiza, umzekelo, amaxesha angama-20 ngenyanga ukusuka kwiintengiso ezahlukeneyo, okanye njengokuba ngamanye amaxesha kuza ngaphandle kweentengiso, kuba ukhumbula le ndawo. Ujonga ezinye iimveliso, uzibeke ebhaskithini, uzikhuphe ebhaskithini. Kwaye, ekugqibeleni, into iyathenga.

Imibuzo enengqiqo: "Ngubani omele ahlawule intengiso, ukuba kuyimfuneko?" kunye "Yeyiphi intengiso eyamphembelelayo, ukuba ikhona?". Oko kukuthi, kwakutheni ukuze athenge nendlela yokufumana abantu abafana nalo mntu ukuba athenge naye?

Ukuze ucombulule le ngxaki, kufuneka udibanise iziganeko ezenzeka kwiwebhusayithi ngendlela efanelekileyo, oko kukuthi, ngandlela-thile yakha uxhumano phakathi kwabo. Emva koko zithunyelwa kuhlalutyo kwi-DWH. Kwaye ngokusekwe kolu hlalutyo, yakha iimodeli zokuba ngubani kwaye zeziphi iintengiso eziza kubonisa.

Intengiselwano yentengiso yiseti yeziganeko ezinxulumene nomsebenzisi eziqala ekuboniseni intengiso, emva koko kwenzeka into, mhlawumbi kuthengwe, kwaye emva koko kunokuthengwa ngaphakathi kokuthenga. Ngokomzekelo, ukuba oku kuyisicelo seselula okanye umdlalo weselula, ngoko ngokuqhelekileyo ukufakwa kwesicelo kwenzeka simahla, kwaye ukuba kukho into eyenziwa apho, imali ingafuneka kule nto. Kwaye okukhona umntu echitha imali kwisicelo, kokukhona kuxabiseke ngakumbi. Kodwa kule nto kufuneka udibanise yonke into.

Kukho iimodeli ezininzi ezibophelelayo.

Ezona zidumileyo zezi:

• Intsebenziswano yokugqibela, apho unxibelelwano lunokuba lucofa okanye luphawu.
• Intsebenziswano yokuQala, o.t. into yokuqala ezise umntu kwindawo.
• Indibaniselwano yomgca - konke ngokulinganayo.
• Ukuthotywa.
• Kwaye nangokunjalo.

Kwaye kwakusebenza njani konke kwasekuqaleni? Kwakukho iRuntime kunye noCassandra. I-Cassandra yayisetyenziselwa ukugcinwa kwentengiselwano, oko kukuthi zonke iintengiselwano ezinxulumene nazo zigcinwe kuyo. Kwaye xa isiganeko esithile sifika kwi-Runtime, umzekelo, ukubonisa iphepha okanye enye into, emva koko isicelo senziwe kuCassandra - ngaba kukho umntu onjalo okanye akunjalo. Emva koko iintengiselwano ezinxulumene nayo zafunyanwa. Kwaye unxibelelwano lwenziwa.

Kwaye ukuba unethamsanqa ukuba isicelo sine id transaction, ngoko kulula. Kodwa ngokuqhelekileyo akukho luck. Ngoko ke, kwakuyimfuneko ukufumana ukuthengiselana kokugqibela okanye ukuthengiselana ngokucofa kokugqibela, njl.

Kwaye konke kusebenze kakuhle kakhulu nje ukuba ukubophelela bekukucofa okokugqibela. Kuba kukho, yithi, ucofa izigidi ezili-10 ngosuku, izigidi ezingama-300 ngenyanga, ukuba sibeka ifestile inyanga. Kwaye ekubeni eCassandra kufuneka yonke imemori ukuze iqhube ngokukhawuleza, kuba i-Runtime idinga ukuphendula ngokukhawuleza, ithathe malunga ne-10-15 amaseva.

Kwaye xa babefuna ukudibanisa intengiselwano kwisiboniso, ngokukhawuleza kwabonakala kungemnandi. Kwaye kutheni? Kunokubonwa ukuba amaxesha angama-30 ngaphezulu kweziganeko kufuneka zigcinwe. Kwaye, ngokufanelekileyo, udinga amaxesha angama-30 ngaphezulu kweeseva. Kwaye kuvela ukuba lo luhlobo oluthile lwe-astronomical figure. Ukugcina ukuya kuthi ga kwiiseva ezingama-500 ukuze wenze uqhakamshelwano, nangona kukho iiseva ezimbalwa kakhulu kwi-Runtime, ke olu luhlobo oluthile lomfanekiso ongalunganga. Baqala ke ukucinga ukuba mabathini na.

Kwaye saya eClickHouse. Kwaye ungayenza njani kwiClickHouse? Ekuboneni kokuqala, kubonakala ngathi le yisethi yeepatheni ezichasayo.

• Intengiselwano iyakhula, sidibanisa imicimbi emininzi kuyo, okt iyaguquguquka, kwaye iClickHouse ayisebenzi kakuhle kakhulu ngezinto eziguquguqukayo.
• Xa undwendwe lusiza kuthi, kufuneka sikhuphe iintengiselwano zakhe ngesitshixo, ngesazisi sotyelelo lwakhe. Oku kwakhona umbuzo point, abakwenzi oko ClickHouse. Ngesiqhelo iClickHouse inee…scan ezinkulu, kodwa apha kufuneka sifumane iirekhodi ezithile. Kwakhona i-antipattern.
• Ukongezelela, ukuthengiselana kwaku-json, kodwa abazange bafune ukuyibhala kwakhona, ngoko babefuna ukugcina i-json ngendlela engacwangciswanga, kwaye ukuba kuyimfuneko, khupha into ethile kuyo. Kwaye oku kwakhona antipattern.

Oko kukuthi, isethi yee-antipatterns.

Kodwa nangona kunjalo kwavela ukwenza inkqubo esebenza kakuhle kakhulu.

Kwenziwa ntoni? I-ClickHouse yavela, apho iinkuni zaphoswa khona, zahlulwe zibe ziirekhodi. Inkonzo enxulumeneyo ibonakala ifumene iilog kwiClickHouse. Emva koko, kwingeniso nganye, nge-id yokundwendwela, ndifumene iintengiselwano ezinokuthi zingakhange zisetyenzwe okwangoku kunye ne-snapshots, o.k.intengiselwano esele iqhagamshelwe, oko kukuthi isiphumo somsebenzi wangaphambili. Sele ndenze ingqiqo kubo, ndakhetha ukuthengiselana okuchanekileyo, ndadibanisa imicimbi emitsha. Iloga kwakhona. Ilog ibuyele kwiClickHouse, okt yinkqubo ehlala ijikeleza. Kwaye ngaphandle koko, ndiye kwi-DWH ukuya kuyihlalutya apho.

Kwakule fomu ukuba ayizange isebenze kakuhle. Kwaye ukwenza kube lula kwiClickHouse, xa kukho isicelo nge-id yokundwendwela, badibanisa ezi zicelo kwiibhloko ze-ID zokutyelela eziyi-1-000 kwaye bakhupha zonke iintengiselwano ze-2-000 yabantu. Kwaye ke konke kwasebenza.

Ukuba ujonga ngaphakathi kweClickHouse, kukho iitafile ezi-3 kuphela ezisebenzela konke oku.

Itafile yokuqala apho iilogi zilayishwa khona, kwaye iilogi zilayishwa phantse ngaphandle kokucubungula.

Itafile yesibini. Ngombono wezinto ezibonakalayo, iziganeko ezingeke zibalwe, oko kukuthi, ezingahambelaniyo, zalunywa ngaphandle kwezi zigodo. Kwaye ngombono owenziweyo, iintengiselwano zakhutshwa kwezi zigodo ukwakha umfanekiso omfutshane. Oko kukuthi, umbono okhethekileyo wezinto ezibonakalayo wakha i-snapshot, oko kukuthi imo yokugqibela eqokelelweyo yentengiselwano.

Nantsi isicatshulwa esibhalwe kwi-SQL. Ndingathanda ukuhlomla kwizinto ezimbalwa ezibalulekileyo kuyo.

Into yokuqala ebalulekileyo kukukwazi ukukhupha iikholomu kunye namasimi ukusuka kwi-json kwi-ClickHouse. Oko kukuthi, iClickHouse ineendlela ezithile zokusebenza kunye nejson. Bakudala kakhulu, kakhulu.

visitParamExtractInt ikuvumela ukuba ukhuphe iimpawu kwi-json, oko kukuthi ukubetha kokuqala kuyasebenza. Kwaye ngale ndlela unokukhupha i-id yentengiselwano okanye undwendwele i-id. Ngeli xesha.

Okwesibini, intsimi enenkohliso isetyenziswa apha. Ingaba ithetha ntoni? Oku kuthetha ukuba awukwazi ukuyifaka etafileni, oko kukuthi ayifakwanga, ibalwa kwaye igcinwe xa ifakiwe. Xa uncamathisela, iClickHouse ikwenzela umsebenzi. Kwaye into oyifunayo kamva sele ikhutshiwe kwi-json.

Kule meko, imbonakalo ebonakalayo yenzelwe imiqolo ekrwada. Kwaye itafile yokuqala enezigodo ezikrwada isetyenziswa nje. Kwaye wenza ntoni? Okokuqala, iyatshintsha indlela yokuhlela, o.k.t. ukuhlela ngoku kuhamba nge-id yotyelelo, kuba kufuneka sikhuphe ngokukhawuleza intengiselwano yakhe kumntu othile.

Into yesibini ebalulekileyo yi-index_granularity. Ukuba ubonile i-MergeTree, ihlala i-8 ngokungagqibekanga index_granularity. Yintoni? Esi sisalathisi sokwahlulwa kweparamitha. KwiClickHouse isalathiso sinqabile, ayisokuze ibonise lonke ungeno. Ikwenza oku rhoqo ngo-192. Kwaye oku kulungile xa kufuneka ukubalwa kwedatha eninzi, kodwa imbi xa incinci, kuba kukho i-overhead enkulu. Kwaye ukuba sinciphisa i-index granularity, ngoko sinciphisa i-overhead. Ayinakuncitshiswa ibe nye, kuba isenokungabikho ngokwaneleyo inkumbulo. Isalathiso sihlala sigcinwe kwinkumbulo.

I-Snapshot ikwasebenzisa ezinye izinto ezinomdla zeClickHouse.

Okokuqala, yi-AggregatingMergeTree. Kwaye iAggregatingMergeTree igcina i-argMax, o.k.t. le yimo yentengiselwano ehambelana nesitampu sexesha sokugqibela. Iintengiselwano zenziwa ngalo lonke ixesha kundwendwe olunikiweyo. Kwaye kwimeko yokugqibela yale ntengiselwano, songeze umsitho kwaye sinelizwe elitsha. Ibethe iClickHouse kwakhona. Kwaye nge-argMax kule mbono yenziweyo, sinokuhlala sifumana imeko yangoku.

• Isibophelelo "siqhawukene" ukusuka kwiXesha lokuSebenza.
• Ukuya kuthi ga kwi-3 yeebhiliyoni zentengiselwano ngenyanga zigcinwe kwaye ziqwalaselwe. Lo myalelo wobukhulu ngaphezu kwakunjalo eCassandra, o.k.e. kwinkqubo eqhelekileyo yentengiselwano.
• Iqela le-2x5 ClickHouse abancedisi. Iiseva ezi-5 kwaye iseva nganye ine-replica. Oku kungaphantsi kunokuba kwakunjalo eCassandra ukuze wenze uphawu lokucofa olusekwe, kwaye apha sinoluvo olusekwe. Oko kukuthi, endaweni yokwandisa inani lamaseva ngamaxesha angama-30, bakwazile ukuwanciphisa.

Kwaye umzekelo wokugqibela yinkampani yezemali ye-Y, eyahlalutya ulungelelwaniso lotshintsho kumaxabiso esitokhwe.

Kwaye umsebenzi waba:

• Kukho malunga nezabelo ezingama-5.
• Iingcaphuno rhoqo nge-100 millisecond ziyaziwa.
• Idatha iqokelelwe kwiminyaka eyi-10. Ngokucacileyo, kwezinye iinkampani ngaphezulu, kwezinye ngaphantsi.
• Kukho malunga ne-100 leebhiliyoni zemigca iyonke.

Kwaye kwakuyimfuneko ukubala ulungelelwaniso lweenguqu.

Nazi izitokhwe ezibini kunye neengcaphuno zazo. Ukuba enye inyuke kwaye enye inyuke, ke oku kukuhambelana kwe-positive, o.k.t. enye inyuka kwaye enye inyuke. Ukuba enye inyuka, njengasekupheleni kwegrafu, kwaye enye ihla, ke oku kuhambelana okungalunganga, oko kukuthi xa enye iphakama, enye iyawa.

Ukuhlalutya olu tshintsho lufanayo, umntu unokwenza izibikezelo kwimarike yezemali.

Kodwa umsebenzi unzima. Kwenziwa ntoni oku? Sine-100 yeebhiliyoni zeerekhodi ezine: ixesha, isitokhwe, kunye nexabiso. Kufuneka sibale okokuqala 100 billion amaxesha ukubaleka Difference ukusuka algorithm ixabiso. I-RunningDifference ngumsebenzi kwiClickHouse obala ngokulandelelana umahluko phakathi kwemitya emibini.

Kwaye emva koko, kufuneka ubale ulungelelwaniso, kwaye ukudibanisa kufuneka kubalwe kwisibini ngasinye. Kwizabelo ze-5, izibini ziyi-000 yezigidi. Kwaye oku kuninzi, oko kukuthi amaxesha angama-12,5 kuyimfuneko ukubala nje umsebenzi wokudibanisa onjalo.

Kwaye ukuba umntu ulibele, ke u-͞x kunye no- ͞y yi-checkmate. ulindelo lwesampulu. Oko kukuthi, akufuneki ukubala kuphela iingcambu kunye nezibalo, kodwa kunye nesinye isibalo ngaphakathi kwezi zibalo. Ubalo oluninzi kufuneka lwenziwe amaxesha ezigidi ezili-12,5, kwaye ludityaniswe ngeeyure. Kwakhona sineeyure ezininzi. Kwaye kufuneka uyenze kwimizuzwana engama-60. Yintlekisa.

Kwakufuneka ukuba ube nexesha ubuncinci ngandlela thile, kuba konke oku kusebenze kakhulu, kancinci kakhulu ngaphambi kokuba iClickHouse ifike.

Bazama ukuyibala eHadoop, eSpark, eGreenplum. Kwaye konke oku kwakucotha kakhulu okanye kubiza kakhulu. Oko kukuthi, kwakunokwenzeka ukubala ngandlela-thile, kodwa ke kwakubiza.

Kwaye emva koko kwafika uClickHouse kwaye izinto zaba ngcono kakhulu.

Ndiyakukhumbuza ukuba sinengxaki yendawo yedatha, kuba ulungelelwaniso alukwazi ukwenziwa lwasekhaya. Asikwazi ukubeka enye idatha kwiseva enye, enye kwenye kwaye sibale, kufuneka sibe nayo yonke idatha kuyo yonke indawo.

Benza ntoni? Ekuqaleni, idatha ifakwe kwindawo. Umncedisi ngamnye ugcina idatha kwixabiso leseti ethile yezabelo. Kwaye azihambelani. Ngoko ke, kunokwenzeka ukubala i-logBuyisela ngokuhambelana kunye nokuzimela, konke oku kwenzeka ngoku ngokuhambelanayo kunye nokusabalalisa.

Emva koko siye sagqiba ekubeni sinciphise ezi datha, ngelixa singaphulukani novakaliso. Ukunciphisa ukusebenzisa ii-arrays, oko kukuthi kwixesha ngalinye, yenza uluhlu lweempahla kunye noluhlu lwamaxabiso. Ke ngoko, kuthatha indawo encinci kakhulu yedatha. Kwaye zilula kancinci ukusebenza nazo. Le yimisebenzi ephantse ifane, o.k.t. ngokuyinxenye sifunda ngokunxuseneyo kwaye sibhalele kumncedisi.

Emva koko, inokuphinda iphindwe. Unobumba "r" uthetha ukuba siyiphindaphinda le datha. Oko kukuthi, sinedatha efanayo kuzo zontathu iiseva - ezi zii-arrays.

Kwaye ke kunye neskripthi esikhethekileyo kule setethi ye-12,5 yezigidi ezidityanisiweyo kufuneka zibalwe, unokwenza iipakethi. Oko kukuthi, 2 imisebenzi kunye 500 izibini zolungelelaniso. Kwaye lo msebenzi uza kubalwa kumncedisi othile weClickHouse. Unayo yonke idatha, kuba idatha iyafana kwaye unokuyibala ngokulandelelana.

Kwakhona, oku kubonakala ngathi. Okokuqala, sinayo yonke idatha kwesi sakhiwo: ixesha, izabelo, ixabiso. Emva koko sibale i-logReturn, oko kukuthi idatha yesakhiwo esifanayo, kodwa endaweni yexabiso esele sinelogReturn. Emva koko zenziwa ngokutsha, o.k.t. sifumene ixesha kunye neqelaArray kwizitokhwe kunye namaxabiso. Iphindaphindwe. Kwaye emva koko, siye savelisa iqela lemisebenzi kwaye sondla kwiClickHouse ukuze ibale. Kwaye iyasebenza.

Ngobungqina bengqiqo, umsebenzi wawungumsebenzi ongaphantsi, oko kukuthi, idatha encinci yathathwa. Kwaye abathathu kuphela abancedisi.

Ezi zigaba zimbini zokuqala: ukubala i-Log_return kunye nokusonga ngoluhlu kuthathe malunga neyure.

Kwaye ukubalwa kolungelelwaniso malunga neeyure ezingama-50. Kodwa iiyure ezingama-50 azonelanga, kuba bebedla ngokusebenza iiveki. Yaba yimpumelelo enkulu. Kwaye ukuba uyabala, amaxesha angama-70 ngesekhondi yonke into ibalwe kweli qela.

Kodwa eyona nto ibaluleke kakhulu kukuba le nkqubo ingokoqobo ngaphandle kweebhotile, oko kukuthi, ilinganisa ngokulinganayo. Kwaye bayijonga. Uyinyuse ngempumelelo.

• Iskimu esifanelekileyo sisiqingatha sedabi. Kwaye iskimu esifanelekileyo kukusetyenziswa kwazo zonke itekhnoloji eziyimfuneko zeClickHouse.
• I-Summing/AggregatingMergeTrees ziitekhnoloji ezikuvumela ukuba udibanise okanye uthathele ingqalelo umfanekiso welizwe njengemeko ekhethekileyo. Kwaye yenza lula kakhulu izinto ezininzi.
• Iimbonakalo ezenziweyo zikuvumela ukuba ugqithe kumda wesalathiso esinye. Mhlawumbi andizange ndithethe ngokucacileyo, kodwa xa silayisha izigodo, izigodo eziluhlaza zazisetafileni kunye nesalathisi esinye, kwaye iimpawu zeempawu zazisetafileni, oko kukuthi idatha efanayo, ihluziwe kuphela, kodwa isalathisi sasiphelele ngokupheleleyo. abanye. Kubonakala ngathi yidatha efanayo, kodwa ukuhlelwa okwahlukileyo. Kwaye iiMbono zeMathiriyeli zikuvumela ukuba, ukuba uyayifuna, udlule umda weClickHouse onjalo.
• Nciphisa isalathisi granularity kwimibuzo yeengongoma.
• Kwaye usasaze idatha ngobuchule, zama ukubeka idatha ngaphakathi kwiseva kangangoko kunokwenzeka. Kwaye zama ukuqinisekisa ukuba izicelo zikwasebenzisa ukwenziwa kwendawo apho kunokwenzeka kangangoko.

Kwaye ukushwankathela le ntetho imfutshane, sinokuthi i-ClickHouse ngoku ibambe ngokuqinileyo ummandla weenkcukacha zorhwebo zombini kunye nedatha evulekileyo yomthombo, oko kukuthi, ngokukodwa uhlalutyo. Ungena ngokugqibeleleyo kulo mhlaba. Kwaye ngaphezu koko, iqala kancinci ukuxina abanye, kuba xa uneClickHouse, awudingi i-InfiniDB. IVertika isenokungafuneki kwakamsinya ukuba benza inkxaso yesiqhelo yeSQL. Yonwabela!

-Enkosi ngengxelo! Inika umdla kakhulu! Ngaba kukho naluphi na uthelekiso kunye neApache Phoenix?

Hayi khange ndive mntu uthelekisa. Thina kunye ne-Yandex sizama ukugcina umkhondo wazo zonke izifaniso ze-ClickHouse kunye noovimba beenkcukacha ezahlukeneyo. Ngenxa yokuba ukuba ngequbuliso into ethile ijika ikhawuleze kuneClickHouse, ke uLesha Milovidov akakwazi ukulala ebusuku kwaye uqala ukukhawuleza ngokukhawuleza. Andizange ndeva ngothelekiso olunjalo.

• (Aleksey Milovidov) IApache Phoenix yinjini yeSQL eqhutywa yiHbase. I-Hbase yenzelwe ubukhulu becala-ixabiso lemeko yomsebenzi. Apho, kumgca ngamnye, kunokubakho inani elingenasizathu leekholomu ezinamagama angaqhelekanga. Oku kunokuthiwa malunga neenkqubo ezifana neHbase, Cassandra. Kwaye yimibuzo yocazululo enzima kanye engayi kusebenza ngokwesiqhelo kubo. Okanye unokucinga ukuba basebenza kakuhle ukuba awuzange ube namava ngeClickHouse.

• Спасибо

  • Mholo Sele ndinomdla kakhulu kwesi sihloko, kuba ndinenkqubo engaphantsi yohlalutyo. Kodwa xa ndijonga kwiClickHouse, ndifumana imvakalelo yokuba iClickHouse ilunge kakhulu kuhlalutyo lomnyhadala, oguqukayo. Kwaye ukuba ndifuna ukuhlalutya idatha eninzi yeshishini kunye neqela leetafile ezinkulu, ngoko i-ClickHouse, njengoko ndiqondayo, ayifanelekanga kakhulu kum? Ngokukodwa ukuba bayatshintsha. Ngaba oku kuchanekile okanye kukho imizekelo enokuphikisa oku?

  • Oku kulungile. Kwaye oku kuyinyani koovimba beenkcukacha abakhethekileyo bohlalutyo. Zilungiselelwe into yokuba kukho itafile enye okanye ngaphezulu ezinkulu eziguquguqukayo, kwaye ezininzi ezincinci eziguqukayo zicotha. Oko kukuthi, i-ClickHouse ayifani ne-Oracle, apho unokubeka yonke into kwaye wakhe imibuzo enzima kakhulu. Ukuze usebenzise iClickHouse ngokufanelekileyo, kufuneka wakhe iskimu ngendlela esebenza kakuhle kwiClickHouse. Oko kukuthi, ukuphepha ukuqheleka okugqithisileyo, sebenzisa izichazi-magama, zama ukwenza amakhonkco amade ambalwa. Kwaye ukuba iskimu sakhiwe ngolu hlobo, ke imisebenzi efanayo yoshishino ingasombululwa kwi-ClickHouse ngokufanelekileyo ngakumbi kunesiseko sedatha yobudlelwane bendabuko.

Enkosi ngengxelo! Ndinombuzo malunga netyala lemali lamva nje. Babene-analytics. Kwakufuneka kuthelekiswe indlela abahla benyuka ngayo. Kwaye ndiyaqonda ukuba uye wakha inkqubo yolu hlalutyo? Ukuba ngomso, umzekelo, bafuna enye ingxelo kule datha, ngaba kufuneka baphinde bakhe iskimu kwaye balayishe idatha? Oko kukuthi, ukwenza uhlobo oluthile lokucubungula ukuze ufumane isicelo?

Ewe, oku kukusetyenziswa kweClickHouse ngomsebenzi othe ngqo. Inokusombulula ngokwesiko ngakumbi ngaphakathi kweHadoop. KwiHadoop, lo ngumsebenzi ofanelekileyo. Kodwa kwiHadoop iyacotha kakhulu. Kwaye injongo yam kukubonisa ukuba iClickHouse inokusombulula imisebenzi ehlala isonjululwa ngeendlela ezahlukeneyo, kodwa kwangaxeshanye yenza oko ngokufanelekileyo. Oku kulungiselelwe umsebenzi othile. Kucacile ukuba ukuba kukho ingxaki ngento efanayo, ngoko inokulungiswa ngendlela efanayo.

Icacile. Uthe iiyure ezingama-50 zenziwe. Ngaba kwasekuqaleni, ulayishe nini idatha okanye ufumane iziphumo?

Ewe Ewe.

Kulungile enkosi kakhulu.

Oku kuluhlu lweseva ezi-3.

Molweni! Enkosi ngengxelo! Yonke into inomdla kakhulu. Andiyi kubuza kancinci malunga nokusebenza, kodwa malunga nokusetyenziswa kweClickHouse ngokwemiqathango yokuzinza. Oko kukuthi, ubunayo na, ubunyanzelekile ukuba uyibuyisele? Ngaba iClickHouse iziphatha njani kule meko? Kwaye kwenzekile ukuba nawe ube nekopi? Umzekelo, sidibene nengxaki ngeClickHouse xa isaphuma kumda wayo kwaye iwe.

Ewe, akukho zinkqubo zifanelekileyo. Kwaye iClickHouse nayo ineengxaki zayo. Kodwa ngaba uvile malunga neYandex.Metrica ayisebenzi ixesha elide? Mhlawumbi akunjalo. Isebenze ngokuthembekileyo ukususela ngo-2012-2013 kwi-ClickHouse. Ndiyakwazi ukuthetha okufanayo malunga namava am. Asizange sibe nokusilela ngokupheleleyo. Ezinye izinto zinokwenzeka, kodwa azizange zibaluleke ngokwaneleyo ukuba zichaphazele ishishini. Ayizange yenzeke loo nto. I-ClickHouse ithembekile kwaye ayintlitheka ngokungacwangciswanga. Awunaxhala ngayo. Asiyonto ikrwada. Oku kuye kwangqinwa ziinkampani ezininzi.

Mholo! Uthe kufuneka ucinge nge-schema yedatha kwangoko. Kuthekani ukuba kwenzekile? Idata yam iyaphalala iyaphalala. Iinyanga ezintandathu zidlula, kwaye ndiyaqonda ukuba akunakwenzeka ukuhlala ngolu hlobo, kufuneka ndiphinde ndilayishe idatha kwaye ndenze okuthile nabo.

Oku kuxhomekeke ngokuqinisekileyo kwindlela yakho. Kukho iindlela ezininzi zokwenza oku ngaphandle kokumisa. Umzekelo, unokwenza iMbono eyiMpahla apho ungenza khona ulwakhiwo lwedatha olwahlukileyo ukuba lunokwenziwa kwimaphu ekhethekileyo. Oko kukuthi, ukuba ivumela imephu usebenzisa i-ClickHouse, okt khupha ezinye izinto, tshintsha isitshixo sokuqala, tshintsha ulwahlulo, emva koko unokwenza iMbono yeMpahla. Bhala phezu kwedatha yakho endala apho, ezintsha ziya kubhalwa ngokuzenzekelayo. Kwaye ke vele utshintshele ekusebenziseni iMbono yeMaterialized, emva koko utshintshe irekhodi kwaye ubulale itafile endala. Le yindlela eqhelekileyo engayekiyo.

Спасибо.

umthombo: www.habr.com