Ngo-Epreli, iinjineli ze-Avito ziqokelele iintlanganiso ze-intanethi kunye nomqhubi oyintloko we-ClickHouse u-Alexey Milovidov kunye noKirill Shvakov, umphuhlisi weGolang ovela kwi-Integros. Sixoxe ngendlela esisebenzisa ngayo isistim yolawulo lwedatha kwaye bubuphi ubunzima esidibana nabo.
Ngokusekelwe kwintlanganiso, siye saqulunqa inqaku ngeempendulo zeengcali zethu kunye nemibuzo yabaphulaphuli malunga ne-backups, ukwabelana ngokutsha kwedatha, izichazi-magama zangaphandle, umqhubi weGolang kunye nokuhlaziya iinguqulelo zeClickHouse. Kunokuba luncedo kubaphuhlisi abasele besebenza ngenkuthalo kunye neYandex DBMS kwaye banomdla ngoku nakwixesha elizayo. Ngokungagqibekanga, iimpendulo ngu-Alexey Milovidov, ngaphandle kokuba kubhalwe ngenye indlela.
Qaphela, kukho itekisi eninzi phantsi kokusikwa. Siyathemba ukuba umxholo onemibuzo uza kukunceda ukhangele.
Iziqulatho
Ukuba awufuni ukufunda isicatshulwa, unokubukela ukurekhodwa kweendibano . Iikhowudi zexesha zikwizimvo zokuqala phantsi kwevidiyo.
I-ClickHouse ihlaziywa rhoqo, kodwa idatha yethu ayikho. Yintoni onokuyenza ngayo?
I-ClickHouse ihlaziywa rhoqo, kwaye idatha yethu, eyaye yaphuculwa ekugqibeleni, ayihlaziywa kwaye ikwikopi yokugcina.
Masithi sibe nengxaki kwaye idatha yalahleka. Sagqiba ekubeni sibuyisele, kwaye kwavela ukuba izahlulo ezindala, ezigcinwe kwiiseva zokulondoloza, zihluke kakhulu kwinguqulo esetyenziswayo yeClickHouse. Yintoni enokwenziwa kwimeko enjalo, yaye ngaba kunokwenzeka?
Imeko apho ubuyisele idatha kwi-backup kwifomathi endala, kodwa ayidibanisi nenguqulo entsha, ayinakwenzeka. Siyaqinisekisa ukuba ifomathi yedatha kwiClickHouse ihlala ibuyela ngasemva ihambelana. Oku kubaluleke kakhulu kunokuhambelana okubuyela umva ekusebenzeni ukuba indlela yokuziphatha yomnye umsebenzi ongafane isetyenziswe itshintshile. Uguqulelo olutsha lweClickHouse kufuneka luhlale lukwazi ukufunda idatha egcinwe kwidiski. Lo ngumthetho.
Zeziphi ezona ndlela zisebenzayo ngoku zokuxhasa idatha esuka kwiClickHouse?
Indlela yokwenza i-backups, ngokuqwalasela ukuba siye salungisa imisebenzi yokugqibela, i-database enkulu ye-terabytes, kunye nedatha ehlaziywayo, yithi, kwiintsuku ezintathu zokugqibela, kwaye akukho nkqubo eyenzekayo kuyo?
Singenza ezethu isisombululo kwaye sibhale kwi-bash: qokelela ezi kopi zogcino ngendlela ethile. Mhlawumbi akukho mfuneko yokuqhwaba nantoni na, kwaye ibhayisekile yasungulwa kudala?
Masiqale ngezona zenzo zingcono. Oogxa bam bahlala becebisa, ekuphenduleni imibuzo malunga ne-backups, ukubakhumbuza malunga nenkonzo yeYandex.Cloud, apho le ngxaki sele isonjululwe. Ngoko sebenzisa ukuba kunokwenzeka.
Akukho sisombululo sipheleleyo se-backups, ikhulu leepesenti elakhiwe kwiClickHouse. Kukho ezinye iindawo ezingenanto ezinokusetyenziswa. Ukufumana isisombululo esipheleleyo, kuya kufuneka ucofe kancinci ngesandla, okanye wenze ii-wrappers ngohlobo lwezikripthi.
Ndiza kuqala ngezona zisombululo ezilula kwaye ndigqibe ngezona ziyinkimbinkimbi, kuxhomekeke kumthamo wedatha kunye nobukhulu beqela. Iqela elikhulu, kokukhona isisombululo siba nzima.
Ukuba itheyibhile enedatha ithatha kuphela iigigabytes ezimbalwa, i-backup ingenziwa ngolu hlobo:
- Gcina inkcazo yetheyibhile o.k. imetadata − bonisa itafile yokudala.
- Yenza inkunkuma usebenzisa iClickHouse client - khetha * ukusuka etafileni ukwenza ifayile. Ngokungagqibekanga uya kufumana ifayile kwi-TabSeparated format. Ukuba ufuna ukusebenza kakuhle, ungayenza ngefomathi yeNative.
Ukuba inani ledatha likhulu, ngoko ukugcinwa kwe-backup kuya kuthatha ixesha elide kunye nendawo eninzi. Oku kubizwa ngokuba yi-backup enengqiqo; ayibotshwanga kwifomati yedatha yeClickHouse. Ukuba kunjalo, ke njengendlela yokugqibela ungathatha i-backup kwaye uyilayishe kwi-MySQL yokubuyisela.
Kwimiba ephambili, iClickHouse inamandla owakhelwe ngaphakathi ukwenza umfanekiso okhawulezayo wezahlulo kwinkqubo yefayile yendawo. Olu phawu luyafumaneka njengesicelo guqula isahlulelo somkhenkce wetafile. Okanye ngokulula guqula umkhenkce wetafile - lo ngumfanekiso wetafile iyonke.
I-snapshot iyakwenziwa ngokuhambelanayo kwitafile enye kwi-shard enye, oko kukuthi, akunakwenzeka ukwenza umfanekiso okhawulezayo weqela lonke ngale ndlela. Kodwa kwimisebenzi emininzi akukho mfuneko injalo, kwaye kwanele ukwenza isicelo kwishard nganye kwaye ufumane umfanekiso okhawulezayo. Idalwe ngendlela yee-hardlinks kwaye ngoko ayithathi indawo eyongezelelweyo. Okulandelayo, ukhuphela le snapshot kwiseva yogcino okanye kwindawo yokugcina oyisebenzisela ii-backups.
Ukubuyisela i-backup enjalo kulula kakhulu. Okokuqala, yenza iitafile usebenzisa iinkcazo zetafile ezikhoyo. Okulandelayo, khuphela iifoto ezigciniweyo zezahlulelo kwi-Directory-Detached kwezi tafile kwaye uqhube umbuzo. ncamathela isahlulelo. Esi sisombululo sifanelekile kumthamo omkhulu wedatha.
Ngamanye amaxesha udinga into epholileyo nangakumbi - kwiimeko apho unamashumi okanye amakhulu e-terabytes kwiseva nganye kunye namakhulu amaseva. Kukho isisombululo apha endisithathileyo kubalingane bam baseYandex.Metrica. Andiyi kucebisa wonke umntu - yifunde kwaye uzenzele isigqibo malunga nokuba ifanelekile na okanye hayi.
Okokuqala kufuneka udale iiseva ezininzi ngeeshelufu zediski ezinkulu. Okulandelayo, kwezi seva, phakamisa iiseva ezininzi zeClickHouse kwaye uziqwalasele ukuze zisebenze njengenye ikopi yeeshadi ezifanayo. Kwaye emva koko sebenzisa inkqubo yefayile okanye isixhobo esithile kwezi seva ekuvumela ukuba wenze izifinyezo. Kukho iinketho ezimbini apha. Inketho yokuqala yi-LVM snapshots, ukhetho lwesibini yi-ZFS kwi-Linux.
Emva koko, yonke imihla kufuneka udale i-snapshot, iya kulala kwaye ithathe indawo. Ngokwemvelo, ukuba idatha iyatshintsha, inani lendawo liya kwanda ngokuhamba kwexesha. Le snapshot inokukhutshwa nangaliphi na ixesha kwaye idatha ibuyiselwe, isisombululo esinjalo esingaqhelekanga. Kwaye, kufuneka sinciphise ezi replicas kwi-config ukuze bangazami ukuba ziinkokeli.
Ngaba kuya kwenzeka ukuba uququzelele i-lag elawulwayo yee-replicas kwi-shafts?
Kulo nyaka uceba ukwenza i-shafts kwi-ClickHouse. Ngaba kuya kwenzeka ukuba uququzelele i-lag elawulwayo yee-replicas kuzo? Singathanda ukuyisebenzisela ukuzikhusela kwiimeko ezimbi ezinotshintsho kunye nolunye utshintsho.
Ngaba kunokwenzeka ukwenza uhlobo oluthile lokubuyela umva kwiialters? Ngokomzekelo, kwi-shaft ekhoyo, thatha kwaye uthi kude kube lo mzuzu usebenzisa utshintsho, kwaye ukususela kulo mzuzu uyeka ukusebenzisa utshintsho?
Ukuba umyalelo weza kwiqela lethu kwaye waliqhekeza, ngoko sine-replica enemiqathango ene-lag yeyure, apho sinokuthi masiyisebenzise okwangoku, kodwa asiyi kusebenzisa utshintsho kuyo kwimizuzu elishumi yokugqibela?
Okokuqala, malunga ne-lag elawulwayo yee-replicas. Bekukho isicelo esinjalo kubasebenzisi, kwaye senze umba kwiGithub ngesicelo: "Ukuba umntu ufuna oku, njengayo, beka intliziyo." Akukho mntu uzisiwe, kwaye umcimbi wavalwa. Nangona kunjalo, unokufumana eli thuba ngokumisela i-ClickHouse. Yinyani, iqala kuphela kwinguqulo 20.3.
I-ClickHouse ihlala yenza ukudibanisa kwedatha ngasemva. Xa ukudibanisa kugqityiwe, isethi ethile yedatha yedatha ithathelwa indawo ngeqhekeza elikhulu. Kwangaxeshanye, iziqwenga zedatha ezazikho ngaphambili ziyaqhubeka nokuhlala kwidiski kangangexesha elithile.
Okokuqala, ziyaqhubeka zigcinwa nje ukuba kukho imibuzo ekhethiweyo ezisebenzisayo, ukwenzela ukubonelela ngokusebenza okungathinteliyo. Imibuzo ekhethiweyo ifundeka ngokulula kwiziqwenga ezindala.
Okwesibini, kukho umda wexesha - amaqhekeza amadala edatha alele kwidiski imizuzu esibhozo. Le mizuzu isibhozo inokwenziwa ngokwezifiso kwaye ijikwe ibe lusuku olunye. Oku kuya kubiza indawo yediski: kuxhomekeke ekuhambeni kwedatha, kuvela ukuba ngosuku lokugqibela idatha ayiyi kuphinda kabini, ingaba ngamaxesha amahlanu. Kodwa ukuba kukho ingxaki enkulu, unokumisa iseva yeClickHouse kwaye uhlele yonke into.
Ngoku umbuzo uvela malunga nokuba le nto ikhusela njani kwiinguqu. Kufanelekile ukujonga nzulu apha, kuba kwiinguqulelo ezindala zeClickHouse, isitshintshi sisebenze ngendlela yokuba sitshintshe iziqwenga ngokuthe ngqo. Kukho iqhekeza ledatha kunye neefayile ezithile, kwaye senza, umzekelo, tshintsha ikholamu yokulahla. Emva koko le kholamu isuswe ngokwasemzimbeni kuzo zonke iinqununu.
Kodwa ukuqala ngenguqulo 20.3, indlela yokutshintsha itshintshiwe ngokupheleleyo, kwaye ngoku amaqhekeza edatha ahlala engaguquki. Azitshintshi konke konke-iinguqu ngoku zisebenza ngendlela efanayo nokudibanisa. Esikhundleni sokutshintsha isiqwenga kwindawo, sidala entsha. Kwi-chunk entsha, iifayile ezingatshintshanga zibe yi-hardlinks, kwaye ukuba sicima ikholamu, iya kulahleka kwi-chunk entsha. Isiqwenga esidala siya kususwa ngokungagqibekanga emva kwemizuzu esibhozo, kwaye apha unokwenza izicwangciso ezikhankanywe ngasentla.
Oku kuyasebenza nakwiinguqu ezifana notshintsho. Xa usenza guqula ukucima okanye guqula uhlaziyo, ayitshintshi isiqwenga, kodwa idala entsha. Kwaye ke ucima endala.
Kuthekani ukuba isakhiwo setafile sitshintshile?
Indlela yokubuyisela i-backup eyenziwe ngeskimu esidala? Kwaye umbuzo wesibini malunga nemeko kunye ne-snapshots kunye nezixhobo zenkqubo yefayile. Ngaba i-Btrfs ilungile apha endaweni ye-ZFS kwi-Linux LVM?
Ukuba wenza njalo ncamathela isahlulelo izahlulelo ezinesakhiwo esahlukileyo, emva koko uClickHouse uya kukuxelela ukuba oku akunakwenzeka. Esi sisisombululo. Eyokuqala kukudala itheyibhile yethutyana yohlobo lweMergeTree ngesakhiwo esidala, uncamathisele idatha apho usebenzisa uncamathiselo, kwaye wenze isicelo sokutshintsha. Emva koko unokukopa okanye uthumele le datha kwaye uncamathisele kwakhona, okanye usebenzise isicelo guqula isahlulelo setafile.
Ngoku umbuzo wesibini kukuba ingaba iiBtrfs zingasetyenziswa. Ukuqala, ukuba une-LVM, emva koko ii-snapshots ze-LVM zanele, kwaye inkqubo yefayile inokuba yi-ext4, ayinamsebenzi. Nge-Btrts, yonke into ixhomekeke kumava akho ekuyisebenziseni. Le yinkqubo yefayile ekhulileyo, kodwa kusekho ukukrokrelana malunga nokuba yonke into iya kusebenza njani kwi-scenario ethile. Andinakucebisa ukuba usebenzise oku ngaphandle kokuba une-Btrfs kwimveliso.
Zeziphi ezona zenzo zilungileyo ngoku ekwabelweni kwakhona kwedatha?
Umba wokwabelana ngokutsha untsonkothile kwaye unamacala amaninzi. Kukho iimpendulo ezininzi ezinokubakho apha. Ungahamba kwicala elinye kwaye uthethe oku-ClickHouse ayinayo into eyakhelweyo yokwabelana kwakhona. Kodwa ndiyoyika ukuba le mpendulo ayizukulungela nabani na. Ngoko ke, unokuhamba ukusuka kwelinye icala kwaye uthi i-ClickHouse ineendlela ezininzi zokwabelana kwakhona kwedatha.
Ukuba iqela liphelelwa yindawo okanye alikwazi ukuphatha umthwalo, wongeza amaseva amatsha. Kodwa ezi seva zingenanto ngokungagqibekanga, akukho datha kuzo, akukho mthwalo. Kufuneka uhlengahlengise idatha ukuze isasazeke ngokulinganayo kwiqela elitsha, elikhulu.
Indlela yokuqala oku kunokwenziwa ngayo kukukopa inxalenye yezahlulo kwiiseva ezintsha usebenzisa isicelo guqula isahlulelo sokulanda itafile. Ngokomzekelo, ube nezahlulo ngenyanga, kwaye uthatha inyanga yokuqala ye-2017 kwaye uyikopishe kwiseva entsha, emva koko ukopishe inyanga yesithathu kwenye iseva entsha. Kwaye uyenza oku de ibe ngaphezulu okanye ngaphantsi ngokulinganayo.
Ukudluliselwa kunokuqhutywa kuphela kwezo zahlulo ezingatshintshiyo ngexesha lokurekhoda. Kwizahlulo ezitsha, ukurekhoda kuya kufuneka kucinywe, kuba ukuhanjiswa kwabo akuyo-athomu. Ngaphandle koko, uya kugqiba ngokuphindaphinda okanye izikhewu kwidatha. Nangona kunjalo, le ndlela iyasebenza kwaye isebenza ngokufanelekileyo. Izahlulo ezilungelelanisiweyo esele zenziwe zigqithiselwa kwinethiwekhi, oko kukuthi, idatha ayicinezelekanga okanye ifakwe kwakhona.
Le ndlela ine-drawback enye, kwaye ixhomekeke kwi-sharding scheme, nokuba ubophelele kwesi sikimu se-sharding, sesiphi isitshixo sokwabelana onaso. Kumzekelo wakho wemeko eneemetrics, isitshixo sokwabelana yi-hash yendlela. Xa ukhetha itafile eSasazwayo, iya kuzo zonke iishadi kwiqela kanye kwaye ithatha idatha ukusuka apho.
Oku kuthetha ukuba eneneni akunamsebenzi kuwe ukuba yeyiphi idatha egqibe ukuba yeyiphi i-shard. Into ephambili kukuba idatha ecaleni kwendlela enye iphelela kwi-shard enye, kodwa yintoni engabalulekanga. Kule meko, ukudlulisa izahlulelo esele zenziwe zilungile, kuba ngemibuzo ekhethiweyo uya kufumana idatha epheleleyo - nokuba ngaphambi kokwabelana kwakhona okanye emva koko, iskimu asinamsebenzi.
Kodwa kukho iimeko ezinzima ngakumbi. Ukuba kwinqanaba le logic yesicelo uthembele kwiskimu esikhethekileyo sokwabelana, ukuba lo mxhasi ubekwe kwindawo enjalo kunye ne-shard enjalo, kwaye isicelo sinokuthunyelwa ngqo apho, kwaye kungekhona kwitafile eSasaziwe. Okanye usebenzisa inguqulelo yamva nje yeClickHouse kwaye wenze useto lusebenze khulisa ukutsiba iishadi ezingasetyenziswanga. Kule meko, ngexesha lombuzo okhethiweyo, ukubonakaliswa kwindawo apho icandelo liza kuhlalutywa kwaye liza kubalwa ukuba yiyiphi i-shards ekufuneka isetyenziswe ngokuhambelana neskimu se-sharding. Oku kusebenza ngokubonelelwa ukuba idatha yahlulahlulwe ngokuthe ngqo ngokwesi skim sokwabelana. Ukuba uzilungelelanise ngokutsha ngesandla, imbalelwano inokutshintsha.
Ngoko le yindlela yokuqala. Kwaye ndilinde impendulo yakho, ukuba indlela ifanelekile, okanye masiqhubele phambili.
UVladimir Kolobaev, umlawuli wenkqubo ekhokelayo kwi-Avito: U-Alexey, indlela oyikhankanyileyo ayisebenzi kakuhle xa kufuneka usasaze umthwalo, kuquka nokufunda. Singathatha isahlulo esiyinyanga kwaye sinokuthatha inyanga yangaphambili kwenye i-node, kodwa xa isicelo sifika kule datha, siya kuyilayisha kuphela. Kodwa singathanda ukulayisha yonke i-cluster, kuba kungenjalo, ixesha elithile wonke umthwalo wokufunda uya kucutshungulwa ngamaqhekeza amabini.
UAlexey Milovidov: Impendulo apha iyamangalisa-ewe, imbi, kodwa inokusebenza. Ndiza kucacisa ukuba njani. Kufanelekile ukujonga imeko yomthwalo oza emva kwedatha yakho. Ukuba le datha yokubeka iliso, ngoko sinokuthi ngokuqinisekileyo sinokuthi uninzi lwezicelo zedatha entsha.
Ufake iiseva ezitsha, wafuduka izahlulelo ezindala, kodwa utshintshe nendlela idatha entsha erekhodwa ngayo. Kwaye idatha entsha iya kusasazwa kulo lonke iqela. Ke, emva kwemizuzu nje emihlanu, izicelo zemizuzu emihlanu yokugqibela ziya kulayisha ngokulinganayo iqela; emva kosuku, izicelo zeeyure ezingama-24 ziya kulayisha ngokulinganayo iqela. Kwaye izicelo zenyanga edlulileyo, ngelishwa, ziya kuphela kwinxalenye yeeseva zeqela.
Kodwa rhoqo awuyi kuba nezicelo ngokuthe ngqo ngoFebruwari 2019. Okunokwenzeka, ukuba izicelo zingena kwi-2019, ngoko ziya kuba ze-2019 yonke - ixesha elide, kwaye kungekhona kuluhlu oluncinci. Kwaye izicelo ezinjalo ziya kuba nako ukulayisha iqela ngokulinganayo. Kodwa ngokubanzi, inkcazo yakho ichanekile ngokupheleleyo ukuba esi sisisombululo se-ad hoc esingasasazi ngokupheleleyo idatha ngokulinganayo.
Ndineengongoma ezimbalwa zokuphendula umbuzo. Enye yazo imalunga nendlela yokuyila ekuqaleni iskimu sokwahlula ukuze ukwahlula kwakhona kunokubangela iintlungu ezincinci. Oku akusoloko kusenzeka.
Umzekelo, unedatha yokubeka iliso. Idatha yokubeka iliso iyakhula ngenxa yezizathu ezithathu. Eyokuqala yingqokelela yedatha yembali. Okwesibini kukukhula kwezithuthi. Kwaye okwesithathu kukwanda kwenani lezinto ezixhomekeke ekubekweni iliso. Kukho iimicroservices ezintsha kunye neemetrics ezifuna ukugcinwa.
Kungenzeka ukuba kwezi, ukwanda okukhulu kuhambelana nesizathu sesithathu - ukwanda kokusetyenziswa kokubeka iliso. Kwaye kule meko, kuyafaneleka ukujonga ubunjani bomthwalo, yiyiphi imibuzo ephambili ekhethiweyo. Imibuzo esisiseko ekhethiweyo iyakuthi isekwe kwiseti esezantsi ye-metrics.
Umzekelo, ukusetyenziswa kwe-CPU kwezinye iiseva ngenkonzo ethile. Kuvela ukuba kukho iseti ethile yezitshixo ofumana ngayo le datha. Kwaye isicelo ngokwaso sale datha sinokwenzeka ukuba silula kwaye sigqitywe kumashumi eemilliseconds. Isetyenziselwa ukubeka iliso kwiinkonzo kunye needeshibhodi. Ndiyathemba ukuba ndiyiqonda kakuhle le nto.
UVladimir Kolobaev: Inyani kukuba sihlala sibhenela kwidatha yembali, kuba sithelekisa imeko yangoku kunye nembali ngexesha lokwenyani. Kwaye kubalulekile ukuba sifikelele ngokukhawuleza kwinani elikhulu ledatha, kwaye i-ClickHouse yenza umsebenzi obalaseleyo ngale nto.
Unyanisile ngokupheleleyo, sifumana uninzi lwezicelo ezifundwe kusuku lokugqibela, njengayo nayiphi na inkqubo yokubeka iliso. Kodwa kwangaxeshanye, umthwalo kwidatha yembali nayo inkulu kakhulu. Isuka kwinkqubo yokwazisa ehamba rhoqo kwimizuzwana engamashumi amathathu kwaye ithi kuClickHouse: “Ndinike idatha kwezi veki zintandathu zidlulileyo. Ngoku ndakhele uhlobo oluthile lomndilili ohambahambayo kubo, kwaye masithelekise ixabiso langoku kunye nembali.”
Ndingathanda ukuthi kwizicelo zamva nje sinenye itafile encinci esigcina kuyo iintsuku ezimbini kuphela zedatha, kwaye izicelo eziphambili zibhabhela kuyo. Sithumela kuphela imibuzo yembali emikhulu kwitafile enkulu ekwabelwana ngayo.
UAlexey Milovidov: Ngelishwa, ibonakale ingasebenzi kakuhle kwimeko yakho, kodwa ndiza kukuxelela inkcazo yeendlela ezimbini ezimbi nezintsonkothileyo zokudityaniswa okungafunekiyo ukuba zisetyenziswe, kodwa ezisetyenziswa kwinkonzo yabahlobo bam.
Kukho iqela eliphambili elineziganeko zeYandex.Metrica. Izehlo kukujongwa kwephepha, ucofa, kunye noguqulelo. Uninzi lwezicelo luya kwiwebhusayithi ethile. Uvula inkonzo yeYandex.Metrica, unayo iwebhusayithi - avito.ru, yiya kwingxelo, kwaye isicelo senziwe kwiwebhusayithi yakho.
Kodwa kukho ezinye izicelo - uhlalutyo kunye nehlabathi jikelele - ezenziwa ngabahlalutyi bangaphakathi. Kanye kwimeko, ndiqaphela ukuba abahlalutyi bangaphakathi benza izicelo kuphela kwiinkonzo zeYandex. Kodwa nangona kunjalo, iinkonzo zeYandex zithatha isabelo esibalulekileyo sayo yonke idatha. Ezi zizicelo ezingezizo izixhobo zokubala ezithile, kodwa zezohluzo olubanzi.
Indlela yokulungelelanisa idatha ngendlela yokuba yonke into isebenze ngokufanelekileyo kwikhawunta enye, kunye nemibuzo yehlabathi? Obunye ubunzima kukuba inani lezicelo kwiClickHouse yeqela leMetrics lingamawaka aliqela ngomzuzwana. Kwangaxeshanye, umncedisi omnye weClickHouse akakwazi ukusingatha izicelo ezingeyomfuneko, umzekelo, amawaka amaninzi ngomzuzwana.
Ubungakanani beqela ngamakhulu amathandathu-abanye abancedisi. Ukuba utsala ngokulula itafile eSasazo ngaphezulu kweli qela kwaye uthumele amawaka aliqela ezicelo apho, iyakuba mbi ngakumbi kunokubathumela kumncedisi omnye. Ngakolunye uhlangothi, ukhetho lokuba idatha isasazwa ngokulinganayo, kwaye sihamba kwaye sicele kuzo zonke iiseva, zichithwa ngokukhawuleza.
Kukho ukhetho oluchasene nediametrically. Khawufane ucinge ukuba sabelana ngedatha kuzo zonke iisayithi, kwaye isicelo sesiza esinye siya kwindawo enye. Ngoku iqela liya kukwazi ukusingatha izicelo ezingamawaka alishumi ngesekhondi, kodwa kwelinye icala nasiphi na isicelo siya kusebenza ngokucotha kakhulu. Ayisayi kuphinda ikhule ngokwemiqathango yokwenziwa komsebenzi. Ngokukodwa ukuba le ndawo i-avito.ru. Andiyi kutyhila imfihlelo ukuba ndithi i-Avito yenye yezona ndawo zityelelwe kakhulu kwi-RuNet. Kwaye ukusetyenzwa kwindawo enye iya kuba yimpambano.
Ngoko ke, iskimu se-sharding senziwe ngendlela enobuqili ngakumbi. Iqela lonke lahlulwe laba ngamaqela amaninzi, esiwabiza ngokuba ziileya. Iqela ngalinye liqulathe ukusuka kwishumi elinesibini ukuya kwishumi elinesibini leeshadi. Kukho amaqela anjalo angamashumi amathathu anesithoba xa ewonke.
Kwenzeka njani konke oku? Inani lamaqela alitshintshi - njengoko lalingamashumi amathathu anesithoba kwiminyaka embalwa edlulileyo, lihlala linjalo. Kodwa ngaphakathi nganye kuzo, ngokuthe ngcembe sonyusa inani leeshadi njengoko siqokelela idatha. Kwaye i-scheme ye-sharding iyonke ifana nale: la maqela ahlulwe kwiiwebhusayithi, kwaye ukuze uqonde ukuba yeyiphi i-website yeyiphi i-cluster, i-metabase eyahlukileyo kwi-MySQL isetyenzisiweyo. Indawo enye - kwiqela elinye. Kwaye ngaphakathi kwayo, ukwahlula kwenzeka ngokwee-ID zabatyeleli.
Xa urekhoda, siyazahlula ngentsalela yolwahlulo lwe-ID yondwendwe. Kodwa xa ufaka i-shard entsha, iskimu sokwabelana siyatshintsha; siyaqhubeka nokwahlula, kodwa ngentsalela yolwahlulo ngelinye inani. Oku kuthetha ukuba umtyeleli omnye sele ebekwe kwiiseva ezininzi, kwaye awukwazi ukuxhomekeka koku. Oku kwenziwa kuphela ukuqinisekisa ukuba idatha ixinaniswe ngcono. Kwaye xa sisenza izicelo, siya kwitheyibhile eSasaziwe, ejonga iqela kwaye ifikelele kuninzi lweeseva. Le yinkqubo yobudenge.
Kodwa ibali lam liya kuba lingaphelelanga ukuba anditsho ukuba silehlile eli qhinga. Kwiskimu esitsha, sitshintshe yonke into kwaye sikopisha yonke idatha usebenzisa i-clickhouse-copier.
Kwiskimu esitsha, zonke iziza zahlulwe zibe ziindidi ezimbini - ezinkulu nezincinci. Andazi ukuba kukhethwe njani umda, kodwa umphumo waba kukuba iisayithi ezinkulu zirekhodwa kwiqela elinye, apho kukho iishadi ezingama-120 ezineekopi ezintathu nganye - oko kukuthi, iiseva ezingama-360. Kwaye i-sharding scheme injalo ukuba nasiphi na isicelo siya kuzo zonke iishadi kanye. Ukuba ngoku uvula naliphi na iphepha lengxelo ye-avito.ru kwiYandex.Metrica, isicelo siya kumaseva angama-120. Kukho iisayithi ezinkulu ezimbalwa kwi-RuNet. Kwaye izicelo azikho iwaka ngomzuzwana, kodwa nangaphantsi kwekhulu. Konke oku kuhlafunwa ngokuzolileyo yitafile eSasazo, leyo nganye kuzo iqhuba ngeeseva ezingama-120.
Kwaye iqela lesibini leleendawo ezincinci. Nasi iskimu sokwabelana ngokusekwe kwi-ID yesayithi, kwaye isicelo ngasinye siya ngqo kwishadi enye.
I-ClickHouse ine-clickhouse-copier eluncedo. Ngaba ungasixelela ngaye?
Ndiza kuthetha ngoko nangoko ukuba esi sisombululo sinzima ngakumbi kwaye asivelisi kangako. Inzuzo kukuba ihlamba idatha ngokupheleleyo ngokwepateni oyichazayo. Kodwa i-drawback ye-utility kukuba ayitshintshi kwakhona. Ikopa idatha ukusuka kwi-schema yeqela ukuya kwenye i-schema yeqela.
Oku kuthetha ukuba ukuze isebenze kufuneka ube namaqela amabini. Zinokufumaneka kwiiseva ezifanayo, kodwa, nangona kunjalo, idatha ayiyi kuhanjiswa ngokunyukayo, kodwa iya kukhutshelwa.
Umzekelo, bekukho iiseva ezine, ngoku zisibhozo. Udala itafile entsha eSabalaliswayo kuzo zonke iiseva, iitafile zasekhaya ezintsha kwaye uqalise i-clickhouse-copier, ebonisa kuyo iskimu somsebenzi ekufuneka sifunde ukusuka apho, samkele iskimu esitsha se-sharding kwaye udlulisele idatha apho. Kwaye kwiiseva ezindala uya kufuna indawo enesiqingatha esiphindwe kabini kunangoku, kuba idatha endala kufuneka ihlale kubo, kwaye isiqingatha sedatha endala iya kufika phezu kwabo. Ukuba ucinga kwangaphambili ukuba idatha idinga ukwabelwa kwakhona kwaye kukho indawo, ke le ndlela ifanelekile.
Isebenza njani i-clickhouse-copier ngaphakathi? Kwahlulahlula wonke umsebenzi ube yiseti yemisebenzi ukulungiselela isahlulelo esinye setafile kwishadi enye. Yonke le misebenzi inokuphunyezwa ngokunxuseneyo, kwaye i-clickhouse-copier inokuqhutywa koomatshini abahlukeneyo kwiimeko ezininzi, kodwa into eyenzayo kwisahlulelo esinye asiyonto ingaphezulu kokufakwa okukhethiweyo. Idatha iyafundwa, ithotywe, iphinde yahlulwe, emva koko icinezelwe kwakhona, ibhalwe kwenye indawo, kwaye ihlelwe kwakhona. Esi sisigqibo esinzima.
Ubunento yokulinga ebizwa ngokuba kukwabelana ngokutsha. Kuthekani ngaye?
Emuva ngo-2017, ubunento yokulinga ebizwa ngokuba yi-resharding. Kukho nokhetho kwiClickHouse. Njengoko ndiyiqonda, ayizange ihambe. Ngaba unokundixelela ukuba kutheni kwenzeke oku? Kubonakala kuhambelana kakhulu.
Ingxaki yonke kukuba ukuba kuyimfuneko ukwahlula kwakhona idatha endaweni, ungqamaniso olunzima kakhulu luyafuneka ukuze kwenziwe oku ngeatom. Xa saqala ukujonga indlela olu ngqamaniso lusebenza ngayo, kwacaca ukuba kukho iingxaki ezibalulekileyo. Kwaye ezi ngxaki zisisiseko azikho ithiyori kuphela, kodwa ngoko nangoko zaqalisa ukuzibonakalisa ngokusebenza ngendlela enokuchazwa ngokulula - akukho nto isebenzayo.
Ngaba kuyenzeka ukudibanisa onke amaqhekeza edatha kunye phambi kokuba uyihambise kwiidiski ezicothayo?
Umbuzo malunga ne-TTL kunye nokhetho lokucotha kwidiski kumxholo wodibaniso. Ngaba kukho indlela, ngaphandle kwe-cron, yokudibanisa onke amalungu kwindawo enye ngaphambi kokuwahambisa kwiidiski ezicothayo?
Impendulo yombuzo kunokwenzeka ukuba ngandlela-thile uncamathelise ngokuzenzekelayo zonke iziqwenga zibe nye ngaphambi kokuba zidluliselwe - hayi. Andiqondi ukuba oku kuyimfuneko. Awudingi ukudibanisa onke amalungu kwindawo enye, kodwa bala ngokulula kwinto yokuba ziya kudluliselwa kwiidiski ezicothayo ngokuzenzekelayo.
Sinemilinganiselo emibini yemigaqo yokudluliselwa. Eyokuqala injengoko izaliswe. Ukuba inqanaba lokugcina langoku linepesenteji ethile yendawo ekhululekileyo, sikhetha i-chunk enye kwaye siyihambisa kwindawo yokugcina ecothayo. Okanye kunoko, hayi kancinci, kodwa elandelayo - njengoko uqwalasela.
Ikhrayitheriya yesibini bubukhulu. Imalunga nokuhambisa amaqhekeza amakhulu. Unokwenza uhlengahlengiso lomqobo ngokwesithuba sasimahla kwidiski ekhawulezayo, kwaye idatha iya kudluliselwa ngokuzenzekelayo.
Ufudukela njani kwiinguqulelo ezintsha zeClickHouse ukuba akukho ndlela yokujonga ukuhambelana kwangaphambili?
Lo mbandela uxutyushwa rhoqo kuthathelwa ingqalelo iinguqulelo ezahlukeneyo, kwaye nangoku. Kukhuselekile kangakanani ukuphucula ukusuka kwi-19.11 ukuya kwi-19.16 kwaye, umzekelo, ukusuka kwi-19.16 ukuya kwi-20.3. Yeyiphi eyona ndlela ingcono yokufudukela kwiinguqulelo ezintsha ngaphandle kokukwazi ukujonga ukuhambelana kwibhokisi yesanti kwangaphambili?
Kukho imithetho emininzi "yegolide" apha. Ekuqaleni - . Mkhulu, kodwa kukho imihlathi eyahlukileyo malunga notshintsho olungahambelaniyo ngasemva. Musa ukuphatha ezi ngongoma njengeflegi ebomvu. Oku kuqhelekile kukungahambelani okuncinci okubandakanya ukusebenza komphetho onokuthi ungasebenzisi.
Okwesibini, ukuba akukho ndlela yokujonga ukuhambelana kwibhokisi yesanti, kwaye ufuna ukuhlaziya ngokukhawuleza kwimveliso, isincomo kukuba awudingi ukwenza oku. Qala ngokwenza ibhokisi yesanti kunye novavanyo. Ukuba akukho ndawo yovavanyo, ngoko ke kusenokwenzeka ukuba awunayo inkampani enkulu kakhulu, nto leyo ethetha ukuba unokukopa enye yedatha kwilaptop yakho kwaye uqiniseke ukuba yonke into isebenza ngokuchanekileyo kuyo. Unokuphakamisa iireplicas ezininzi kwindawo yakho kumatshini wakho. Okanye unokuthatha inguqulelo entsha kwindawo ekufutshane kwaye ulayishe enye yedatha apho - oko kukuthi, udale indawo yovavanyo ephuculweyo.
Omnye umgaqo awuyikuhlaziya iveki emva kokukhululwa kwenguqulo ngenxa yokubamba iibhugi kwimveliso kunye nezilungiso ezikhawulezayo ezilandelayo. Makhe sijonge amanani eenguqulelo zeClickHouse ukuze ungabhideki.
Kukho inguqulelo 20.3.4. Inombolo ye-20 ibonisa unyaka wokuvelisa - 2020. Ukususela kwindawo yokujonga into engaphakathi, oku akunandaba, ngoko asiyi kuqwalasela. Okulandelayo - 20.3. Sandisa inani lesibini - kule meko 3 - rhoqo xa sikhulula ukukhululwa kunye nokusebenza okutsha. Ukuba sifuna ukongeza into ethile kwiClickHouse, kufuneka sinyuse eli nani. Oko kukuthi, kwinguqulo 20.4 ClickHouse iya kusebenza ngcono. Idijithi yesithathu ngu-20.3.4. Nantsi i-4 inani lokukhutshwa kwepatch apho singakhange songeze iimpawu ezintsha, kodwa silungise ezinye iibhugi. Kwaye u-4 uthetha ukuba siyenze amatyeli amane.
Musani ukucinga ukuba yinto embi le. Ngokuqhelekileyo umsebenzisi unokufaka inguqulelo yamva nje kwaye iya kusebenza ngaphandle kweengxaki kunye nexesha lokuphumla ngonyaka. Kodwa khawufane ucinge ukuba komnye umsebenzi wokusetyenzwa kwe-bitmaps, eyongezwe ngamaqabane ethu aseTshayina, iseva iyantlitheka xa idlulisa iingxoxo ezingachanekanga. Sinoxanduva lokuyilungisa le nto. Siza kukhupha i-patch version entsha kwaye i-ClickHouse iya kuzinza ngakumbi.
Ukuba uneClickHouse esebenzayo kwimveliso, kwaye inguqulelo entsha yeClickHouse ikhutshwe kunye neempawu ezongezelelweyo - umzekelo, i-20.4.1 iyona yokuqala kakhulu, musa ukukhawuleza ukuyibeka kwimveliso ngosuku lokuqala. Kutheni ifuneka kwaphela? Ukuba awusebenzisi iClickHouse okwangoku, ungayifaka, kwaye kusenokwenzeka ukuba yonke into izakulunga. Kodwa ukuba iClickHouse sele isebenza ngokuzinzileyo, ke hlala ubeke iliso kwiipetshi kunye nohlaziyo ukuze ubone ukuba zeziphi iingxaki esizilungisayo.
Kirill Shvakov: Ndingathanda ukongeza kancinci malunga neemeko zovavanyo. Wonke umntu uyoyika kakhulu iimeko zovavanyo kwaye ngenxa yesizathu esithile bakholelwa ukuba ukuba uneClickHouse enkulu kakhulu yeCluster, ke imeko yovavanyo kufuneka ingabi ngaphantsi okanye ubuncinane amaxesha alishumi amancinci. Akunjalo kwaphela.
Ndingakuxelela kowam umzekelo. Ndineprojekthi, kwaye kukho iClickHouse. Indawo yethu yovavanyo yenzelwe yena kuphela - lo ngumatshini omncinci obonakalayo e-Hetzner ngee-euro ezingamashumi amabini, apho yonke into ibekwe khona. Ukwenza oku, sine-automation epheleleyo kwi-Ansible, kwaye ke ngoko, ngokomgaqo, akukho mahluko apho uya khona - kwiiseva ze-hardware okanye ukuhambisa nje koomatshini benyani.
Yintoni enokwenziwa? Kuya kuba kuhle ukubonelela ngomzekelo kumaxwebhu eClickHouse malunga nendlela yokuhambisa i-cluster encinci kwikhaya lakho - kwi-Docker, kwi-LXC, mhlawumbi udale i-Ansible playbook, kuba abantu abahlukeneyo banokusasazwa okuhlukeneyo. Oku kuya kwenza lula kakhulu. Xa uthatha kwaye uhambisa iqela kwimizuzu emihlanu, kulula kakhulu ukuzama ukufumanisa into ethile. Oku kulula ngakumbi, kuba ukuqengqeleka kwinguqulelo yemveliso ongakhange uyivavanye yindlela eya ndawo. Ngamanye amaxesha iyasebenza kwaye ngamanye amaxesha ayisebenzi. Kwaye ke, ithemba lempumelelo libi.
UMaxim Kotyakov, injineli ephezulu ngasemva uAvito: Ndiza kongeza kancinci malunga neendawo zovavanyo ukusuka kuluhlu lweengxaki ezijongene neenkampani ezinkulu. Sineqela elipheleleyo lokwamkelwa kweClickHouse; ngokwezicwangciso zedatha kunye noseto, yikopi echanekileyo yento ekwimveliso. Eli qela libekwe kwizikhongozelo ezinobulungisa ezinobuncinane bezibonelelo. Sibhala ipesenti ethile yedatha yemveliso apho, ngethamsanqa kunokwenzeka ukuphindaphinda umlambo eKafka. Yonke into ekhoyo ilungelelaniswe kwaye ilinganiswe - kokubini ngokomthamo kunye nokuhamba, kwaye, kwithiyori, zonke ezinye izinto ziyalingana, kufuneka ziziphathe njengemveliso ngokweemethrikhi. Yonke into enokudubula iqala iqengqeleke kule ndawo kwaye ishiywe apho kangangeentsuku ezininzi de ilungile. Kodwa ngokwemvelo, esi sisombululo siyabiza, sinzima kwaye sineendleko zenkxaso ezingekho zero.
UAlexey Milovidov: Ndiza kukuxelela ukuba yintoni indawo yokuvavanya abahlobo bethu baseYandex.Metrica. Iqela elinye lalinamaseva angama-600, elinye linama-360, kwaye kukho elesithathu kunye namaqela amaninzi. Indawo yovavanyo yenye yazo ziingceba nje ezimbini ezineekopi ezimbini kwindawo nganye. Kutheni iziqwenga ezimbini? Ukuze awuwedwa. Kwaye kufuneka kubekho ii-replicas. Ubuncinci bemali onokuthi uyihlawule.
Le meko-bume yovavanyo ikuvumela ukuba ujonge ukuba imibuzo yakho iyasebenza kwaye ukuba kukho nantoni na enkulu eyophukileyo. Kodwa kaninzi iingxaki zivela kwindalo eyahlukileyo ngokupheleleyo, xa yonke into isebenza, kodwa kukho utshintsho oluncinci kumthwalo.
Makhe ndikunike umzekelo. Sigqibe kwelokuba sifake inguqulelo entsha yeClickHouse. Ithunyelwe kwindawo yokuvavanya, iimvavanyo ezizenzekelayo ziye zagqitywa kwi-Yandex.Metrica ngokwayo, ethelekisa idatha kwinguqulo endala kunye nentsha, eqhuba yonke ipayipi. Kwaye ke, iimvavanyo eziluhlaza zeCI yethu. Kungenjalo ngesingade siyicebise le nguqulelo.
Yonke into iilungile. Siqala ukuqhubela phambili kwimveliso. Ndifumana umyalezo wokuba umthwalo kwiigrafu uye wanda amaxesha amaninzi. Sibuyisela umva inguqulelo. Ndijonge igrafu kwaye ndibone: umthwalo ngokwenene unyuke amaxesha amaninzi ngexesha lokukhutshwa, kwaye wehla umva xa bephuma. Emva koko saqala ukubuyisela umva inguqulelo. Nomthwalo wenyuka kwangolo hlobo, wawa kwangolo hlobo. Ngoko ke isiphelo yile: umthwalo uye wanda ngenxa yokuma, akukho nto imangalisayo.
Emva koko kwakunzima ukukholisa oogxa ukuba bafake inguqulelo entsha. Ndathi: "Kulungile, phuma. Gcina iminwe yakho, yonke into iya kusebenza. Ngoku umthwalo kwiigrafu uye wanda, kodwa yonke into ilungile. Bambelela apho." Ngokubanzi, senze oku, kwaye yiyo loo nto-inguqulelo yakhululwa ukuba ivelise. Kodwa phantse kulo lonke uyilo iingxaki ezifanayo zivela.
Ukubulala umbuzo kufanele ukuba ubulale imibuzo, kodwa akunjalo. Ngoba?
Umsebenzisi, uhlobo oluthile lomhlalutyi, weza kum kwaye wenza isicelo esibeka iCluster yam yeClickHouse. Enye indawo okanye iqela elipheleleyo, kuxhomekeke ekubeni yeyiphi ikopi okanye i-shard isicelo esiye kuyo. Ndiyabona ukuba zonke izixhobo ze-CPU kule seva ziseshelufini, yonke into ibomvu. Kwangaxeshanye, iClickHouse ngokwayo iphendula izicelo. Kwaye ndibhala: "Nceda undibonise, uludwe lwenkqubo, sesiphi isicelo esibangele obu budenge."
Ndifumana esi sicelo kwaye ndibhala ukubulala kuyo. Kwaye ndiyabona ukuba akukho nto yenzekayo. Umncedisi wam ukwishelufu, iClickHouse indinika imiyalelo ethile, ibonisa ukuba umncedisi uyaphila, kwaye yonke into ilungile. Kodwa ndinokuthotywa kuzo zonke izicelo zabasebenzisi, ukuthotywa kuqala ngeerekhodi kwiClickHouse, kwaye umbuzo wam wokubulala awusebenzi. Ngoba? Ndicinga ukuba umbuzo wokubulala bekufanele ubulale imibuzo, kodwa akunjalo.
Ngoku kuya kubakho impendulo engaqhelekanga. Inqaku lelokuba umbuzo wokubulala awubulali imibuzo.
Umbuzo wokubulala ujonga ibhokisi encinci ebizwa "Ndifuna lo mbuzo ubulawe." Kwaye isicelo ngokwaso sijonga le flegi xa kusenziwa ibhloko nganye. Ukuba isetiwe, isicelo siyayeka ukusebenza. Kuvela ukuba akukho mntu ubulala isicelo, yena ngokwakhe kufuneka ahlole yonke into kwaye ayeke. Kwaye oku kufuneka kusebenze kuzo zonke iimeko apho isicelo sikwimeko yokulungiswa kweebhloko zedatha. Iya kuqhuba ibhloko elandelayo yedatha, khangela iflegi, kwaye uyeke.
Oku akusebenzi kwiimeko apho isicelo sivaliwe kumsebenzi othile. Kuyinyani, kusenokwenzeka ukuba oku akusiyo imeko yakho, kuba, ngokutsho kwakho, isebenzisa itoni yemithombo yomncedisi. Kungenzeka ukuba oku akusebenzi kwimeko yokuhlelwa kwangaphandle kunye nakwezinye iinkcukacha. Kodwa ngokubanzi oku akufanele kwenzeke, yimpazamo. Kwaye ekuphela kwento endinokuyicebisa kukuhlaziya iClickHouse.
Indlela yokubala ixesha lokuphendula phantsi komthwalo wokufunda?
Kukho itafile egcina izinto ezidityanisiweyo - izixhobo zokubala ezahlukeneyo. Inani lemigca limalunga nezigidi ezilikhulu. Ngaba kunokwenzeka ukubala kwixesha lokuphendula eliqikelelweyo ukuba ugalela i-1K RPS kwizinto ze-1K?
Ukuqwalasela umxholo, sithetha ngomthwalo wokufunda, kuba akukho ngxaki ngokubhala - nokuba iwaka, nokuba ikhulu lamawaka, kwaye ngamanye amaxesha imiqolo emininzi yezigidi ingafakwa.
Izicelo zokufunda zahluke kakhulu. Ekukhetheni u-1, iClickHouse inokwenza malunga namashumi amawaka ezicelo ngomzuzwana, ngoko ke izicelo zesitshixo esinye ziya kuba sele zifuna ezinye izixhobo. Kwaye imibuzo enjalo yamanqaku iya kuba nzima ngakumbi kunezinye iziseko zexabiso elingundoqo, kuba kufundo ngalunye kuyimfuneko ukufunda ibhloko yedatha ngesalathisi. Isalathisi sethu asijongi irekhodi nganye, kodwa uluhlu ngalunye. Oko kukuthi, kuya kufuneka ufunde lonke uluhlu - le yimigca engama-8192 ngokungagqibekanga. Kwaye kuya kufuneka ucofe ibhloko yedatha ecinezelweyo ukusuka kwi-64 KB ukuya kwi-1 MB. Ngokwesiqhelo, imibuzo ekujoliswe kuyo ithatha imizuzwana embalwa ukuyigqiba. Kodwa olu lolona khetho lulula.
Makhe sizame i-arithmetic elula. Ukuba uphinda-phinda iimillisecond ezimbalwa ngewaka, ufumana imizuzwana embalwa. Kufana nokungathi akunakwenzeka ukugcina izicelo eziliwaka ngomzuzwana, kodwa eneneni inokwenzeka, kuba sinezinto ezininzi zeprosesa. Ke, ngokomgaqo, iClickHouse ngamanye amaxesha inokubamba i-1000 RPS, kodwa ngezicelo ezimfutshane, ezijolise ngokukodwa.
Ukuba ufuna ukulinganisa iClickHouse cluster ngenani lezicelo ezilula, ngoko ke ndincoma into elula kakhulu - ukwandisa inani leekopi kwaye uthumele izicelo kwi-random replica. Ukuba i-replica enye ibamba izicelo ezingamakhulu amahlanu ngesekhondi, eyinyani ngokupheleleyo, ii-replicas ezintathu ziya kusingatha iwaka elinesiqingatha.
Ngamanye amaxesha, ngokuqinisekileyo, unokuqwalasela i-ClickHouse ngenani eliphezulu leengongoma zokufunda. Yintoni efunekayo kule nto? Eyokuqala kukunciphisa i-granularity yesalathisi. Kule meko, akufanele kuncitshiswe kwenye, kodwa ngesiseko sokuba inani lokungeniswa kwisalathiso liya kuba zizigidi ezininzi okanye amashumi ezigidi kwiseva nganye. Ukuba itheyibhile inemiqolo yezigidi ezilikhulu, ke i-granularity inokusetwa ku-64.
Unokunciphisa ubungakanani bebhloko ecinezelweyo. Kukho iisetingi zale nto min ubungakanani block block, ubukhulu bebhloko yebhloko enkulu. Zinokuncitshiswa, zizaliswe kwakhona ngedatha, kwaye ke imibuzo ekujoliswe kuyo iya kukhawuleza. Kodwa okwangoku, iClickHouse ayisiyosiseko sexabiso ledatha. Inani elikhulu lezicelo ezincinci ngumthwalo we-antipattern.
Kirill Shvakov: Ndiza kunika iingcebiso ukuba kukho iiakhawunti eziqhelekileyo apho. Le yimeko eqhelekileyo xa iClickHouse igcina uhlobo oluthile lwekhawunta. Ndinomsebenzisi, uvela kwilizwe elinjalo nelinjalo, kunye nentsimi yesithathu, kwaye kufuneka ndonyuse into ngokunyukayo. Thatha i-MySQL, yenza isitshixo esikhethekileyo-kwi-MySQL sisitshixo esiphindiweyo, kwaye kwi-PostgreSQL yimpixano- kwaye wongeze uphawu lokudibanisa. Oku kuya kusebenza ngcono kakhulu.
Xa ungenayo idatha eninzi, akukho nto ingako ekusebenziseni i-ClickHouse. Kukho oovimba beenkcukacha eziqhelekileyo kwaye bakwenza oku kakuhle.
Ndingenza ntoni kwiClickHouse ukuze idatha engakumbi ikwi-cache?
Makhe sicinge ngemeko - amaseva ane-256 GB ye-RAM, kwi-ClickHouse yemihla ngemihla ithatha malunga ne-60-80 GB, kwi-peak - ukuya kwi-130. Yintoni enokuthi inikwe amandla kunye ne-tweaked ukwenzela ukuba idatha eninzi igcinwe kwi-cache kwaye, ngokufanelekileyo, kukho iihambo ezimbalwa kwidiski?
Ngokwesiqhelo, i-cache yekhasi lenkqubo yokusebenza yenza umsebenzi olungileyo woku. Ukuba uvula nje phezulu, jonga apho i-cached okanye ikhululekile - ikwatsho ukuba ingakanani i-cache - ngoko uya kuqaphela ukuba yonke imemori yamahhala isetyenziselwa i-cache. Kwaye xa ufunda le datha, ayiyi kufundwa kwidiski, kodwa kwi-RAM. Kwangaxeshanye, ndingatsho ukuba i-cache isetyenziswe ngokufanelekileyo kuba yidatha ecinezelweyo egcinwe kwi-cache.
Nangona kunjalo, ukuba ufuna ukukhawulezisa imibuzo elula ngakumbi, kuyenzeka ukuba uvule i-cache kwidatha ethotyiweyo ngaphakathi kweClickHouse. Ibizwa ngokuba i-cache engaxinzelelwanga. Kwifayile yoqwalaselo ye-config.xml, setha ubungakanani be-cache obunganyanzelwanga kwixabiso elifunekayo - ndincoma ukuba akukho ngaphezu kwesiqingatha se-RAM yamahhala, kuba abanye baya kungena phantsi kwekhasi lekhasi.
Ukongeza, kukho iisetingi zenqanaba lesicelo ezimbini. Isilungiselelo sokuqala - sebenzisa i-cache engaxinzelelwanga - kubandakanya ukusetyenziswa kwayo. Kucetyiswa ukuba iyenze kuzo zonke izicelo, ngaphandle kwezo zinzima, ezinokufunda zonke iinkcukacha kwaye zigungxule i-cache. Kwaye isicwangciso sesibini siyinto efana nenani eliphezulu lemigca yokusebenzisa i-cache. Inciphisa ngokuzenzekelayo imibuzo emikhulu ukuze idlule i-cache.
Ndingenza njani ukugcinwa_uqwalaselo logcino kwi-RAM?
Kuxwebhu olutsha lweClickHouse ndifunde icandelo elinxulumeneyo . Inkcazo iqulethe umzekelo nge-SSD ekhawulezayo.
Ndiyazibuza ukuba le nto inye inokuqwalaselwa njani ngememori eshushu yevolumu. Kwaye omnye umbuzo. Ukukhetha kusebenza njani nalo mbutho wedatha, iyakufunda iseti yonke okanye kuphela leyo ikwidiski, kwaye ingaba le datha icinezelwe kwinkumbulo? Kwaye icandelo langaphambili lisebenza njani kunye nombutho onjalo wedatha?
Esi sicwangciso sichaphazela ukugcinwa kweengceba zedatha, kwaye ifomathi yazo ayitshintshi nangayiphi na indlela.
Makhe sihlolisise.
Ungaqwalasela ukugcinwa kwedatha kwi-RAM. Yonke into elungiselelwe idiski yindlela yayo. Wenza isahlulelo se-tmpfs esinyuselwe kwindlela ethile kwindlela yefayile. Ukhankanya le ndlela njengendlela yokugcina idatha yesahlulelo esishushu, amaqhekeza edatha aqala ukufika kwaye abhalwe apho, yonke into ilungile.
Kodwa andikucebisi ukwenza oku ngenxa yokuthembeka okuphantsi, nangona ukuba ubuncinci ii-replicas ezintathu kumaziko ahlukeneyo edatha, ngoko kunokwenzeka. Ukuba kukho nantoni na eyenzekayo, idatha iya kubuyiselwa. Makhe sicinge ukuba iseva icinyiwe ngequbuliso kwaye yaphinda yavula. Ukwahlula kwaphinda kwafakwa, kodwa kwakungekho nto apho. Xa iseva yeClickHouse iqala, ibona ukuba ayinazo ezi ziqwenga, nangona, ngokwe-metadata ye-ZooKeeper, kufuneka zibe khona. Ujonga ukuba zeziphi iikopi ezinazo, azicele aze azikhuphele. Ngale ndlela idatha iya kubuyiselwa.
Ngale ndlela, ukugcina idatha kwi-RAM akwahlukanga ngokusisiseko ekuyigcineni kwidiski, kuba xa idatha ibhalwe kwidiski, iphinda iphele kwicache yephepha kwaye ibhalwe ngokwasemzimbeni kamva. Oku kuxhomekeke kukhetho lokunyuswa kwendlela yefayile. Kodwa ukuba kunokwenzeka, ndiza kuthi iClickHouse ayifani xa ufaka.
Kule meko, idatha kwi-RAM igcinwe ngendlela efanayo nediski. Umbuzo okhethiweyo ngendlela efanayo ukhetha iziqwenga ezifuna ukufundwa, ukhethe uluhlu oluyimfuneko lwedatha kwiinqununu, kwaye uzifunde. Kwaye ngaphambili isebenza ngokufanayo, nokuba idatha ibikwi-RAM okanye kwidiski.
Kude kube leliphi inani lamaxabiso ahlukileyo apho iKhardinali ephantsi iyasebenza?
I-Cardinality ephantsi iyilwe ngobuchule. Iqokelela izichazi-magama zedatha, kodwa zezasekhaya. Okokuqala, kukho izichazi-magama ezahlukeneyo kwisiqwenga ngasinye, kwaye okwesibini, nangaphakathi kwisiqwenga esinye zinokwahluka kuluhlu ngalunye. Xa inani lamaxabiso akhethekileyo lifikelela kwinani lomda-isigidi esinye, ndicinga ukuba-isichazi-magama sibekwe eshelufini kwaye kwenziwa esitsha.
Impendulo ibanzi: kuluhlu ngalunye lwendawo - yithi, kusuku ngalunye - kwindawo ethile ukuya kuthi ga kwisigidi samaxabiso akhethekileyo iKhardinali ephantsi iyasebenza. Emva koko kuya kuba lula ukubuyisela umva, apho izichazi-magama ezininzi ezahlukeneyo ziya kusetyenziswa, hayi nje esinye. Iza kusebenza phantse ngokufana nekholamu yomtya eqhelekileyo, mhlawumbi ingasebenzi kakuhle, kodwa akusayi kubakho kuthotywa kokusebenza okunzulu.
Zeziphi ezona zenzo zilungileyo zokukhangela umbhalo opheleleyo kwitheyibhile eneebhiliyoni ezintlanu zeerowu?
Kukho iimpendulo ezahlukeneyo. Eyokuqala kukuba iClickHouse ayisiyiyo injini yokukhangela epheleleyo. Kukho iinkqubo ezikhethekileyo zoku, umzekelo, и . Nangona kunjalo, ndiya ndibona abantu besithi bayatshintsha kwi-Elasticsearch ukuya kwiClickHouse.
Kutheni le nto isenzeka? Bachaza oku ngento yokuba i-Elasticsearch iyayeka ukujamelana nomthwalo kwimiqulu ethile, ngokuqala ngokwakhiwa kwezalathisi. Izalathisi ziba nzima kakhulu, kwaye ukuba uthumela ngokulula idatha kwi-ClickHouse, kuvela ukuba zigcinwa izihlandlo ezininzi ngokufanelekileyo ngokwevolumu. Kwangaxeshanye, imibuzo yokukhangela yayisoloko ingekho kangangokuba kwakuyimfuneko ukufumana ibinzana elithile kumthamo wonke wedatha, kuthathelwa ingqalelo i-morphology, kodwa eyahlukileyo ngokupheleleyo. Umzekelo, fumana ezinye ii-bytes ezilandelelanayo kwiilog kwiiyure ezimbalwa ezidlulileyo.
Kule meko, wenza isalathisi kwi-ClickHouse, indawo yokuqala eya kuba ngumhla kunye nexesha. Kwaye eyona datha inkulu inqunyulwe iya kusekelwa kuluhlu lomhla. Ngaphakathi koluhlu lomhla olukhethiweyo, njengomthetho, sele kuyenzeka ukwenza uphendlo olupheleleyo lokubhaliweyo, nokuba usebenzisa indlela ye-brute force usebenzisa i-like. Umsebenzisi onje kwiClickHouse ngoyena osebenza kakuhle njengomsebenzisi onokumfumana. Ukuba ufumana into engcono, ndixelele.
Kodwa kunjalo, kufana nokuskena ngokupheleleyo. Kwaye ukuskena okupheleleyo kunokucotha kungekuphela nje kwi-CPU, kodwa nakwidiski. Ukuba ngequbuliso une-terabyte enye yedatha ngosuku, kwaye ukhangela igama ngexesha lasemini, kuya kufuneka uskene i-terabyte. Kwaye mhlawumbi kwii-hard drives eziqhelekileyo, kwaye ekugqibeleni ziya kulayishwa ngendlela yokuba awuyi kukwazi ukufikelela kule seva nge-SSH.
Kule meko, ndikulungele ukunika enye iqhinga elincinci. Lulingo- lunokusebenza, lusenokungasebenzi. I-ClickHouse inezalathisi ezipheleleyo ezibhalwe ngohlobo lwetrigram Bloom filters. Oogxa bethu eArenadata sele bezamile ezi indexes, kwaye zihlala zisebenza kanye njengoko bekucetyiwe.
Ukuze uzisebenzise ngokuchanekileyo, kufuneka uqonde kakuhle ukuba zisebenza njani: yintoni i-trigram Bloom filter kunye nendlela yokukhetha ubungakanani bayo. Ndingatsho ukuba baya kunceda imibuzo kumanye amabinzana anqabileyo, ama-substrings angafane afumaneke kwidatha. Kule meko, i-subranges iya kukhethwa ngezalathisi kunye nedatha encinci iya kufundwa.
Kutshanje, iClickHouse yongeze imisebenzi ephambili ngakumbi yokukhangela okubhaliweyo okugcweleyo. Oku, okokuqala, ukukhangela i-substrings ye-substrings ngexesha elinye, kubandakanywa iinketho ezithintekayo, ezingenakuvakalelwa, kunye nenkxaso ye-UTF-8 okanye kuphela i-ASCII. Khetha eyona isebenzayo oyifunayo.
Khangela amabinzana aqhelekileyo amaninzi kwipasi enye nayo ivele. Awudingi ukubhala u-X njengomtya ongezantsi omnye okanye u-X njengomtya ongezantsi. Ubhala kwangoko, kwaye yonke into yenziwa ngokufanelekileyo ngokusemandleni.
Okwesithathu, ngoku kukho uphendlo oluqikelelweyo lwe-regexps kunye nophendlo oluqikelelweyo lwemitya engaphantsi. Ukuba umntu upelwe kakubi igama, liya kukhangelwa owona mlinganiselo uphezulu.
Yeyiphi eyona ndlela ilungileyo yokulungiselela ukufikelela kwiClickHouse kwinani elikhulu labasebenzisi?
Sixelele indlela engcono yokulungiselela ukufikelela kwinani elikhulu labathengi kunye nabahlalutyi. Uwenza njani umgca, ukubeka phambili imibuzo emininzi ngexesha elinye, kwaye ngaziphi izixhobo?
Ukuba i-cluster inkulu ngokwaneleyo, ke isisombululo esihle siya kuba kukuphakamisa ezinye iiseva ezimbini, eziza kuba yindawo yokungena kubahlalutyi. Oko kukuthi, musa ukuvumela abahlalutyi ukuba bafikelele kwi-shards ethile kwi-cluster, kodwa ngokulula benze iiseva ezimbini ezingenanto, ngaphandle kwedatha, kwaye uqwalasele amalungelo okufikelela kubo. Kule meko, izicwangciso zabasebenzisi bezicelo ezisasazwayo zidluliselwa kwiiseva ezikude. Oko kukuthi, uqwalasela yonke into kwezi mncedisi zimbini, kwaye izicwangciso zinefuthe kwiqela lonke.
Ngokomgaqo, ezi seva azinadatha, kodwa isixa se-RAM kubo sibaluleke kakhulu ekwenzeni izicelo. Idiski inokusetyenziselwa idata yethutyana ukuba udibaniso lwangaphandle okanye uhlengahlengiso lwangaphandle luvuliwe.
Kubalulekile ukujonga izicwangciso ezihambelana nayo yonke imida enokwenzeka. Ukuba ngoku ndiya kwiqela leYandex.Metrica njengomhlalutyi kwaye ndicele isicelo khetha count ukusuka hits, ngoko ke ngoko nangoko ndiya kunikwa ngaphandle kokuba andinako ukwenza isicelo. Elona nani liphezulu lemiqolo endivumeleke ukuba ndiyiskene likhulu leebhiliyoni, kwaye xa lilonke kukho amashumi amahlanu eetriliyoni kuyo kwitheyibhile enye kwiqela. Lo ngumda wokuqala.
Masithi ndisusa umda womqolo ndiphinde ndiqhube umbuzo kwakhona. Emva koko ndiza kubona oku kulandelayo - ukusetha kwenziwe ukuba kusebenze nyanzela isalathisi ngokomhla. Andikwazi ukugqiba umbuzo ukuba khange ndiluchaze uluhlu lomhla. Awunyanzelekanga ukuba uthembele kubahlalutyi ukuyichaza ngesandla. Imeko eqhelekileyo kuxa uluhlu lomhla lubhalwa apho umhla wesiganeko phakathi kweveki. Kwaye emva koko bachaza ngokulula isibiyeli kwindawo engafanelekanga, kwaye endaweni kwaye kwavela ukuba okanye-okanye umdlalo we-URL. Ukuba akukho mda, iya kugaqa kwikholamu ye-URL kwaye ichithe nje itoni yezibonelelo.
Ukongeza, iClickHouse inesetingi ezimbini eziphambili. Ngelishwa, bakudala kakhulu. Omnye ubizwa ngokulula phambili. Ukuba okuphambili ≠ 0, kunye nezicelo ezinokubaluleka okuthile ziphunyezwa, kodwa isicelo esinexabiso lokuqala elingaphantsi kwe, elithetha ukuba umba ophambili ongaphezulu, uyaphunyezwa, ngoko isicelo esinexabiso eliphambili langaphezulu, elithetha into ephambili esezantsi. , inqunyanyisiwe kwaye ayizukusebenza kwaphela ngeli xesha.
Esi sisimo esikrwada kakhulu kwaye asifanelekanga kwiimeko apho iqela linomthwalo oqhubekayo. Kodwa ukuba unezicelo ezimfutshane, eziqhumayo ezibalulekileyo, kwaye iqoqo lihlala lingasebenzi, olu seto lufanelekile.
Ulungiselelo olulandelayo oluphambili olubizwa ngokuba Ukubaluleka komsonto we-OS. Ibeka ngokulula ixabiso elimnandi layo yonke imisonto yokwenziwa kwesicelo yomcwangcisi weLinux. Isebenza njalo-njalo, kodwa isasebenza. Ukuba ubeka elona xabiso lincinci limnandi - lelona xabiso likhulu, kwaye ke ngoko eyona nto iphantsi - kwaye ibeke -19 yezicelo ezinokubaluleka okuphezulu, ngoko ke iCPU iya kudla izicelo eziphambili eziphantsi malunga namaxesha amane ngaphantsi kwezo ziphezulu.
Kufuneka kwakhona uqwalasele ixesha lokwenziwa kwesicelo esiphezulu - yithi, imizuzu emihlanu. Esona santya sincinci sokwenziwa kombuzo yeyona nto ipholileyo. Olu seto sele lukhona ixesha elide, kwaye ayifuneki kuphela ukungqina ukuba iClickHouse ayicothi, kodwa ukuyinyanzela.
Khawufane ucinge, useta: ukuba ezinye iinkqubo zemibuzo zingaphantsi kwesigidi semigca ngomzuzwana, awukwazi ukwenza oko. Oku kulihlazisa igama lethu elihle, ugcino-lwazi lwethu oluhle. Masiyivale nje lento. Kukho iisetingi ezimbini ngokwenene. Omnye ubizwa isantya sokwenza imiz - kwimigca ngesekondi nganye, kwaye eyesibini ibizwa ngokuba lixesha lokuvala phambi kokujonga isantya sokwenza imiz - imizuzwana elishumi elinesihlanu ngokungagqibekanga. Oko kukuthi, imizuzwana elishumi elinesihlanu inokwenzeka, kwaye ke, ukuba iyacotha, vele ulahle ngaphandle kwaye usikhuphe isicelo.
Kananjalo kufuneka usete izabelo. I-ClickHouse inomsebenzi we-quota eyakhelwe-ngaphakathi obala ukusetyenziswa kobutyebi. Kodwa, ngelishwa, kungekhona izixhobo ze-hardware ezifana ne-CPU, iidiski, kodwa ezinengqiqo - inani lezicelo ezicutshungulwayo, imigca kunye neebhayithi ezifundwayo. Kwaye ungaqwalasela, umzekelo, ubuninzi bezicelo ezilikhulu kwimizuzu emihlanu kunye nezicelo eziliwaka ngeyure.
Kutheni ibalulekile nje? Ngenxa yokuba eminye imibuzo yohlalutyo iya kwenziwa ngesandla ngokuthe ngqo kwiClickHouse client. Kwaye konke kuya kulunga. Kodwa ukuba unabahlalutyi abaphambili kwinkampani yakho, baya kubhala iskripthi, kwaye kunokubakho impazamo kwiskripthi. Kwaye le mpazamo iya kubangela ukuba isicelo senziwe kwilophu engapheliyo. Yile nto kufuneka sizikhusele kuyo.
Ngaba kunokwenzeka ukunika iziphumo zombuzo omnye kubathengi abalishumi?
Sinabasebenzisi abaninzi abathanda ukuza nezicelo ezinkulu kakhulu ngexesha elinye. Isicelo sikhulu kwaye, ngokomgaqo, senziwe ngokukhawuleza, kodwa ngenxa yokuba zininzi izicelo ezinjalo ngexesha elifanayo, kuba buhlungu kakhulu. Ngaba kunokwenzeka ukwenza isicelo esifanayo, esifike izihlandlo ezilishumi ngokulandelelana, kanye, kwaye unike umphumo kubaxhasi abalishumi?
Ingxaki kukuba asinazo iziphumo ze-cache okanye i-cache yedatha ephakathi. Kukho i-cache yekhasi lenkqubo yokusebenza, eya kukuthintela ukuba ufunde idatha kwidiski kwakhona, kodwa, ngelishwa, idatha iya kucinywa, ikhutshwe kwaye iphinde iphinde ihlaziywe.
Ndingathanda ngandlela thile ukunqanda oku, nokuba kungokugcina idatha ephakathi, okanye ngokulungelelanisa imibuzo efanayo kuhlobo oluthile lomgca kunye nokongeza i-cache yeziphumo. Okwangoku sinesicelo esinye sokutsala kuphuhliso esongeza i-cache yesicelo, kodwa kuphela kwi-subqueries ngaphakathi kunye nokujoyina amacandelo - oko kukuthi, isisombululo asiphelelanga.
Noko ke, sikwajamelene nemeko enjalo. Umzekelo okhethekileyo we-canonical yimibuzo enephepha. Kukho ingxelo, inamaphepha amaninzi, kwaye kukho isicelo somda we-10. Emva koko into efanayo, kodwa umda we-10,10. Kulandele elinye iphepha elilandelayo. Kwaye umbuzo ngulo, kutheni sibala yonke le nto ngalo lonke ixesha? Kodwa ngoku asikho isisombululo, kwaye akukho ndlela yakukuphepha.
Kukho esinye isisombululo esibekwe njengemoto ecaleni kweClickHouse - .
Kirill Shvakov: I-ClickHouse Proxy ine-limiter eyakhelwe-ngaphakathi kunye ne-cache yeziphumo ezakhelwe ngaphakathi. Izicwangciso ezininzi zenziwa apho ngenxa yokuba ingxaki efanayo yayisonjululwa. Ummeli ikuvumela ukuba unciphise izicelo ngokuzibeka emgceni kwaye uqwalasele ukuba i-cache yesicelo ihlala ixesha elingakanani. Ukuba izicelo bezifana ngenene, uMmeli uza kuzithumela amaxesha amaninzi, kodwa uya kuClickHouse kube kanye kuphela.
I-Nginx nayo ine-cache kwinguqulo yamahhala, kwaye oku kuya kusebenza. I-Nginx ide ibe nezicwangciso ukuba izicelo zifika ngaxeshanye, ziya kuthothisa abanye de kugqitywe enye. Kodwa kuClickHouse Proxy ukuba ukuseta kwenziwe ngcono kakhulu. Yenziwe ngokukodwa kwiClickHouse, ngokukodwa kwezi zicelo, ngoko ifanelekile ngakumbi. Ewe, kulula ukuyifaka.
Kuthekani ngemisebenzi engahambelaniyo kunye neembono ezenziweyo?
Kukho ingxaki yokuba ukusebenza kunye ne-injini yokubuyisela i-asynchronous - okokuqala idatha ibhaliwe, emva koko iyawa. Ukuba i-tablet ebonakalayo kunye nezinye ii-aggregates zihlala phantsi kophawu, okuphindwe kabini kuya kubhalwa kuyo. Kwaye ukuba akukho ngqiqweni entsonkothileyo, ngoko ke idatha iya kuphindwa. Yintoni onokuyenza ngayo?
Kukho isisombululo esicacileyo - ukuphumeza i-trigger kudidi oluthile lwe-matviews ngexesha lokusebenza kokuwa kwe-asynchronous. Ngaba kukho iibhulethi zesilivere okanye izicwangciso zokuphumeza ukusebenza okufanayo?
Kufanelekile ukuqonda ukuba kusebenza njani ukubuyisela. Into endiza kukuxelela yona ngoku ayihambelani nombuzo, kodwa ukuba kufanelekile ukukhumbula.
Xa ufaka kwitafile ephindwe kabini, kukho ukuphindaphinda kweebhloko zonke ezifakiweyo. Ukuba uphinde ufake ibhloko enye equlethe inani elifanayo lemiqolo efanayo ngokulandelelana okufanayo, ngoko idatha iyakhutshwa. Uya kufumana "Ok" ekuphenduleni ukufaka, kodwa ngokwenene ipakethi enye yedatha iya kubhalwa, kwaye ayiyi kuphinda iphindwe.
Oku kuyimfuneko ngokuqinisekileyo. Ukuba ufumana "Kulungile" ngexesha lokufakwa, ngoko idatha yakho ifakiwe. Ukuba ufumana impazamo kwiClickHouse, oko kuthetha ukuba abafakwanga kwaye kufuneka uphinde ufakelo. Kodwa ukuba uxhulumaniso luphukile ngexesha lokufakwa, ngoko awuyazi ukuba idatha ifakiwe okanye hayi. Ekuphela kwenketho kukuphinda ufakelo kwakhona. Ukuba idatha ifakwe ngenene kwaye uphinde wayifaka, kukho udibaniso lwebhloko. Oku kuyafuneka ukuphepha ukuphinda-phinda.
Kwaye ibalulekile indlela esebenza ngayo kwiimbono ezibonakalayo. Ukuba idatha ikhutshiwe xa ifakwe kwitheyibhile engundoqo, ngoko ayiyi kungena kumbono wezinto ezibonakalayo.
Ngoku malunga nombuzo. Imeko yakho intsonkothe ngakumbi kuba urekhoda iimpinda zemigca nganye. Oko kukuthi, ayisiyiyo yonke ipakethi ephindwe kabini, kodwa imigca ethile, kwaye iwa ngasemva. Enyanisweni, idatha iya kuwela kwitheyibhile eyintloko, kodwa idatha engagqitywanga iya kumbono wezinto ezibonakalayo, kwaye ngexesha lokudibanisa akukho nto iya kwenzeka kwiimbono eziphathekayo. Ngenxa yokuba umbono owenziweyo awuyonto ingaphezulu kwe-insert trigger. Ngexesha leminye imisebenzi, akukho nto eyongezelelweyo eyenzekayo kuyo.
Kwaye andinakukwenza wonwabe apha. Kufuneka nje ujonge isisombululo esithile kule meko. Umzekelo, ngaba kunokwenzeka ukuyidlala kwakhona kwimbono eyenziweyo, kwaye indlela yokudibanisa inokusebenza ngendlela efanayo. Kodwa ngelishwa, akunjalo rhoqo. Ukuba iyadityaniswa, ayizukusebenza.
Kirill Shvakov: Sasikwakhiwe ngezinto zokudondolozela emva phaya. Bekukho ingxaki yokuba kukho intengiso yentengiso, kwaye kukho idata esinokuyibonisa ngexesha lokwenyani- ezi zizimvo nje. Azifane ziphindaphindwe, kodwa ukuba oku kuyenzeka, siya kuzitshonisa kamva. Kwaye kukho izinto ezingenakuphinda ziphindwe-ukucofa kunye neli bali lonke. Kodwa ndandifuna nokubabonisa ngokukhawuleza.
Zenziwa njani iimbono ezenziweyo? Kwakukho iimbono apho kubhalwe ngokuthe ngqo - kubhalwe kwidatha eluhlaza, kwaye kubhalwe kwiimbono. Apho, ngexesha elithile idatha ayichanekanga kakhulu, iphindwe kabini, njalo njalo. Kwaye kukho isahlulo sesibini setafile, apho zikhangeleka zifana ncakasana neembono ezenziweyo, oko kukuthi, ziyafana ngokupheleleyo ngesakhiwo. Kanye ngexesha siphinda sibala idatha, sibale idatha ngaphandle kokuphindaphinda, bhala kwezo tafile.
Sahamba nge-API - oku akuyi kusebenza kwi-ClickHouse ngesandla. Kwaye i-API ibonakala: xa ndinomhla wokongezwa kokugqibela kwitheyibhile, apho kuqinisekisiwe ukuba idatha echanekileyo sele ibaliwe, kwaye yenza isicelo kwitafile enye nakwenye itafile. Ukusuka kwesinye isicelo sikhetha ukuya kuthi ga kwixesha elithile, kwaye kwelinye lifumana into engekabalwa. Kwaye iyasebenza, kodwa hayi ngeClickHouse kuphela.
Ukuba unolunye uhlobo lwe-API - kubahlalutyi, kubasebenzisi - ngoko, ngokomgaqo, oku kukhetho. Uhlala ubala, ubala rhoqo. Oku kunokwenziwa kanye ngosuku okanye ngelinye ixesha. Uzikhethela uluhlu ongaludingiyo kwaye alubalulekanga.
I-ClickHouse inezigodo ezininzi. Ndingayibona njani yonke into eyenzekayo kumncedisi ngokujonga nje?
I-ClickHouse inenani elikhulu kakhulu leelogi ezahlukeneyo, kwaye eli nani liyanda. Kwiinguqulelo ezintsha, ezinye zazo zide zenziwe ngokuzenzekelayo; kwiinguqulelo ezindala kufuneka zenziwe ukuba zenziwe xa zihlaziya. Noko ke, zisanda ngakumbi. Ekugqibeleni, ndingathanda ukubona ukuba kwenzeka ntoni ngomncedisi wam ngoku, mhlawumbi kuhlobo oluthile lwesishwankathelo sedeshibhodi.
Ngaba unalo iqela leClickHouse, okanye amaqela abahlobo bakho, axhasa umsebenzi othile wedeshibhodi esele ilungile eza kubonisa ezi zigodo njengemveliso egqityiweyo? Ekugqibeleni, ukujonga nje iilog kwiClickHouse kulungile. Kodwa kuya kuba kuhle kakhulu ukuba sele ilungiselelwe ngendlela yedeshibhodi. Bendiya kufumana ukukhaba kuyo.
Kukho iideshibhodi, nangona zingekho emgangathweni. Kwinkampani yethu, malunga namaqela angama-60 asebenzisa i-ClickHouse, kwaye eyona nto imangalisayo kukuba uninzi lwazo luneedeshibhodi abazenzele zona, kunye nezahluke kancinci. Amanye amaqela asebenzisa ukufakwa kweYandex.Cloud yangaphakathi. Kukho iingxelo esele zenziwe, nangona ingezizo zonke eziyimfuneko. Abanye banezabo.
Oogxa bam baseMetrica banedeshibhodi yabo eGrafana, nam ndineyam yeqela labo. Ndijonge izinto ezifana ne-cache hit ye-serif cache. Kwaye kunzima ngakumbi kukuba sisebenzisa izixhobo ezahlukeneyo. Ndidale ideshibhodi yam ndisebenzisa isixhobo esidala kakhulu esibizwa ngokuba yiGraphite-web. Umbi kwaphela. Kwaye ndisayisebenzisa ngale ndlela, nangona iGrafana inokuba lula kwaye intle.
Into esisiseko kwiideshibhodi iyafana. Ezi ziimetriki zesixokelelwano zeqela: CPU, imemori, idiski, inethiwekhi. Abanye - inani lezicelo zangaxeshanye, inani lokudityaniswa kwangaxeshanye, inani lezicelo ngesekhondi, inani eliphezulu lee-chunks zeMergeTree table partitions, i-replication lag, ubungakanani bomgca wokuphindaphinda, inani lemiqolo efakiweyo ngesekhondi, inani leebhloko ezifakiweyo ngomzuzwana. Yiyo yonke le nto ayifumanekanga kwiilog, kodwa kwiimethrikhi.
UVladimir Kolobaev: U-Alexey, ndingathanda ukuyilungisa kancinci. Kukho iGrafana. IGrafana inomthombo wedatha, eyiClickHouse. Oko kukuthi, ndingenza izicelo ezivela eGrafana ngqo kwiClickHouse. I-ClickHouse inetafile enezigodo, iyafana kumntu wonke. Ngenxa yoko, ndifuna ukufikelela kule tafile yelog eGrafana kwaye ndibone izicelo ezenziwa ngumncedisi wam. Kuya kuba kuhle ukuba nedashboard efana nale.
Ndiyikhwele ngokwam ibhayisekile. Kodwa ndinombuzo - ukuba konke kusemgangathweni, kwaye iGrafana isetyenziswe ngumntu wonke, kutheni i-Yandex ingenayo ideshibhodi esemthethweni?
Kirill Shvakov: Enyanisweni, umthombo wedatha oya kwiClickHouse ngoku uxhasa i-Altinity. Kwaye ndifuna nje ukunika i-vector yendawo yokumba kunye nokuba ngubani oza kutyhala. Unokubabuza, kuba iYandex isenza iClickHouse, kwaye hayi ibali elijikelezayo. IAltinity yeyona nkampani iphambili ngoku ekhuthaza iClickHouse. Abayi kumshiya, kodwa baya kumxhasa. Ngenxa yokuba, ngokomgaqo, ukulayisha ideshibhodi kwiwebhusayithi yeGrafana, kufuneka ubhalise kuphela kwaye uyilayishe - akukho ngxaki zikhethekileyo.
UAlexey Milovidov: Kunyaka ophelileyo, iClickHouse yongeze amandla amaninzi okuphendula imibuzo. Kukho iimetriki zesicelo ngasinye ekusetyenzisweni kobutyebi. Kwaye kutsha nje, songeze iprofayile yombuzo ekumgangatho ophantsi ukubona apho umbuzo uchitha khona yonke imillisecond. Kodwa ukusebenzisa lo msebenzi, kufuneka ndivule umxhasi we-console kwaye ndichwetheze isicelo, endihlala ndisilibala. Ndayigcina ndaweni ithile kwaye ndihlala ndilibala ukuba kuphi kanye kanye.
Ndinqwenela ukuba bekukho isixhobo esisandula ukuthi, nantsi imibuzo yakho enzima, ehlanganiswe ngokweklasi yombuzo. Ndacinezela enye, kwaye babendixelela ukuba yiyo loo nto inzima. Akukho sisombululo sinjalo ngoku. Kwaye iyamangalisa ngokwenene into yokuba xa abantu bendibuza: “Khawutsho, ngaba zikhona iideshibhodi esele zenziwe eGrafana?”, ndithi: “Yiya kwiwebhusayithi yaseGrafana, kukho “iiDashboards” zasekuhlaleni, kukho nedashboard. ukusuka eDimka, kukho ideshibhodi evela eKostyan. Andazi ukuba yintoni, khange ndiyisebenzise ngokwam. "
Uluphembelela njani udibaniso ukuze umncedisi angawisi kwi-OOM?
Ndinetafile, kukho isahlulelo esinye kuphela kwitafile, yiReplaceMergeTree. Ndineminyaka emine ndibhala idatha kuyo. Kwafuneka ndenze utshintsho kuyo kwaye ndicime idatha ethile.
Ndenze oku, kwaye ngexesha lokuqhutyelwa kwesi sicelo, yonke imemori kuzo zonke iiseva kwiqela idliwe, kwaye zonke iiseva kwi-cluster zangena kwi-OOM. Emva koko bonke bavuka kunye, baqala ukudibanisa lo msebenzi mnye, le block yedatha, kwaye bawela kwi-OOM kwakhona. Baphinda baphakama bawa kwakhona. Kwaye le nto ayizange iyeke.
Emva koko kwavela ukuba le yayiyi-bug eneneni eyalungiswa ngabafana. Oku kuhle kakhulu, enkosi kakhulu. Kodwa intsalela yasala. Kwaye ngoku, xa ndicinga ngokwenza uhlobo oluthile lokudibanisa etafileni, ndinombuzo - kutheni ndingenako ukuphembelela oku kudityaniswa? Ngokomzekelo, banqande ngenani le-RAM efunekayo, okanye, ngokomgaqo, ngexabiso eliya kuqhuba le theyibhile.
Ndinetafile ebizwa ngokuba yi "Metrics", nceda uyiqhubele kum ngemisonto emibini. Akukho sidingo sokwenza udibaniso ezilishumi okanye ezintlanu ngokunxuseneyo, yenze kabini. Ndicinga ukuba ndinenkumbulo eyaneleyo yesibini, kodwa isenokunganeli ukwenza ishumi. Kutheni uloyiko luhleli? Ngenxa yokuba itheyibhile iyakhula, kwaye ngenye imini ndiya kujongana nemeko ethi, ngokomgaqo, ayisekho ngenxa ye-bug, kodwa ngenxa yokuba idatha iya kutshintsha kwisixa esikhulu kangangokuba andiyi kuba nememori eyaneleyo kwi-bug. umncedisi. Kwaye ke iseva iya kuntlitheka kwi-OOM xa idibanisa. Ngaphezu koko, ndingalurhoxisa utshintsho, kodwa uMerji akasekho.
Uyazi, xa udibanisa, umncedisi akayi kuwela kwi-OOM, kuba xa udibanisa, inani le-RAM lisetyenziselwa kuphela uluhlu oluncinci lwedatha. Ngoko yonke into iya kulunga kungakhathaliseki inani ledatha.
UVladimir Kolobaev: Kulungile. Nanku umzuzu wokuba emva kokuba i-bug ilungisiwe, ndizikhuphele inguqulelo entsha, kwaye kwenye itafile, encinci, apho kukho izahlulo ezininzi, ndenze umsebenzi ofanayo. Kwaye ngexesha lokudibanisa, malunga ne-100 GB ye-RAM yatshiswa kumncedisi. Ndine-150 ehlala, i-100 idliwe, kunye ne-50 GB yefestile eshiywe, ngoko andizange ndiwele kwi-OOM.
Yintoni ngoku endikhuselayo ekuweleni kwi-OOM ukuba ngokwenene idla i-100 GB ye-RAM? Yintoni enokuyenza ukuba ngequbuliso i-RAM ekudibaneni iyaphela?
UAlexey Milovidov: Kukho ingxaki enjalo ukuba ukusetyenziswa kwe-RAM ngokukodwa ukudibanisa akukhawulelwanga. Kwaye ingxaki yesibini kukuba ukuba uhlobo oluthile lokudibanisa lunikezelwe, ngoko kufuneka lwenziwe kuba lurekhodwe kwilog yophindaphindo. Ilog yophindaphindo zizenzo ezifunekayo ukuzisa umfanekiso kwisimo esingaguqukiyo. Ukuba awenzi ujikelo lwezandla oluzakubuyisela umva olushicilelo lophindaphindo, udibaniso kuyakufuneka lwenziwe ngendlela enye okanye enye.
Ewe kunjalo, akuyi kuba yinto engafanelekanga ukuba nomda we-RAM "ukuba nje" ukhusela kwi-OOM. Akuyi kunceda ukudibanisa ukugqiba, kuya kuqala kwakhona, ukufikelela kumda othile, ukuphosa okungafaniyo, kwaye emva koko uqale kwakhona - akukho nto ilungileyo iya kuza kule nto. Kodwa ngokomgaqo, kuya kuba luncedo ukwazisa esi sithintelo.
Uzakuphuhliswa njani umqhubi weGolang weClickHouse?
Umqhubi weGolang, owabhalwa nguKirill Shvakov, ngoku uxhaswa ngokusemthethweni liqela leClickHouse. Yena , ngoku umkhulu kwaye uyinyani.
Inqaku elincinci. Kukho uvimba omangalisayo kunye nothandekayo weendlela eziqhelekileyo zocwangco olungenasiphelo - le yiVertica. Bakwanabo nomqhubi wabo we-python osemthethweni, oxhaswa ngabaphuhlisi beVertica. Kwaye amaxesha amaninzi kwenzeka ukuba iinguqulelo zokugcina kunye neenguqulelo zomqhubi zahlukana kakhulu, kwaye umqhubi ngaxa lithile wayeka ukusebenza. Kwaye inqaku lesibini. Inkxaso yalo mqhubi osemthethweni, kubonakala kum, iqhutywa yinkqubo "yengono" - ubabhalela umcimbi, kwaye ixhomekeke ngonaphakade.
Ndinemibuzo emibini. Ngoku umqhubi weGolang kaKirill phantse uyindlela engagqibekanga yokunxibelelana neGolang ngeClickHouse. Ngaphandle kokuba umntu usanxibelelana nge-http ujongano kuba ethanda ngolo hlobo. Ingaba uphuhliso lwalo mqhubi luya kuqhubeka njani? Ngaba iya kulungelelaniswa nalo naluphi na utshintsho oluqhekezayo kwindawo yokugcina? Kwaye yintoni inkqubo yokuqwalasela umcimbi?
Kirill Shvakov: Eyokuqala yindlela yonke into ehlelwe ngayo ngokusemthethweni. Le ngongoma ayizange ixoxwe, ngoko andinanto yokuphendula.
Ukuphendula umbuzo malunga nomba, sifuna imbali encinci yomqhubi. Ndasebenza kwinkampani enedatha eninzi. Kwakuyi-advertising spinner enenani elikhulu leziganeko ezifuna ukugcinwa kwindawo ethile. Kwaye ngaxa lithile kwavela iClickHouse. Sayigcwalisa ngedatha, kwaye ekuqaleni yonke into yayilungile, kodwa emva koko iClickHouse yawa. Ngelo xesha sagqiba ekubeni asiyidingi.
Kunyaka kamva, sabuyela kumbono wokusebenzisa iClickHouse, kwaye kwafuneka sibhale idatha apho ngandlela thile. Umyalezo wentshayelelo wawulo: i-hardware ibuthathaka kakhulu, kukho izixhobo ezimbalwa. Kodwa besihlala sisebenza ngale ndlela, kwaye ke sijonge kwiprotocol yasekhaya.
Oko sasisebenza eGo, kwacaca ukuba sifuna umqhubi weGo. Ndiyenze phantse ixesha elizeleyo - yayingumsebenzi wam. Sayizisa kwinqanaba elithile, kwaye ngokomgaqo akukho mntu wayecinga ukuba nabani na ngaphandle kwethu uya kuyisebenzisa. Emva koko i-CloudFlare yeza nengxaki efanayo kanye, kwaye ixesha elithile sisebenza nabo ngokutyibilikayo, kuba babenemisebenzi efanayo. Ngaphezu koko, sikwenzile oku kokubini kwiClickHouse ngokwethu nakumqhubi.
Ngexesha elithile, ndiyekile ukuyenza, kuba umsebenzi wam ngokweClickHouse kunye nomsebenzi watshintsha kancinci. Ngoko ke imiba ayivalwanga. Ngamaxesha athile, abantu abafuna into ngokwabo bazibophelele kwindawo yokugcina. Emva koko ndijonge isicelo sokutsala kwaye ngamanye amaxesha ndide ndihlele into ngokwam, kodwa oku kwenzeka kunqabile.
Ndifuna ukubuyela kumqhubi. Kwiminyaka eliqela eyadlulayo, xa yonke le nto yaqalayo, iClickHouse yayahlukile kwaye inamandla ahlukeneyo. Ngoku sinokuqonda malunga nendlela yokwenza kwakhona umqhubi ukuze asebenze kakuhle. Ukuba oku kuyenzeka, ke uguqulelo 2 aluyi kuhambelani kuyo nayiphi na imeko ngenxa yeentonga eziqokelelweyo.
Andazi ukuba ndingawuququzelela njani lo mba. Andinaxesha lininzi mna. Ukuba abanye abantu bamgqibile umqhubi, ndingabanceda ndibaxelele ukuba benze ntoni. Kodwa inxaxheba esebenzayo yeYandex ekuphuhlisweni kweprojekthi ayikaxoxwa.
UAlexey Milovidov: Enyanisweni, akukho bureferensi malunga naba baqhubi okwangoku. Into kuphela kukuba zingeniswa kumbutho osemthethweni, oko kukuthi, lo mqhubi uqatshelwa njengesisombululo esisemthethweni sokuhamba. Kukho abanye abaqhubi, kodwa beza ngokwahlukeneyo.
Asinalo naluphi na uphuhliso lwangaphakathi lwaba baqhubi. Umbuzo kukuba ngaba sinokuqesha umntu ngamnye, kungekhona kulo mqhubi, kodwa kuphuhliso lwabo bonke abaqhubi basekuhlaleni, okanye sinokufumana umntu ovela ngaphandle.
Isichazi-magama sangaphandle asilayishi emva kokuqalisa ngokutsha nge-lazy_load setting evuliwe. Kwenziwe ntoni?
Sine-lazy_load setting enikwe amandla, kwaye emva kokuba iseva iqalwe ngokutsha, isichazi-magama asilayishi ngokwaso. Iphakanyiswa kuphela emva kokuba umsebenzisi efikelele kwesi sichazi-magama. Kwaye okokuqala ndifikelela kuyo, inika impazamo. Ngaba kunokwenzeka ukuba ngandlela ithile ulayishe ngokuzenzekelayo izichazi-magama usebenzisa iClickHouse, okanye ngaba kufuneka uhlale ulawula ukulungela kwazo ngokwakho ukuze abasebenzisi bangafumani iimpazamo?
Mhlawumbi sinoguqulelo oludala lweClickHouse, ke isichazi-magama asizange silayishe ngokuzenzekelayo. Ngaba oku kunokuba njalo?
Okokuqala, izichazi-magama zinokunyanzelwa ukuba zilayishwe kusetyenziswa umbuzo inkqubo yokulayisha kwakhona izichazi-magama. Okwesibini, malunga nempazamo - ukuba isichazi-magama sele silayishiwe, ngoko imibuzo iya kusebenza ngokusekelwe kwidatha elayishiweyo. Ukuba isichazi-magama asikalayishwa, siya kulayishwa ngokuthe ngqo ngexesha lesicelo.
Oku akulunganga kakhulu kwizichazi-magama ezinzima. Umzekelo, kufuneka utsale imiqolo yesigidi ukusuka kwi-MySQL. Umntu wenza ukhetho olulula, kodwa oku kukhetha kuya kulinda imiqolo yesigidi esifanayo. Kukho izisombululo ezibini apha. Eyokuqala kukucima lazy_load. Okwesibini, xa iseva iphezulu, ngaphambi kokubeka umthwalo kuyo, yenza inkqubo yokulayisha kwakhona isichazi-magama okanye wenze nje umbuzo osebenzisa isichazi-magama. Emva koko isichazi-magama siya kulayishwa. Kufuneka ulawule ukufumaneka kwezichazi-magama ngolungiselelo lwe-lazy_load, kuba i-ClickHouse ayizilayishi ngokuzenzekelayo.
Impendulo kumbuzo wokugqibela ingaba inguqulelo indala okanye ifuna ukulungiswa.
Kwenziwa ntoni ngenyaniso yokuba inkqubo yokulayisha ngokutsha izichazi-magama ayilayishi nasiphi na kwezininzi izichazi-magama ukuba enye yazo iye yantlitheka ngempazamo?
Kukho omnye umbuzo malunga nezichazi-magama zenkqubo. Sinezichazi-magama ezibini - enye ayilayishwanga, eyesibini ilayishiwe. Kulo mzekelo, iNkqubo yokulayisha kwakhona izichazi-magama ayilayishi nasiphi na isichazi-magama, kwaye kufuneka ulayishe inqaku ngenqaku elithile ngegama laso usebenzisa inkqubo yokulayisha kwakhona isichazi-magama. Ngaba oku kukwanxulumene noguqulelo lweClickHouse?
Ndifuna ukonwabisa. Le ndlela yokuziphatha yayitshintsha. Oku kuthetha ukuba ukuba uhlaziya iClickHouse, nayo iya kutshintsha. Ukuba awonwabanga yindlela oziphethe ngayo ngoku inkqubo yokulayisha kwakhona izichazi-magama, hlaziya, kwaye masithembe ukuba iya kutshintsha ibe ngcono.
Ngaba kukho indlela yokuqwalasela iinkcukacha kwiClickHouse config, kodwa ungazibonisi xa kukho iimpazamo?
Umbuzo olandelayo umalunga neempazamo ezinxulumene nesichazi-magama, ezizezi iinkcukacha. Sicacise iinkcukacha zoqhagamshelwano kwiClickHouse config kwisichazi-magama, kwaye ukuba kukho impazamo, sifumana ezi nkcukacha kunye negama lokugqitha kwimpendulo.
Siyisombulule le mpazamo ngokongeza iinkcukacha kwi-ODBC umqhubi we-config. Ngaba kukho nayiphi na indlela yokuqwalasela iinkcukacha kwiClickHouse config, kodwa ungabonisi ezi nkcukacha xa kukho iimpazamo?
Isisombululo sangempela apha kukucacisa ezi ziqinisekiso kwi-odbc.ini, kwaye kwi-ClickHouse ngokwayo ichaza kuphela i-ODBC Data Source Name. Oku akuyi kwenzeka kweminye imithombo yesichazi-magama - hayi kwisichazi-magama esine-MySQL, okanye kwabanye, akufanele ubone igama eligqithisiweyo xa ufumana umyalezo wemposiso. Kwi-ODBC, ndiza kujonga-ukuba ikhona, kufuneka uyisuse.
Ibhonasi: imvelaphi yeZoom ukusuka kwiindibano
Ngokucofa umfanekiso, imvelaphi yebhonasi evela kwiindibano iya kuvula abona bafundi bazingisayo. Sicima umlilo kunye ne-mascots ye-teknoloji ye-Avito, sinikezela kunye noogxa bethu kwigumbi lomlawuli wenkqubo okanye iklabhu yekhompyutha yesikolo esidala, kwaye siqhuba iintlanganiso zemihla ngemihla phantsi kwebhuloho ngokubhekiselele kwi-graffiti yangasemva.
umthombo: www.habr.com