Umthwalo oPhezulu ++ eMoscow 2018, kwiHolo yeNkomfa. NgoNovemba 9, 15:00
Amagqabantshintshi kunye nenkcazo:
U-Yuri Nasretdinov (VKontakte): ingxelo iya kuthetha malunga namava okuphumeza i-ClickHouse kwinkampani yethu - kutheni siyidinga, ingakanani idatha esiyigcinayo, indlela esiyibhala ngayo, njalo njalo.
Izinto ezongeziweyo:
UYuri Nasretdinov: - Molweni nonke! Igama lam nguYuri Nasretdinov, njengoko sele ndaziswa. Ndisebenza kwiVKontakte. Ndiza kuthetha malunga nendlela esifaka ngayo idatha kwiClickHouse ukusuka kwi-server yethu (amashumi amawaka).
Ziyintoni iinkuni kwaye kutheni ziqokelelwa?
Yintoni esiya kukuxelela yona: into esiyenzileyo, kutheni sifuna "ClickHouse", ngokulandelelana, kutheni siyikhethile, luhlobo luni lomsebenzi onokulufumana ngaphandle kokumisela nantoni na ngokukodwa. Ndiza kukuxelela ngakumbi malunga neetafile ze-buffer, malunga neengxaki esinazo kunye nezisombululo zethu esiziphuhlise kumthombo ovulekileyo-KittenHouse kunye ne-Lighthouse.
Kutheni kufuneka senze nantoni na (yonke into ilungile kwi-VKontakte, akunjalo?). Sasifuna ukuqokelela iilogi ze-debug (kwaye kwakukho amakhulu e-terabytes yedatha apho), mhlawumbi ngandlela-thile kuya kuba lula ukubala izibalo; kwaye sineqela lamashumi amawaka eeseva apho konke oku kufuneka kwenziwe.
Kwakutheni ukuze senze isigqibo? Mhlawumbi besinazo izicombululo zokugcina iinkuni. Apha - kukho uluntu olunjalo "Backend VK". Ndincoma kakhulu ukubhalisa kuyo.
Yintoni iinkuni? Le yinjini ebuyisela uluhlu olungenanto. Iinjini kwi-VK yinto abanye abayibiza ngokuba yi-microservices. Kwaye nasi isincamathelisi esincumileyo (kakhulu ukuthandwa). Kanjani? Kaloku, mamela ngokubhekele phaya!
Yintoni enokusetyenziswa ukugcina iinkuni? Akunakwenzeka ukukhankanya iHadup. Emva koko, umzekelo, i-Rsyslog (ukugcina ezi logs kwiifayile). LSD. Ngubani owaziyo ukuba yintoni iLSD? Hayi, hayi le LSD. Gcina iifayile, ngokulandelelana, kakhulu. Ewe, iClickHouse lukhetho olungaqhelekanga.
Clickhouse kunye nabakhuphisanayo: iimfuno kunye namathuba
Sifuna ntoni? Sifuna ukuqinisekisa ukuba akukho mfuneko yokuba sikhathazeke kakhulu malunga nokusebenza, ukuze kusebenze ngaphandle kwebhokisi, ngokufanelekileyo kunye noqwalaselo oluncinci. Sifuna ukubhala kakhulu, kwaye sibhale ngokukhawuleza. Kwaye sifuna ukuyigcina kuzo zonke iintlobo zeenyanga, iminyaka, oko kukuthi, ixesha elide. Sinokufuna ukuqonda ingxaki ethile abeze nayo kuthi, "Kukho into engasebenzi apha," kwaye yayiyinyanga ezi-3 ezidlulileyo), kwaye sifuna ukukwazi ukubona okwenzeka kwiinyanga ezi-3 ezidlulileyo " Ukunyanzeliswa kwedatha - kucacile ukuba kutheni kuya kuba yinto edibeneyo - kuba iyanciphisa inani lendawo elithathayo.
Kwaye sinemfuno enomdla: ngamanye amaxesha sibhala imveliso yeminye imiyalelo (umzekelo, iilog), inokuba ngaphezulu kwe-4 kilobytes ngokulula. Kwaye ukuba le nto isebenza ngaphezu kwe-UDP, ngoko ayifuni ukuchitha ... ayiyi kuba nayiphi na "i-overhead" yokudibanisa, kunye nenani elikhulu lamaseva oku kuya kuba kudibanisa.
Makhe sibone ukuba yintoni umthombo ovulekileyo osinika yona. Okokuqala, sinayo i-Logs Engine - le yi-injini yethu; Ngokomgaqo, unokwenza yonke into, unokubhala imigca emide. Ewe, ayicinezeli ngokucacileyo idatha - sinokuzicinezela iikholamu ezinkulu ngokwethu ukuba sifuna ... thina, ewe, asifuni (ukuba kunokwenzeka). Ingxaki kuphela kukuba unokunikela kuphela oko kufanelekileyo kwinkumbulo yakhe; Ukufunda okunye, kufuneka ufumane i-binlog yale njini kwaye, ngokufanelekileyo, kuthatha ixesha elide.
Zeziphi ezinye iindlela ezikhoyo? Umzekelo, "Hadup". Ukulula kokusebenza ... Ngubani ocinga ukuba i-Hadup ilula ukuseta? Ngokuqinisekileyo, akukho ngxaki ngokurekhoda. Xa kufundwa, kuphakama imibuzo maxa wambi. Ngokomgaqo, ndingathi mhlawumbi akunjalo, ngakumbi kwiilogi. Ukugcinwa kwexesha elide - ngokuqinisekileyo, ewe, ukunyanzeliswa kwedatha - ewe, iintambo ezide - kucacile ukuba unokurekhoda. Kodwa ukurekhoda ukusuka kwinani elikhulu lamaseva ... Kusafuneka wenze into ngokwakho!
Rsyslog. Enyanisweni, sisebenzise njengendlela yokugcina ukuze sikwazi ukuyifunda ngaphandle kokulahla i-binlog, kodwa ayikwazi ukubhala imigca emide, ngokomgaqo ayikwazi ukubhala ngaphezu kwe-4 kilobytes. Kufuneka wenze ucinezelo lwedatha ngendlela efanayo ngokwakho. Ukufunda kuya kuvela kwiifayile.
Emva koko kukho "badushka" uphuhliso lwe-LSD. Ngokusisiseko iyafana ne "Rsyslog": ixhasa imitya emide, kodwa ayinakusebenza nge-UDP kwaye, enyanisweni, ngenxa yoku, ngelishwa, zininzi izinto ezifuna ukubhalwa kwakhona apho. I-LSD idinga ukuyilwa ngokutsha ukuze ikwazi ukurekhoda ukusuka kumashumi amawaka eeseva.
Kwaye apha! Ukhetho oluhlekisayo yi-ElasticSearch. Ukutsho njani? Uqhuba kakuhle ngokufunda, oko kukuthi, ufunda ngokukhawuleza, kodwa akakho kakuhle kakhulu ngokubhala. Okokuqala, ukuba icinezela idatha, ibuthathaka kakhulu. Okunokwenzeka, ukukhangela okupheleleyo kufuna ulwakhiwo lwedatha enkulu kunomthamo wokuqala. Kunzima ukusebenza kwaye iingxaki zihlala zivela ngayo. Kwaye, kwakhona, ukurekhoda kwi-Elastic - kufuneka senze yonke into ngokwethu.
Apha i-ClickHouse lukhetho olufanelekileyo, kunjalo. Into kuphela kukuba ukurekhoda ukusuka kumashumi amawaka amaseva yingxaki. Kodwa ubuncinane kukho ingxaki enye, sinokuzama ukuyisombulula ngandlela thile. Kwaye enye ingxelo imalunga nale ngxaki. Loluphi uhlobo lomsebenzi onokululindela kwiClickHouse?
Siza kuyifaka njani? MergeTree
Ngubani phakathi kwenu ongevanga okanye owazi malunga ne "ClickHouse"? Kufuneka ndikuxelele, andithi? Ngesatya esikhulu. Ukufakwa apho - i-1-2 yegigabhithi ngomzuzwana, ukugqabhuka ukuya kuthi ga kwi-10 yegigabhithi ngomzuzwana inokumelana nolu lungelelwaniso - kukho ezimbini i-6-core Xeons (oko kukuthi, hayi neyona inamandla), i-256 gigabytes ye-RAM, i-20 terabytes. kwi-RAID (akukho mntu uqwalaselweyo, useto olungagqibekanga). U-Alexey Milovidov, umthuthukisi weClickHouse, mhlawumbi uhleli apho ekhala ngenxa yokuba asizange silungiselele nantoni na (yonke into yasebenza ngolo hlobo kuthi). Ngokufanelekileyo, isantya sokuskena, yithi, malunga neebhiliyoni ezi-6 zemigca ngesekhondi inokufumaneka ukuba idatha icinezelwe kakuhle. Ukuba uthanda i-% kumtya wokubhaliweyo - 100 lezigidi zemigca ngomzuzwana, oko kukuthi, ibonakala ikhawuleza kakhulu.
Siza kuyifaka njani? Ewe, uyazi ukuba iVK isebenzisa i-PHP. Siya kufaka ukusuka kumsebenzi ngamnye we-PHP nge-HTTP kwi-"ClickHouse", kwitheyibhile ye-MergeTree kwirekhodi nganye. Ngubani obona ingxaki kwesi sikimu? Ngesizathu esithile, asingabo bonke abaphakamisa izandla zabo. Makhe ndikuxelele.
Okokuqala, zininzi iiseva - ngokufanelekileyo, kuya kubakho unxibelelwano oluninzi (olubi). Emva koko kungcono ukufaka idatha kwi-MergeTree akukho rhoqo kunokuba kube kanye ngesekhondi. Yaye ngubani owaziyo ukuba kutheni? Kulungile, kulungile. Ndiza kukuxelela kancinci malunga noku. Omnye umbuzo onomdla kukuba asenzi i-analytics, akudingeki ukuba sityebise idatha, asifuni amaseva aphakathi, sifuna ukufaka ngokuthe ngqo kwi "ClickHouse" (ngokukhethekileyo - ngokuthe ngqo ngakumbi, ngcono).
Ngokufanelekileyo, ukufakwa kwenziwa njani kwiMergeTree? Kutheni le nto kungcono ukufaka kuyo akukho rhoqo ngaphezu kwesihlandlo esinye okanye ngaphantsi rhoqo? Inyani kukuba "ClickHouse" yi-database ye-columnar kwaye ihlela idatha ngokunyuka komyalelo weqhosha eliphambili, kwaye xa ufaka, inani leefayile lenziwa ubuncinane lilingana nenani leekholomu apho idatha ihlelwa khona. kulandelelwano olunyukayo lweqhosha eliphambili (ulawulo olwahlukileyo luyenziwa, iseti yeefayile kwidiski yofakelo ngalunye). Emva koko ukufakwa okulandelayo kuza, kwaye ngasemva zidibaniswe "kwizahlulo" ezinkulu. Ekubeni idatha ihlelwe, kunokwenzeka "ukudibanisa" iifayile ezimbini ezihleliweyo ngaphandle kokutya imemori eninzi.
Kodwa, njengoko unokuthelekelela, ukuba ubhala iifayile ezili-10 kufakelo ngalunye, emva koko uClickHouse (okanye umncedisi wakho) uya kuphelisa ngokukhawuleza, ngoko kuyacetyiswa ukuba ufake kwiibhetshi ezinkulu. Ngokufanelekileyo, asizange siqalise iskimu sokuqala kwimveliso. Ngokukhawuleza sazisa enye, apho iNombolo yesi-2 ine:
Apha khawufane ucinge ukuba kukho malunga newaka leeseva esithe saqalisa kuzo, kukho nje i-PHP. Kwaye kumncedisi ngamnye kukho i-arhente yethu yendawo, esiyibiza ngokuthi "Kittenhouse", egcina uxhulumaniso olunye kunye ne "ClickHouse" kwaye ifake idatha yonke imizuzwana embalwa. Ifaka idatha hayi kwi-MergeTree, kodwa kwitafile yesithinteli, esebenza ngokuchanekileyo ukunqanda ukufaka ngqo kwi-MergeTree kwangoko.
Ukusebenza ngeetafile zesithinteli
Yintoni? Iitheyibhile zesithinteli liqhekeza lenkumbulo elahlulwayo (oko kukuthi, inokufakwa kuyo rhoqo). Ziquka iziqwenga ezininzi, kwaye nganye yeziqwenga zisebenza njenge-buffer ezimeleyo, kwaye zigungxulwa ngokuzimeleyo (ukuba unamaqhekeza amaninzi kwi-buffer, ngoko kuya kubakho ukufakwa okuninzi ngesekhondi). Kuyenzeka ukuba ufunde kwezi tafile - emva koko ufunda umanyano lwemixholo ye-buffer kunye netafile yomzali, kodwa ngalo mzuzu ukubhala kuvaliwe, ngoko ke kungcono ukuba ungafundi ukusuka apho. Kwaye iitafile ze-buffer zibonisa i-QPS entle kakhulu, oko kukuthi, ukuya kuthi ga kwi-3 amawaka e-QPS awuyi kuba nangxaki kwaphela xa ufaka. Kucacile ukuba ukuba umncedisi ulahlekelwa amandla, ngoko idatha ingalahleka, kuba yayigcinwe kuphela kwimemori.
Kwangaxeshanye, iskimu esinesithinteli sidibanisa i-ALTER, kuba kufuneka kuqala ulahle itafile endala ye-buffer kunye neskimu esidala (idatha ayisayi kunyamalala naphi na, kuba iya kugungxulwa phambi kokuba itafile icinywe). Emva koko "utshintshe" itafile oyifunayo kwaye wenze itafile yebuffer kwakhona. Ngokufanelekileyo, ngelixa kungekho tafile ye-buffer, idatha yakho ayizukuhamba naphi na, kodwa unokuba nayo kwidiski ubuncinci basekhaya.
Yintoni iKittenhouse kwaye isebenza njani?
Yintoni iKittenHouse? Lo ngummeli. Qikelela ukuba loluphi ulwimi? Ndiqokelele izihloko ze-hype kwingxelo yam - "Clickhouse", Hamba, mhlawumbi ndiya kukhumbula enye into. Ewe, oku kubhaliwe kwi-Go, kuba andiyazi kakuhle indlela yokubhala kwi-C, andifuni.
Ngokufanelekileyo, igcina unxibelelwano kunye nomncedisi ngamnye kwaye ingabhalela kwimemori. Umzekelo, ukuba sibhala iilogi zempazamo kwiClickhouse, ke ukuba iClickhouse ayinalo ixesha lokufaka idatha (emva koko, ukuba kuninzi kubhaliwe), ngoko asidumbi imemori - silahla nje abanye. Kuba ukuba sibhala iigigabhithi ezininzi ngesekondi yeempazamo, ngoko sinokulahla ezinye. I-Kittenhouse inokukwenza oku. Ngaphezu koko, inokwenza unikezelo oluthembekileyo, oko kukuthi, ukubhala kwidiski kumatshini wendawo kwaye kanye ngamaxesha onke (apho, kanye ngemizuzwana embalwa) izama ukuhambisa idatha kule fayile. Kwaye ekuqaleni sasebenzisa ifomati yeMigangatho eqhelekileyo - hayi ifomathi yokubini, ifomathi yokubhaliweyo (njengakwiSQL eqhelekileyo).
Kodwa ke kwenzeka oku. Sasebenzisa ukuhanjiswa okuthembekileyo, sabhala izigodo, emva koko sagqiba (yayiyi-cluster yovavanyo olunemiqathango)... Yakhutshwa ngaphandle kweeyure eziliqela kwaye yabuyiselwa, kwaye ukufakwa kwaqala kwiiseva eziliwaka - kwavela ukuba iClickhouse isenayo. "Umsonto kunxibelelwano" - ngokufanelekileyo, kuqhagamshelo lwewaka, ukufakwa okusebenzayo kukhokelela kumyinge womthwalo kumncedisi omalunga newaka elinesiqingatha. Okumangalisayo kukuba, umncedisi wamkela izicelo, kodwa idatha yayisafakwa emva kwexesha elithile; kodwa kwakunzima kakhulu kumncedisi ukuba asebenze...
Yongeza nginx
Isisombululo esinjalo soMsonto ngemodeli yoqhagamshelo yi nginx. Sifake nginx phambi kweClickhouse, kwangaxeshanye siseta ulungelelwaniso lweereplicas ezimbini (isantya sethu sokufaka sinyuke ngamaxesha angama-2, nangona ingeyonyani ukuba oku kufanele ukuba kube njalo) kwaye kwanciphisa inani loqhagamshelo kwiClickhouse, phezulu kwaye, ngokufanelekileyo, ngaphezulu, kunonxibelelwano lwe-50, kubonakala ngathi akukho nto yokufaka.
Emva koko siye safumanisa ukuba olu cwangciso luneziphene, kuba sinenginx enye kuphela apha. Ngokufanelekileyo, ukuba le nginx iphahlazeka, nangona kukho ubukho bee-replicas, silahlekelwa yidatha okanye, ubuncinane, ungabhali naphi na. Yiyo loo nto sizenzele eyethu imithwalo yokulinganisa. Saye saqaphela ukuba i-"Clickhouse" isafanelekile kwiigodo, kwaye "idemon" nayo yaqala ukubhala izigodo zayo kwi-"Clickhouse" - ilula kakhulu, ukunyaniseka. Sisayisebenzisa kwezinye "iidemon".
Emva koko siye safumanisa le ngxaki inika umdla: ukuba usebenzisa indlela engekho mgangathweni yokufaka kwimo ye-SQL, inyanzelisa i-AST-based SQL parser epheleleyo, ecotha kakhulu. Ngokufanelekileyo, songeze iisetingi ukuqinisekisa ukuba oku akunakuze kwenzeke. Senze i-load balancing, iisheke zezempilo, ukuze ukuba umntu uyafa, sishiye idatha. Ngoku sineetafile ezininzi ekufuneka sibe neqela leClickhouse ezahlukeneyo. Kwaye siphinde saqala ukucinga malunga nolunye usetyenziso-umzekelo, besifuna ukubhala iilog ukusuka kwiimodyuli ze-nginx, kodwa abayazi indlela yokunxibelelana sisebenzisa i-RPC yethu. Ewe, ndingathanda ukubafundisa indlela yokuthumela nokuba ngandlel' ithile-umzekelo, ukufumana imisitho kwi-hosthost nge-UDP kwaye emva koko uyidlulisele kwiClickhouse.
Inyathelo elinye kude nesisombululo
Iskimu sokugqibela saqala ukujongeka ngolu hlobo (uguqulelo lwesine lwesi sikimu): kumncedisi ngamnye phambi kwe Clickhouse kukho nginx (kwiseva enye) kwaye yenza ngokulula izicelo zokubamba indawo ngomda kwinani lodibaniso lwe 50 amaqhekeza. Kwaye esi sikimu sasisele sisebenza, yonke into yayilungile ngayo.
Sahlala ngolu hlobo isithuba esingangenyanga. Wonke umntu wayonwabile, bongeza iitafile, bongeza, bongeza ... Ngokubanzi, kwavela ukuba indlela esongeza ngayo iitafile ze-buffer yayingekho kakhulu (mayibeke ngolo hlobo). Senze iziqwenga ezili-16 kwitafile nganye kunye nekhefu lemizuzwana embalwa; sineetafile ze-20 kwaye itafile nganye ifumene i-8 efakwe ngomzuzwana - kwaye ngeli nqanaba "Clickhouse" yaqala ... iirekhodi zaqala ukucotha. Abazange badlule nje... Ngokungagqibekanga, i-nginx yayinento enomdla kangangokuba ukuba uxhulumaniso luphela kumlambo, emva koko ibuyisele "502" kuzo zonke izicelo ezitsha.
Kwaye apha siye (ndajonga iinkuni kwiClickhouse ngokwayo) malunga nesiqingatha sepesenti yezicelo aziphumelelanga. Ngokufanelekileyo, ukusetyenziswa kwedisk kwakuphezulu, kwakukho ukudibanisa okuninzi. Kulungile, ndenze ntoni? Ngokwemvelo, andizange ndizikhathaze ngokufumanisa ukuba kutheni unxibelelwano kunye nokunyuka komsinga kuphelile.
Ukubuyisela i-nginx nge-proxy ebuyela umva
Ndaye ndagqiba kwelokuba kufuneka silawule oku ngokwethu, akufuneki siyishiye kwi-nginx - i-nginx ayiyazi ukuba zeziphi iitafile ezikhoyo kwiClickhouse, kwaye ndatshintsha i-nginx nge-proxy eguqukayo, endizibhalele yona.
Inga wenzantoni? Isebenza ngokusekwe kwilayibrari ye-fasthttp "goshnoy", oko kukuthi, ngokukhawuleza, phantse ngokukhawuleza njengenginx. Uxolo, Igor, ukuba ukhona apha (qaphela: Igor Sysoev ngumdwelisi wenkqubo waseRashiya odale iseva yewebhu ye nginx). Inokuqonda ukuba luhlobo luni lwemibuzo ezi - FAKA okanye KHETHA - ngokufanelekileyo, ibamba amachibi okudibanisa ahlukeneyo kwiintlobo ezahlukeneyo zemibuzo.
Ngokufanelekileyo, nangona singenalo ixesha lokugqiba izicelo zofakelo, "ukhetho" luya kudlula, kwaye ngokuphambene. Kwaye idibanisa idatha kwiitafile ze-buffer - nge-buffer encinci: ukuba bekukho naziphi na iimpazamo, iimpazamo ze-syntax, njalo njalo-ukuze zingayichaphazeli kakhulu yonke idatha, kuba xa sifaka nje kwiitafile ze-buffer, yayine "bachi" encinci, kwaye zonke iimpazamo zesintaksi zichaphazele kuphela eli qhekeza lincinci; kwaye apha ziza kuchaphazela isithinteli esikhulu. Incinci yi-megabyte enye, oko kukuthi, ayincinci kangako.
Ukufaka ungqamaniso kunye nokutshintsha indawo ye-nginx, yenza into efanayo naleyo nginx yayenzayo ngaphambili - awudingi ukutshintsha "iKittenhouse" yasekhaya kule nto. Kwaye ekubeni isebenzisa i-fasthttp, ikhawuleza kakhulu - unokwenza izicelo ezingaphezu kwe-100 lamawaka ngomzuzwana wofakelo olulodwa ngeproksi ebuyela umva. Ngokwethiyori, ungafaka umgca omnye ngexesha kwi-kittenhouse reverse proxy, kodwa ngokuqinisekileyo asikwenzi oko.
Iskimu saqala ukubonakala ngolu hlobo: "I-Kittenhouse", amaqela e-proxy abuyela umva acela ezininzi kwiitafile kwaye, ngokuphindaphindiweyo, iitafile ze-buffer zizifaka kwizinto eziphambili.
Umbulali sisisombululo sexeshana, iKitten isisigxina
Le yingxaki enika umdla... Ingaba ukhona kuni okhe wasebenzisa fasthttp? Ngubani osebenzise i-fasthttp ngezicelo ze-POST? Mhlawumbi, oku ngenene bekungafanelanga ukuba kwenziwe, kuba ikhusela umzimba wesicelo ngokungagqibekanga, kwaye ubungakanani bethu bebuffer bumiselwe kwiimegabytes ezili-16. Ufakelo luyekile ukugcina ngexesha elithile, kwaye i-16-megabyte chunks yaqala ukufika ukusuka kuwo onke amashumi amawaka amaseva, kwaye zonke zigcinwe kwimemori ngaphambi kokuba zithunyelwe kwiClickhouse. Ngokufanelekileyo, imemori yaphela, uMbulali we-Of-Of-Memory weza wabulala ummeli obuyela umva (okanye "I-Clickhouse", enokuthi "itye" ngaphezulu kwe-proxy engasemva). Umjikelo waziphinda. Hayi ingxaki emnandi kakhulu. Nangona siye sakhubeka koku kuphela emva kweenyanga eziliqela sisebenza.
Ingaba Ndenze ntoni? Kwakhona, andithandi ukuqonda ukuba kwenzeke ntoni kanye kanye. Ndicinga ukuba kucace gca ukuba akufuneki ubambe inkumbulo. Andikwazanga ukupetch fasthttp, nangona ndizamile. Kodwa ndifumene indlela yokuyenza ukuze kungabikho mfuneko yokupakisha nantoni na, kwaye ndeza nendlela yam kwi-HTTP - ndayibiza ngokuba yi-KITTEN. Ewe, kunengqiqo - "VK", "Kitten"... Yintoni enye?
Ukuba isicelo siza kumncedisi ngendlela yeKitten, ngoko umncedisi kufuneka aphendule "meow" - ngokufanelekileyo. Ukuba uyaphendula kule nto, ngoko kuthathwa ukuba uyayiqonda le protocol, kwaye ke ndiyayinqanda uxhulumaniso (i-fasthttp inendlela enjalo), kwaye uxhumano luya kwimodi "eluhlaza". Kutheni ndiyifuna? Ndifuna ukulawula ukuba ukufunda kuqhagamshelo lwe-TCP kwenzeka njani. I-TCP inepropati emangalisayo: ukuba akukho mntu ufunda kwelinye icala, ngoko ukubhala kuqala ukulinda, kwaye imemori ayichithwanga ngokukodwa kule nto.
Kwaye ke ndafunda malunga nabaxhasi be-50 ngexesha (ukusuka kumashumi amahlanu ngenxa yokuba amashumi amahlanu kufuneka ngokuqinisekileyo abe ngokwaneleyo, nokuba izinga livela kwelinye i-DC) ... Ukusetyenziswa kunciphile ngale ndlela ubuncinane ngamaxesha angama-20, kodwa mna, ukunyaniseka , Andikwazanga ukulinganisa ngokuthe ngqo ukuba yiliphi ixesha, kuba sele lingenanjongo (sele lifikelele kwinqanaba lephutha). Iprothokholi ibini, oko kukuthi, iqulethe igama letafile kunye nedatha; akukho zihloko ze-http, ngoko andizange ndisebenzise i-socket yewebhu (andifuni ukunxibelelana nabaphequluli - ndenze iprotocol ehambelana neemfuno zethu). Kwaye yonke into yayihamba kakuhle naye.
Itafile yesithinteli ilusizi
Kutshanje siye sadibana nenye into enomdla yeetafile zebuffer. Kwaye le ngxaki sele ibuhlungu kakhulu kunezinye. Makhe sicinge ngale meko: sele usebenzisa ngokusebenzayo iClickhouse, unamaseva amaninzi eClickhouse, kwaye unezicelo ezithatha ixesha elide ukufunda (masithi, ngaphezu kwemizuzwana engama-60); kwaye uze wenze i-Alter ngalo mzuzu... Okwangoku, "ikhetha" eyaqala phambi ko"Alter" ayizukufakwa kule theyibhile, "Alter" ayizukuqala - mhlawumbi ezinye iimpawu zendlela "Clickhouse" esebenza kuyo. le ndawo. Mhlawumbi le nto inokulungiswa? Okanye ngaba akunakwenzeka?
Ngokubanzi, kucacile ukuba enyanisweni oku akuyona ingxaki enkulu, kodwa ngeetafile ze-buffer ziba buhlungu ngakumbi. Ngenxa yokuba, ukuba, masithi, "Hlela" ixesha lokuphuma (kwaye linokuphuma kwenye inginginya - hayi kweyakho, kodwa kwikopi, umzekelo), emva koko... Ucime itafile yebuffer, "Alter" yakho ( okanye enye inginginya) iphelelwe lixesha.kwaye kwenzeke imposiso “Luhlu”) - usafuna ukuqinisekisa ukuba idata iyaqhubeka ukubhalwa: udala iitafile ze buffer umva (ngokodweliso olufanayo netafile yomzali), ngoko "Utshintsho" luyadlula, luphele emva kwayo yonke loo nto, kwaye i-buffer yetafile iqala ukwahluka kwi-schema kumzali. Kuxhomekeka ekubeni yayiyintoni na i-“Alter”, into efakiweyo ayinakuphinda iye kule tafile ye-buffer - oku kulusizi kakhulu.
Kukho kwakhona uphawu olunjalo (mhlawumbi umntu ulubonile) - lubizwa ngokuba yi-query_thread_log kwiinguqulelo ezintsha zeClickhouse. Ngokungagqibekanga, kwenye inguqulelo kwakukho enye. Apha siye saqokelela iirekhodi ezizigidi ezingama-840 kwiinyanga ezimbalwa (i-100 gigabytes). Oku kungenxa yokuba "ukufakwa" kwabhalwa apho (mhlawumbi ngoku, ngendlela, ayibhalwanga). Njengoko bendinixelele, "izinto ezifakiweyo" zincinci - sine "insert" ezininzi kwiitafile zebuffer. Kucacile ukuba oku kukhubazekile-ndikuxelela nje into endiyibonile kwiseva yethu. Ngoba? Le yenye impikiswano echasene nokusebenzisa iitafile zesithinteli! Spotty lusizi kakhulu.
Ngubani owayesazi ukuba igama lalo mfo nguSpotty? abasebenzi VK baphakamisa izandla zabo. KULUNGILE.
Malunga nezicwangciso ze "KitttenHouse"
Izicwangciso azihlali kwabelwana ngazo, akunjalo? Ngokukhawuleza awuyi kuzizalisekisa kwaye awuyi kukhangeleka kakuhle emehlweni abanye abantu. Kodwa ndiza kuthatha umngcipheko! Sifuna ukwenza oku kulandelayo: iitafile zesithinteli, kubonakala kum, zisengumqobo kwaye kufuneka sithintele ukufakwa ngokwethu. Kodwa asikafuni ukuyikhusela kwidiski, ngoko ke siza kufakela ukufakwa kwinkumbulo.
Ngokufanelekileyo, xa "ukufakwa" kwenziwa, akusayi kuphinda kuhambelane - iya kuba sele isebenza njengetafile ye-buffer, iya kufaka kwitafile yomzali (kaloku, ngenye imini kamva) kwaye inike ingxelo ngesitishi esahlukileyo esifakelweyo kwaye esidlulile. Andi.
Kutheni ndingalushiyi ulungelelwaniso lofakelo? Iluncedo kakhulu. Inyani kukuba ukuba ufaka ukusuka kwi-10 lamawaka emikhosi, ngoko yonke into ilungile - uya kufumana incinci kumsingathi ngamnye, ufaka khona kanye okwesibini, yonke into ilungile. Kodwa ndingathanda ukuba le nkqubo isebenze, umzekelo, kumatshini amabini, ukuze ukwazi ukukhuphela ngesantya esiphezulu - mhlawumbi ungafumani ubuninzi beClickhouse, kodwa bhala ubuncinane i-megabytes eyi-100 ngesekhondi ukusuka kumatshini omnye ngokusebenzisa ummeleli ojikelezayo - esi sikimu kufuneka silinganise kuzo zombini izixa ezinkulu nezincinci, ngoko ke asinakulinda isekhondi kufakelo ngalunye, ngoko ke kufuneka lube-synchronous. Kwaye ngendlela efanayo, ukuqinisekiswa kwe-asynchronous kufuneka kufike emva kokuba ukufakwa kugqityiwe. Siya kuyazi ukuba idlulile okanye ayiphumelelanga.
Eyona nto ibalulekileyo kukuba kule scheme siyazi ngokuqinisekileyo ukuba ukufakwa kwenzeka okanye hayi. Khawucinge ngale meko: unetafile ye-buffer, ubhale into kuyo, kwaye ke, masithi, itheyibhile yangena kwindlela yokufunda kuphela kwaye yazama ukugungxula isithinteli. Iza kuya phi idatha? Ziya kuhlala kwi-buffer. Kodwa asikwazi ukuqiniseka ngale nto - kuthekani ukuba kukho enye impazamo, ngenxa yokuba idatha ayiyi kuhlala kwi-buffer ... (Iidilesi u-Alexey Milovidov, i-Yandex, i-ClickHouse developer) Okanye ngaba iya kuhlala? Ngalo lonke ixesha? U-Alexey uyasiqinisekisa ukuba yonke into iya kulunga. Asinasizathu sokungamkholelwa. Kodwa zonke ziyafana: ukuba asizisebenzisi iitafile ze-buffer, ke akuyi kubakho naziphi na iingxaki kubo. Ukudala iitafile eziphindwe kabini akunzima, nangona ngokomgaqo akukho zingxaki ezinkulu. Esi sisicwangciso.
Masithethe ngokufunda
Ngoku makhe sithethe ngokufunda. Siphinde sabhala isixhobo sethu apha. Kubonakala ngathi, kutheni ubhala isixhobo sakho apha? .. Kwaye ngubani osebenzise iTabix? Ngandlela-thile abantu abambalwa baphakamisa izandla ... Kwaye ngubani owanelisekile ngokusebenza kweTabix? Ewe, asonwabe ngayo, kwaye ayilungelanga kakhulu ukujonga idatha. Ilungile kuhlahlelo, kodwa nje ukuyijonga ngokucacileyo ayilungiswanga. Ndiye ndabhala eyam, eyam i-interface.
Ilula kakhulu - inokufunda kuphela idatha. Akayazi indlela yokubonisa imizobo, akazi ukwenza nantoni na. Kodwa ingabonisa into esiyidingayo: umzekelo, mingaphi imiqolo ekwitheyibhile, ingakanani isithuba esiyithathayo (ngaphandle kokuyiqhekeza kwiikholamu), oko kukuthi, i-interface esisiseko yinto esiyifunayo.
Kwaye ibonakala ifana kakhulu ne-Sequel Pro, kodwa yenziwe kuphela kwi-Bootstrap ye-Twitter, kunye nenguqulo yesibini. Uyabuza: "Yuri, kutheni kwinguqulelo yesibini?" Ngowuphi unyaka? 2018? Ngokubanzi, ndiyenze le nto ixesha elide elidlulileyo kwi "Muscle" (MySQL) kwaye nditshintshe nje imigca embalwa kwimibuzo apho, kwaye yaqala ukusebenzela "Clickhouse", elowo ngumbulelo okhethekileyo! Ngenxa yokuba i-parser ifana kakhulu ne "imisipha" enye, kwaye imibuzo ifana kakhulu - ilula kakhulu, ngakumbi ekuqaleni.
Ewe, iyakwazi ukuhluza iitafile, ingabonisa ubume kunye nemixholo yetafile, ikuvumela ukuba uhlele, ucofe ngokwemiqolo, ibonisa umbuzo obangele isiphumo, imiqolo echaphazelekayo (ebangaphi na ngenxa yoko), oko kukuthi, izinto ezisisiseko zokujonga idatha. Ukukhawuleza ngokukhawuleza.
Kukwakho nomhleli. Ndizamile ngokunyanisekileyo ukuba umhleli wonke kwiTabix, kodwa andikwazanga. Kodwa ngandlel’ ithile iyasebenza. Ngokomgaqo, yiyo yonke loo nto.
"Clickhouse" ifanelekile kwimiqolomba
Ndifuna ukukuxelela ukuba i-Clickhouse, nangona zonke iingxaki ezichazwe, zifanelekile kakhulu kwiilogi. Okona kubaluleke kakhulu, iyasombulula ingxaki yethu - ikhawuleza kakhulu kwaye ikuvumela ukuba ucofe izingodo ngeekholamu. Ngokomgaqo, iitafile ze-buffer azenzanga kakuhle, kodwa ngokuqhelekileyo akukho mntu wazi ukuba kutheni ... Mhlawumbi ngoku uyazi ngcono apho uya kuba neengxaki.
TCP? Ngokubanzi, kwiVK kuqhelekile ukusebenzisa i-UDP. Kwaye xa ndisebenzisa i-TCP ... Kakade ke, akukho mntu wandixelela: "Yuri, uthetha ngantoni! Awunakukwazi, udinga i-UDP. " Kwavela ukuba i-TCP ayisoyiki kangako. Ekuphela kwento, ukuba unamashumi amawaka eekhompawundi ezisebenzayo ozibhalayo, kufuneka uyilungiselele kancinci kancinci; kodwa kunokwenzeka, kwaye kulula kakhulu.
Ndathembisa ukuthumela "iKittenhouse" kunye ne "Lighthouse" kwi-HighLoad yaseSiberia ukuba wonke umntu ubhalisele uluntu lwethu "i-VK backend" ... Kwaye uyazi, kungekhona wonke umntu obhalisile ... Ngokuqinisekileyo, andiyi kufuna ukuba ubhalisele uluntu. Nisebaninzi kakhulu, umntu usenokukhubeka, kodwa kunjalo, nceda ubhalise (kwaye apha kufuneka ndenze amehlo afana nalawo ekati). Kunjalo . Enkosi kakhulu! IGithub yeyethu . NgeClickhouse iinwele zakho ziya kuba zithambile kwaye zisilika.
Ukukhokela: - Bahlobo, ngoku imibuzo. Kanye emva kokuba sibonise isatifikethi sombulelo kunye nengxelo yakho ngeVHS.
UYuri Nasretdinov (emva koku kuthiwa nguYN): -Ukwazile njani ukurekhoda ingxelo yam kwi-VHS ukuba iphelile?
Ukukhokela: -Awunakukwazi ngokupheleleyo ukuba "iClickhouse" iya kusebenza njani okanye hayi! Bahlobo, imizuzu emi-5 yemibuzo!
Imibuzo yakho
Umbuzo ovela kubaphulaphuli (apha kubhekiselwa kubo njengoMbuzo): - Mholo emvakwemini. Enkosi kakhulu ngengxelo. Ndinemibuzo emibini. Ndiza kuqala ngento engenamsebenzi: ngaba inani leeleta t egameni elithi "Kittenhouse" kwimizobo (3, 4, 7 ...) ichaphazela ukwaneliseka kweekati?
YN: - Ubungakanani bento?
З: – Unobumba t. Kukho oo-t abathathu, kwindawo ethile malunga noo-t abathathu.
YN: - Khange ndiyilungise? Ewe, kunjalo kunjalo! Ezi zimveliso ezahlukeneyo - bendinikhohlisa ngalo lonke eli xesha. Kulungile, ndiyadlala-akunamsebenzi. Ah, kanye apha! Hayi, yinto enye, ndenze i-typo.
З: - Enkosi. Umbuzo wesibini unzulu. Ngokokuqonda kwam, kwiClickhouse, iitafile ze-buffer zihlala kuphela kwimemori, azikhutshelwanga kwidiski kwaye, ngokufanelekileyo, aziqhubeki.
YN: - Ewe.
З: -Kwaye kwangaxeshanye, umxhasi wakho ugcina iibuffers kwidisk, nto leyo ethetha isiqinisekiso sokuhanjiswa kwezi log zinye. Kodwa oku akuqinisekwanga kwaClickhouse. Cacisa ukuba isiqinisekiso siqhutywa njani, ngenxa yantoni?
YN: - Ewe, ngokwethiyori akukho kuphikisana apha, kuba xa uClickhouse ewa, unokuyibona ngesigidi ngeendlela ezahlukeneyo. Ukuba iClickhouse iyawa (ukuba iphetha ngokungachanekanga), ungathetha ngokurhabaxa, ubuyisele umva intwana yelog yakho oyibhalileyo kwaye uqale ukusuka kumzuzu xa yonke into yayilungile. Masithi ubuyisela umva umzuzu, oko kukuthi, kuthathwa ngokuba ugungxule yonke into ngomzuzu.
З: - Oko kukuthi, "i-Kittenhouse" ibamba ifestile ixesha elide kwaye, xa iwa, iyakwazi ukuyiqonda kwaye ibuyisele umva?
YN: - Kodwa oku kwithiyori. Enyanisweni, asikwenzi oku, kwaye ukuhanjiswa okuthembekileyo kuvela kwi-zero ukuya kumaxesha angapheliyo. Kodwa ngokomyinge omnye. Sanelisekile ukuba ukuba iClickhouse iwa ngenxa yesizathu esithile okanye iiseva "ziqalise kwakhona," ngoko silahlekelwa kancinane. Kuzo zonke ezinye iimeko, akukho nto iya kwenzeka.
З: - Mholo. Kwasekuqaleni kwakubonakala kum ukuba ngenene uya kusebenzisa i-UDP kwasekuqaleni kwengxelo. Unayo i-http, yonke loo nto ... Kwaye uninzi lweengxaki ozichazile, njengoko ndiyiqonda, zibangelwa sesi sisombululo ...
YN: – Sisebenzisa ntoni i-TCP?
З: - Ngokusisiseko ewe.
YN: - Hayi.
З: -Kwaku nge-fasthttp ukuba uneengxaki, kunye noqhagamshelwano ube neengxaki. Ukuba ubusandula ukusebenzisa i-UDP ngewuzigcine ixesha elithile. Ewe, kuya kubakho iingxaki ngemiyalezo emide okanye enye into...
YN: - Ngantoni?
З: - Ngemiyalezo emide, ekubeni ingenako ukungena kwi-MTU, enye into ... Ewe, kunokubakho iingxaki zabo. Umbuzo ngulo: kutheni i-UDP?
YN: -Ndiyakholelwa ukuba ababhali abaphuhlise i-TCP / IP bahlakaniphile kakhulu kunam kwaye bazi ngcono kunam indlela yokulinganisa iipakethi (ukuze zihambe), kwangaxeshanye lungisa ifestile yokuthumela, ungalayishi inethiwekhi, unike ingxelo malunga nokuba yintoni na. ayifundwanga, ingabalwanga kwelinye icala ... Zonke ezi ngxaki, ngokombono wam, ziya kubakho kwi-UDP, kuphela kuya kufuneka ndibhale ikhowudi engaphezulu kunokuba sele ndiyibhalile ukuze ndiphumeze into efanayo nam kwaye kunokwenzeka kakubi. Andikuthandi ncam ukubhala ngoC, mandiyeke apho...
З: - Iluncedo nje! Ithunyelwe kulungile kwaye ungalindeli nantoni na - ayihambelani ngokupheleleyo. Kwabuya isaziso sokuba yonke into ilungile - oko kuthetha ukuba ifikile; Ukuba ayifiki, ithetha ukuba imbi.
YN: - Ndifuna zombini - kufuneka ndikwazi ukuthumela zombini ngesiqinisekiso sokuhanjiswa kwaye ngaphandle kwesiqinisekiso sokuhanjiswa. Ezi ziimeko ezimbini ezahlukeneyo. Kufuneka ndingaphulukani nezigodo okanye ndingaphulukani nazo ngesizathu.
З: - Andiyi kuchitha ixesha. Oku kufuneka kuxoxwe ngakumbi. Enkosi.
Ukukhokela: – Ngubani onemibuzo – izandla zisiya esibhakabhakeni!
З: - Molo, ndinguSasha. Kwindawo ethile embindini wengxelo, kwabonakala ukuba, ngaphezu kwe-TCP, kwakunokwenzeka ukusebenzisa isisombululo esele silungisiwe - uhlobo oluthile lweKafka.
YN: - Ewe ... Ndikuxelele ukuba andifuni ukusebenzisa amaseva aphakathi, kuba ... eKafka, kuvela ukuba sinemikhosi yamawaka alishumi; eneneni, sinazo ngaphezulu - amashumi amawaka emikhosi. Kwakhona kunokuba buhlungu ukwenza neKafka ngaphandle kweeproxies. Ukongeza, okona kubaluleke kakhulu, isanika "i-latency", inika imikhosi eyongezelelweyo ekufuneka ubenayo. Kodwa andifuni ukuba nazo - ndifuna ...
З: "Kodwa ekugqibeleni kwaba njalo."
YN: - Hayi, akukho mamkeli! Konke oku kusebenza kwi-Clickhouse host hosts.
З: - Ewe, kunye ne "Kittenhouse", ephikisanayo - uhlala phi?
YN: Kwi-Clickhouse host, ayibhali nto kwidiski.
З: - Makhe sicinge.
Ukukhokela: – Ngaba wanelisekile? Singakunika umvuzo?
З: - Ewe Unga. Enyanisweni, kukho ezininzi iintonga ukuze ufumane into efanayo, kwaye ngoku - impendulo yangaphambili kwisihloko se-TCP iyaphikisana, ngokombono wam, le meko. Kuvakala ngathi yonke into ibinokwenziwa emadolweni am ngexesha elincinci kakhulu.
YN: Kwaye kwakutheni ndingafuni ukusebenzisa iKafka, kuba bekukho izikhalazo ezininzi kwincoko yeClickhouse Telegraph, umzekelo, imiyalezo evela eKafka yalahleka. Hayi ukusuka eKafka ngokwayo, kodwa ekudityanisweni kweKafka kunye neClikhaus; okanye into ayizange idibane apho. Ukuthetha nje, kuya kufuneka ukuba ubhale umxhasi weKafka ngoko. Andiqondi ukuba kunokubakho isisombululo esilula okanye esithembeke ngakumbi.
З: -Ndixelele, kutheni ungazange uzame nayiphi na imigca okanye uhlobo lwebhasi eqhelekileyo? Ukusukela ukuba usithi nge-asynchrony ungazithumela ngokwazo iilog emgceni kwaye ufumane impendulo ngokulandelelanayo emgceni?
YN: – Nceda ucebise ukuba yeyiphi imigca enokusetyenziswa?
З: - Nayiphi na, nangaphandle kwesiqinisekiso sokuba ilungile. Olunye uhlobo lweRedis, RMQ...
YN: - Ndinemvakalelo yokuba iRedis inokwenzeka ukuba ayinakukwazi ukutsala umthamo onjalo wofakelo nakwinginginya enye (ngengqiqo yeeseva ezininzi) ekhupha iClickhouse. Andikwazi ukuyixhasa le nto nangabuphi na ubungqina (andiyilinganiseli), kodwa kubonakala kum ukuba iRedis ayisosisombululo silungileyo apha. Ngokomgaqo, le nkqubo inokuqwalaselwa njengomgca womyalezo ophuculweyo, kodwa olungiselelwe kuphela "Clickhouse"
Ukukhokela: – Yuri, enkosi kakhulu. Ndicebisa ukuba imibuzo neempendulo ipheliswe apha sichaze ukuba ngubani kwabo babuze lo mbuzo sizakumnika incwadi.
YN: – Ndingathanda ukunika incwadi kumntu wokuqala obuze umbuzo.
Ukukhokela: - Intle! Kakhulu! Intle! Ndibulela kakhulu!
Ezinye iintengiso 🙂
Enkosi ngokuhlala nathi. Ngaba uyawathanda amanqaku ethu? Ngaba ufuna ukubona umxholo onomdla ngakumbi? Sixhase ngokufaka iodolo okanye ngokucebisa abahlobo, , i-analogue eyodwa yeeseva zomgangatho wokungena, eyenzelwe wena: (ifumaneka nge-RAID1 kunye ne-RAID10, ukuya kuthi ga kwi-24 cores kunye ne-40GB DDR4).
Dell R730xd 2x ngexabiso eliphantsi kwiziko ledatha le-Equinix Tier IV eAmsterdam? Kuphela apha eNetherlands! Dell R420 - 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB - ukusuka $99! Funda malunga
umthombo: www.habr.com