Ngoko uqokelela iimethrikhi. Njengoko sinjalo. Siphinda siqokelele iimethrikhi. Kakade ke, kuyimfuneko kwishishini. Namhlanje siza kuthetha ngekhonkco lokuqala lenkqubo yethu yokubeka iliso- iseva edibeneyo ye-statsd-ehambelanayo. , isizathu sokuba siyibhale nesizathu sokuba simlahle ubrubeck.
Kumanqaku ethu angaphambili (, ) ungafumanisa ukuba kude kube lixesha elithile siqokelele iithegi sisebenzisa . Kubhaliwe kwi-C. Ukusuka kwikhowudi yokujonga ikhowudi, ilula njengeplagi (oku kubalulekile xa ufuna ukufaka isandla) kwaye, okona kubaluleke kakhulu, iphatha imiqulu yethu ye-2 yezigidi zeemethrikhi ngomzuzwana (MPS) kwindawo ephezulu. ngaphandle kweengxaki. Amaxwebhu achaza inkxaso ye-4 yezigidi ze-MPS ezineenkwenkwezi. Oku kuthetha ukuba uya kufumana inani elichaziweyo ukuba uqwalasela umsebenzi womnatha ngokuchanekileyo kwiLinux. (Asazi ukuba zingaphi iiMPS onokuzifumana ukuba ushiya inethiwekhi njengoko injalo). Phezu kwazo nje ezi ngenelo, saba nezikhalazo ezininzi ezinzulu ngebrubeck.
Ibango 1. I-Github, umphuhlisi weprojekthi, wayeka ukuyixhasa: ukupapasha iipetshi kunye nokulungiswa, ukwamkela kwethu kunye (kungekhona nje yethu) PR. Kwiinyanga ezimbalwa zokugqibela (kwindawo ethile ukusuka ngoFebruwari-Matshi 2018), umsebenzi uqalise kwakhona, kodwa ngaphambi koko kwakukho phantse iminyaka emi-2 yokuzola ngokupheleleyo. Ukongeza, iprojekthi iyaphuhliswa , enokuthi ibe ngumqobo omkhulu ekuqalisweni kweempawu ezintsha.
Ibango 2. Ukuchaneka kwezibalo. UBrubeck uqokelela inani elipheleleyo lama-65536 amaxabiso okudibanisa. Kwimeko yethu, kwezinye iimetriki, ngexesha lokudibanisa (imizuzwana engama-30), amaxabiso angaphezulu anokufika (1 encotsheni). Ngenxa yale sampuli, awona maxabiso aphezulu kunye nawona mancinci abonakala engenamsebenzi. Umzekelo, njengale:
Njengoko kwakunjalo
Bekufanele kube njani
Ngesizathu esifanayo, izixa-mali zibalwa ngokungalunganga. Yongeza apha impazamo ene-32-bit float ephuphumayo, ethi ithumele umncedisi gabalala kwi-segfault xa efumana i-metric ekhangeleka ngathi imsulwa, kwaye yonke into iba ntle. I-bug, ngendlela, ayizange ilungiswe.
Kwaye ekugqibeleni Banga u-X. Ngexesha lokubhala, sikulungele ukuyinikezela kuzo zonke ii-14 ngaphezulu okanye ngaphantsi kokuphunyezwa kwe-statsd esikwaziyo ukuyifumana. Masicinge ukuba ezinye iziseko zophuhliso zikhule kakhulu kangangokuba ukwamkela izigidi ezi-4 zeMPS akusanele. Okanye nokuba ayikakhuli okwangoku, kodwa iimethrikhi sele zibaluleke kakhulu kuwe kangangokuba nokuba mfutshane, imizuzu emi-2 ukuya kwe-3 yokuntywila kwiitshathi inokuba sele ibaluleke kakhulu kwaye ibangele ukudakumba okungenakuthintelwa phakathi kwabaphathi. Ekubeni ukunyanga ukudandatheka kungumsebenzi ongenambulelo, kufuneka izicombululo zobugcisa.
Okokuqala, ukunyamezela impazamo, ukwenzela ukuba ingxaki ngokukhawuleza kumncedisi ingabangeli i-apocalypse ye-zombie yengqondo eofisini. Okwesibini, ukukala ukukwazi ukwamkela ngaphezulu kwe-4 yezigidi ze-MPS, ngaphandle kokugrumba nzulu kwisitaki sothungelwano lweLinux kunye nokukhula ngokuzolileyo "ngobubanzi" ukuya kubungakanani obufunekayo.
Ekubeni sasinendawo yokwandisa, sagqiba ekubeni siqale ngokunyamezela iimpazamo. "MALUNGA! Ukunyamezela iimpazamo! Ilula, singayenza, βsacinga kwaye sazisa iiseva ezi-2, siphakamisa ikopi ye-brubeck nganye. Ukwenza oku, kuye kwafuneka sikope itrafikhi ngeemetriki kuzo zombini iiseva kwaye sibhalele oku . Sisombulule ingxaki yokunyamezela impazamo ngale nto, kodwa... hayi kakuhle kakhulu. Ekuqaleni yonke into yayibonakala inkulu: i-brubeck nganye iqokelela inguqulelo yayo yokudibanisa, ibhala idatha kwiGraphite kanye ngemizuzwana engama-30, ibhala ngaphezulu kwexesha elidala (oku kwenziwa kwicala leGraphite). Ukuba umncedisi omnye uyasilela ngesiquphe, sihlala sineyesibini enekopi yayo yedatha edityanisiweyo. Kodwa nantsi ingxaki: ukuba umncedisi uyasilela, i "saw" ibonakala kwiigrafu. Oku kubangelwa ukuba i-brubeck ye-30-yesibini i-intervals ayihambelani, kwaye ngexesha lokuphazamiseka enye yazo ayibhalwanga. Xa umncedisi wesibini eqala, kwenzeka into efanayo. Kuyanyamezeleka, kodwa ndifuna okungcono! Ingxaki yokwaphuka nayo ayikapheli. Zonke iimetrics "zisabhabha" kwiseva enye, kwaye ke ngoko sinqunyelwe kwi-MPS yezigidi ezi-2-4, kuxhomekeke kwinqanaba lenethiwekhi.
Ukuba ucinga kancinci malunga nengxaki kwaye ngexesha elifanayo ukumba iqhwa ngefosholo, ke le ngcamango ilandelayo inokuthi ifike engqondweni: udinga i-statsd enokusebenza kwimodi yokusabalalisa. Oko kukuthi, lowo uphumeza ungqamaniso phakathi kweenodi ngexesha kunye neemetrics. βEwe, isisombululo esinjalo mhlawumbi sele sikhona,β satsho saya kuGoogleβ¦. Kwaye abazange bafumane nto. Emva kokugqitha kuxwebhu lwestatsd ezahlukeneyo ( ukusukela nge-11.12.2017 kaDisemba XNUMX), asifumananga nto kwaphela. Kuyabonakala ukuba, abaphuhlisi okanye abasebenzisi bezi zisombululo abake badibane neemetrikhi ezininzi, kungenjalo ngokuqinisekileyo baya kuza nento.
Kwaye emva koko sakhumbula malunga ne "toy" statsd - i-bioyino, eyabhalwa kwi-Just for Fun hackathon (igama leprojekthi lenziwe ngumbhalo ngaphambi kokuqala kwe-hackathon) kwaye saqonda ukuba sifuna ngokukhawuleza i-statsd yethu. Yantoni?
- kuba kukho iiclones ezimbalwa kakhulu emhlabeni,
- kuba kunokwenzeka ukubonelela okufunwayo okanye kufutshane nonyamezelo olunqwenelekayo lwesiphoso kunye nokulinganisa (kubandakanya ukuvumelanisa iimetrics ezidityanisiweyo phakathi kweeseva kunye nokusombulula ingxaki yokuthumela iingxabano),
- kuba kunokwenzeka ukubala iimethrikhi ngokuchanekileyo ngakumbi kunobrubeck,
- kuba unokuqokelela iinkcukacha-manani ezithe kratya ngokwakho, uBrubeck angakhange asinike zona,
- kuba bendinethuba lokucwangcisa eyam i-hyperperformance isasazwe isikali selabhu isicelo, esingayi kuphinda ngokupheleleyo uyilo lolunye ulwakhiwo olufanayo...
Ungabhala ngantoni? Ngokuqinisekileyo, kwiRust. Ngoba?
- kuba bekukho isisombululo somzekelo,
- kuba umbhali wenqaku sele esazi iRust ngelo xesha kwaye wayenomdla wokubhala into kuyo ukuze avelise ithuba lokuyibeka kwindawo evulekileyo,
- kuba iilwimi ezine-GC azisifanelanga ngenxa yobume betrafikhi efunyenweyo (phantse ixesha lokwenyani) kunye nokunqumama kwe-GC akwamkelekanga,
- kuba ufuna ukusebenza okuphezulu xa kuthelekiswa noC
- kuba umhlwa usibonelela ngeconcurrency engoyikiyo, kwaye ukuba siqale ukuyibhala ngeC/C++, besiya kuba sesichengeni ngakumbi, ukuphuphuma kwebuffer, iimeko zobuhlanga kunye namanye amagama awoyikisayo kune-brubeck.
Kwabakho nengxabano ngokuchasene noRust. Inkampani yayingenawo amava okudala iiprojekthi kwi-Rust, kwaye ngoku asiyi kucwangcisa ukuyisebenzisa kwiprojekthi ephambili. Ke ngoko, bekukho uloyiko olukhulu lokuba akukho nto iza kulunga, kodwa sagqiba kwelokuba sithathe ithuba kwaye sizame.
Lahamba ixesha...
Ekugqibeleni, emva kwemizamo emininzi engaphumelelanga, inguqulelo yokuqala yokusebenza yayilungile. Kweneke ntoni? Kwenzeka oku.
Indawo nganye ifumana iiseti zayo zeemetrics kwaye iziqokelele, kwaye ayihlanganisi iimetrics kwezo ntlobo apho isethi yazo epheleleyo ifuneka khona ukuze kuhlanganiswe okokugqibela. Iinqununu zixhunywe omnye komnye ngohlobo oluthile lweprotocol yokutshixa esasazwayo, ekuvumela ukuba ukhethe phakathi kwabo kuphela (apha sakhala) ofanelekileyo ukuthumela i-metrics kuMkhulu. Le ngxaki ngoku iyasonjululwa ngu , kodwa kwixesha elizayo amabhongo ombhali anabela I-Raft, apho oyena ufanelekileyo uya kuthi, ngokuqinisekileyo, abe yinkokeli yesivumelwano. Ukongeza kwimvumelwano, iindawo zokuhlala rhoqo (kanye ngesekhondi ngokungagqibekanga) zithumela kubamelwane babo ezo nxalenye zeemetrics ezidityanisiweyo abathe bakwazi ukuziqokelela kweso sibini. Kuvela ukuba ukukala kunye nokunyamezelana kwempazamo kuyagcinwa - i-node nganye isabambe isethi epheleleyo yeemetriki, kodwa iimetriki zithunyelwa sele zihlanganisiwe, nge-TCP kwaye zifakwe kwi-protocol ye-binary, ngoko ke iindleko zokuphindaphinda ziyancipha kakhulu xa kuthelekiswa ne-UDP. Ngaphandle kwenani elikhulu ngokufanelekileyo leemethrikhi ezingenayo, ukuqokelela kufuna imemori encinci kakhulu kunye ne-CPU encinci. Kwi-mertics yethu ecinezeleka kakhulu, oku ngamashumi ambalwa eemegabytes zedatha. Njengebhonasi eyongezelelweyo, asifumani iinkcukacha ezingeyomfuneko zokuphinda sibhale kwiGraphite, njengoko kwakunjalo nge-burbeck.
Iipakethi ze-UDP ezineemetriki azilingani phakathi kwee-nodes kwizixhobo zenethiwekhi ngokusebenzisa i-Round Robin elula. Ewe kunjalo, i-hardware yenethiwekhi ayicazululi imixholo yeepakethi kwaye ngenxa yoko inokutsala ngaphezulu kweepakethi ze-4M ngomzuzwana, singasathethi ke ngeemetrics engazi nto ngazo. Ukuba sithathela ingqalelo ukuba iimetriki azifiki enye ngexesha kwipakethi nganye, ngoko asiboni kwangaphambili naziphi na iingxaki zokusebenza kule ndawo. Ukuba umncedisi uphahlazeka, isixhobo sothungelwano ngokukhawuleza (ngaphakathi kwe-1-2 imizuzwana) sifumanisa le nyaniso kwaye isusa iseva ephukileyo ekujikelezeni. Ngenxa yoku, i-passive (o.k.t., i-non-leader) iindawo zokuhlala zinokuvulwa kwaye zicinywe ngokubonakalayo ngaphandle kokuqaphela ukutsalwa kweetshathi. Ubuninzi esiphulukene nabo yinxalenye yeemetrics ezingene kwisekondi yokugqibela. Ilahleko ngequbuliso/ukuvalwa/ukutshintsha kwenkokeli kusaza kudala ukudida okuncinci (isithuba sesibini se-30 sisaphumile kwi-sync), kodwa ukuba kukho unxibelelwano phakathi kweenodi, ezi ngxaki zinokuncitshiswa, umzekelo, ngokuthumela iipakethi zongqamaniso. .
Kancinci malunga nesakhiwo sangaphakathi. Isicelo, ngokuqinisekileyo, si-multithreaded, kodwa i-architecture ye-threading yahlukile kuleyo isetyenziswe kwi-brubeck. Imisonto kwi-brubeck iyafana - ngamnye kubo unoxanduva lokuqokelela ulwazi kunye nokudibanisa. Kwi-bioyino, abasebenzi bahlula ngamaqela amabini: abo bajongene nenethiwekhi kunye nabo banoxanduva lokudibanisa. Olu lwahlulo lukuvumela ukuba ulawule ngokulula usetyenziso ngokuxhomekeke kuhlobo lweemethrikhi: apho udityaniso olunzulu lufuneka khona, unokongeza ii-aggregators, apho kukho i-traffic eninzi yenethiwekhi, unokongeza inani lokuhamba kwenethiwekhi. Okwangoku, kwiiseva zethu sisebenza kwinethiwekhi ye-8 kunye ne-4 aggregation flows.
Ukubala (okunoxanduva lokudityaniswa) inxalenye iyadika. Izithinteli ezizaliswe ngokuqukuqela kothungelwano zisasazwa phakathi koqukuqelo lokubala, apho zithi emva koko zahlulwe kwaye zidityaniswe. Ngesicelo, ii-metrics zinikezelwa ukuba zithunyelwe kwezinye iindawo. Konke oku, kubandakanywa nokuthumela idatha phakathi kweendawo zokuhlala kunye nokusebenza no-Consul, kwenziwa ngokungafaniyo, kusebenza kwisakhelo. .
Iingxaki ezininzi ngakumbi ngexesha lophuhliso zabangelwa yinxalenye yenethiwekhi enoxanduva lokufumana iimethrikhi. Eyona njongo iphambili yokwahlula ukuqukuqela komnatha kumaziko ahlukeneyo yayingumnqweno wokunciphisa ixesha elichithwa yi-flow hayi ukufunda idatha kwisokethi. Ukhetho olusebenzisa i-UDP engahambelaniyo kunye ne-recvmsg eqhelekileyo yanyamalala ngokukhawuleza: eyokuqala itya kakhulu indawo yomsebenzisi ye-CPU yokusetyenzwa komnyhadala, eyesibini ifuna utshintsho lweemeko ezininzi. Ngoko ke isetyenziswa ngoku kunye nezithinteli ezinkulu (kunye nezithinteli, amagosa amanene, akukho nto kuwe!). Inkxaso ye-UDP eqhelekileyo igcinelwe iimeko zokukhanya apho i-recvmmsg ingafunekiyo. Kwimowudi yomyalezo omninzi, kuyenzeka ukufezekisa eyona nto iphambili: uninzi lwexesha, umsonto womnatha urhuqa umgca we-OS-ufunda idatha ukusuka kwisokethi kwaye uyidlulisele kwi-buffer yendawo yomsebenzisi, utshintshela ngamaxesha athile ukunika isithinteli esigcweleyo. aggregators. Umgca okwisokethi awuqokeleli, inani leepakethi eziwisiweyo alikhuli.
Qaphela:
Kuseto olungagqibekanga, ubungakanani bebuffer bumiselwe ukuba bukhulu kakhulu. Ukuba ngokukhawuleza uthatha isigqibo sokuzama iseva ngokwakho, unokuhlangabezana nenyaniso yokuba emva kokuthumela inani elincinci leemethrikhi, aziyi kufika kwi-Graphite, eseleyo kwi-buffer ye-network stream. Ukuze usebenze ngenani elincinci leemethrikhi, kufuneka usete i-bufsize kunye nomsebenzi-ubungakanani bomgca kumaxabiso amancinci kuqwalaselo.
Ekugqibeleni, ezinye iitshathi zabathandi betshathi.
Izibalo ngenani leemetrikhi ezingenayo kwiseva nganye: ngaphezulu kwe-2 yezigidi zeMPS.
Ikhubaza enye yeendawo kunye nokusasazwa kwakhona kweemetrics ezingenayo.
Izibalo kwiimetrics eziphumayo: inye kuphela indawo ehlala ithumela - umphathi wogqogqo.
Izibalo zokusebenza kwe-node nganye, kuthathelwa ingqalelo iimpazamo kwiimodyuli zenkqubo ezahlukeneyo.
Iinkcukacha zeemetrics ezingenayo (amagama eemetriki afihliwe).
Siceba ukwenza ntoni ngako konke oku ngokulandelayo? Ewe, bhala ikhowudi, damn ...! Iprojekthi yayicetywe ekuqaleni ukuba ibe ngumthombo ovulekileyo kwaye iya kuhlala injalo ubomi bayo bonke. Izicwangciso zethu zangoku zibandakanya ukutshintshela kwinguqu yethu yeRaft, ukutshintsha iprotocol yoontanga ibe yinto ephathekayo, ukwazisa izibalo zangaphakathi ezongezelelweyo, iintlobo ezintsha zeemetriki, ukulungiswa kwebug kunye nolunye uphuculo.
Ngokuqinisekileyo, wonke umntu wamkelekile ukuba ancede ekuphuhlisweni kweprojekthi: ukudala i-PR, iMiba, ukuba kunokwenzeka siya kuphendula, siphucule, njl.
Xa kusithiwa, kuphelele apho bantu, thenga iindlovu zethu!
umthombo: www.habr.com