Ngakho uqoqa amamethrikhi. Njengoba sinjalo. Siphinde siqoqe amamethrikhi. Yiqiniso, ezidingekayo ibhizinisi. Namuhla sizokhuluma ngesixhumanisi sokuqala ngqa sesistimu yethu yokuqapha - iseva yokuhlanganisa ehambisana ne-statsd , kungani sayibhala nokuthi kungani samlahla u-brubeck.
Kusuka ezihlokweni zethu ezedlule (, ) ungathola ukuthi kwaze kwaba yisikhathi siqoqa amamaki sisebenzisa . Ibhalwe ngo-C. Ngokombono wekhodi, ilula njengepulaki (lokhu kubalulekile uma ufuna ukufaka isandla) futhi, okubaluleke kakhulu, iphatha imiqulu yethu yamamethrikhi ayizigidi ezingu-2 ngomzuzwana (MPS) phezulu. ngaphandle kwezinkinga. Imibhalo ithi ukwesekwa kwe-MPS eyizigidi ezi-4 ngenkanyezi. Lokhu kusho ukuthi uzothola isibalo esishiwo uma umisa inethiwekhi ngendlela efanele ku-Linux. (Asazi ukuthi mangaki ama-MPS ongawathola uma ushiya inethiwekhi njengoba injalo). Naphezu kwalezi zinzuzo, saba nezikhalo eziningana ezinzima mayelana ne-brubeck.
Isimangalo 1. U-Github, umthuthukisi wephrojekthi, uyekile ukuyisekela: ukushicilela iziqephu nezilungiso, ukwamukela okwethu kanye (hhayi nje okwethu) i-PR. Ezinyangeni ezimbalwa ezedlule (endaweni ethile kusukela ngoFebhuwari-Mashi 2018), umsebenzi usuqalile, kodwa ngaphambi kwalokho bekukhona cishe iminyaka emi-2 yokuzola okuphelele. Ngaphezu kwalokho, iphrojekthi iyathuthukiswa , okungaba isithiyo esingathi sΓna ekwethulweni kwezici ezintsha.
Isimangalo 2. Ukunemba kwezibalo. I-Brubeck iqoqa inani lamanani angu-65536 wokuhlanganisa. Esimweni sethu, kwamanye amamethrikhi, ngesikhathi sokuhlanganisa (imizuzwana engama-30), amanani amaningi angafika (1 phezulu). Njengomphumela walokhu kusampula, amanani aphezulu kanye nobuncane abonakala engenamsebenzi. Ngokwesibonelo, kanje:
Njengoba kwakunjalo
Bekufanele kube kanjani
Ngesizathu esifanayo, amanani ngokuvamile abalwa ngokungalungile. Engeza lapha iphutha elinokuchichima kwe-32-bit float, okuvamise ukuthumela iseva ku-segfault lapho ithola imethrikhi ebonakala ingenacala, futhi yonke into iba yinhle. I-bug, ngendlela, ayizange ilungiswe.
Futhi ekugcineni Mangalela u-X. Ngesikhathi sokubhala, sesilungele ukuyethula kukho konke oku-14 okusebenzayo okungaphezulu noma okuncane kokusetshenziswa kwe-statsd esikwazile ukukuthola. Ake sicabange ukuthi ingqalasizinda eyodwa isikhule kakhulu kangangokuthi ukwamukela ama-MPS ayizigidi ezi-4 akusanele. Noma ngabe ingakakhuli okwamanje, kodwa amamethrikhi asevele ebaluleke kakhulu kuwe kangangokuthi ngisho nokufushane, ukucwiliswa kwemizuzu engu-2-3 kumashadi kungase kuvele kubucayi futhi kubangele ukucindezeleka okungenakunqotshwa phakathi kwabaphathi. Njengoba ukwelapha ukucindezeleka kuwumsebenzi ongabongi, kudingeka izixazululo zobuchwepheshe.
Okokuqala, ukubekezelelana kwamaphutha, ukuze inkinga engazelelwe kuseva ingabangeli i-apocalypse ye-zombie yengqondo ehhovisi. Okwesibili, ukukala ukuze ukwazi ukwamukela ama-MPS angaphezu kwezigidi ezi-4, ngaphandle kokumba ujule esitakini senethiwekhi ye-Linux futhi ukhule ngokuzolile βngobubanziβ kusayizi odingekayo.
Njengoba sasinendawo yokukala, sanquma ukuqala ngokubekezelela amaphutha. "MAYELANA! Ukubekezelela iphutha! Kulula, singakwenza,β sacabanga futhi sethula amaseva ama-2, siphakamisa ikhophi ye-brubeck ngayinye. Ukuze senze lokhu, bekufanele sikopishe ithrafikhi ngamamethrikhi kuwo womabili amaseva futhi sibhalele lokhu . Sixazulule inkinga yokubekezelela amaphutha ngalokhu, kodwa... hhayi kahle kakhulu. Ekuqaleni konke kwakubonakala kukuhle: i-brubeck ngayinye iqoqa inguqulo yayo yokuhlanganisa, ibhala idatha ku-Graphite kanye njalo ngemizuzwana engu-30, ibhala ngaphezulu isikhawu esidala (lokhu kwenziwa ohlangothini lwe-Graphite). Uma iseva eyodwa ihluleka kungazelelwe, sihlala sineyesibili enekhophi yayo yedatha ehlanganisiwe. Kodwa nansi inkinga: uma iseva ihluleka, i-"saw" ivela kumagrafu. Lokhu kungenxa yokuthi izikhawu ze-brubeck ezingamasekhondi angu-30 azivumelaniswa, futhi ngesikhathi sokuphahlazeka esinye sazo asibhalwanga ngaphezulu. Lapho iseva yesibili iqala, kwenzeka okufanayo. Kuyabekezelelwa impela, kepha ngifuna okungcono! Inkinga yokuqina nayo ayikapheli. Wonke amamethrikhi "asandiza" kuseva eyodwa, ngakho-ke sikhawulelwe kuma-MPS ayizigidi ezi-2-4, kuye ngezinga lenethiwekhi.
Uma ucabanga kancane ngenkinga futhi ngesikhathi esifanayo ukumba iqhwa ngefosholo, khona-ke umqondo olandelayo osobala ungase ufike engqondweni: udinga i-statsd engasebenza kwimodi yokusabalalisa. Okungukuthi, leyo esebenzisa ukuvumelanisa phakathi kwamanodi ngesikhathi namamethrikhi. βYebo, isixazululo esinjalo cishe sesivele sikhona,β sasho saya ku-Googleβ¦. Futhi abatholanga lutho. Ngemuva kokudlula imibhalo yezibalo ezahlukene ( kusukela ngoDisemba 11.12.2017, XNUMX), asitholanga lutho nhlobo. Ngokusobala, akuyena onjiniyela noma abasebenzisi balezi zixazululo abangakaze bahlangabezane namamethrikhi amaningi KANJALO, ngaphandle kwalokho bebezoqhamuka nokuthile.
Sabe sesikhumbula ngezibalo βzethoyiziβ - i-bioyino, eyabhalwa ku-Just for Fun hackathon (igama lephrojekthi lakhiqizwa umbhalo ngaphambi kokuqala kwe-hackathon) futhi sabona ukuthi sasizidinga ngokuphuthumayo izibalo zethu. Kwani?
- ngoba zimbalwa izibalo zama-clones emhlabeni,
- ngoba kungenzeka ukunikeza okufunayo noma okusondele ekubekezeleleni amaphutha kanye nokulinganisa (okuhlanganisa ukuvumelanisa amamethrikhi ahlanganisiwe phakathi kwamaseva nokuxazulula inkinga yokuthumela ukungqubuzana),
- ngoba kungenzeka ukubala amamethrikhi anembe kakhulu kunobrubeck,
- ngoba ungakwazi ukuqoqa izibalo ezinemininingwane eyengeziwe ngokwakho, u-brubeck angasinikezanga,
- ngoba ngibe nethuba lokuhlela eyami i-hyperperformance distributed scale lab application, engeke iphinde iphinde iphinde iphinde iphinde yenzeke enye i-hyperfor... kahle, yilokho kuphela.
Yini ongabhala kuyo? Yiqiniso, ku-Rust. Kungani?
- ngoba kwase kunesixazululo se-prototype,
- ngoba umbhali wendatshana wayeseyazi kakade uRust ngaleso sikhathi futhi wayemagange ukubhala okuthile kuyo ukuze kukhiqizwe nethuba lokuyibeka endaweni evulekile,
- ngoba izilimi ezine-GC azisifanele ngenxa yemvelo yethrafikhi etholiwe (cishe isikhathi sangempela) nokumiswa kwe-GC cishe akwamukelekile,
- ngoba udinga ukusebenza okuphezulu okuqhathaniswa no-C
- ngenxa yokuthi i-Rust isinika ukuvumelana okungesabi, futhi uma siqale ukuyibhala nge-C/C++, ngabe sesithole ubungozi obuningi, ukuchichima kwe-buffer, izimo zomjaho namanye amagama asabekayo kune-brubeck.
Kuphinde kwaba nengxabano ngokumelene noRust. Inkampani yayingenaso isipiliyoni sokudala amaphrojekthi eRust, futhi manje asihleli futhi ukuyisebenzisa kuphrojekthi enkulu. Ngakho-ke, kwakunokwesaba okukhulu kokuthi akukho okuzolunga, kodwa sanquma ukuthatha ithuba futhi sizame.
Sahamba isikhathi...
Ekugcineni, ngemva kwemizamo eminingana ehlulekile, inguqulo yokuqala yokusebenza yayisilungile. Kwenzenjani? Kwenzeka lokhu.
Inodi ngayinye ithola isethi yayo yamamethrikhi futhi iwaqongelele, futhi ayihlanganisi amamethrikhi alezo zinhlobo lapho isethi yawo ephelele idingeka khona ukuze kuhlanganiswe okokugcina. Ama-node axhunywe komunye nomunye ngohlobo oluthile lwephrothokholi yokukhiya, evumela ukuthi ukhethe phakathi kwabo eyodwa kuphela (lapha sakhala) efanelwe ukuthumela amamethrikhi kuMkhulu. Le nkinga okwamanje ixazululwa ngu , kodwa esikhathini esizayo izifiso zombhali zidlulela ku I-Raft, lapho ofaneleka kakhulu, kuzoba yi-node yomholi wokuvumelana. Ngokungeziwe ekuvumelaneni, ama-node ngokuvamile (kanye ngesekhondi ngokuzenzakalelayo) athumela komakhelwane babo lezo zingxenye zamamethrikhi ahlanganiswe ngaphambilini abakwazile ukuziqoqa kulowo mzuzwana. Kuvele ukuthi ukukala nokubekezelela amaphutha kuyagcinwa - inodi ngayinye isaphethe isethi egcwele yamamethrikhi, kodwa amamethrikhi athunyelwa asevele ehlanganisiwe, nge-TCP futhi afakwe ikhodi kuphrothokholi kanambambili, ngakho izindleko zokuphindaphinda zehliswa kakhulu uma kuqhathaniswa ne-UDP. Ngaphandle kwenani elikhulu kakhulu lamamethrikhi angenayo, ukunqwabelana kudinga inkumbulo encane kakhulu kanye ne-CPU encane. Kuma-mertics ethu acindezelwa kakhulu, lawa amashumi ambalwa wamamegabhayithi edatha. Njengebhonasi eyengeziwe, asitholi idatha ebhalwe kabusha engadingekile ku-Graphite, njengoba kwakunjalo nge-burbeck.
Amaphakethe e-UDP anamamethrikhi awalingani phakathi kwamanodi kumishini yenethiwekhi ngokusebenzisa i-Round Robin elula. Vele, i-hardware yenethiwekhi ayikuncozululi okuqukethwe kwamaphakethe ngakho-ke ingadonsa amaphakethe angaphezu kuka-4M ngomzuzwana, ingasaphathwa eye-metrics engazi lutho ngawo. Uma sicabangela ukuthi amamethrikhi awafiki kanye kanye ephaketheni ngalinye, lapho-ke asiboni kusengaphambili noma yiziphi izinkinga zokusebenza kule ndawo. Uma iseva iphahlazeka, idivayisi yenethiwekhi ngokushesha (phakathi kwamasekhondi angu-1-2) ithola leli qiniso futhi isuse iseva ephukile ekuzungeziseni. Njengomphumela walokhu, amanodi angenzi lutho (okungukuthi, okungeyena umholi) angavulwa futhi acinywe ngaphandle kokuqaphela ukudonswa phansi emashadini. Ubuningi esilahlekelwayo buyingxenye yamamethrikhi angene ngesekhondi lokugcina. Ukulahlekelwa kungazelelwe/ukuvalwa/ukushintsha komholi kusazodala ukudida okuncane (isikhawu sesibili esingama-30 sisaxoxisana), kodwa uma kukhona ukuxhumana phakathi kwamanodi, lezi zinkinga zingancishiswa, ngokwesibonelo, ngokuthumela amaphakethe okuvumelanisa. .
Okuncane mayelana nesakhiwo sangaphakathi. Uhlelo lokusebenza, yiqiniso, lunezintambo eziningi, kodwa i-architecture yokuxhuma ihlukile kunaleyo esetshenziswe ku-brubeck. Imicu ku-brubeck iyafana - ngayinye yazo inesibopho sakho kokubili ukuqoqwa kolwazi nokuhlanganisa. Ku-bioyino, abasebenzi bahlukaniswe ngamaqembu amabili: labo ababhekene nenethiwekhi kanye nalabo abanomthwalo wemfanelo wokuhlanganisa. Lesi sigaba sikuvumela ukuthi ulawule kalula uhlelo lokusebenza ngokuya ngohlobo lwamamethrikhi: lapho kudingeka khona ukuhlanganisa okujulile, ungangeza izihlanganisi, lapho kunokuningi kwethrafikhi yenethiwekhi, ungangeza inani lokugeleza kwenethiwekhi. Okwamanje, kumaseva ethu sisebenza kunethiwekhi eyi-8 kanye nokugeleza kokuhlanganisa okungu-4.
Ingxenye yokubala (ebhekele ukuhlanganisa) iyabhora. Amabhafa agcwaliswe ukugeleza kwenethiwekhi asakazwa phakathi kokugeleza kokubala, lapho abuye ahlukaniswe futhi ahlanganiswe. Ngesicelo, amamethrikhi anikezwa ukuze athunyelwe kwamanye amanodi. Konke lokhu, okuhlanganisa ukuthumela idatha phakathi kwama-node nokusebenza ne-Consul, kwenziwa ngokuhambisanayo, kusebenza kuhlaka. .
Izinkinga eziningi ezengeziwe ngesikhathi sokuthuthukiswa zidalwe ingxenye yenethiwekhi enesibopho sokwamukela amamethrikhi. Umgomo oyinhloko wokuhlukanisa ukugeleza kwenethiwekhi kumabhizinisi ahlukene kwaba isifiso sokunciphisa isikhathi esichithwa ukugeleza hhayi ukufunda idatha kusuka kusokhethi. Izinketho ezisebenzisa i-Asynchronous UDP kanye ne-recvmsg evamile zanyamalala ngokushesha: eyokuqala idla kakhulu i-CPU yesikhala somsebenzisi ukuze kucutshungulwe umcimbi, okwesibili kudinga ukushintshwa komongo okuningi kakhulu. Ngakho-ke manje isetshenziswa nama-buffers amakhulu (kanye nama-buffers, izikhulu zamadoda, azilutho kuwe!). Usekelo lwe-UDP evamile lugcinelwe izimo ezikhanyayo lapho i-recvmmsg ingadingeki. Kumodi ye-multimessage, kungenzeka ukufeza into esemqoka: isikhathi esiningi, intambo yenethiwekhi idonsa ulayini we-OS - ifunda idatha kusuka kusokhethi bese iyidlulisela ku-buffer yendawo yomsebenzisi, kuphela ngezikhathi ezithile ishintshela ekunikezeni isilondolozi esigcwele. ama-aggregators. Ulayini kusokhethi empeleni awuqoqi, inani lamaphakethe awehlile alikhuli.
Ukubhala
Kuzilungiselelo ezizenzakalelayo, usayizi webhafa usethwe ukuthi ube mkhulu impela. Uma ngokungazelelwe unquma ukuzama iseva ngokwakho, ungase uhlangabezane neqiniso lokuthi ngemva kokuthumela inombolo encane yamamethrikhi, ngeke afike ku-Graphite, esele ku-buffer yokusakaza kwenethiwekhi. Ukuze usebenze ngenani elincane lamamethrikhi, udinga ukusetha usayizi we-bufsize kanye nosayizi womugqa womsebenzi ube amanani amancane ekulungiselelweni.
Ekugcineni, amanye amashadi abathandi beshadi.
Izibalo zenombolo yamamethrikhi angenayo eseva ngayinye: ngaphezu kwezigidi ezingu-2 MPS.
Ikhubaza enye yamanodi nokusabalalisa kabusha amamethrikhi angenayo.
Izibalo zamamethrikhi aphumayo: kuthunyelwa inodi eyodwa kuphela - umphathi wokuhlasela.
Izibalo zokusebenza kwe-node ngayinye, kucatshangelwa amaphutha kumamojula wesistimu ahlukahlukene.
Imininingwane yamamethrikhi angenayo (amagama e-metric afihliwe).
Sihlela ukwenzani ngakho konke lokhu ngokulandelayo? Yebo, bhala ikhodi, damn...! Iphrojekthi ekuqaleni yayihlelelwe ukuthi ibe ngumthombo ovulekile futhi izohlala injalo impilo yayo yonke. Izinhlelo zethu ezisheshayo zihlanganisa ukushintshela enguqulweni yethu ye-Raft, ukushintsha iphrothokholi yontanga iye kwephatheka kakhudlwana, ukwethula izibalo zangaphakathi ezengeziwe, izinhlobo ezintsha zamamethrikhi, ukulungiswa kweziphazamisi nokunye ukuthuthukiswa.
Yiqiniso, wonke umuntu wamukelekile ukusiza ekuthuthukiseni iphrojekthi: dala i-PR, Izinkinga, uma kungenzeka sizophendula, sithuthukise, njll.
Sekushiwo lokho, bakwethu, thenga izindlovu zethu!
Source: www.habr.com