Njengoko sisuka kwisicelo se-monolithic ukuya kwi-architecture ye-microservices, sijongene nemingeni emitsha.

Kwisicelo se-monolithic, ngokuqhelekileyo kulula ukumisela ukuba yeyiphi indawo yenkqubo impazamo eyenzekileyo. Inokwenzeka ukuba, ingxaki ikwikhowudi ye-monolith ngokwayo, okanye kwisiseko sedatha. Kodwa xa siqala ukukhangela ingxaki kwi-microservice architecture, yonke into ayisekho ngokucacileyo. Kufuneka sifumane indlela yonke esicelwe ngayo ukusuka ekuqaleni ukuya ekugqibeleni kwaye siyikhethe kumakhulu eenkonzo ezincinci. Ngaphezu koko, ezininzi zazo zineendawo zazo zokugcina, ezinokuthi zibangele iimpazamo ezinengqiqo, kunye neengxaki zokusebenza kunye nokunyamezela impazamo.

Kudala ndikhangela isixhobo esiya kunceda ukujamelana neengxaki ezinjalo (ndibhale malunga noku kuHabré: 1, 2), kodwa ekugqibeleni ndenze eyam isisombululo somthombo ovulekileyo. Kule nqaku ndithetha ngeenzuzo zendlela ye-mesh yenkonzo kwaye ndabelana ngesixhobo esitsha sokuphunyezwa kwayo.

Ukuhanjiswa kokulandelela isisombululo esiqhelekileyo kwingxaki yokufumana iimpazamo kwiinkqubo ezisasazwayo. Kodwa kuthekani ukuba le ndlela yokuqokelela ulwazi malunga nokusebenzisana kwenethiwekhi ayikakaphunyezwa kwinkqubo, okanye, ngakumbi, inxalenye yenkqubo sele isebenza ngokufanelekileyo, kodwa inxalenye ayifuni, ekubeni ayizange ifakwe kwiinkonzo zakudala. ? Ukuchonga eyona nto ibangela ingxaki, kuyimfuneko ukuba ube nomfanekiso opheleleyo wento eyenzekayo kwinkqubo. Kubaluleke ngakumbi ukuqonda ukuba zeziphi ii-microservices ezibandakanyekayo kwiindlela eziphambili zeshishini.

Apha indlela ye-mesh yenkonzo inokusincedisa, eya kujongana nabo bonke oomatshini bokuqokelela ulwazi lwenethiwekhi kwinqanaba elingaphantsi kuneenkonzo ezisebenza ngokwazo. Le ndlela ivumela ukuba sithintele yonke i-traffic kwaye siyihlalutye kwi-fly. Ngaphezu koko, izicelo akufuneki nokuba zazi nantoni na ngayo.

Indlela yenkonzo yomnatha

Umbono ophambili wendlela yemesh yenkonzo kukongeza omnye umaleko weziseko ezingundoqo ngaphezulu kwenethiwekhi, eya kusivumela ukuba senze nantoni na ngonxibelelwano phakathi kwenkonzo. Uninzi lokuphunyezwa lusebenza ngolu hlobo lulandelayo: i-container eyongezelelweyo ye-sidecar ene-proxy ecacileyo yongezwa kwi-microservice nganye, apho yonke i-traffic engenayo nephumayo yenkonzo idluliswa. Kwaye le yindawo apho sinokwenza khona ukulinganisa umxhasi, sisebenzise imigaqo-nkqubo yokhuseleko, sibeke izithintelo kwinani lezicelo kwaye siqokelele ulwazi olubalulekileyo ekusebenzisaneni kweenkonzo kwimveliso.

Izisombululo

Sele kukho iindlela ezininzi zokuphunyezwa kwale ndlela: Istio и linkrd2. Banikezela ngeempawu ezininzi ngaphandle kwebhokisi. Kodwa kwangaxeshanye, kuza i-overhead enkulu kwizibonelelo. Ngaphezu koko, okukhona likhulu iqela apho inkqubo enjalo isebenza khona, kokukhona kuya kufuneka izibonelelo ezininzi zokugcina iziseko ezitsha. Kwi-Avito, sisebenza amaqela e-kubernetes aqulethe amawaka eemeko zenkonzo (kwaye inani labo liyaqhubeka likhula ngokukhawuleza). Ekuphunyezweni kwayo kwangoku, i-Istio idla ~ 300Mb ye-RAM ngomzekelo wenkonzo. Ngenxa yenani elikhulu lamathuba, ukulinganisa okucacileyo kukwachaphazela ixesha lokuphendula lonke leenkonzo (ukuya kwi-10ms).

Ngenxa yoko, siye sajonga ngqo ukuba yeyiphi izakhono esizidingayo ngoku, kwaye sagqiba ekubeni esona sizathu siphambili sokuba siqalise ukusebenzisa izisombululo ezinjalo kukukwazi ukuqokelela ulwazi lokulandela umkhondo kwinkqubo yonke ngokuphandle. Sikwafuna ukuba nolawulo kunxibelelwano lweenkonzo kwaye senze iindlela ezahlukeneyo zokukhohlisa ngeeheader ezikhutshelwa phakathi kweenkonzo.

Ngenxa yoko, sifikelele kwisigqibo sethu:  Netramesh.

Netramesh

Netramesh sisisombululo se-mesh yenkonzo ekhaphukhaphu kunye nokukwazi ukulinganisa ngokungenasiphelo, kungakhathaliseki ukuba inani leenkonzo kwinkqubo.

Iinjongo eziphambili zesisombululo esitsha zaziphantsi kwemithombo ephantsi kunye nokusebenza okuphezulu. Phakathi kweempawu eziphambili, ngokukhawuleza sasifuna ukukwazi ukuthumela ngokubonakalayo iindawo zokulandela umkhondo kwinkqubo yethu yeJaeger.

Namhlanje, ezininzi izisombululo zamafu ziphunyezwa eGolang. Kwaye, ngokuqinisekileyo, kukho izizathu zoku. Ukubhala usetyenziso lwenethiwekhi kwi-Golang esebenza ngokuhambelana ne-I/O kunye nesikali kwii-cores njengoko kufuneka kuluncedo kwaye kulula kakhulu. Kwaye, okubaluleke kakhulu, ukusebenza kwanele ukusombulula le ngxaki. Yiyo loo nto nathi sikhethe iGolang.

Imveliso

Sijolise iinzame zethu ekuphumezeni imveliso ephezulu. Ukufumana isisombululo esibekwe ecaleni komzekelo ngamnye wenkonzo, ukusetyenziswa okuncinci kwe-RAM kunye nexesha le-CPU liyafuneka. Kwaye, ngokuqinisekileyo, ukulibaziseka kwempendulo kufuneka kube kuncinci.

Makhe sibone ukuba zeziphi iziphumo esizifumeneyo.

i-ram

I-Netramesh idla i- ~ 10Mb ngaphandle kwe-traffic kunye ne-50Mb ephezulu kunye nomthwalo ofikelela kwi-10000 RPS ngomzekelo ngamnye.

Umthunywa we-Istio uhlala esitya ~ 300Mb kumaqela ethu ngamawaka eemeko. Oku akuvumeli ukuba ilinganiswe kwiqela lonke.

NgeNetramesh sifumene ~ 10x yokunciphisa ukusetyenziswa kwememori.

ICPU

Ukusetyenziswa kwe-CPU kuyalingana phantsi komthwalo. Kuxhomekeke kwinani lezicelo kwiyunithi yexesha kwi-sidecar. Amaxabiso kwizicelo ezingama-3000 ngesekhondi kwincopho:

Kukho enye ingongoma ebalulekileyo: I-Netramesh - isisombululo ngaphandle kwendiza yokulawula kwaye ngaphandle komthwalo ayidli ixesha le-CPU. Nge-Istio, ii-sidecars zihlala zihlaziya iziphelo zenkonzo. Ngenxa yoko, sinokubona lo mfanekiso ngaphandle komthwalo:

Sisebenzisa i-HTTP/1 kunxibelelwano phakathi kweenkonzo. Ukonyuka kwexesha lokuphendula le-Istio xa i-proxying ngokusebenzisa umthunywa ifikelele kwi-5-10ms, into eninzi kakhulu kwiinkonzo ezilungele ukuphendula kwi-millisecond. NgeNetramesh eli xesha liye lehla ukuya kwi-0.5-2ms.

Ukubaleka

Ubungakanani obuncinci bezibonelelo ezisetyenziswa ngummeli ngamnye zenza kube lula ukubeka ecaleni kwenkonzo nganye. I-Netramesh yenziwe ngabom ngaphandle kwecandelo lolawulo lwenqwelomoya ukugcina nje inqwelo esecaleni ilula. Rhoqo kwizisombululo zemesh yenkonzo, inqwelomoya yolawulo isasaza ulwazi lokufunyanwa kwenkonzo kwikhareji nganye esecaleni. Ngokuhamba nayo kuza ulwazi malunga nokuphuma kwexesha kunye nezicwangciso zokulinganisa. Konke oku kukuvumela ukuba wenze izinto ezininzi eziluncedo, kodwa, ngelishwa, ibhula ii-sidecars ngobukhulu.

Ukufunyanwa kwenkonzo

I-Netramesh ayongezi naziphi na iindlela ezongezelelweyo zokufunyanwa kwenkonzo. Zonke itrafikhi zifakwe elubala kwi-netra sidecar.

INetramesh ixhasa iprotocol yesicelo seHTTP/1. Ukuyichaza, uluhlu oluqwalaselweyo lwamazibuko luyasetyenziswa. Ngokuqhelekileyo, inkqubo inamazibuko amaninzi apho unxibelelwano lwe-HTTP lwenzeka khona. Ngokomzekelo, sisebenzisa i-80, i-8890, i-8080 yokusebenzisana phakathi kweenkonzo kunye nezicelo zangaphandle Kulo mzekelo, zinokusetwa ngokusebenzisa ukuguquguquka kokusingqongileyo NETRA_HTTP_PORTS.

Ukuba usebenzisa i-Kubernetes njenge-okhestra kunye ne-service entity mechanism yonxibelelwano phakathi kweqela phakathi kweenkonzo, ke indlela yokusebenza ihlala ifana kanye. Okokuqala, i-microservice ifumana idilesi ye-IP yenkonzo isebenzisa i-kube-dns kwaye ivula uxhulumaniso olutsha kuyo. Olu xhulumaniso lusekwe okokuqala nge-netra-sidecar yendawo kwaye zonke iipakethi ze-TCP zifika kuqala kwi-netra. Okulandelayo, i-netra-sidecar iseka umdibaniso kunye nendawo yokuqala. I-NAT kwi-pod IP kwi-node ihlala ifana ngqo nangaphandle kwe-netra.

Ukuhanjiswa komkhondo kunye nokuthunyelwa kwemeko

I-Netramesh ibonelela ngokusebenza okufunekayo ukuthumela iindawo zokulandela umkhondo malunga nokusebenzisana kwe-HTTP. I-Netra-sidecar iyacalula i-HTTP protocol, imilinganiselo yesicelo sokulibaziseka, kwaye ikhupha ulwazi oluyimfuneko kwi-header ye-HTTP. Ekugqibeleni, sifumana yonke imikhondo kwinkqubo enye yeJaeger. Kuqwalaselo olucokisekileyo, unokusebenzisa iinguqu zemo engqongileyo ezibonelelwa lithala leencwadi elisemthethweni jaeger go ilayibrari.

Kodwa kukho ingxaki. De iinkonzo zivelise kwaye zithumele i-header ye-uber ekhethekileyo, asiyi kubona iindawo zokulandela umkhondo ezidityanisiweyo kwinkqubo. Kwaye yiloo nto esiyidingayo ukuze sifumane unobangela weengxaki ngokukhawuleza. Apha kwakhona iNetramesh inesisombululo. Abameli bafunda iiheader zeHTTP kwaye, ukuba aziqulathanga i-id ye-uber trace, yenza enye. I-Netramesh ikwagcina ulwazi malunga nezicelo ezingenayo neziphumayo kwi-sidecar kwaye izitshatise ngokuzityebisa ngeeheader eziyimfuneko eziphumayo. Konke okufuneka ukwenze kwiinkonzo kukuthumela umbhalo omnye kuphela X-Request-Id, enokuthi iqwalaselwe kusetyenziswa imo eguquguqukayo NETRA_HTTP_REQUEST_ID_HEADER_NAME. Ukulawula ubungakanani bomxholo kwiNetramesh, unokuseta oku kuguquguquka kwemeko-bume kulandelayo: NETRA_TRACING_CONTEXT_EXPIRATION_MILLISECONDS (ixesha apho umxholo uya kugcinwa) kunye NETRA_TRACING_CONTEXT_CLEANUP_INTERVAL (izihlandlo zokucocwa komxholo).

Kuyenzeka kwakhona ukudibanisa iindlela ezininzi kwindlela yakho ngokumakisha ngophawu lweseshoni ekhethekileyo. I-Netra ikuvumela ukuba ufake HTTP_HEADER_TAG_MAP ukuguqula iiheader zeHTTP zibe ziithegi ezihambelanayo zokulandela umkhondo. Oku kunokuba luncedo ngakumbi kuvavanyo. Emva kokuphumelela uvavanyo olusebenzayo, unokubona ukuba yeyiphi inxalenye yenkqubo echatshazelwe kukuhluzwa ngeqhosha leseshoni elihambelanayo.

Ukumisela uMthombo wesicelo

Ukumisela apho isicelo sivela khona, ungasebenzisa usebenziso lokudibanisa ngokuzenzekelayo isihloko esinesihloko kunye nomthombo. Ukusebenzisa imo eguquguqukayo NETRA_HTTP_X_SOURCE_HEADER_NAME Ungakhankanya igama leheader eliza kufakwa ngokuzenzekelayo. Ngokusebenzisa NETRA_HTTP_X_SOURCE_VALUE ungacwangcisa ixabiso apho i-X-Umthombo weheader izakumiselwa kuzo zonke izicelo eziphumayo.

Oku kuvumela unikezelo lwale ntloko iluncedo ukuba isasazwe ngokufanayo kuwo wonke umsebenzi womnatha. Emva koko unokuyisebenzisa kwiinkonzo kwaye uyongeze kwiilog kunye neemetrics.

Indlela yeTrafikhi kunye ne-Netramesh yangaphakathi

I-Netramesh iqulathe amacandelo amabini aphambili. Eyokuqala, i-netra-init, ibeka imithetho yenethiwekhi ukunqanda i-traffic. Usebenzisa iptables redirect rules ukuthintela yonke okanye inxalenye yetrafikhi kwi-sidecar, eyona nxalenye yesibini ingundoqo ye-Netramesh. Ungaqwalasela ukuba ngawaphi amazibuko afuna ukuvalelwa kwiiseshoni zeTCP ezingenayo neziphumayo: INBOUND_INTERCEPT_PORTS, OUTBOUND_INTERCEPT_PORTS.

Isixhobo sikwanalo uphawu olunomdla - umzila onokwenzeka. Ukuba usebenzisa iNetramesh ngokukhethekileyo ukuqokelela iindawo zokukhangela umkhondo, ngoko kwimeko-bume yemveliso ungagcina izixhobo kwaye wenze umzila onokwenzeka usebenzisa izinto eziguquguqukayo. NETRA_INBOUND_PROBABILITY и NETRA_OUTBOUND_PROBABILITY (ukusuka kwi-0 ukuya kwi-1). Ixabiso elimiselweyo ngu-1 (zonke iitrafikhi zibaliwe).

Emva kokungenelela okuyimpumelelo, i-netra sidecar yamkela uqhagamshelwano olutsha kunye nokusetyenziswa SO_ORIGINAL_DST ukhetho lwesokhethi ukufumana indawo yokuqala. Emva koko i-Netra ivula uxhulumaniso olutsha kwidilesi ye-IP yasekuqaleni kwaye iseke unxibelelwano lwe-TCP yeendlela ezimbini phakathi kwamaqela, ukuphulaphula zonke izithuthi ezidlulayo. Ukuba izibuko lichazwa njenge HTTP, iNetra izama ukuyicazulula kwaye iyilandele. Ukuba ulwahlulo lwe-HTTP aluphumeleli, iNetra iwela umva kwi-TCP kwaye ngokuphandle yenza i-bytes.

Ukwakha igrafu yokuxhomekeka

Emva kokufumana inani elikhulu lolwazi lokulandelela kwiJaeger, ndifuna ukufumana igrafu epheleleyo yokusebenzisana kwinkqubo. Kodwa ukuba inkqubo yakho ilayishiwe kwaye iibhiliyoni zezithuba zokulandela ziyaqokelelana ngosuku, ukuzihlanganisa akungomsebenzi ulula kangako. Kukho indlela esemthethweni yokwenza oku: ukuxhomekeka kwentlantsi. Nangona kunjalo, kuya kuthatha iiyure ukwakha igrafu epheleleyo kwaye iya kukunyanzela ukuba ukhuphele yonke idatha yedatha kwi-Jaeger kwiiyure ezidlulileyo ze-24.

Ukuba usebenzisa i-Elasticsearch ukugcina izithuba zokulandela umkhondo, ungasebenzisa usetyenziso olulula lweGolang, eya kwakha igrafu efanayo kwimizuzu usebenzisa iimpawu kunye nezakhono ze-Elasticsearch.

Uyisebenzisa njani iNetramesh

I-Netra inokongezwa ngokulula kuyo nayiphi na inkonzo eqhuba nayiphi na iokhestra. Ungawubona umzekelo apha.

Okwangoku, i-Netra ayinakho ukukwazi ukuphumeza ngokuzenzekelayo ii-sidecars kwiinkonzo, kodwa kukho izicwangciso zokuphunyezwa.

Ikamva leNetramesh

Injongo ephambili Netramesh kukuzuza iindleko ezincinci zezibonelelo kunye nokusebenza okuphezulu, ukubonelela ngezakhono ezisisiseko zokubonwa kunye nokulawula unxibelelwano phakathi kweenkonzo.

Kwixesha elizayo, iNetramesh iya kuxhasa ezinye iiprothokholi zomaleko wesicelo ngaphandle kweHTTP. Indlela ye-L7 iya kufumaneka kungekudala.

Sebenzisa iNetramesh ukuba uhlangabezana neengxaki ezifanayo kwaye usibhalele ngemibuzo kunye neengcebiso.

umthombo: www.habr.com