ProHoster > Blog > Utongi > [Shanduro] Mumiriri wenhumwa

[Shanduro] Mumiriri wenhumwa

Dudziro yechinyorwa: Envoy threading modhi - https://blog.envoyproxy.io/envoy-threading-model-a8d44b922310

Ndakaona chinyorwa ichi chinonakidza, uye sezvo Envoy ichinyanya kushandiswa sechikamu che "istio" kana kungoita se "ingress controller" ye kubernetes, vanhu vazhinji havana kudyidzana kwakananga nayo se, semuenzaniso, neyakajairwa. Nginx kana Haproxy kuiswa. Zvisinei, kana chimwe chinhu chichiputsika, zvingava zvakanaka kuti unzwisise kuti chinoshanda sei kubva mukati. Ndakaedza kushandurira mashoko akawanda mumutauro wechiRussian sezvinobvira, kusanganisira mazwi anokosha; kune avo vanozviona zvichirwadza kutarisa izvi, ndakasiya zvinyorwa mumabharanzi. Kugamuchirwa kukati.

Yakaderera-level tekinoroji zvinyorwa zveiyo Envoy codebase parizvino zvishoma. Kugadzirisa izvi, ini ndinoronga kuita nhevedzano yeblog blog nezve akasiyana subsystems eEnvoy. Sezvo iyi iri nyaya yokutanga, ndapota ndizivise zvaunofunga uye zvaungafarira munyaya dzomunguva yemberi.

Imwe yemibvunzo yakajairika yehunyanzvi yandinogamuchira nezve Nhume ndeyekubvunza yakaderera-pamwero tsananguro yemhando yeshinda yainoshandisa. Mune ino positi, ini ndichatsanangura mamepu eNhume yekubatanidza kune tambo, pamwe neThread Yenzvimbo Yekuchengetera sisitimu yainoshandisa mukati kugadzira kodhi inoenderana uye yakakwirira-kuita.

Threading overview

[Shanduro] Mumiriri wenhumwa

Envoy inoshandisa marudzi matatu akasiyana enzizi:

  • Main: Iyi tambo inodzora maitiro ekutanga uye kumisa, kuseta kweXDS (xDiscovery Service) API, kusanganisira DNS, kutarisa hutano, general cluster uye runtime management, statistics reset, manejimendi uye general process management - Linux masaini. hot restart, nezvimwe zvese izvo zvinoitika mune iyi shinda ndeye asynchronous uye "isina-kuvhara". Kazhinji, iyo tambo huru inoronga ese akakosha mashandiro maitiro ayo asingade yakakura yeCPU kuti iite. Izvi zvinobvumira kodhi yekudzora yakawanda kuti inyorwe sekunge ine shinda imwe chete.
  • Mushandi: Nekumisikidza, Nhume inogadzira tambo yevashandi kune yega yega tambo muhurongwa, izvi zvinogona kudzorwa uchishandisa sarudzo --concurrency. Imwe neimwe tambo yevashandi inomhanyisa "isina-kuvharira" chiitiko loop, iyo ine basa rekuteerera kune mumwe nemumwe mutereri; panguva yekunyora (Chikunguru 29, 2017) hapana sharding yemuteereri, kugamuchira kutsva kunobatanidza, kusimbisa stack yefirita. iyo yekubatanidza, uye kugadzirisa ese ekuisa / kubuda (IO) mashandiro panguva yehupenyu hwekubatanidza. Zvekare, izvi zvinobvumira yakawanda yekubatanidza kubata kodhi kuti inyorwe sekunge yakasungwa imwechete.
  • File flusher: Imwe neimwe faira inonyora Envoy, kunyanya kuwana matanda, parizvino ine yakazvimirira yekuvharira tambo. Izvi zvinokonzerwa nekuti kunyora kune mafaera akachengetwa nefaira system kunyangwe uchishandisa O_NONBLOCK dzimwe nguva anogona kuvharwa (kugomera). Kana tambo dzevashandi dzichida kunyorera kufaira, iyo data inofambiswa kune buffer mundangariro uko inozopedzisira yapeperetswa neshinda. faira flush. Iyi inzvimbo imwe yekodhi iyo tekinoroji tambo dzese dzevashandi dzinogona kuvhara kiyi imwechete uchiedza kuzadza memory buffer.

Kubatanidza kubata

Sezvakakurukurwa muchidimbu pamusoro, tambo dzese dzevashandi dzinoteerera kune vese vateereri pasina kugovana. Nekudaro, iyo kernel inoshandiswa kutumira nenyasha zvigadziko zvinogamuchirwa kune tambo dzevashandi. Mbeu dzemazuvano dzinowanzo nyanya kunaka pane izvi, dzinoshandisa maficha senge input/output (IO) inosimudzira kuyedza kuzadza shinda nebasa vasati vatanga kushandisa dzimwe shinda dziri kuteererawo pachisoketi chimwe chete, uyezve kusashandisa robin yakatenderera. kukiya (Spinlock) kugadzirisa chikumbiro chega chega.
Kana kubatanidza kwagamuchirwa pashinda yevashandi, haimbosiyi shinda iyo. Yese kuenderera mberi kwekugadzirisa kwekubatanidza kunobatwa zvachose mushinda yevashandi, kusanganisira chero maitiro ekutumira.

Izvi zvine mhedzisiro yakakosha yakawanda:

  • Ese madziva ekubatanidza muEnvoy anopihwa kune shinda yevashandi. Saka, kunyangwe madziva ekubatanidza eHTTP/2 achingoita chinongedzo chimwe chete kune imwe neimwe inokwidza kumusoro panguva, kana paine tambo ina dzevashandi, pachave neina HTTP/2 yekubatanidza pane imwe nhanho inokwidza munzvimbo yakatsiga.
  • Chikonzero chinoita kuti Nhume ishande nenzira iyi ndechekuti nekuchengeta zvese pane imwechete yevashandi shinda, dzinenge kodhi yese inogona kunyorwa isina kuvharika uye sekunge yakasungwa imwechete. Iyi dhizaini inoita kuti zvive nyore kunyora yakawanda kodhi uye zvikero zvinokatyamadza kusvika kune isingaverengeki nhamba yeshinda yevashandi.
  • Nekudaro, chimwe chezvinonyanya kutora ndechekuti kubva mundangariro dziva uye yekubatanidza mashandiro ekuona, zvinonyanya kukosha kugadzirisa iyo --concurrency. Kuve netambo dzakawanda dzevashandi kupfuura zvinodiwa kuchaparadza ndangariro, kugadzira mamwe majoini asina basa, uye kuderedza chiyero chekubatanidza. KuLyft, midziyo yedu yenhumwa yepadivi inomhanya ine yakaderera kwazvo concurrency kuitira kuti kuita kunoenderana nemasevhisi avagere pedyo nawo. Isu tinomhanyisa Envoy seyekumucheto proxy chete panguva yepamusoro concurrency.

Ko kusavharisa kunorevei?

Izwi rekuti "kusavharira" rakashandiswa kakawanda kusvika pari zvino pakukurukura kuti tambo huru nedzevashandi dzinoshanda sei. Yese kodhi yakanyorwa pakufungidzira kuti hapana chinombovharwa. Zvisinei, ichi hachisi chokwadi zvachose (chii chisiri chokwadi zvachose?).

Envoy inoshandisa akati wandei marefu makiyi:

  • Sezvakakurukurwa, kana uchinyora matanda ekuwana, tambo dzese dzevashandi dzinowana kiyi imwechete iyo in-memory log buffer isati yazadzwa. Iyo yekukiya inobata nguva inofanirwa kunge yakadzikira, asi zvinogoneka kuti kukiya kukwikwidzwe nepamusoro concurrency uye yakakwira throughput.
  • Envoy inoshandisa hurongwa hwakaoma kwazvo kubata nhamba dziri munharaunda kune shinda. Iyi ichava musoro weimwe nzvimbo yakasiyana. Nekudaro, ini ndichataura muchidimbu kuti sechikamu chekugadzirisa tambo manhamba munharaunda, dzimwe nguva zvinodikanwa kuwana kiyi pane yepakati "stats store". Kukiya uku hakufanirwe kudiwa.
  • Iyo tambo huru nguva nenguva inoda kubatana neshinda dzese dzevashandi. Izvi zvinoitwa ne "kuburitsa" kubva pashinda huru kuenda kune shinda dzevashandi, uye dzimwe nguva kubva kune shinda dzevashandi kudzokera kune shinda huru. Kutumira kunoda kukiyiwa kuitira kuti meseji yakadhindwa igoiswa mumutsetse kuti igozotumirwa. Aya makiyi haafanire kukwikwidzwa zvakanyanya, asi anogona kuvharika nehunyanzvi.
  • Kana Envoy inonyora danda kune system yekukanganisa rwizi (yakajairwa kukanganisa), inowana kukiya pane yese maitiro. Kazhinji, kutema matanda kweEnvoy kunoonekwa sekutyisa kubva pamaonero ekuita, saka hapasati pave nekutariswa kwakawanda kwakapihwa mukunatsiridza.
  • Kune mamwe mashoma makiyi asina kurongeka, asi hapana kana imwe yadzo yakakosha uye haifanirwe kupikiswa.

Thread yenzvimbo yekuchengetedza

Nekuda kwenzira iyo Nhume inoparadzanisa nayo mitoro yeshinda huru kubva pamabasa eshinda yevashandi, pane chinodiwa kuti kugadzirisa kwakaoma kunogona kuitwa pashinda huru uye ipapo kupihwa kune yega yega shinda yemushandi nenzira inowirirana. Ichi chikamu chinotsanangura Envoy Thread Local Storage (TLS) pamwero wepamusoro. Muchikamu chinotevera ini ndichatsanangura kuti rinoshandiswa sei kubata sumbu.
[Shanduro] Mumiriri wenhumwa

Sezvakatsanangurwa kare, tambo huru inobata dzinenge dzese manejimendi uye kudzora mashandiro endege muEvoy process. Iyo ndege yekudzora yakaremerwa pano, asi kana iwe ukaitarisa mukati meiyo Envoy process pachayo woienzanisa nekutumira kunoitwa neshinda dzevashandi, zvine musoro. Mutemo mukuru ndewekuti tambo huru inoita rimwe basa, uyezve inoda kuvandudza shinda yega yega yevashandi zvinoenderana nemhedzisiro yebasa iroro. mune iyi kesi, shinda yevashandi haidi kuwana kiyi pane yega yega yekuwana.

Envoy's TLS (Thread yenzvimbo yekuchengetedza) system inoshanda seinotevera:

  • Kodhi inomhanya patambo huru inogona kugovera TLS slot yemaitiro ese. Kunyangwe izvi zvichibviswa, mukuita indekisi muvheta, inopa O(1) kuwana.
  • Iyo tambo huru inogona kuisa dhairekitori data mune yayo slot. Kana izvi zvaitwa, iyo data inoburitswa kune yega yega yevashandi tambo seyakajairika chiitiko loop chiitiko.
  • Shinda dzevashandi dzinogona kuverenga kubva kune yavo TLS slot uye kutora chero shinda-yenzvimbo data iripo.

Kunyangwe iri paradigm yakapusa uye ine simba rinoshamisa, yakafanana neiyo pfungwa yeRCU (Read-Copy-Update) ichivharira. Chaizvoizvo, tambo dzevashandi hadzina kumboona chero shanduko yedata muTLS slots apo basa riri kushanda. Shanduko inoitika chete panguva yekuzorora pakati pezviitiko zvebasa.

Mumiriri anoshandisa izvi nenzira mbiri dzakasiyana:

  • Nekuchengetedza data rakasiyana pane yega yega yevashandi tambo, iyo data inogona kuwanikwa pasina chero kuvharira.
  • Nekuchengetedza yakagovaniswa pointer kune yepasirese data mune yekuverenga-chete modhi pane yega yega yevashandi shinda. Saka, tambo yega yega yevashandi ine data referenzi kuverenga iyo isingagone kudzikiswa basa richiri kushanda. Chete kana vashandi vese vadzikama uye kurodha nyowani yakagovaniswa data iyo yekare data ichaparadzwa. Izvi zvakafanana neRCU.

Cluster update threading

Muchikamu chino, ini ndichatsanangura nzira iyo TLS (Thread yenzvimbo yekuchengetedza) inoshandiswa kubata sumbu. Cluster management inosanganisira xDS API uye/kana DNS kugadzirisa, pamwe nekutarisa hutano.
[Shanduro] Mumiriri wenhumwa

Cluster flow management inosanganisira zvinotevera zvikamu uye matanho:

  1. Cluster Maneja chikamu chiri mukati meNhume inotarisira ese anozivikanwa cluster kumusoro, iyo Cluster Discovery Service (CDS) API, iyo Chakavanzika Discovery Service (SDS) uye Endpoint Discovery Service (EDS) APIs, DNS, uye inoshanda yekunze macheki. Iyo ine basa rekugadzira iyo "inozoenderana" maonero ega ega sumbu rekumusoro, iro rinosanganisira vanowanikwa vaenzi pamwe nemamiriro ehutano.
  2. Muongorori wehutano anoita cheki yehutano inoshanda uye anoshuma shanduko yehutano kune cluster manager.
  3. CDS (Cluster Discovery Service) / SDS (Secret Discovery Service) / EDS (Endpoint Discovery Service) / DNS inoitwa kuti ione nhengo dzeboka. Kuchinja kwenyika kunodzoserwa kune maneja weboka.
  4. Imwe neimwe tambo yevashandi inoramba ichiita chiitiko loop.
  5. Kana maneja wecluster aona kuti nyika yesumbu yachinja, inogadzira mutsva wekuverenga-chete snapshot yenzvimbo yecluster uye inotumira kune yega yega shinda yevashandi.
  6. Munguva inotevera yakanyarara, shinda yevashandi inovandudza mufananidzo mune yakagoverwa TLS slot.
  7. Panguva yechiitiko cheI/O chinofanirwa kutarisisa mugadziri kuti atakure chiyero, muyedzo wemutoro unokumbira TLS (Thread yenzvimbo yekuchengetedza) slot kuti uwane ruzivo nezve mugamuchiri. Izvi hazvidi zvekukiya. Ziva zvakare kuti TLS inogona zvakare kukonzeresa zviitiko kuitira kuti mitoro inoremedza uye zvimwe zvikamu zvinogona kuverengera cache, zvimiro zvedata, nezvimwe. Izvi zvinopfuura chiyero chechinyorwa ichi, asi chinoshandiswa munzvimbo dzakasiyana siyana mukodhi.

Uchishandisa maitiro ari pamusoro apa, Nhume inogona kugadzirisa chikumbiro chese pasina kuvharira (kunze sekutsanangurwa kwakamboitwa). Kunze kwekuoma kweiyo TLS kodhi pachayo, yakawanda yekodhi haidi kunzwisisa kuti multithreading inoshanda sei uye inogona kunyorwa imwe-tambo. Izvi zvinoita kuti yakawanda yekodhi ive nyore kunyora mukuwedzera kune yepamusoro kuita.

Mamwe ma subsystems anoshandisa TLS

TLS (Thread local storage) uye RCU (Verenga Copy Update) anoshandiswa zvakanyanya muEnvoy.

Mienzaniso yekushandisa:

  • Mechanism yekuchinja mashandiro panguva yekuuraya: Rondedzero yazvino yemashandiro akagoneswa akaverengerwa mushinda huru. Imwe neimwe tambo yevashandi inopihwa yekuverenga-chete snapshot uchishandisa RCU semantics.
  • Kutsiva matafura enzira: Pamatafura enzira anopiwa neRSS (Route Discovery Service), matafura enzira anogadzirwa pashinda huru. Iyo yekuverenga-chete snapshot inozopihwa kune yega yega shinda yemushandi uchishandisa RCU (Verenga Copy Update) semantics. Izvi zvinoita kuti kuchinja matafura enzira ashande zvakanaka.
  • HTTP musoro we caching: Sezvazvinozoitika, kuverenga iyo HTTP musoro wechikumbiro chega chega (uchiri kumhanya ~ 25K + RPS pamusimboti) inodhura zvakanyanya. Nhume yepakati inoverengera musoro ingangoita hafu yesekondi uye inopa kune wese mushandi kuburikidza neTLS neRCU.

Pane zvimwe zviitiko, asi mienzaniso yapfuura inofanira kupa kunzwisisa kwakanaka kweiyo TLS inoshandiswa.

Kuzivikanwa kwekuita pitfalls

Nepo Nhume ichiita zvakanaka chose, kune dzimwe nzvimbo dzinocherechedzwa dzinoda kutariswa kana ichishandiswa neyakakura kwazvo concurrency uye kubuda:

  • Sezvinotsanangurwa muchinyorwa chino, parizvino tambo dzese dzevashandi dzinowana kiyi painonyorera kune yekuwana log memory buffer. Pakuwirirana kwepamusoro uye nekubuda kwepamusoro, iwe unozofanirwa kubheta matanda ekuwana kune yega yega shinda yemushandi uchibhadhara kunze-kwe-odha yekutumira kana uchinyorera kune yekupedzisira faira. Neimwe nzira, iwe unogona kugadzira yakaparadzana yekuwana log kune yega yega yevashandi shinda.
  • Kunyangwe manhamba akagadziridzwa zvakanyanya, pamari yakakwira zvakanyanya uye kuburikidza nekushandisa panogona kunge paine gakava reatomu pahuwandu hwega. Mhinduro kudambudziko iri icounters per worker thread ine periodic reset yemacentral counters. Izvi zvichakurukurwa mune inotevera post.
  • Iyo yekuvakisa yazvino haishande zvakanaka kana Envoy ikaiswa mumamiriro ezvinhu apo kune mashoma ekubatanidza ayo anoda zvakakosha kugadzirisa zviwanikwa. Iko hakuna vimbiso yekuti zvinongedzo zvichagovaniswa zvakaenzana pakati petambo dzevashandi. Izvi zvinogona kugadziriswa nekushandisa mushandi wekubatanidza kuenzanisa, izvo zvinobvumira kuchinjana kwekubatana pakati petambo dzevashandi.

Mhedziso

Envoy's threading modhi yakagadzirirwa kupa kureruka kwehurongwa uye hukuru hwekufananidza pamutengo wekuparadza ndangariro nekubatanidza kana isina kugadzirwa nemazvo. Iyi modhi inoibvumira kuti iite zvakanaka kwazvo pahupamhi hweshinda uye kubuda.
Sezvandakataura muchidimbu pa Twitter, dhizaini yacho inogonawo kumhanya pamusoro peiyo yakazara-mushandisi-modhi networking stack seDPDK (Data Plane Development Kit), izvo zvinogona kuguma neakajairwa maseva anobata mamirioni ezvikumbiro pasekondi yakazara L7 kugadzirisa. Zvichave zvinonakidza kuona izvo zvinovakwa mumakore mashoma anotevera.
Imwe yekupedzisira inokurumidza kutaura: Ndakabvunzwa kakawanda kuti sei takasarudza C ++ yeEnvoy. Chikonzero chinoramba chiri chekuti ndiyo chete ichiri kushandiswa zvakanyanya giredhi giredhi mutauro umo mavakirwo anotsanangurwa mune ino positi anogona kuvakwa. C ++ haina kufanira kune ese kana akawanda mapurojekiti, asi kune mamwe mashandisirwo enyaya ndiyo chete chishandiso chekuita kuti basa riitwe.

Links kune kodhi

Zvinongedzo kune mafaera ane mainterfaces uye emusoro mashandisirwo anokurukurwa mune ino positi:

Source: www.habr.com

Voeg