Kumasulira kwa nkhaniyi: Mtundu wa ulusi wa nthumwi - https://blog.envoyproxy.io/envoy-threading-model-a8d44b922310

Nkhaniyi idandisangalatsa kwambiri, ndipo popeza Envoy imagwiritsidwa ntchito nthawi zambiri ngati gawo la "istio" kapena kungoti "ingress controller" ya kubernetes, anthu ambiri samalumikizana nawo mwachindunji monga, mwachitsanzo, Kuyika kwa Nginx kapena Haproxy. Komabe, ngati chinachake chisweka, zingakhale bwino kumvetsetsa momwe zimagwirira ntchito kuchokera mkati. Ndinayesera kumasulira zambiri za malemba m'Chirasha monga momwe ndingathere, kuphatikizapo mawu apadera; kwa iwo omwe amawona kuti ndizowawa kuyang'ana izi, ndinasiya zoyambazo m'makolo. Takulandilani kumphaka.

Zolemba zotsika zaukadaulo za Envoy codebase pakadali pano ndizochepa. Kuti ndithetse izi, ndikukonzekera kupanga zolemba zingapo zamabulogu zamagulu osiyanasiyana a Envoy. Popeza iyi ndi nkhani yoyamba, chonde ndidziwitseni zomwe mukuganiza komanso zomwe mungasangalale nazo m'nkhani zamtsogolo.

Limodzi mwamafunso aukadaulo omwe ndimalandira okhudza Envoy ndikufunsa kufotokozera kwapang'onopang'ono kwa mtundu wa ulusi womwe umagwiritsa ntchito. Mu positi iyi, ndifotokoza momwe Evoy imalumikizirana ndi ulusi, komanso Thread Local Storage system yomwe imagwiritsa ntchito mkati kuti ipangitse kachidindo kufanana kwambiri komanso kuchita bwino kwambiri.

Kufotokozera mwachidule

Othandizira amagwiritsa ntchito mitundu itatu yosiyanasiyana ya mitsinje:

Chachikulu: Ulusi uwu umayang'anira kuyambika ndi kutha, kukonza zonse za XDS (xDiscovery Service) API, kuphatikiza DNS, kuyang'anira thanzi, gulu lonse ndi kasamalidwe ka nthawi, kukonzanso ziwerengero, kuyang'anira ndi kasamalidwe kazinthu zonse - zizindikiro za Linux. kuyambitsanso kotentha, ndi zina. Zomwe zimachitika mu ulusi uwu ndizosasinthika komanso "zopanda kutsekereza". Kawirikawiri, ulusi waukulu umagwirizanitsa machitidwe onse ovuta omwe safuna kuchuluka kwa CPU kuti ayendetse. Izi zimalola ma code ambiri owongolera kuti alembedwe ngati kuti ali ndi ulusi umodzi.
Wantchito: Mwachikhazikitso, Envoy imapanga ulusi wogwira ntchito pa ulusi uliwonse wa hardware mu dongosolo, izi zikhoza kuwongoleredwa pogwiritsa ntchito njirayo --concurrency. Ulusi uliwonse wogwira ntchito umakhala ndi "chosatsekereza" chochitika, chomwe chimakhala ndi udindo womvera womvera aliyense; panthawi yolemba (Julayi 29, 2017) palibe kuphwanya kwa omvera, kuvomereza kulumikizidwa kwatsopano, kulimbikitsa zosefera. kulumikiza, ndi kukonza ntchito zonse zolowetsa/zotulutsa (IO) panthawi yonse ya kulumikizidwa. Apanso, izi zimalola ma code ambiri olumikizirana kuti alembedwe ngati kuti ndi ulusi umodzi.
Fayilo Flusher: Fayilo iliyonse yomwe Envoy imalemba, makamaka zolowa zolowa, pakadali pano ili ndi ulusi wodziletsa wodziletsa. Izi ndichifukwa choti kulembera mafayilo osungidwa ndi fayilo ngakhale mukugwiritsa ntchito O_NONBLOCK nthawi zina amatha kutsekeka (kuusa moyo). Pamene ulusi wa antchito ukufunika kulembera ku fayilo, deta imasunthidwa ku buffer mu kukumbukira komwe pamapeto pake imatulutsidwa mu ulusi. kutsitsa fayilo. Ili ndi gawo limodzi la code komwe mwaukadaulo ulusi wonse wogwira ntchito ukhoza kutsekereza loko poyesa kudzaza chosungira kukumbukira.

Kusamalira kugwirizana

Monga tafotokozera mwachidule pamwambapa, ulusi wonse wogwira ntchito umamvera omvera onse popanda kugawana. Chifukwa chake, kernel imagwiritsidwa ntchito kutumiza mwachisomo zitsulo zovomerezeka ku ulusi wa antchito. Maso amakono nthawi zambiri amakhala abwino kwambiri pa izi, amagwiritsa ntchito zinthu monga input/output (IO) kulimbikitsa patsogolo kuyesa kudzaza ulusi ndi ntchito asanayambe kugwiritsa ntchito ulusi wina womwe umamvetseranso pa socket yomweyo, komanso osagwiritsa ntchito robin yozungulira. kutseka (Spinlock) kuti mugwiritse ntchito pempho lililonse.
Kulumikizana kukavomerezedwa pa ulusi wogwira ntchito, sikuchoka ulusi umenewo. Kukonzekera kwina konse kwa kulumikizana kumayendetsedwa kwathunthu mu ulusi wa ogwira ntchito, kuphatikiza machitidwe aliwonse otumizira.

Izi zili ndi zotsatila zingapo zofunika:

Maiwe olumikizana onse mu Envoy amaperekedwa kwa ulusi wogwira ntchito. Chifukwa chake, ngakhale maiwe olumikizirana a HTTP/2 amangolumikizana ndi munthu aliyense wakumtunda panthawi imodzi, ngati pali ulusi wa antchito anayi, padzakhala maulumikizidwe anayi a HTTP/2 pa wolandila wakumtunda mokhazikika.
Chifukwa chomwe Mtumiki amagwirira ntchito motere ndikuti posunga chilichonse pa ulusi umodzi wogwira ntchito, pafupifupi ma code onse amatha kulembedwa popanda kutsekereza komanso ngati kuti ndi ulusi umodzi. Kupanga uku kumapangitsa kuti zikhale zosavuta kulemba ma code ambiri ndi masikelo modabwitsa mpaka pafupifupi ulusi wopanda malire wa ulusi wa antchito.
Komabe, chimodzi mwazofunikira kwambiri ndichakuti kuchokera pakukumbukira komanso kulumikizidwa bwino, ndikofunikira kwambiri kukhazikitsa --concurrency. Kukhala ndi ulusi wochuluka wa antchito kuposa momwe kumafunikira kumawononga kukumbukira, kupanga zolumikizira zopanda ntchito, ndikuchepetsa kuchuluka kwa kulumikizana. Ku Lyft, zotengera zathu zam'mbali mwa nthumwi zimayenda ndi ndalama zochepa kwambiri kotero kuti magwiridwe antchito amafanana ndi ntchito zomwe amakhala pafupi nazo. Timayendetsa Envoy ngati proxy ya m'mphepete pokhapokha ngati pali ndalama zambiri.

Kusatsekereza kumatanthauza chiyani?

Mawu oti "osatsekereza" akhala akugwiritsidwa ntchito kangapo mpaka pano pokambirana momwe ulusi waukulu ndi wogwira ntchito umagwirira ntchito. Ma code onse amalembedwa poganiza kuti palibe chomwe chatsekedwa. Komabe, izi sizowona kwathunthu (zomwe sizowona kwathunthu?).

Envoy amagwiritsa ntchito maloko angapo aatali:

Monga momwe tafotokozera, polemba zipika zolowera, ulusi wonse wa ogwira ntchito umakhala ndi loko yofananayo buffer yokumbukira kukumbukira isanadzazidwe. Nthawi yogwira loko iyenera kukhala yotsika kwambiri, koma ndizotheka kuti loko ikhale yotsutsana ndi ndalama zambiri komanso kutulutsa kwakukulu.
Mtumiki amagwiritsa ntchito dongosolo lovuta kwambiri kuthana ndi ziwerengero zomwe zili pafupi ndi ulusi. Uwu ukhala mutu wa positi yosiyana. Komabe, nditchula mwachidule kuti monga gawo la ziwerengero za ulusi kwanuko, nthawi zina zimakhala zofunikira kupeza loko pa "stats store" yapakati. Kutseka kumeneku sikuyenera kufunidwa.
Ulusi waukulu nthawi ndi nthawi umayenera kugwirizanitsa ndi ulusi wonse wa antchito. Izi zimachitika ndi "kusindikiza" kuchokera ku ulusi waukulu kupita ku ulusi wa antchito, ndipo nthawi zina kuchokera ku ulusi wa antchito kubwerera ku ulusi waukulu. Kutumiza kumafuna loko kuti uthenga womwe wasindikizidwa ukhoze kuikidwa pamzere kuti utumizidwe mtsogolo. Maloko awa sayenera kutsutsidwa kwambiri, komabe amatha kutsekedwa mwaukadaulo.
Pamene Mtumiki alemba chipika ku mtsinje wolakwika wa dongosolo (zolakwika zokhazikika), zimapeza loko pazochitika zonse. Nthawi zambiri, kudula mitengo kwanuko kwa Envoy kumaonedwa kuti ndi koyipa kwambiri, chifukwa chake sipanakhalepo chidwi chachikulu pakuwongolera.
Palinso maloko ena ochepa mwachisawawa, koma palibe omwe ali ofunikira kwambiri ndipo sayenera kutsutsidwa.

Ulusi yosungirako m'deralo

Chifukwa cha momwe Mtumiki amalekanitsira maudindo a ulusi waukulu kuchokera ku maudindo a ulusi wogwira ntchito, pali kufunikira kuti kukonza zovuta kuchitidwe pa ulusi waukulu ndiyeno kuperekedwa kwa ulusi aliyense wogwira ntchito mofanana kwambiri. Gawoli likufotokoza za Envoy Thread Local Storage (TLS) pamlingo wapamwamba. Mu gawo lotsatira ndifotokoza momwe amagwiritsidwira ntchito poyang'anira masango.

Monga tafotokozera kale, ulusi waukulu umagwira ntchito zonse zoyendetsera ndi kuyendetsa ndege mu ndondomeko ya Evoy. Ndege yoyang'anira imakhala yodzaza kwambiri pano, koma mukayang'ana mkati mwa Evoy process yokha ndikuiyerekeza ndi kutumiza komwe ulusi wa ogwira ntchito amachita, ndizomveka. Lamulo lalikulu ndiloti ulusi waukulu umagwira ntchito, ndiyeno umafunika kusintha ulusi uliwonse wa wogwira ntchito molingana ndi zotsatira za ntchitoyo. Pankhaniyi, ulusi wogwira ntchito sufunika kupeza loko panjira iliyonse.

Dongosolo la Envoy's TLS (Thread local storage) limagwira ntchito motere:

Khodi yomwe ikuyenda pa ulusi waukulu imatha kugawa malo a TLS panjira yonseyi. Ngakhale izi ndizosawerengeka, m'machitidwe ndi index mu vekitala, kupereka mwayi wa O(1).
Ulusi waukulu ukhoza kukhazikitsa deta yosasinthika mu slot yake. Izi zikachitika, deta imasindikizidwa ku ulusi uliwonse wa ogwira ntchito ngati chochitika chokhazikika.
Ulusi wa ogwira ntchito amatha kuwerenga kuchokera ku TLS yawo ndikupezanso ulusi uliwonse wapamalo omwe ulipo.

Ngakhale ndi njira yosavuta komanso yamphamvu kwambiri, ndiyofanana kwambiri ndi lingaliro la kutsekereza kwa RCU(Read-Copy-Update). Kwenikweni, ulusi wa ogwira ntchito sawona kusintha kulikonse mu malo a TLS pomwe ntchito ikugwira ntchito. Kusintha kumachitika kokha panthawi yopuma pakati pa zochitika za ntchito.

Othandizira amagwiritsa ntchito izi m'njira ziwiri:

Mwa kusunga deta yosiyana pa ulusi uliwonse wogwira ntchito, deta imatha kupezeka popanda kutsekereza.
Pokhala ndi cholozera chogawana ku data yapadziko lonse lapansi mumayendedwe owerengera okha pa ulusi uliwonse wa ogwira ntchito. Choncho, ulusi uliwonse wogwira ntchito uli ndi chiwerengero cha deta chomwe sichikhoza kuchepetsedwa pamene ntchito ikugwira ntchito. Pokhapokha antchito onse akadekha ndikuyika zatsopano zomwe adagawana ndizomwe zidachitika kale. Izi ndizofanana ndi RCU.

Kusintha kwa Cluster

M'chigawo chino, ndikufotokozera momwe TLS (Thread local storage) imagwiritsidwira ntchito poyang'anira gulu. Kuwongolera magulu kumaphatikizapo xDS API ndi/kapena kukonza kwa DNS, komanso kuyang'anira thanzi.

Kasamalidwe ka Cluster flow kamakhala ndi zigawo ndi masitepe awa:

Cluster Manager ndi gawo lomwe lili mkati mwa Envoy lomwe limayang'anira magulu onse odziwika bwino, Cluster Discovery Service (CDS) API, Secret Discovery Service (SDS) ndi Endpoint Discovery Service (EDS) APIs, DNS, ndi macheke akunja akunja. Ili ndi udindo wopanga mawonedwe "osasinthika" a gulu lililonse lakumtunda, lomwe limaphatikizapo omwe apezeka komanso thanzi.
Woyang'anira zaumoyo amawunika zaumoyo ndikuwonetsa kusintha kwaumoyo kwa oyang'anira magulu.
CDS (Cluster Discovery Service) / SDS (Secret Discovery Service) / EDS (Endpoint Discovery Service) / DNS amachitidwa kuti adziwe umembala wamagulu. Kusintha kwa dziko kumabwezeretsedwa kwa woyang'anira gulu.
Ulusi uliwonse wa ogwira ntchito mosalekeza umagwiritsa ntchito loop ya zochitika.
Pamene cluster manager awona kuti dziko la cluster lasintha, limapanga chithunzithunzi chatsopano cha cluster's state ndikuchitumiza ku ulusi uliwonse wa ogwira ntchito.
Munthawi yabata yotsatira, ulusi wa ogwira ntchito udzasintha chithunzithunzi cha malo operekedwa a TLS.
Pamwambo wa I/O womwe ukuyenera kutsimikizira wolandirayo kuti akweze bwino, woyezera katunduyo adzapempha malo a TLS (Thread local storage) kuti adziwe zambiri za wolandirayo. Izi sizifuna maloko. Zindikiraninso kuti TLS imathanso kuyambitsa zochitika zosintha kuti ma balancers ndi zinthu zina athe kuwerengeranso ma cache, ma data, ndi zina. Izi ndizoposa kukula kwa positiyi, koma zimagwiritsidwa ntchito m'malo osiyanasiyana mu code.

Pogwiritsa ntchito njira yomwe ili pamwambapa, Envoy imatha kukonza pempho lililonse popanda kutsekereza (kupatula momwe tafotokozera kale). Kupatula pazovuta za code ya TLS yokha, ma code ambiri safunikira kumvetsetsa momwe multithreading imagwirira ntchito ndipo imatha kulembedwa ndi ulusi umodzi. Izi zimapangitsa ma code ambiri kukhala osavuta kulemba kuwonjezera pakuchita bwino kwambiri.

Ma subsystems ena omwe amagwiritsa ntchito TLS

TLS (Thread local storage) ndi RCU (Read Copy Update) amagwiritsidwa ntchito kwambiri mu Envoy.

Zitsanzo zamagwiritsidwe:

Njira yosinthira magwiridwe antchito panthawi yakupha: Mndandanda wapano wa magwiridwe antchito amawerengedwa mu ulusi waukulu. Ulusi uliwonse wogwira ntchito umapatsidwa chithunzi chowerengera chokha pogwiritsa ntchito semantics ya RCU.
Kusintha matebulo anjira: Kwa matebulo apanjira operekedwa ndi RDS (Route Discovery Service), matebulo anjira amapangidwa pa ulusi waukulu. Chithunzi chowerengera chokha chidzaperekedwa kwa aliyense wogwira ntchito pogwiritsa ntchito semantics ya RCU (Read Copy Update). Izi zimapangitsa kusintha matebulo anjira kukhala kothandiza kwambiri.
Kusungitsa mutu wa HTTP: Zotsatira zake, kuwerengera mutu wa HTTP pa pempho lililonse (pothamanga ~ 25K + RPS pachimake) ndikokwera mtengo kwambiri. Evoy imawerengera mutuwo pafupifupi theka la sekondi iliyonse ndikuupereka kwa wogwira ntchito aliyense kudzera pa TLS ndi RCU.

Palinso milandu ina, koma zitsanzo zam'mbuyomu ziyenera kupereka chidziwitso chabwino cha zomwe TLS imagwiritsidwa ntchito.

Zodziwika zovuta zamasewera

Ngakhale kuti Envoy imagwira bwino ntchito yonse, pali madera ochepa omwe amafunikira chidwi akagwiritsidwa ntchito ndi ndalama zambiri komanso zotuluka:

Monga tafotokozera m'nkhaniyi, pakali pano ulusi wonse wogwira ntchito umakhala ndi loko polemba ku buffer yofikira. Pokhala ndi ndalama zambiri komanso kutulutsa kwakukulu, mudzafunika kusonkhanitsa zipika zofikira pa ulusi uliwonse wa wogwira ntchitoyo ndikuwononga ndalama zomwe simunatumize polemba fayilo yomaliza. Kapenanso, mutha kupanga chipika cholowera chosiyana cha ulusi uliwonse wa ogwira ntchito.
Ngakhale ziwerengerozo ndizokongoletsedwa kwambiri, pakugwirizana kwakukulu komanso kupitilira kwake padzakhala mikangano ya atomiki pa ziwerengero zapayekha. Yankho lavutoli ndi zowerengera pa ulusi uliwonse wa ogwira ntchito omwe amakhazikitsanso zowerengera zapakati nthawi ndi nthawi. Izi zidzakambidwa mu post yotsatira.
Zomangamanga zamakono sizingagwire ntchito bwino ngati Mtumiki atumizidwa muzochitika zomwe pali malumikizano ochepa omwe amafunikira zofunikira zothandizira. Palibe chitsimikizo chakuti malumikizano adzagawidwa mofanana pakati pa ulusi wa ogwira ntchito. Izi zitha kuthetsedwa ndikugwiritsa ntchito kusanja kulumikizana kwa ogwira ntchito, zomwe zingathandize kusinthana kwa kulumikizana pakati pa ulusi wogwira ntchito.

Mapeto

Mtundu wa ulusi wa Envoy udapangidwa kuti ukhale wosavuta pamapulogalamu komanso kufananiza kwakukulu ndikuwononga kukumbukira komwe kungawononge komanso kulumikizana ngati sikunakonzedwe bwino. Chitsanzochi chimapangitsa kuti chizigwira ntchito bwino kwambiri paziwerengero zokwera kwambiri komanso zotuluka.
Monga ndanenera mwachidule pa Twitter, mapangidwewo amathanso kuthamanga pamwamba pa malo ochezera a pa intaneti monga DPDK (Data Plane Development Kit), zomwe zingapangitse ma seva ochiritsira akugwira zopempha mamiliyoni pa sekondi imodzi ndi kukonza kwathunthu kwa L7. Zidzakhala zosangalatsa kwambiri kuwona zomwe zidzamangidwe m'zaka zingapo zikubwerazi.
Ndemanga imodzi yomaliza yomaliza: Ndafunsidwa nthawi zambiri chifukwa chomwe tidasankha C++ kwa Envoy. Chifukwa chake ndi chakuti akadali chilankhulo chokhacho chomwe chimagwiritsidwa ntchito kwambiri m'mafakitale momwe zomanga zomwe zafotokozedwa patsamba lino zitha kumangidwa. C ++ siyoyenera kuma projekiti onse kapena ambiri, koma pamagwiritsidwe ena akadali chida chokhacho kuti ntchitoyi ichitike.

Links kuti kodi

Maulalo kumafayilo omwe ali ndi zolumikizirana ndi kuyika pamutu zomwe zafotokozedwa mu positi iyi:

Source: www.habr.com

[Kumasulira] Mtundu wolumikizira nthumwi