Cube-on-cube, metaclusters, mizinga, kugovera zviwanikwa

Mupunga. 1. Kubernetes ecosystem paAlibaba Cloud

Kubva 2015, Alibaba Cloud Container Service yeKubernetes (ACK) yanga iri imwe yeanokurumidza kukura masevhisi emakore muAlibaba Cloud. Iyo inoshandira vatengi vakawanda uye zvakare inotsigira Alibaba yemukati masisitimu uye mamwe ekambani masevhisi emakore.

Sezvineiwo masevhisi emidziyo akafanana kubva kupasi-kirasi gore vanopa, zvatinokoshesa zvakanyanya kuvimbika uye kuwanikwa. Naizvozvo, chikuva chinotyisa uye chinowanikwa pasi rose chakagadzirirwa makumi ezviuru emasumbu eKubernetes.

Muchikamu chino, tichagovera ruzivo rwedu rwekugadzirisa nhamba huru yeKubernetes masumbu pa cloud infrastructure, pamwe chete nekuvakwa kwepuratifomu.

kupinda

Kubernetes yave iyo de facto chiyero cheakasiyana emabasa ari mugore. Sezvinoratidzwa muFig. 1 pamusoro, kuwedzera uye kuwanda kweAlibaba Cloud zvikumbiro zvave kushanda paKubernetes zvikwata: zvikumbiro uye zvisingaverengeki zvikumbiro, pamwe nemamaneja ekushandisa. Kubernetes manejimendi yagara iri inonakidza uye yakakomba musoro wehurukuro kune mainjiniya anovaka nekuchengetedza zvivakwa. Kana zvasvika kune vanopa makore seAlibaba Cloud, nyaya yekuyera inouya pamberi. Maitiro ekugadzirisa Kubernetes masumbu pachiyero ichi? Isu takatovhara maitiro akanakisa ekutonga 10-node Kubernetes masumbu makuru. Zvechokwadi, iyi idambudziko rinonakidza rekuyera. Asi pane chimwe chikero: uwandu masumbu acho pachawo.

Takakurukura nyaya iyi nevakawanda vashandisi veACK. Vazhinji vavo vanosarudza kumhanya akawanda, kana asiri mazana, emadiki kana epakati-saizi Kubernetes masumbu. Pane zvikonzero zvakanaka zveizvi: kudzikisira kukuvara kunogona kuitika, kupatsanura masumbu ezvikwata zvakasiyana, kugadzira masumbu chaiwo ekuedzwa. Kana ACK ichivavarira kushandira vateereri vepasirese neiyi modhi yekushandisa, inofanirwa kuvimbika uye nemazvo kutonga nhamba huru yemasumbu munzvimbo dzinopfuura makumi maviri.

Mupunga. 2. Matambudziko ekugadzirisa nhamba huru yeKubernetes masumbu

Ndeapi matambudziko makuru ekutarisira masumbu pachikero ichi? Sezvinoratidzwa mumufananidzo, pane zvinhu zvina zvekugadzirisa:

• Heterogeneity

ACK inofanirwa kutsigira akasiyana marudzi emasumbu, anosanganisira akajairwa, serverless, Edge, Windows, uye mamwe akati wandei. Masumbu akasiyana anoda akasiyana sarudzo, zvikamu, uye mhando dzekutambira. Vamwe vatengi vanoda rubatsiro nekugadzirisa kune avo chaiwo nyaya.

• Saizi dzakasiyana dzemasumbu

Masumbu anosiyana muhukuru: kubva pamatinji ane mapodhi akati wandei kusvika kumakumi ezviuru zvemanodhi ane zviuru zvepodhi. Resource zvinodiwa zvakare zvinosiyana zvakanyanya. Kugovewa kwezviwanikwa zvisina kunaka kunogona kukanganisa kuita kana kukonzeresa kutadza.

• Shanduro dzakasiyana

Kubernetes iri kubuda nekukurumidza. Shanduro itsva dzinoburitswa mwedzi mishoma yega yega. Vatengi vanogara vachida kuedza zvitsva. Saka ivo vanoda kuisa bvunzo mutoro pane itsva shanduro Kubernetes uye kugadzirwa mutoro pane dzakagadzikana. Kuti isangane nechinodiwa ichi, ACK inofanirwa kuramba ichiendesa shanduro itsva dzeKubernetes kune vatengi uku ichichengetedza shanduro dzakagadzikana.

• Chengetedzo Kutevedzera

Masumbu anoparadzirwa munzvimbo dzakasiyana. Saka nekudaro, ivo vanofanirwa kutevedzera zvakasiyana-siyana zvekuchengetedza zvinodiwa uye mirairo yepamutemo. Semuenzaniso, sumbu muEurope rinofanira kuenderana neGDPR, nepo gore rezvemari muChina rinofanira kunge riine mamwe matinji edziviriro. Izvi zvinodikanwa zvinosungirwa uye hazvitenderwe kuzvifuratira, sezvo izvi zvichigadzira njodzi huru kune vatengi vepapuratifomu yegore.

Iyo ACK chikuva chakagadzirirwa kugadzirisa mazhinji ematambudziko ari pamusoro. Parizvino yakavimbika uye yakadzikama inobata anopfuura zviuru gumi Kubernetes masumbu kutenderera pasirese. Ngatitarisei kuti izvi zvakawanikwa sei, kusanganisira kuburikidza akati wandei makiyi dhizaini / misimboti yekuvaka.

patani

Cube-on-cube uye muzinga weuchi

Kusiyana nehumambo hwepakati, sero-yakavakirwa architecture inowanzo shandiswa kuyera chikuva kupfuura imwe chete data data kana kuwedzera chiyero chekudzoreredza njodzi.

Dunhu rega rega riri muAlibaba Cloud rine akati wandei (AZ) uye rinowanzoenderana nenzvimbo yakatarwa yedata. Mudunhu rakakura (semuenzaniso, Huangzhou), kazhinji kune zviuru zveKubernetes zvikwata zvevatengi zvinomhanya ACK.

ACK inobata aya maKubernetes masumbu achishandisa Kubernetes pachayo, zvichireva kuti isu tine Kubernetes metacluster inomhanya kubata mutengi Kubernetes masumbu. Ichi chivakwa chinonzi zvakare "kube-on-kube" (KoK). Iyo KoK dhizaini inorerutsa manejimendi emaklasta evatengi nekuti kuendesa masumbu ari nyore uye anotemerwa. Zvinotonyanya kukosha, tinogona kushandisa zvekare maficha eKubernetes. Semuenzaniso, kutonga maseva eAPI kuburikidza nekutumira, uchishandisa iyo etcd opareta kubata akawanda etcds. Kudzokorora kwakadaro nguva dzose kunounza mufaro unokosha.

Anoverengeka Kubernetes metaclusters anoiswa mukati medunhu rimwe, zvichienderana nehuwandu hwevatengi. Tinodaidza aya maseru maseru. Kuti udzivirire pakutadza kwenzvimbo yese, ACK inotsigira akawanda-anoshanda deployments munharaunda imwe chete: iyo metacluster inogovera Kubernetes mutengi cluster master zvikamu munzvimbo dzakawanda uye inomhanya iwo panguva imwe chete, ndiko kuti, mune yakawanda-inoshanda mode. Kuti ive nechokwadi chekuvimbika uye kushanda zvakanaka kwatenzi, ACK inogonesa kuiswa kwezvikamu uye inova nechokwadi chekuti API server uye etcd zviri padyo nemumwe.

Iyi modhi inobvumidza iwe kubata Kubernetes zvakanaka, zviri nyore uye zvakavimbika.

Metacluster zviwanikwa kuronga

Sezvatakambotaura, huwandu hwemetaclusters mudunhu rega rega zvinoenderana nehuwandu hwevatengi. Asi panguva ipi yekuwedzera metacluster itsva? Iri ndiro dambudziko rekugadzirisa zviwanikwa. Sezvo mutemo, itsika yekugadzira imwe itsva apo metacluster iripo yakapedza zviwanikwa zvavo zvose.

Ngatitorei network zviwanikwa, semuenzaniso. Mukuvaka kweKoK, Kubernetes zvikamu kubva kumapoka evatengi zvinoiswa semapodhi mune metacluster. Isu tinoshandisa Terway (Fig. 3) ndeyepamusoro-inoshanda plugin yakagadzirwa neAlibaba Cloud yekutakura network network. Iyo inopa yakapfuma seti yekuchengetedza uye inobvumidza iwe kuti ubatanidze kune vatengi' virtual private makore (VPCs) kuburikidza neAlibaba Cloud Elastic Networking Interface (ENI). Kugovera zvinobudirira zviwanikwa zvenetiweki munzvimbo dzese, mapodhi uye masevhisi mune metacluster, isu tinofanirwa kunyatso tarisa mashandisiro azvo mukati memetacluster yemakore akavanzika. Kana zviwanikwa zvetiweki zvasvika pakupera, sero idzva rinogadzirwa.

Kuti titarise huwandu hwakakwana hwemasumbu evatengi mune imwe neimwe metacluster, isu tinofungawo mutengo wedu, density zvinodiwa, resource quota, kuvimbika zvinodiwa uye manhamba. Sarudzo yekugadzira metacluster itsva inoitwa zvichienderana neruzivo urwu rwese. Ndokumbira utarise kuti masumbu madiki anogona kuwedzera zvakanyanya mune ramangwana, saka kushandiswa kwezviwanikwa kunowedzera kunyangwe huwandu hwemasumbu hukaramba husina kuchinjika. Tinowanzosiya nzvimbo yakakwana yemahara kuti sumbu rega rega rikure.

Mupunga. 3. Terway network architecture

Kuyera zvikamu zvewizard pane ese macustomer masumbu

Wizard zvikamu zvine zvakasiyana resource zvinodiwa. Vanovimba nehuwandu hwemanodhi uye pods musumbu, nhamba yevasiri-standard controllers/operators vanobatana ne APIServer.

MuACK, imwe neimwe Kubernetes mutengi cluster inosiyana muhukuru uye nguva yekumhanya zvinodiwa. Iko hakuna gadziriso yepasirese yekuisa zvikamu zvewizard. Kana isu tikanganisa kuisa yakaderera resource muganho kune mutengi akakura, ipapo cluster yayo haizokwanisa kubata nemutoro. Kana iwe ukaisa muganho wekuchengetedza wakakwira kune ese masumbu, zviwanikwa zvichaparara.

Kuti uwane kutengeserana kusinganzwisisike pakati pekuvimbika uye mutengo, ACK inoshandisa rudzi system. Zvichireva, tinotsanangura marudzi matatu emasumbu: madiki, epakati uye makuru. Mhando yega yega ine yakaparadzana sosi yekugovera mbiri. Rudzi rwakatemerwa zvichienderana nekuremerwa kwezvikamu zvewizard, nhamba yemanodhi, uye zvimwe zvinhu. Iyo cluster type inogona kuchinja nekufamba kwenguva. ACK inoramba ichitarisa zvinhu izvi uye inogona kukwira/pasi kunyora saizvozvo. Kana rudzi rwemasumbu rwashandurwa, kugoverwa kwezviwanikwa kunovandudzwa otomatiki nekupindira kushoma kwemushandisi.

Tiri kushanda kuti tivandudze sisitimu iyi nekuyera-grained scaling uye kunyatso gadziridza mhando kuitira kuti shanduko idzi dziitike zviri nani uye kuti ive nehungwaru hwehupfumi.

Mupunga. 4. Intelligent multi-stage type switching

Evolution yevatengi masumbu pachiyero

Zvikamu zvakapfuura zvakafukidza zvimwe zvinhu zvekutonga nhamba huru dzeKubernetes masumbu. Zvisinei, pane rimwe dambudziko rinoda kugadziriswa: kushanduka kwemasumbu.

Kubernetes ndiyo "Linux" yenyika yegore. Inoramba ichivandudzwa uye inowedzera modular. Isu tinofanirwa kugara tichiendesa mavhezheni matsva kune vatengi vedu, kugadzirisa kusagadzikana uye kugadzirisa masumbu aripo, pamwe nekutonga huwandu hukuru hwezvinhu zvine hukama (CSI, CNI, Chishandiso Plugin, Scheduler Plugin uye zvimwe zvakawanda).

Ngatitore Kubernetes chikamu manejimendi semuenzaniso. Kutanga, takagadzira yepakati system yekunyoresa uye kutonga zvese izvi zvakabatana zvikamu.

Mupunga. 5. Flexible uye pluggable zvikamu

Usati waenda kumberi, unofanirwa kuve nechokwadi kuti iyo update yakabudirira. Kuti tiite izvi, takagadzira hurongwa hwekutarisa kushanda kwezvikamu. Cheki inoitwa pamberi uye mushure mekuvandudza.

Mupunga. 6. Preliminary check of cluster components

Kukurumidza uye nekuvimbika kugadzirisa izvi zvikamu, inoenderera yekuendesa sisitimu inoshanda nerutsigiro rwekufambira mberi zvishoma (greyscale), kumbomira uye mamwe mabasa. Standard Kubernetes controllers haina kunyatsokodzera iyi kesi yekushandisa. Naizvozvo, kubata zvikamu zvemasumbu, isu takagadzira seti yeakasarudzika ma controller, anosanganisira plugin uye yekubatsira control module (sidecar management).

Semuenzaniso, BroadcastJob controller yakagadzirirwa kugadzirisa zvikamu pamushini wega wega wevashandi kana kutarisa node pamushini wega wega. Basa reNhepfenyuro rinomhanyisa podhi pane imwe neimwe node musumbu, seDaemonSet. Nekudaro, DaemonSet inogara ichichengeta iyo pod ichimhanya kwenguva yakareba, nepo BroadcastJob ichiiputsa. Iyo Broadcast controller inotangisawo mapodhi pane achangobva kujoinwa node uye inotanga node nezvinodiwa zvinhu. Muna Chikumi 2019, takavhura kodhi kodhi yeOpenKruise otomatiki injini, iyo isu pachedu tinoshandisa mukati mekambani.

Mupunga. 7. OpenKurise inoronga kuitwa kwebasa reBroadcast pamanodhi ese

Kubatsira vatengi kusarudza iyo chaiyo cluster masisitimu, isu tinopawo seti yeakafanotsanangurwa profiles, anosanganisira Serverless, Edge, Windows, uye Bare Metal profiles. Sezvo nzvimbo yacho ichiwedzera uye zvinodiwa nevatengi vedu zvichikura, isu tichawedzera mamwe maprofile kuti kurerutsa iyo inonetesa yekuseta maitiro.

Mupunga. 8. Yepamberi uye inochinjika cluster profiles kune akasiyana siyana

Kuonekwa kwepasi rose munzvimbo dzese dzedata

Sezvinoratidzwa pazasi fig. 9, Alibaba Cloud Container Cloud sevhisi yakaiswa mumatunhu makumi maviri kutenderera pasirese. Zvichipa chiyero ichi, chimwe chezvinangwa zvakakosha zve ACK ndeyekutarisa zviri nyore mamiriro ekumhanya masumbu kuitira kuti kana mutengi sumbu asangana nedambudziko, tinogona kukurumidza kupindura mamiriro acho. Mune mamwe mazwi, iwe unofanirwa kuuya nemhinduro inozokubvumidza kuti utore zvine hunyanzvi uye zvakachengeteka zviverengero munguva chaiyo kubva kumapoka evatengi mumatunhu ese - uye nemaziso kuratidza mhedzisiro.

Mupunga. 9. Global deployment yeAlibaba Cloud Container service mumatunhu makumi maviri

Kufanana neakawanda Kubernetes ekutarisa masisitimu, isu tinoshandisa Prometheus sechishandiso chedu chikuru. Kune imwe neimwe metacluster, Prometheus vamiririri vanotora anotevera metrics:

• OS metrics senge zviwanikwa zvekugamuchira (CPU, ndangariro, dhisiki, nezvimwewo) uye network bandwidth.
• Metrics yemetacluster uye mutengi cluster manejimendi system, senge kube-apiserver, kube-controller-maneja uye kube-scheduler.
• Metrics kubva kubernetes-state-metrics uye cadvisor.
• etcd metrics akadai sedhisiki yekunyora nguva, saizi yedhatabhesi, kubuda kwezvinongedzo pakati pemanodhi, nezvimwe.

Nhamba dzepasi rose dzinounganidzwa uchishandisa yakajairwa akawanda-layer aggregation modhi. Yekutarisisa data kubva kune yega metacluster inotanga kuunganidzwa munharaunda yega yega uye yozotumirwa kune yepakati server inoratidza iyo yakazara mufananidzo. Zvese zvinoshanda kuburikidza nemubatanidzwa meshini. Iyo Prometheus sevha mune yega yega data data inounganidza metrics kubva iyo data data, uye yepakati Prometheus server ine basa rekuunganidza data rekutarisa. AlertManager inobatanidza nechepakati Prometheus uye inotumira chenjedzo sezvinodiwa kuburikidza neDingTalk, email, SMS, nezvimwe. Visualization - Kushandisa Grafana.

Mumufananidzo 10, iyo yekutarisa system inogona kukamurwa kuita nhanho nhatu:

• muganhu wenhanho

Iyo layer iri kure nepakati. Iyo Prometheus Edge Server inomhanya mune imwe neimwe metacluster, ichiunganidza meta kubva meta uye mutengi masumbu mukati meiyo imwechete network domain.

• Cascade level

Basa rePrometheus cascade layer ndere kuunganidza data rekutarisa kubva kumatunhu akawanda. Aya maseva anoshanda pamwero wezvikamu zvakakura zvenzvimbo seChina, Asia, Europe neAmerica. Sezvo masumbu achikura, dunhu rinogona kukamurwa, uye ipapo cascade-level Prometheus server ichaonekwa munzvimbo imwe neimwe yakakura. Neichi zano, unogona kuyera mushe sezvinodiwa.

• Central level

Iyo yepakati Prometheus server inobatanidza kune ese cascade maseva uye inoita yekupedzisira data aggregation. Nekuvimbika, maviri epakati Prometheus zviitiko akasimudzwa munzvimbo dzakasiyana, akabatana kune imwechete cascade maseva.

Mupunga. 10. Global multi-level monitoring architecture inobva paPrometheus federation mechanism

Summary

Kubernetes-yakavakirwa gore mhinduro dzinoramba dzichichinja indasitiri yedu. Alibaba Cloud mudziyo sevhisi inopa yakachengeteka, yakavimbika uye yepamusoro-inoshanda yekutambira - ndiyo imwe yeakanakisa Kubernetes cloud hosting. Iyo Alibaba Cloud timu inotenda zvakasimba mumisimboti yeOpen Source uye yakavhurika sosi nharaunda. Isu zvirokwazvo ticharamba tichigovera ruzivo rwedu mumunda wekushanda nekugadzirisa makore matekinoroji.

Source: www.habr.com