Cube-on-cube, metaclusters, saƙar zuma, rarraba albarkatu

Shinkafa 1. Kubernetes muhalli akan Alibaba Cloud

Tun daga 2015, Alibaba Cloud Container Service don Kubernetes (ACK) ya kasance ɗayan sabis na girgije mafi girma a cikin Alibaba Cloud. Yana hidima ga abokan ciniki da yawa kuma yana tallafawa abubuwan more rayuwa na cikin gida na Alibaba da sauran sabis na girgije na kamfanin.

Kamar yadda yake tare da sabis na kwantena iri ɗaya daga masu samar da girgije na duniya, manyan abubuwan da muka fi ba da fifiko sune dogaro da samuwa. Saboda haka, an ƙirƙiri wani dandamali mai daidaitawa kuma mai isa ga duniya don dubun-dubatar gungu na Kubernetes.

A cikin wannan labarin, za mu raba kwarewarmu na sarrafa babban adadin Kubernetes gungu akan kayan aikin girgije, da kuma gine-ginen dandamali na tushe.

Gabatarwa

Kubernetes ya zama ma'auni na gaskiya don nau'ikan ayyukan aiki a cikin gajimare. Kamar yadda aka nuna a cikin Fig. 1 a sama, ƙarin aikace-aikacen Alibaba Cloud yanzu suna gudana akan gungu na Kubernetes: aikace-aikacen jihohi da marasa jiha, da kuma manajan aikace-aikacen. Gudanarwar Kubernetes koyaushe ya kasance batu mai ban sha'awa da mahimmanci na tattaunawa ga injiniyoyi waɗanda ke ginawa da kiyaye ababen more rayuwa. Idan ya zo ga masu samar da girgije kamar Alibaba Cloud, batun ƙaddamarwa ya zo kan gaba. Yadda ake sarrafa gungu na Kubernetes a wannan sikelin? Mun riga mun rufe mafi kyawun ayyuka don sarrafa manyan gungun Kubernetes-node 10. Tabbas, wannan matsala ce mai ban sha'awa. Amma akwai wani sikelin: yawa gungu kansu.

Mun tattauna wannan batu tare da masu amfani da ACK da yawa. Yawancinsu sun zaɓi gudanar da daruruwa, idan ba ɗaruruwa ba, na gungu na Kubernetes kanana ko matsakaita. Akwai kyawawan dalilai na wannan: iyakance yuwuwar lalacewa, raba gungu don ƙungiyoyi daban-daban, ƙirƙirar gungu na gani don gwaji. Idan ACK yana nufin yin hidima ga masu sauraro na duniya tare da wannan samfurin amfani, dole ne ya dogara da inganci da sarrafa babban adadin gungu a cikin yankuna sama da 20.

Shinkafa 2. Matsalolin sarrafa babban adadin gungu na Kubernetes

Menene manyan kalubalen sarrafa gungu a wannan sikelin? Kamar yadda aka nuna a cikin adadi, akwai batutuwa guda huɗu da za a magance su:

• Bambance-bambance

ACK yakamata ya goyi bayan nau'ikan gungu daban-daban, gami da daidaitattun, maras sabar, Edge, Windows, da sauran su. Tari daban-daban suna buƙatar zaɓuɓɓuka daban-daban, abubuwan haɗin gwiwa, da samfuran baƙi. Wasu abokan ciniki suna buƙatar taimako tare da keɓancewa don takamaiman al'amuransu.

• Girman gungu daban-daban

Tari sun bambanta da girma, daga nau'i-nau'i biyu masu ƴan kwasfa zuwa dubun-dubatar nodes masu dubunnan kwasfa. Bukatun albarkatun kuma sun bambanta sosai. Rarraba albarkatun da ba daidai ba na iya yin tasiri ga aiki ko ma haifar da gazawa.

• Daban-daban iri

Kubernetes yana tasowa da sauri. Ana fitar da sabbin sigogi kowane ƴan watanni. Abokan ciniki koyaushe suna shirye don gwada sabbin abubuwa. Don haka suna so su sanya nauyin gwaji a kan sababbin nau'ikan Kubernetes da kuma samar da kayan aiki a kan barga. Don saduwa da wannan buƙatun, ACK dole ne ya ci gaba da isar da sabbin nau'ikan Kubernetes ga abokan ciniki yayin da suke kiyaye juzu'i.

• Yarda da Tsaro

Ana rarraba gungu a yankuna daban-daban. Don haka, dole ne su bi ka'idodin aminci daban-daban da ƙa'idodin hukuma. Misali, gungu a Turai dole ne ya zama mai yarda da GDPR, yayin da girgijen kuɗi a China dole ne ya sami ƙarin matakan kariya. Waɗannan buƙatun sun zama tilas kuma ba za a yarda da yin watsi da su ba, saboda wannan yana haifar da babbar haɗari ga abokan cinikin dandamalin girgije.

An tsara dandalin ACK don magance yawancin matsalolin da ke sama. A halin yanzu yana dogara kuma yana sarrafa gungu na Kubernetes sama da dubu 10 a duniya. Bari mu kalli yadda aka cimma hakan, gami da ta hanyar ƙira da ƙa'idodin gine-gine da yawa.

Zane

Cube-on-cube da zumar zuma

Ba kamar matsayi na tsakiya ba, tsarin gine-ginen tantanin halitta yawanci ana amfani da shi don auna dandali fiye da cibiyar bayanai guda ɗaya ko don faɗaɗa iyakar dawo da bala'i.

Kowane yanki a cikin Alibaba Cloud ya ƙunshi yankuna da yawa (AZ) kuma yawanci yana dacewa da takamaiman cibiyar bayanai. A cikin babban yanki (misali Huangzhou), galibi ana samun dubban gungun abokan cinikin Kubernetes da ke gudana ACK.

ACK tana sarrafa waɗannan gungu na Kubernetes ta amfani da Kubernetes kanta, ma'ana muna da kubernetes metacluster da ke gudana don sarrafa gungu na Kubernetes abokin ciniki. Ana kuma kiran wannan gine-ginen "kube-on-kube" (KoK). Tsarin gine-ginen KoK yana sauƙaƙe gudanar da ƙungiyoyin abokan ciniki saboda ƙaddamar da gungu yana da sauƙi kuma mai ƙididdigewa. Mafi mahimmanci, za mu iya sake amfani da fasalin Kubernetes na asali. Misali, sarrafa sabar API ta hanyar turawa, ta amfani da mai aiki da sauransu don sarrafa mahara da sauransu. Irin wannan maimaitawa koyaushe yana kawo jin daɗi na musamman.

Yawancin kubernetes metaclusters ana tura su a cikin yanki ɗaya, ya danganta da adadin abokan ciniki. Muna kiran waɗannan ƙwayoyin metaclusters. Don karewa daga gazawar yanki gabaɗaya, ACK yana goyan bayan ƙaddamar da aiki da yawa a cikin yanki ɗaya: metacluster yana rarraba abubuwan haɗin gwiwar abokin ciniki na Kubernetes a cikin yankuna da yawa kuma yana gudanar da su lokaci guda, wato, a cikin yanayin aiki da yawa. Don tabbatar da aminci da inganci na maigidan, ACK yana inganta sanyawa na kayan aiki kuma yana tabbatar da cewa uwar garken API da sauransu suna kusa da juna.

Wannan samfurin yana ba ku damar sarrafa Kubernetes da kyau, sassauƙa da dogaro.

Tsarin albarkatun Metacluster

Kamar yadda muka riga muka ambata, adadin metaclusters a kowane yanki ya dogara da adadin abokan ciniki. Amma a wane lokaci don ƙara sabon metacluster? Wannan wata matsala ce ta tsara kayan aiki. A matsayinka na al'ada, al'ada ce don ƙirƙirar sabo lokacin da metaclusters da ke akwai sun ƙare duk albarkatun su.

Bari mu ɗauki albarkatun cibiyar sadarwa, alal misali. A cikin gine-ginen KoK, abubuwan Kubernetes daga gungu na abokin ciniki ana tura su azaman kwasfa a cikin metacluster. Muna amfani Terway (Fig. 3) babban kayan aiki ne wanda Alibaba Cloud ya haɓaka don sarrafa cibiyar sadarwar kwantena. Yana ba da ingantaccen tsarin tsare-tsaren tsaro kuma yana ba ku damar haɗawa da gajimare masu zaman kansu na abokan ciniki (VPCs) ta hanyar Intanet na Alibaba Cloud Elastic Networking Interface (ENI). Don rarraba albarkatun cibiyar sadarwa yadda ya kamata a cikin nodes, pods da kuma ayyuka a cikin metacluster, dole ne mu sanya ido a hankali yadda ake amfani da su a cikin metacluster na girgije masu zaman kansu. Lokacin da albarkatun cibiyar sadarwa suka ƙare, an ƙirƙiri sabon tantanin halitta.

Don ƙayyade mafi kyawun adadin gungu na abokin ciniki a cikin kowane metacluster, muna kuma yin la'akari da farashin mu, buƙatun yawa, adadin albarkatun, buƙatun aminci da ƙididdiga. An yanke shawarar ƙirƙirar sabon metacluster bisa duk wannan bayanin. Lura cewa ƙananan gungu na iya faɗaɗa sosai a nan gaba, don haka amfani da albarkatu yana ƙaruwa ko da adadin gungu ya ragu. Yawancin lokaci muna barin isasshen sarari kyauta don kowane gungu ya girma.

Shinkafa 3. Terway cibiyar sadarwa gine

Haɓaka abubuwan maye a cikin gungu na abokin ciniki

Abubuwan haɗin wizard suna da buƙatun albarkatu daban-daban. Sun dogara da adadin nodes da pods a cikin tari, adadin masu sarrafawa/masu aiki da ba daidai ba da ke hulɗa da APIServer.

A cikin ACK, kowane gungu abokin ciniki na Kubernetes ya bambanta cikin girman da buƙatun lokacin aiki. Babu wani tsari na duniya don sanya abubuwan maye. Idan muka yi kuskure saita ƙarancin albarkatu don babban abokin ciniki, to tarin sa ba zai iya jure wa lodin ba. Idan kun saita iyaka mai girma na ra'ayin mazan jiya ga duk gungu, za a yi asarar albarkatu.

Don nemo ciniki mai dabara tsakanin dogaro da farashi, ACK yana amfani da tsarin nau'in. Wato, mun ayyana nau'ikan gungu guda uku: kanana, matsakaici da babba. Kowane nau'i yana da bayanin martabar rabon albarkatu daban. An ƙayyade nau'in bisa la'akari da nauyin abubuwan maye, adadin nodes, da sauran dalilai. Nau'in tari na iya canzawa akan lokaci. ACK yana ci gaba da lura da waɗannan abubuwan kuma yana iya yin sama/ƙasa daidai da haka. Da zarar an canza nau'in tari, ana sabunta rabon albarkatun ta atomatik tare da ƙaramin sa hannun mai amfani.

Muna aiki don inganta wannan tsarin tare da mafi kyawun sikeli da ingantaccen nau'in sabuntawa don waɗannan canje-canjen su faru cikin sauƙi kuma su sami ƙarin ma'anar tattalin arziki.

Shinkafa 4. Canjin nau'in nau'in mataki-mataki mai hankali

Juyin Juyin Halitta na abokin ciniki a sikelin

Sassan da suka gabata sun rufe wasu fannoni na sarrafa manyan lambobi na gungu na Kubernetes. Duk da haka, akwai wata matsala da ke buƙatar warwarewa: juyin halitta na gungu.

Kubernetes shine "Linux" na duniyar girgije. Ana ci gaba da sabunta shi kuma yana ƙara haɓakawa. Dole ne mu ci gaba da isar da sabbin nau'ikan ga abokan cinikinmu, mu gyara lahani da sabunta ƙungiyoyin da ke akwai, da kuma sarrafa ɗimbin abubuwan da ke da alaƙa (CSI, CNI, Na'ura Plugin, Mai tsara Plugin da sauran su).

Bari mu ɗauki sarrafa bangaren Kubernetes a matsayin misali. Da farko, mun ƙirƙiri wani tsari na musamman don yin rajista da sarrafa duk waɗannan abubuwan haɗin gwiwa.

Shinkafa 5. Abubuwan sassauƙan sassauƙa da pluggable

Kafin ci gaba, kuna buƙatar tabbatar da sabuntawa ya yi nasara. Don yin wannan, mun ƙirƙiri tsarin don duba ayyukan abubuwan da aka gyara. Ana yin rajistan kafin da bayan sabuntawa.

Shinkafa 6. Binciken farko na abubuwan gungun

Don sabunta waɗannan abubuwan da sauri da dogaro, tsarin ƙaddamar da ci gaba yana aiki tare da goyan baya don ci gaba na ɗan lokaci (maunin toka), tsayawa da sauran ayyuka. Standard Kubernetes masu kula ba su dace da wannan yanayin amfani ba. Don haka, don sarrafa abubuwan da suka shafi gungu, mun ƙirƙira saitin na'urori na musamman, gami da plugin da tsarin kulawa na taimako (gudanar da motar gefe).

Misali, an tsara mai sarrafa BroadcastJob don sabunta abubuwan da aka gyara akan kowane injin ma'aikaci ko duba nodes akan kowace na'ura. Aikin Watsa shirye-shiryen yana gudanar da kwasfa akan kowane kumburi a cikin tari, kamar DaemonSet. Koyaya, DaemonSet koyaushe yana kiyaye kwandon yana gudana na dogon lokaci, yayin da BroadcastJob ya rushe shi. Mai kula da Watsa shirye-shiryen kuma yana ƙaddamar da kwas ɗin akan sabbin nodes ɗin da aka haɗa kuma yana ƙaddamar da nodes tare da abubuwan da suka dace. A watan Yuni 2019, mun buɗe lambar tushe na injin sarrafa kansa na OpenKruise, wanda mu kanmu muke amfani da shi a cikin kamfanin.

Shinkafa 7. OpenKurise yana shirya aiwatar da aikin Watsa shirye-shiryen akan duk nodes

Don taimaka wa abokan ciniki su zaɓi daidaitattun saiti na gungu, muna kuma samar da saitin bayanan martaba, gami da bayanan martaba na Serverless, Edge, Windows, da Bare Metal profile. Yayin da shimfidar wuri ke faɗaɗa kuma bukatun abokan cinikinmu suna girma, za mu ƙara ƙarin bayanan martaba don sauƙaƙe tsarin saitin mai wahala.

Shinkafa 8. Na ci gaba da sassauƙan bayanan bayanan tari don yanayi daban-daban

Alamar duniya a cikin cibiyoyin bayanai

Kamar yadda aka nuna a kasa fig. 9, Alibaba Cloud Container sabis na girgije an tura shi cikin yankuna ashirin a duniya. Idan aka yi la’akari da wannan sikelin, ɗaya daga cikin mahimman manufofin ACK shine a sauƙaƙe sanya ido kan yanayin tafiyar da gungu ta yadda idan rukunin abokin ciniki ya gamu da matsala, za mu iya ba da amsa cikin sauri ga lamarin. A takaice dai, kuna buƙatar fito da hanyar da za ta ba ku damar tattara ƙididdiga cikin inganci da aminci a cikin ainihin lokaci daga ƙungiyoyin abokan ciniki a duk yankuna - kuma ku gabatar da sakamakon gani.

Shinkafa 9. Ƙaddamar da sabis na Container Alibaba Cloud a duniya a yankuna ashirin

Kamar yawancin tsarin sa ido na Kubernetes, muna amfani da Prometheus azaman babban kayan aikin mu. Ga kowane metacluster, wakilan Prometheus suna tattara ma'auni masu zuwa:

• Ma'auni na OS kamar albarkatun mai watsa shiri (CPU, ƙwaƙwalwar ajiya, faifai, da sauransu) da bandwidth na cibiyar sadarwa.
• Ma'auni na metacluster da tsarin gudanarwar gungu na abokin ciniki, kamar kube-apiserver, kube-controller-manager da kube-scheduler.
• Ma'auni daga kubernetes-state-metrics da cadvisor.
• da sauransu.

Ana tattara ƙididdiga na duniya ta amfani da samfurin tara yawan Layer na yau da kullun. Ana fara tattara bayanan sa ido daga kowane metacluster a kowane yanki sannan a aika zuwa uwar garken tsakiya wanda ke nuna cikakken hoto. Komai yana aiki ta hanyar tsarin tarayya. Sabar Prometheus a kowace cibiyar bayanai tana tattara ma'auni daga wannan cibiyar bayanai, kuma uwar garken Prometheus ta tsakiya ce ke da alhakin tattara bayanan sa ido. AlertManager yana haɗi zuwa tsakiyar Prometheus kuma, idan ya cancanta, yana aika faɗakarwa ta hanyar DingTalk, imel, SMS, da sauransu. Kayayyakin gani - ta amfani da Grafana.

A cikin hoto na 10, ana iya raba tsarin sa ido zuwa matakai uku:

• Matsayin iyaka

Layer mafi nisa daga tsakiya. Prometheus Edge Server yana gudana a cikin kowane metacluster, yana tattara awo daga meta da gungun abokan ciniki a cikin yanki ɗaya na cibiyar sadarwa.

• Babban darajar

Ayyukan Layer na Cascade na Prometheus shine tattara bayanan kulawa daga yankuna da yawa. Waɗannan sabobin suna aiki a matakin manyan sassan yanki kamar China, Asiya, Turai da Amurka. Yayin da gungu ke girma, ana iya raba yankin, sannan uwar garken Prometheus mai matakin cascade zai bayyana a kowane sabon babban yanki. Tare da wannan dabarun, zaku iya daidaita ma'auni daidai yadda ake buƙata.

• Matsayin tsakiya

Sabar Prometheus ta tsakiya tana haɗi zuwa duk sabar cascade kuma tana aiwatar da tattara bayanan ƙarshe. Don amintacce, an ɗaga lokuttan Prometheus na tsakiya guda biyu a yankuna daban-daban, an haɗa su da sabar cascade iri ɗaya.

Shinkafa 10. Gine-gine na saka idanu masu yawa na duniya bisa tsarin tsarin tarayya na Prometheus

Takaitaccen

Hanyoyin girgije na tushen Kubernetes suna ci gaba da canza masana'antar mu. Sabis ɗin kwantena na Alibaba Cloud yana ba da amintaccen, abin dogaro da babban aiki - yana ɗaya daga cikin mafi kyawun Kubernetes Cloud hosting. Ƙungiyar Alibaba Cloud ta yi imani da ƙa'idodin Buɗaɗɗen tushe da buɗaɗɗen al'umma. Tabbas za mu ci gaba da raba iliminmu a fagen aiki da sarrafa fasahar girgije.

source: www.habr.com