Ukwandisa nokwengeza i-Kubernetes (uhlolojikelele kanye nombiko wevidiyo)

Ephreli 8 engqungqutheleni I-Saint HighLoad++ 2019, njengengxenye yesigaba esithi "DevOps and Operations", umbiko "Ukwandisa nokugcwalisa i-Kubernetes" yanikezwa, ekudalweni lapho abasebenzi abathathu benkampani ye-Flant babambe iqhaza. Kuyo, sikhuluma ngezimo eziningi lapho besifuna ukwandisa futhi sigcwalise amakhono e-Kubernetes, kodwa esingazange sithole isisombululo esenziwe ngomumo nesilula. Sinezixazululo ezidingekayo ngendlela yamaphrojekthi womthombo ovulekile, futhi le nkulumo inikezelwe kubo.

Ngokwesiko, siyajabula ukwethula ividiyo yombiko (imizuzu engama-50, inolwazi oluningi kakhulu kune-athikili) kanye nesifinyezo esikhulu sefomu lombhalo. Hamba!

Okubalulekile kanye nezengezo kuma-K8s

I-Kubernetes ishintsha imboni kanye nezindlela zokuphatha esezinesikhathi eside zasungulwa:

• Ngibonga yena abstractions, asisasebenzi ngemiqondo efana nokusetha ukucushwa noma ukusebenzisa umyalo (Umpheki, Ongavumelekile...), kodwa sebenzisa ukuqoqwa kweziqukathi, amasevisi, njll.
• Singalungiselela izicelo ngaphandle kokucabanga ngama-nuances we indawo ethile, lapho izokwethulwa khona: insimbi engenalutho, ifu lomunye wabahlinzeki, njll.
• Ngama-K8 awukaze ufinyeleleke kakhudlwana imikhuba engcono kakhulu ekuhleleni ingqalasizinda: amasu okukala, ukuzelapha, ukubekezelela amaphutha, njll.

Kodwa-ke, vele, yonke into ayisheleli kangako: I-Kubernetes nayo ilethe izinselelo zayo ezintsha.

Kubernetes hhayi iyinhlanganisela exazulula zonke izinkinga zabo bonke abasebenzisi. Isibindi I-Kubernetes inesibopho kuphela sesethi yemisebenzi encane edingekayo ekhona wonke umuntu iqoqo:

I-Kubernetes core ichaza isethi eyisisekelo yezinto zokuqala zokuhlanganisa iziqukathi, ukuphatha ithrafikhi, nokunye. Sikhulume ngazo kabanzi ku kubika eminyakeni engu-2 edlule.

Ngakolunye uhlangothi, i-K8s inikeza amathuba amahle okwandisa imisebenzi etholakalayo, esiza ukuvala eminye - ethize - izidingo zomsebenzisi. Ukwengezwa ku-Kubernetes kuwumthwalo wabalawuli beqoqo, okufanele bafake futhi balungiselele yonke into edingekayo ukuze bathole iqoqo labo “ngesimo esifanele” [ukuxazulula izinkinga zabo ezithile]. Hlobo luni lwezengezo lezi? Ake sibheke ezinye izibonelo.

Izibonelo zezengezo

Ngemva kokufaka i-Kubernetes, singamangala ukuthi ukuxhumana kwenethiwekhi okudingeka kakhulu ekuxhumaneni kwama-pods kokubili ngaphakathi kwe-node naphakathi kwama-node akusebenzi ngokwawo. I-Kubernetes kernel ayiqinisekisi ukuxhumana okudingekayo; kunalokho, inquma inethiwekhi isikhombimsebenzisi (CNI) ezengezo zezinkampani zangaphandle. Kufanele sifake esisodwa salezi zengezo, esizoba nesibopho sokucushwa kwenethiwekhi.

Isibonelo esiseduze yizixazululo zokugcina idatha (idiski yendawo, idivayisi yokuvimba inethiwekhi, i-Ceph ...). Ekuqaleni babephakathi, kodwa ngokufika CSI isimo sishintshela kokuthile okufana nalokhu osekuchaziwe: i-interface iku-Kubernetes, futhi ukuqaliswa kwayo kumamojula ezinkampani zangaphandle.

Ezinye izibonelo zihlanganisa:

• Ingress-abalawuli (bheka isibuyekezo sabo ku isihloko sethu sakamuva).
• umphathi we-cert:

  

• Abasebenzisi iyisigaba sonke sezengezo (okuhlanganisa isiphathi-magama esishiwo), sichaza ama-primitive(ama) kanye nesilawuli. I-logic yomsebenzi wabo inqunyelwe kuphela umcabango wethu futhi isivumela ukuthi siguqule izingxenye zengqalasizinda esezilungile (isibonelo, i-DBMS) zibe izinto zokuqala, okulula kakhulu ukusebenza nazo (kuneqoqo leziqukathi nezilungiselelo zazo). Inani elikhulu lama-opharetha libhaliwe - noma ngabe abaningi babo bengakalungeli ukukhiqizwa, kuyindaba yesikhathi kuphela:

  

• Amamethrikhi - omunye umfanekiso wokuthi i-Kubernetes ihlukanise kanjani isixhumi esibonakalayo (Metrics API) kusukela ekusetshenzisweni (izengezo zenkampani yangaphandle ezifana ne-adaptha ye-Prometheus, i-ejenti yeqoqo le-Datadog...).
• Ukuze ukuqapha kanye nezibalo, lapho ekusebenzeni kungadingeki nje kuphela Prometheus futhi Grafana, kodwa futhi kube-state-metrics, node-exporter, njll.

Futhi lokhu akulona uhlu oluphelele lwezengezo ... Isibonelo, enkampanini ye-Flant esiyifaka njengamanje 29 izengezo (konke okudala ingqikithi yezinto ezingama-249 Kubernetes). Kalula nje, asikwazi ukubona impilo yeqoqo ngaphandle kokwengezwa.

Ukuzenzakalela

Ama-opharetha aklanyelwe ukwenza ngokuzenzakalelayo imisebenzi yenjwayelo esihlangana nayo nsuku zonke. Nazi izibonelo zangempela lapho ukubhala umsebenzi kungaba yisixazululo esihle kakhulu:

1. Kukhona indawo eyimfihlo (okungukuthi edinga ukungena ngemvume) enezithombe zohlelo lokusebenza. Kucatshangwa ukuthi i-pod ngayinye inikezwe imfihlo ekhethekile evumela ukuqinisekiswa kurejista. Umsebenzi wethu uwukuqinisekisa ukuthi le mfihlo iyatholakala endaweni yamagama ukuze ama-pods akwazi ukulanda izithombe. Kungaba nezinhlelo zokusebenza eziningi (ngayinye yazo idinga imfihlo), futhi kuyasiza ukuvuselela izimfihlo ngokwazo njalo, ngakho-ke inketho yokubeka izimfihlo ngesandla iyasuswa. Kulapho u-opharetha esiza khona: sidala isilawuli esizolinda ukuthi indawo yegama ivele futhi, ngokusekelwe kulo mcimbi, sizofaka imfihlo endaweni yamagama.
2. Ukuvumela ngokuzenzakalelayo ukufinyelela kusuka ku-pod kuya ku-inthanethi akuvunyelwe. Kodwa ngezinye izikhathi kungase kudingeke: kunengqondo ukuthi indlela yemvume yokufinyelela isebenze kalula, ngaphandle kokudinga amakhono athile, isibonelo, ngokuba khona kwelebula ethile endaweni yamagama. Umsebenzisi angasisiza kanjani lapha? Kudalwe isilawuli esilinda ukuthi ilebula ivele endaweni yamagama bese sengeza inqubomgomo efanelekile yokufinyelela ku-inthanethi.
3. Isimo esifanayo: ake sithi sidinga ukungeza okuthile ukungcola, uma inelebula efanayo (nohlobo oluthile lwesiqalo). Izenzo no-opharetha zisobala...

Kunoma iyiphi iqoqo, imisebenzi evamile kufanele ixazululwe, futhi kwesokudla lokhu kungenziwa ngokusebenzisa opharetha.

Sifingqa zonke izindaba ezichaziwe, sifinyelele esiphethweni sokuthi ukuze uthole umsebenzi othokomele ku-Kubernetes owudingayo: A) faka izengezo, b) ukuthuthukisa opharetha (yokuxazulula imisebenzi yansuku zonke yomlawuli).

Ungasibhala kanjani isitatimende se-Kubernetes?

Ngokuvamile, uhlelo lulula:

... kodwa bese kuvela ukuthi:

• I-Kubernetes API iyinto engeyona into encane ethatha isikhathi esiningi ukuyiqonda;
• ukuhlela futhi akukona okwawo wonke umuntu (ulimi lwesiGo lukhethwe njengolimi olukhethwayo ngoba kunohlaka olukhethekile lwalo - I-Opharetha ye-SDK);
• Isimo siyefana nangohlaka ngokwalo.

Isiphetho sendaba: ukubhala isilawuli (opharetha) kufanele sebenzisa izinsiza ezibalulekile ukufunda impahla. Lokhu kuzolunga kuma-opharetha “amakhulu” - yithi, ku-MySQL DBMS. Kodwa uma sikhumbula izibonelo ezichazwe ngenhla (izimfihlo ezivezwayo, ukufinyelela ama-pods ku-Inthanethi ...), esifuna futhi ukukwenza ngendlela efanele, khona-ke sizoqonda ukuthi umzamo owenziwe uzodlula umphumela esiwudingayo manje:

Ngokuvamile, kuphakama inkinga: sebenzisa izinsiza eziningi futhi uthole ithuluzi elifanele lokubhala izitatimende, noma ukwenze ngendlela yakudala (kodwa ngokushesha). Ukuyixazulula - ukuthola ukuvumelana phakathi kwalokhu okweqisayo - sidale iphrojekthi yethu: igobolondo-opharetha (bheka futhi eyakhe isimemezelo sakamuva endaweni).

I-Shell-opharetha

Usebenza kanjani? Iqoqo line-pod equkethe i-Go binary ene-shell-operator. Eduze kwakhe kukhona isethi izingwegwe (imininingwane eyengeziwe ngabo - bheka ngezansi). I-shell-opharetha ngokwayo ibhalisela okuthile izenzakalo ku-Kubernetes API, lapho kwenzeka yethula izingwegwe ezihambisanayo.

I-shell-operator yazi kanjani ukuthi yiziphi izingwegwe okufanele zibize kuziphi izehlakalo? Lolu lwazi ludluliselwa ku-shell-opharetha ngamahhuku ngokwawo, futhi akwenza kalula.

I-hook iwumbhalo we-Bash nanoma yiliphi elinye ifayela elisebenzisekayo elamukela ingxabano eyodwa --config futhi iphendula nge-JSON. Lesi sakamuva sinquma ukuthi yiziphi izinto ezithakaselwayo futhi yiziphi izehlakalo (zalezi zinto) okufanele ziphendulwe:

Ngizobonisa ukuqaliswa ku-shell-opharetha kwesinye sezibonelo zethu - izimfihlo zokubola zokufinyelela ukubhaliswa kwangasese ngezithombe zohlelo lokusebenza. Iqukethe izigaba ezimbili.

Prakthiza: 1. Bhala ihhuku

Okokuqala, ku-hook sizocubungula --config, okubonisa ukuthi sinentshisekelo kuzikhala zamagama, futhi ikakhulukazi, isikhathi sokudalwa kwazo:

[[ $1 == "--config" ]] ; then
  cat << EOF
{
  "onKubernetesEvent": [
    {
      "kind": "namespace",
      "event": ["add"]
    }
  ]
}
EOF
…

I-logic izobukeka kanjani? Futhi kulula kakhulu:

…
else
  createdNamespace=$(jq -r '.[0].resourceName' $BINDING_CONTEXT_PATH)
  kubectl create -n ${createdNamespace} -f - << EOF
Kind: Secret
...
EOF
fi

Isinyathelo sokuqala ukuthola ukuthi iyiphi indawo yamagama eyadalwa, kanti eyesibili wukuyenza usebenzisa kubectl imfihlo yalesi sikhala samagama.

Prakthiza: 2. Ukuhlanganisa isithombe

Okusele nje ukudlulisa ingwegwe edaliwe ku-shell-opharetha - ungakwenza kanjani lokhu? I-shell-opharetha ngokwayo iza njengesithombe se-Docker, ngakho-ke umsebenzi wethu ukungeza ihhuku kumkhombandlela okhethekile kulesi sithombe:

FROM flant/shell-operator:v1.0.0-beta.1
ADD my-handler.sh /hooks

Okusele nje ukuwuhlanganisa bese uwuphusha:

$ docker build -t registry.example.com/my-operator:v1 .
$ docker push registry.example.com/my-operator:v1

Ukuthinta kokugcina ukuhambisa isithombe kuqoqo. Ukwenza lokhu, asibhale Ukuthunyelwa:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-operator
spec:
  template:
    spec:
      containers:
      - name: my-operator
        image: registry.example.com/my-operator:v1 # 1
      serviceAccountName: my-operator              # 2

Kunamaphuzu amabili okufanele unakwe:

1. inkomba yesithombe esisanda kwakhiwa;
2. Lena ingxenye yesistimu (okungenani) edinga amalungelo okubhalisa emicimbini eku-Kubernetes kanye nokwaba izimfihlo kuzikhala zamagama, ngakho-ke sakha i-ServiceAccount (kanye nesethi yemithetho) yehuku.

Umphumela - siyixazulule inkinga yethu izihlobo ye-Kubernetes ngendlela edala u-opharetha wokubola izimfihlo.

Ezinye izici ze-shell-opharetha

Ukukhawulela izinto zohlobo olukhethile ihuku elizosebenza ngazo, zingahlungwa, ukukhetha ngokwamalebula athile (noma usebenzisa matchExpressions):

"onKubernetesEvent": [
  {
    "selector": {
      "matchLabels": {
        "foo": "bar",
       },
       "matchExpressions": [
         {
           "key": "allow",
           "operation": "In",
           "values": ["wan", "warehouse"],
         },
       ],
     }
     …
  }
]

Kuhlinzekiwe indlela yokunciphisa, okuthi - usebenzisa isihlungi se-jq - ikuvumela ukuthi uguqule izinto ezinkulu ze-JSON zibe ezincane, lapho kuphela lawo mapharamitha esifuna ukuqapha izinguquko.

Uma kubizwa ihhuku, i-shell-opharetha iyayidlulisa idatha yento, engasetshenziswa kunoma yisiphi isidingo.

Izehlakalo ezicupha izingwegwe azikhawulelwe kumicimbi ye-Kubernetes: i-shell-opharetha inikeza usekelo ukubiza izingwegwe ngesikhathi (efana ne-crontab kusihleli sendabuko), kanye nomcimbi okhethekile ku-Startup. Zonke lezi zenzakalo zingahlanganiswa futhi zinikezwe i-hook efanayo.

Futhi izici ezimbili ezengeziwe ze-shell-opharetha:

1. Kuyasebenza ngokulinganayo. Njengoba umcimbi we-Kubernetes (ofana nento edalwayo) wamukelwe, eminye imicimbi (efana nento efanayo esuswayo) ibingenzeka kuqoqo, futhi izingwegwe zidinga ukulandisa ngalokhu. Uma ihhuku yenziwe ngephutha, khona-ke ngokuzenzakalela kuzoba njalo phinda ushayele ucingo kuze kuqedwe ngempumelelo (lokhu kuziphatha kungashintshwa).
2. Ithumela ngaphandle amamethrikhi ye-Prometheus, ongaqonda ngayo ukuthi i-shell-opharetha iyasebenza, thola inani lamaphutha wehuku ngalinye kanye nosayizi wamanje womugqa.

Ukufingqa le ngxenye yombiko:

Ukufaka izengezo

Ngomsebenzi onethezekile ne-Kubernetes, isidingo sokufaka izengezo siphinde sashiwo. Ngizokutshela ngakho ngisebenzisa isibonelo sendlela yenkampani yethu yokuthi sikwenza kanjani manje.

Saqala ukusebenza noKubernetes ngamaqoqo amaningana, okuwukuphela kokwengezwa okwakuyi-Ingress. Yayidinga ukufakwa ngokuhlukile kuqoqo ngalinye, futhi senze ukulungiselelwa kwe-YAML okuningana kwezindawo ezihlukene: insimbi engenalutho, i-AWS...

Njengoba kwakukhona amaqoqo amaningi, kwakukhona ukucushwa okwengeziwe. Ngaphezu kwalokho, sizithuthukise ngokwazo lezi zilungiselelo, ngenxa yalokho zahluka kakhulu:

Ukuze sibeke yonke into ngononina, siqale ngombhalo (install-ingress.sh), okuthathe njengengxabano uhlobo lweqoqo esizothumela kulo, ikhiqize ukulungiselelwa kwe-YAML okudingekayo futhi ikukhiphele ku-Kubernetes.

Ngamafuphi, indlela yethu eyengeziwe kanye nokucabanga okuhlotshaniswa nayo kwakungokulandelayo:

• ukuze usebenze ngokucushwa kwe-YAML, injini yesifanekiso iyadingeka (ezigabeni zokuqala lena i-sed elula);
• ngokunyuka kwenani lamaqoqo, isidingo sokuvuselela okuzenzakalelayo safika (isixazululo sokuqala kwakuwukubeka iskripthi ku-Git, sibuyekeze usebenzisa i-cron futhi siyiqhube);
• kwakudingeka umbhalo ofanayo ku-Prometheus (install-prometheus.sh), nokho, kuyaphawuleka ngeqiniso lokuthi idinga idatha yokufaka eyengeziwe, kanye nokugcinwa kwayo (ngendlela enhle - phakathi nendawo kanye neqoqo), futhi enye idatha (amaphasiwedi) ingakhiqizwa ngokuzenzakalelayo:

  

• ingozi yokukhipha okuthile okungalungile enanini elikhulayo lamaqoqo yayilokhu ikhula, ngakho-ke sabona ukuthi abafaki (okungukuthi imibhalo emibili: ye-Ingress ne-Prometheus) i-staging yayidingeka (amagatsha amaningana ku-Git, ama-crons amaningana ukuze awabuyekeze ngokuhambisanayo: amaqoqo azinzile noma okuhlola);
• с kubectl apply sekunzima ukusebenza ngakho ngoba ayisho futhi ingakha izinto kuphela, kodwa hhayi ukuthatha izinqumo ngesimo sazo / ukuzisusa;
• Besigeja imisebenzi ethile ebesingakayisebenzisi nhlobo ngaleso sikhathi:
  • ukulawula okugcwele komphumela wezibuyekezo zeqoqo,
  • ukunqunywa okuzenzakalelayo kwamanye amapharamitha (okokufaka izikripthi zokufaka) ngokusekelwe kudatha engatholwa kuqoqo (ukutholwa),
  • ukuthuthukiswa kwayo okunengqondo ngendlela yokutholakala okuqhubekayo.

Sisebenzise konke lokhu okuhlangenwe nakho okuqoqiwe ngaphakathi kohlaka lweminye iphrojekthi yethu - i-addon-opharetha.

I-Addon-opharetha

Isekelwe ku-shell-opharetha esishiwo kakade. Lonke uhlelo lubukeka kanje:

Okulandelayo kwengezwe kumahhuku e-shell-opharetha:

• isitoreji samagugu,
• Ishadi lesigqoko,
• ingxenye leyo iqapha amanani esitolo futhi - uma kwenzeka kuba noshintsho - ucela u-Helm ukuthi aphinde akhiphe ishadi.

Ngakho-ke, singasabela kumcimbi ku-Kubernetes, siqalise i-hook, futhi kusukela kule hook singenza izinguquko kwisitoreji, ngemva kwalokho ishadi lizolandwa kabusha. Emdwebeni ophumela, sihlukanisa isethi yezingwegwe kanye neshadi zibe ingxenye eyodwa, esiyibizayo module:

Kungaba namamojula amaningi, futhi kuwo sengeza izingwegwe zomhlaba wonke, isitolo samanani omhlaba, kanye nengxenye eqapha lesi sitolo somhlaba wonke.

Manje, uma kwenzeka okuthile ku-Kubernetes, singasabela kukho sisebenzisa ihuku lomhlaba wonke futhi sishintshe okuthile esitolo somhlaba. Lolu shintsho luzoqashelwa futhi luzobangela ukuthi wonke amamojula akuqoqo akhishwe:

Lolu hlelo lunelisa zonke izidingo zokufaka izengezo ezishiwo ngenhla:

• I-Helm inesibopho sokwenza isifanekiso nokumemezela.
• Inkinga yokubuyekeza okuzenzakalelayo yaxazululwa kusetshenziswa i-hook yomhlaba wonke, eya esibhalisini ngeshejuli futhi, uma ibona isithombe esisha sesistimu lapho, iyasikhipha (okungukuthi "ngokwaso").
• Ukugcina izilungiselelo ku-cluster kusetshenziswa kusetshenziswa ConfigMap, equkethe idatha eyinhloko yesitoreji (ekuqaliseni zilayishwa kusitoreji).
• Izinkinga ngokukhiqiza amaphasiwedi, ukutholwa kanye nokutholakala okuqhubekayo kwaxazululwa kusetshenziswa izingwegwe.
• Isiteji sitholwa ngenxa yomaka, abasekelwa yi-Docker ngaphandle kwebhokisi.
• Umphumela ugadwa kusetshenziswa amamethrikhi esingaqonda ngawo isimo.

Lonke lolu hlelo lusetshenziswa ngendlela kanambambili eyodwa ku-Go, ebizwa nge-addon-operator. Lokhu kwenza umdwebo ubukeke ulula:

Ingxenye eyinhloko kulo mdwebo isethi yamamojula (kugqanyiswe ngokumpunga ngezansi). Manje singabhala imojula yesengezo esidingekayo ngomzamo omncane futhi siqiniseke ukuthi izofakwa kuqoqo ngalinye, izobuyekezwa futhi iphendule ezehlakalweni eziyidingayo kuqoqo.

"Flant" isetshenziswa i-addon-opharetha kumaqoqo angu-70+ e-Kubernetes. Isimo samanje - inguqulo ye-alpha. Manje silungiselela imibhalo ukuze sikhulule i-beta, kodwa okwamanje endaweni yokugcina izibonelo ezikhona, ngesisekelo lapho ungakha i-addon yakho.

Ngingawatholaphi amamojula we-addon-opharetha? Ukushicilela umtapo wethu wezincwadi kuyisigaba esilandelayo kithi; sihlela ukwenza lokhu ehlobo.

Amavidiyo namaslayidi

Ividiyo evela ekusebenzeni (~ imizuzu engama-50):

Ukwethulwa kombiko:

PS

Eminye imibiko kubhulogi yethu:

• «Databases kanye Kubernetes"; (Dmitry Stolyarov; Novemba 8, 2018 ku-HighLoad++);
• «Ukuqapha kanye ne-Kubernetes"; (Dmitry Stolyarov; May 28, 2018 at RootConf);
• «I-CI/CD Imikhuba Engcono Kakhulu nge-Kubernetes kanye ne-GitLab"; (Dmitry Stolyarov; Novemba 7, 2017 ku-HighLoad++);
• «Okuhlangenwe nakho kwethu ne-Kubernetes kumaphrojekthi amancane"; (Dmitry Stolyarov; Juni 6, 2017 e-RootConf).

Ungase futhi ube nentshisekelo kokushicilelwe okulandelayo:

• «Sethula i-shell-opharetha: ukudala ama-opharetha e-Kubernetes sekulula".

Source: www.habr.com