Kukulitsa ndi kukwaniritsa Kubernetes (ndemanga ndi lipoti lamavidiyo)

April 8 kumsonkhanowu Saint HighLoad++ 2019, monga gawo la gawo la "DevOps and Operations", lipoti la "Kukulitsa ndi Kuonjezera Kubernetes" linaperekedwa, popanga omwe antchito atatu a kampani ya Flant adagwira nawo ntchito. M'menemo, timalankhula za zochitika zambiri zomwe tinkafuna kukulitsa ndi kukwaniritsa mphamvu za Kubernetes, koma zomwe sitinapeze yankho lokonzekera komanso losavuta. Tili ndi mayankho ofunikira monga mapulojekiti a Open Source, ndipo mawu awa adaperekedwanso kwa iwo.

Mwa mwambo, ndife okondwa kupereka kanema wa lipoti (Mphindi 50, yophunzitsa zambiri kuposa nkhaniyo) komanso chidule chachikulu m'mawu. Pitani!

Core ndi zowonjezera mu K8s

Kubernetes akusintha makampani ndi njira zoyendetsera ntchito zomwe zakhazikitsidwa kalekale:

• Zikomo kwa iye zotsalira, sitigwiranso ntchito ndi malingaliro monga kukhazikitsa config kapena kuyendetsa lamulo (Chef, Ansible...), koma gwiritsani ntchito magulu a zotengera, mautumiki, ndi zina zotero.
• Titha kukonzekera mapulogalamu osaganizira zamitundu yosiyanasiyana malo enieni, yomwe idzayambitsidwe: chitsulo chopanda kanthu, mtambo wa mmodzi wa opereka chithandizo, ndi zina zotero.
• Ndi ma K8 simunapezekepo machitidwe abwino pakukonzekera zomangamanga: njira zokulira, kudzichiritsa nokha, kulolerana ndi zolakwika, ndi zina.

Komabe, zowona, zonse sizili bwino: Kubernetes adabweretsanso zovuta zake zatsopano.

Kubernetes osati ndi kuphatikiza komwe kumathetsa mavuto onse a ogwiritsa ntchito. Pakatikati Kubernetes amangoyang'anira ntchito zingapo zofunika zomwe zilipo aliyense gulu:

Choyambira cha Kubernetes chimatanthawuza zoyambira zoyambira m'magulumagulu, kuyang'anira magalimoto, ndi zina zotero. Tinakambirana za iwo mwatsatanetsatane mu lipoti zaka 2 zapitazo.

Kumbali ina, K8s imapereka mwayi waukulu wowonjezera ntchito zomwe zilipo, zomwe zimathandiza kutseka ena - mwachindunji - Zosowa za ogwiritsa. Zowonjezera ku Kubernetes ndi udindo wa oyang'anira magulu, omwe ayenera kukhazikitsa ndi kukonza zonse zofunika kuti gulu lawo likhale "loyenera" [kuti athetse mavuto awo enieni]. Ndi zowonjezera zotani izi? Tiyeni tione zitsanzo zina.

Zitsanzo za zowonjezera

Titayika Kubernetes, titha kudabwa kuti maukonde omwe ndi ofunikira kwambiri pakulumikizana kwa ma pod mkati mwa node komanso pakati pa node sikugwira ntchito pawokha. Kubernetes kernel sikutsimikizira kulumikizana kofunikira; m'malo mwake, imatsimikizira maukonde mawonekedwe (CNI) pazowonjezera za gulu lachitatu. Tiyenera kukhazikitsa imodzi mwazowonjezera izi, zomwe zidzayang'anire kasinthidwe ka netiweki.

Chitsanzo chapafupi ndi njira zosungiramo deta (disk ya m'deralo, chipangizo cha block block, Ceph ...). Poyamba, iwo anali mu chikhalidwe, koma ndi kubwera CSI zinthu zikusintha kukhala zofanana ndi zomwe zafotokozedwa kale: mawonekedwe ali ku Kubernetes, ndipo kukhazikitsidwa kwake kuli m'ma module a chipani chachitatu.

Zitsanzo zina ndi izi:

• Ingress-owongolera (onani ndemanga yawo mu nkhani yathu yaposachedwa).
• cert-woyang'anira:

  

• Ogwira ntchito ndi gulu lonse la zowonjezera (zomwe zikuphatikizapo cert-manager), amatanthauzira akale ndi olamulira. Lingaliro la ntchito yawo ndi lochepa chabe ndi malingaliro athu ndipo limatithandiza kutembenuza zida zokonzekera (mwachitsanzo, DBMS) kukhala zoyamba, zomwe zimakhala zosavuta kugwira ntchito (kusiyana ndi zotengera ndi zoikamo). Ogwiritsa ntchito ambiri alembedwa - ngakhale ambiri aiwo sanakonzekere kupanga, ndi nkhani yanthawi yake:

  

• Metrics - fanizo lina la momwe Kubernetes adalekanitsira mawonekedwe (Metrics API) kuchokera pakukhazikitsa (zowonjezera za chipani chachitatu monga adaputala ya Prometheus, wothandizira datadog cluster ...).
• chifukwa kuyang'anira ndi ziwerengero, kumene m’zochita osati kokha zofunika Prometheus ndi Grafana, komanso kube-state-metrics, node-exporter, etc.

Ndipo iyi si mndandanda wathunthu wa zowonjezera ... Mwachitsanzo, pa kampani ya Flant yomwe timayika panopa 29 zowonjezera (zonsezi zimapanga zinthu zonse za 249 Kubernetes). Mwachidule, sitingathe kuwona moyo wa tsango popanda zowonjezera.

Zodzichitira

Othandizira adapangidwa kuti azingogwiritsa ntchito nthawi zonse zomwe timakumana nazo tsiku lililonse. Nazi zitsanzo zenizeni zomwe kulemba wogwiritsa ntchito kungakhale yankho labwino kwambiri:

1. Pali kaundula wachinsinsi (i.e. wofuna kulowa) wokhala ndi zithunzi za pulogalamuyo. Zimaganiziridwa kuti pod iliyonse imapatsidwa chinsinsi chapadera chomwe chimalola kutsimikizika mu registry. Ntchito yathu ndikuwonetsetsa kuti chinsinsichi chikupezeka m'malo a mayina kuti ma pod amatha kutsitsa zithunzi. Pakhoza kukhala ntchito zambiri (iliyonse yomwe imafunikira chinsinsi), ndipo ndizothandiza kusinthira zinsinsizo nthawi zonse, kotero kuti mwayi woyika zinsinsi ndi dzanja umachotsedwa. Apa ndi pamene woyendetsa amabwera kudzapulumutsa: timapanga wolamulira yemwe adzadikirira kuti dzina la dzina liwonekere ndipo, pogwiritsa ntchito chochitika ichi, adzawonjezera chinsinsi ku malo a mayina.
2. Lolani mwachisawawa kupeza kuchokera ku ma pod kupita pa intaneti ndikoletsedwa. Koma nthawi zina zingafunike: ndi zomveka kuti njira yopezera chilolezo igwire ntchito mophweka, popanda kufunikira luso linalake, mwachitsanzo, ndi kupezeka kwa chizindikiro mu malo a mayina. Kodi wogwira ntchitoyo angatithandize bwanji apa? Chowongolera chimapangidwa chomwe chimadikirira kuti chizindikirocho chiwoneke m'malo a mayina ndikuwonjezera ndondomeko yoyenera yofikira pa intaneti.
3. Momwemonso: tiyerekeze kuti tikufunika kuwonjezera zina kuyipa, ngati ili ndi chizindikiro chofanana (ndi mtundu wina wa chiyambi). Zochita ndi opareshoni ndizodziwikiratu ...

M'magulu aliwonse, ntchito zanthawi zonse ziyenera kuthetsedwa, ndi kulondola izi zitha kuchitika pogwiritsa ntchito opareta.

Pofotokoza mwachidule nkhani zonse zomwe zafotokozedwa, tinafika potsimikiza kuti kuti mugwire ntchito yabwino ku Kubernetes yomwe mukufuna: A) kukhazikitsa zowonjezera, b) khazikitsani ogwira ntchito (pothetsa ntchito za admin zatsiku ndi tsiku).

Momwe mungalembe mawu a Kubernetes?

Kawirikawiri, ndondomekoyi ndi yosavuta:

.. koma zinapezeka kuti:

• Kubernetes API ndi chinthu chopanda pake chomwe chimatenga nthawi yochuluka kuti chidziwe bwino;
• mapulogalamu nawonso si a aliyense (chiyankhulo cha Go chinasankhidwa kukhala chilankhulo chokondedwa chifukwa pali dongosolo lapadera la icho - Othandizira SDK);
• Zinthu zilinso chimodzimodzi ndi chimango chokha.

Pansi mzere: kulemba woyang'anira (woyendetsa) ayenera wononga ndalama zambiri kuphunzira zinthu. Izi zitha kukhala zomveka kwa ogwiritsa ntchito "akuluakulu" - nenani, pa MySQL DBMS. Koma ngati tikumbukira zitsanzo zomwe tafotokozazi (zobisika zinsinsi, kupeza ma pods pa intaneti ...), zomwe tikufunanso kuchita molondola, ndiye kuti tidzamvetsetsa kuti khama lomwe likugwiritsidwa ntchito lidzaposa zotsatira zomwe tikufunikira tsopano:

Nthawi zambiri, pamakhala vuto: gwiritsani ntchito zinthu zambiri ndikupeza chida choyenera cholembera mawu, kapena chitani mwanjira yachikale (koma mwachangu). Kuti tithetse - kuti tipeze kusagwirizana pakati pa izi - tidapanga pulojekiti yathu: chipolopolo-woyendetsa (onaninso ake chilengezo chaposachedwa pa gombe).

Shell-operator

Kodi amagwira ntchito bwanji? Gululi lili ndi poto yokhala ndi Go binary yokhala ndi chipolopolo. Pafupi naye pali gulu la mbedza (zambiri za iwo - onani pansipa). The shell-operator palokha amalembetsa zina zochitika mu Kubernetes API, zikachitika zomwe zimayambitsa zingwe zofananira.

Kodi woyendetsa zipolopolo amadziwa bwanji mbedza kuti atchule zochitika ziti? Chidziwitsochi chimaperekedwa kwa oyendetsa chipolopolo ndi mbedza zokha, ndipo amazichita mophweka.

Hook ndi script ya Bash kapena fayilo ina iliyonse yomwe ingatheke yomwe imavomereza mkangano umodzi --config ndikuyankha ndi JSON. Chotsatiracho chimatsimikizira kuti ndi zinthu ziti zomwe zili ndi chidwi ndi zomwe zikuchitika (zazinthu izi) zomwe ziyenera kuyankhidwa:

Ndikuwonetsa kukhazikitsidwa kwa chipolopolo-oyendetsa chimodzi mwa zitsanzo zathu - zinsinsi zowola kuti mupeze zolembera zachinsinsi ndi zithunzi zofunsira. Limakhala ndi magawo awiri.

Yesani: 1. Lembani mbedza

Choyamba, mu mbedza tidzakonza --config, kusonyeza kuti tili ndi chidwi ndi malo, makamaka nthawi yomwe adalengedwa:

[[ $1 == "--config" ]] ; then
  cat << EOF
{
  "onKubernetesEvent": [
    {
      "kind": "namespace",
      "event": ["add"]
    }
  ]
}
EOF
…

Kodi malingaliro angawoneke bwanji? Komanso yosavuta:

…
else
  createdNamespace=$(jq -r '.[0].resourceName' $BINDING_CONTEXT_PATH)
  kubectl create -n ${createdNamespace} -f - << EOF
Kind: Secret
...
EOF
fi

Gawo loyamba ndikupeza malo omwe adapangidwa, ndipo chachiwiri ndikulipanga pogwiritsa ntchito kubectl chinsinsi cha malo awa.

Yesani: 2. Kusonkhanitsa chithunzi

Zomwe zatsala ndikudutsa mbedza yopangidwa kwa oyendetsa chipolopolo - momwe mungachitire izi? Wogwiritsa ntchito chipolopolo amabwera ngati chithunzi cha Docker, ndiye ntchito yathu ndikuwonjezera mbedza ku chikwatu chapadera pachithunzichi:

FROM flant/shell-operator:v1.0.0-beta.1
ADD my-handler.sh /hooks

Chotsalira ndikuchisonkhanitsa ndikuchikankhira:

$ docker build -t registry.example.com/my-operator:v1 .
$ docker push registry.example.com/my-operator:v1

Kukhudza komaliza ndikutumiza chithunzicho kumagulu. Kuti tichite izi, tiyeni tilembe Kutumizidwa:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-operator
spec:
  template:
    spec:
      containers:
      - name: my-operator
        image: registry.example.com/my-operator:v1 # 1
      serviceAccountName: my-operator              # 2

Pali mfundo ziwiri zofunika kuziganizira:

1. chiwonetsero cha chithunzi chopangidwa chatsopano;
2. Ichi ndi gawo la dongosolo lomwe (pang'onopang'ono) limafunikira ufulu wolembetsa ku zochitika ku Kubernetes ndikugawa zinsinsi ku malo a mayina, kotero timapanga ServiceAccount (ndi ndondomeko ya malamulo) ya mbedza.

Zotsatira - tathetsa vuto lathu achibale kwa Kubernetes m'njira yomwe imapanga wogwiritsa ntchito kuti awononge zinsinsi.

Zochita zina za shell-operator

Kuti muchepetse zinthu zamtundu womwe mwasankha zomwe mbedza ingagwire ntchito, akhoza kusefedwa, kusankha molingana ndi zilembo zina (kapena kugwiritsa ntchito matchExpressions):

"onKubernetesEvent": [
  {
    "selector": {
      "matchLabels": {
        "foo": "bar",
       },
       "matchExpressions": [
         {
           "key": "allow",
           "operation": "In",
           "values": ["wan", "warehouse"],
         },
       ],
     }
     …
  }
]

Zaperekedwa deduplication ndondomeko, yomwe - pogwiritsa ntchito fyuluta ya jq - imakulolani kuti mutembenuzire zinthu zazikulu za JSON kukhala zazing'ono, kumene magawo okhawo amatsalira omwe tikufuna kuyang'anira kusintha.

Pamene mbeza imatchedwa, woyendetsa chipolopolo amadutsa data ya chinthu, yomwe ingagwiritsidwe ntchito pa zosowa zilizonse.

Zochitika zomwe zimayambitsa mbedza sizingokhala zochitika za Kubernetes: wogwiritsa ntchito chipolopolo amapereka chithandizo kuitana mbedza ndi nthawi (zofanana ndi crontab mu ndondomeko yachikhalidwe), komanso chochitika chapadera paStartup. Zochitika zonsezi zikhoza kuphatikizidwa ndikupatsidwa mbedza imodzi.

Ndipo zina ziwiri za chipolopolo-operator:

1. Zikugwira asynchronously. Popeza chochitika cha Kubernetes (monga chinthu chopangidwa) chinalandiridwa, zochitika zina (monga chinthu chomwecho chikuchotsedwa) zikanatha kuchitika mgululi, ndipo mbedza ziyenera kuwerengera izi. Ngati mbedza idachitidwa ndi cholakwika, ndiye kuti mwachisawawa idzakhala kuitananso mpaka kumaliza bwino (khalidweli likhoza kusinthidwa).
2. Imatumiza kunja metrics kwa Prometheus, yomwe mungamvetsetse ngati chipolopolo-woyendetsa ntchito, fufuzani chiwerengero cha zolakwika pa mbedza iliyonse ndi kukula kwa mzere wamakono.

Kufotokozera mwachidule gawo ili la lipoti:

Kuyika zowonjezera

Kuti mugwire ntchito yabwino ndi Kubernetes, kufunika kokhazikitsa zowonjezera kudanenedwanso. Ndikuuzani za izi pogwiritsa ntchito chitsanzo cha njira ya kampani yathu momwe timachitira tsopano.

Tinayamba kugwira ntchito ndi Kubernetes ndi magulu angapo, chowonjezera chokha chomwe chinali Ingress. Zimafunika kukhazikitsidwa mosiyana mgulu lililonse, ndipo tidapanga masinthidwe angapo a YAML m'malo osiyanasiyana: chitsulo chopanda kanthu, AWS ...

Popeza panali magulu ambiri, panali masinthidwe ambiri. Kuphatikiza apo, tidakonza zosintha izi zokha, zomwe zidapangitsa kuti zikhale zosiyana kwambiri:

Kuti tikonze zonse, tidayamba ndi script (install-ingress.sh), yomwe idatenga ngati mkangano mtundu wamagulu omwe titumizireko, idapanga kasinthidwe koyenera kwa YAML ndikuyika Kubernetes.

Mwachidule, njira yathu yowonjezera komanso malingaliro okhudzana nawo anali motere:

• kuti mugwire ntchito ndi masanjidwe a YAML, injini ya template ikufunika (pamagawo oyamba izi ndi sed yosavuta);
• ndi kuwonjezeka kwa masango, kufunikira kosinthika kwadzidzidzi kunabwera (yankho loyambirira linali kuyika script ku Git, kuyisintha pogwiritsa ntchito cron ndikuyendetsa);
• script yofananayo idafunikira kwa Prometheus (install-prometheus.sh), komabe, ndizodziwikiratu chifukwa zimafunikira zambiri zolowera, komanso kusungidwa kwawo (m'njira yabwino - yapakati komanso pagulu), ndipo data ina (machinsinsi) imatha kupangidwa yokha:

  

• chiwopsezo chotulutsa china cholakwika kumagulu ochulukirachulukira chikukula mosalekeza, kotero tidazindikira kuti okhazikitsa (ie zolembedwa ziwiri: za Ingress ndi Prometheus) masitepe amafunikira (nthambi zingapo ku Git, ma crons angapo kuti asinthe molingana: masango okhazikika kapena oyesa);
• с kubectl apply zakhala zovuta kugwira nawo ntchito chifukwa sizikulengeza ndipo zimatha kupanga zinthu zokha, koma osapanga zisankho pamikhalidwe yawo / kuzichotsa;
• Tinali kusowa ntchito zina zomwe sitinazigwiritse ntchito panthawiyo:
  • kulamulira kwathunthu zotsatira za zosintha zamagulu,
  • kutsimikiza kodziwikiratu kwa magawo ena (zolowera pazolemba zoyika) kutengera zomwe zingapezeke pagulu (kutulukira),
  • chitukuko chake chomveka mu mawonekedwe a kutulukira mosalekeza.

Tinagwiritsa ntchito zonse zomwe tapezazi mkati mwa projekiti yathu ina - addon-wothandizira.

Wowonjezera-wothandizira

Zimatengera chipolopolo-oyendetsa chomwe chatchulidwa kale. Dongosolo lonse likuwoneka motere:

Zotsatirazi zikuwonjezedwa ku ndowe za shell-operator:

• values ​​yosungirako,
• Tchati cha helm,
• chigawo kuti imayang'anira sitolo yamtengo wapatali ndipo - pakasintha - apempha Helm kuti akonzenso tchati.

Choncho, tikhoza kuchitapo kanthu pazochitika ku Kubernetes, kuyambitsa mbedza, ndipo kuchokera ku ndowe iyi tikhoza kusintha zosungirako, pambuyo pake tchaticho chidzatsitsidwanso. Pachithunzi chotsatira, timalekanitsa ndowe ndi tchati kukhala chigawo chimodzi, chomwe timachitcha module:

Pakhoza kukhala ma module ambiri, ndipo kwa iwo timawonjezera mbedza zapadziko lonse lapansi, sitolo yapadziko lonse lapansi, ndi gawo lomwe limayang'anira sitolo yapadziko lonse lapansi.

Tsopano, chinachake chikachitika ku Kubernetes, tikhoza kuchitapo kanthu pogwiritsa ntchito mbedza yapadziko lonse ndikusintha chinachake mu sitolo yapadziko lonse. Kusintha kumeneku kudzazindikirika ndipo kupangitsa kuti ma module onse omwe ali mgululi atulutsidwe:

Dongosololi limakwaniritsa zofunikira zonse pakuyika zowonjezera zomwe zanenedwa pamwambapa:

• Helm ali ndi udindo pa templating ndi declarativeness.
• Nkhani ya auto-update inathetsedwa pogwiritsa ntchito mbedza yapadziko lonse, yomwe imapita ku registry pa ndandanda ndipo, ngati iwona chithunzi chatsopano cha dongosolo pamenepo, imatulutsa (ie "yokha").
• Kusungirako makonda mu cluster kumayendetsedwa pogwiritsa ntchito ConfigMap, yomwe ili ndi deta yoyambirira ya storages (pakuyambitsa iwo amalowetsedwa mu storages).
• Mavuto ndi kupanga mawu achinsinsi, kutulukira ndi kupeza mosalekeza anathetsedwa pogwiritsa ntchito mbedza.
• Masitepe amatheka chifukwa cha ma tag, omwe Docker amathandizira kunja kwa bokosi.
• Zotsatira zake zimawunikidwa pogwiritsa ntchito ma metric omwe tingamvetsetse momwe zilili.

Dongosolo lonseli limayendetsedwa ngati binary imodzi mu Go, yomwe imatchedwa addon-operator. Izi zimapangitsa kuti chithunzichi chiwoneke chosavuta:

Chigawo chachikulu mu chithunzichi ndi ma modules (zowonetsedwa mu imvi pansipa). Tsopano titha kulemba gawo lazowonjezera zofunikira ndi khama pang'ono ndikutsimikiza kuti zidzayikidwa mumagulu aliwonse, zidzasinthidwa ndikuyankha zochitika zomwe zimafunikira mumagulu.

"Flant" amagwiritsidwa ntchito addon-wothandizira pamagulu 70+ a Kubernetes. Mkhalidwe wapano - mtundu wa alpha. Tsopano tikukonzekera zolembedwa kuti titulutse beta, koma pakadali pano munkhokwe zitsanzo zilipo, pamaziko omwe mutha kupanga addon yanu.

Kodi ma module a addon-operator ndingapeze kuti? Kusindikiza laibulale yathu ndi gawo lotsatira kwa ife; tikukonzekera kuchita izi m'chilimwe.

Makanema ndi zithunzi

Kanema wakuchitako (~ mphindi 50):

Kafotokozedwe ka lipoti:

PS

Malipoti ena pabulogu yathu:

• «Databases ndi Kubernetes"; (Dmitry Stolyarov; Novembala 8, 2018 pa HighLoad++);
• «Monitoring ndi Kubernetes"; (Dmitry Stolyarov; May 28, 2018 ku RootConf);
• «CI/CD Best Practices ndi Kubernetes ndi GitLab"; (Dmitry Stolyarov; Novembala 7, 2017 pa HighLoad++);
• «Zomwe takumana nazo ndi Kubernetes mumapulojekiti ang'onoang'ono"; (Dmitry Stolyarov; June 6, 2017 ku RootConf).

Mukhozanso kukhala ndi chidwi ndi zofalitsa zotsatirazi:

• «Kuyambitsa zipolopolo: kupanga ogwiritsa ntchito Kubernetes kwakhala kosavuta".

Source: www.habr.com