🥇 Fadadawa da haɓaka Kubernetes (rahoton bita da bidiyo)

Afrilu 8 a taron Saint HighLoad++ 2019, A matsayin wani ɓangare na sashin "DevOps da Ayyuka", an ba da rahoton "Faɗawa da haɓaka Kubernetes", a cikin ƙirƙirar wanda ma'aikata uku na kamfanin Flant suka shiga. A ciki, muna magana game da yanayi da yawa waɗanda muke son fadadawa da haɓaka damar Kubernetes, amma waɗanda ba mu sami mafita mai sauƙi ba. Muna da hanyoyin da suka dace ta hanyar ayyukan Buɗaɗɗen Madogararsa, kuma wannan jawabin kuma an sadaukar da shi gare su.

Bisa al'ada, mun ji daɗin gabatarwa bidiyon rahoton (minti 50, ƙarin bayani fiye da labarin) da babban taƙaitawa a cikin sigar rubutu. Tafi!

Core da ƙari a cikin K8s

Kubernetes yana canza masana'antu da hanyoyin gudanarwa waɗanda aka daɗe da kafawa:

Godiya gare shi abstractions, Ba mu ƙara yin aiki tare da ra'ayoyi kamar kafa saiti ko gudanar da umarni (Chef, Mai yiwuwa...), amma muna amfani da rukunin kwantena, ayyuka, da sauransu.
Za mu iya shirya aikace-aikace ba tare da tunani game da nuances na takamaiman site, wanda za a kaddamar da shi: karfe maras tushe, gajimare na daya daga cikin masu samarwa, da dai sauransu.
Tare da K8s ba ku taɓa samun damar samun dama ba mafi kyawun ayyuka akan tsara abubuwan more rayuwa: dabarun ƙira, warkar da kai, haƙurin kuskure, da sauransu.

Duk da haka, ba shakka, duk abin da ba shi da kyau sosai: Kubernetes kuma ya kawo sabon kalubale.

Kubernetes ba haɗakarwa ce da ke magance duk matsalolin duk masu amfani. Ainihin Kubernetes yana da alhakin kawai saitin mafi ƙarancin ayyuka masu mahimmanci waɗanda ke cikin su kowa da kowa tari:

Kubernetes core yana bayyana ainihin saiti na farko don haɗa kwantena, sarrafa zirga-zirga, da sauransu. Mun yi magana game da su dalla-dalla rahoton shekaru 2 da suka gabata.

A gefe guda, K8s yana ba da babbar dama don faɗaɗa ayyukan da ake da su, waɗanda ke taimakawa rufe wasu - takamaiman - buƙatun mai amfani. Ƙarin zuwa Kubernetes alhakin masu gudanar da gungu ne, waɗanda dole ne su shigar da kuma daidaita duk abin da ya dace don samun gungu "a cikin siffar da ta dace" [don magance matsalolin su na musamman]. Wane irin kari ne waɗannan? Bari mu kalli wasu misalai.

Misalai na add-ons

Bayan shigar da Kubernetes, zamu iya mamakin cewa sadarwar da ke da mahimmanci don hulɗar kwasfa biyu a cikin kumburi da tsakanin nodes ba ya aiki da kansa. Kwayar Kubernetes baya bada garantin haɗin da ake buƙata; maimakon haka, yana ƙayyade hanyar sadarwa dubawa (CNI) don ƙara-kan ɓangare na uku. Dole ne mu shigar da ɗayan waɗannan add-kan, waɗanda za su ɗauki alhakin daidaitawar hanyar sadarwa.

Misali na kusa shine hanyoyin adana bayanai (faifan gida, na'urar toshe hanyar sadarwa, Ceph...). Da farko sun kasance a cikin ainihin, amma tare da zuwan CSI halin da ake ciki ya canza zuwa wani abu mai kama da wanda aka riga aka bayyana: dubawa yana cikin Kubernetes, kuma aiwatar da shi yana cikin sassan ɓangare na uku.

Sauran misalan sun haɗa da:

Ingress- masu sarrafawa (duba nazarin su a cikin labarin mu na kwanan nan).
cert-sarrafa:
Masu aiki babban aji ne na add-ons (wanda ya haɗa da mai sarrafa da aka ambata), suna ayyana (s) na farko da masu sarrafawa. Ma'anar aikin su yana iyakance ne kawai ta tunaninmu kuma yana ba mu damar kunna kayan aikin da aka yi da shirye-shiryen (alal misali, DBMS) a cikin primitives, waɗanda suke da sauƙin aiki tare (fiye da saitin kwantena da saitunan su). An rubuta adadi mai yawa na masu aiki - ko da yawancinsu ba su riga sun shirya don samarwa ba, lokaci ne kawai:
Ma'auni - wani kwatanci na yadda Kubernetes ya raba mahaɗin (Metrics API) daga aiwatarwa (ƙarancin ɓangare na uku kamar Adaftar Prometheus, Wakilin cluster Datadog ...).
domin saka idanu da kididdiga, inda a aikace muke bukata ba kawai Prometheus dan Grafana, amma kuma kube-state-metrics, node-exporter, da dai sauransu.

Kuma wannan ba cikakken jerin abubuwan kari bane... Misali, a kamfanin Flant da muke girka a halin yanzu kari 29 (duk wanda ke haifar da jimlar 249 Kubernetes abubuwa). A taƙaice, ba za mu iya ganin rayuwar tari ba tare da ƙari ba.

Autom

An ƙera masu aiki don sarrafa ayyukan yau da kullun waɗanda muke ci karo da su kowace rana. Anan akwai misalan rayuwa na gaske waɗanda rubuta ma'aikaci zai zama kyakkyawan mafita:

Akwai keɓaɓɓen (watau buƙatar shiga) rajista tare da hotuna don aikace-aikacen. Ana ɗauka cewa an ba wa kowane kwasfa asiri na musamman wanda ke ba da damar tantancewa a cikin rajista. Ayyukanmu shine tabbatar da cewa an gano wannan sirrin a cikin sararin sunan don kwas ɗin su iya sauke hotuna. Ana iya samun aikace-aikacen da yawa (kowannensu yana buƙatar sirri), kuma yana da amfani don sabunta sirrin kansu akai-akai, don haka zaɓin shimfiɗa asirin da hannu yana kawar da shi. Wannan shi ne inda mai aiki ya zo don ceto: mun ƙirƙiri mai sarrafawa wanda zai jira sararin sunan ya bayyana kuma, bisa ga wannan taron, zai ƙara wani asiri ga sunan suna.
Bari ta tsohuwa samun dama daga kwasfan fayiloli zuwa Intanet an haramta. Amma wani lokacin ana iya buƙata: yana da ma'ana don tsarin izinin samun damar yin aiki kawai, ba tare da buƙatar takamaiman ƙwarewa ba, misali, ta gaban wata tambari a cikin sunan. Ta yaya ma'aikacin zai iya taimaka mana a nan? An ƙirƙiri mai sarrafawa wanda ke jiran alamar ta bayyana a cikin sarari suna kuma ƙara manufofin da suka dace don samun damar Intanet.
Irin wannan yanayin: a ce muna buƙatar ƙara wani abu kazanta, idan yana da lakabi iri ɗaya (tare da wani nau'in prefix). Ayyuka tare da mai aiki a bayyane suke...

A cikin kowane gungu, dole ne a warware ayyukan yau da kullun, kuma dama ana iya yin wannan ta amfani da masu aiki.

Takaita duk labaran da aka bayyana, mun kai ga ƙarshe cewa don jin daɗin aiki a Kubernetes kuna buƙata: A) shigar add-ons, b) haɓaka masu aiki (don warware ayyukan admin na yau da kullun).

Yadda ake rubuta sanarwa don Kubernetes?

Gabaɗaya, tsarin yana da sauƙi:

... amma sai ya zama cewa:

Kubernetes API wani abu ne wanda ba shi da mahimmanci wanda ke ɗaukar lokaci mai yawa don ƙwarewa;
Programming kuma ba na kowa bane (an zaɓi yaren Go a matsayin yaren da aka fi so saboda akwai tsari na musamman akansa - Mai aiki SDK);
Halin yana kama da tsarin kanta.

Kasa line: don rubuta mai sarrafawa (operator) ya kamata kashe gagarumin albarkatu don nazarin kayan aiki. Wannan zai zama barata ga "manyan" masu aiki - a ce, don MySQL DBMS. Amma idan muka tuna da misalan da aka bayyana a sama (bayyana asirin, samun damar shiga Intanet ...), wanda kuma muna so mu yi daidai, to, za mu fahimci cewa ƙoƙarin da aka yi zai wuce sakamakon da muke bukata a yanzu:

Gabaɗaya, matsala ta taso: kashe albarkatu masu yawa kuma nemo kayan aikin da ya dace don rubuta maganganun, ko kuma yi ta hanyar da ta gabata (amma da sauri). Don warware shi - don samun sulhu tsakanin waɗannan matsananci - mun ƙirƙiri namu aikin: mai aiki da harsashi (duba kuma nasa sanarwar kwanan nan na zamba).

Shell-operator

Ta yaya yake aiki? Tarin yana da kwasfa mai ɗauke da Go binary tare da mai sarrafa harsashi. Kusa da shi akwai saitin ƙugiya (ƙarin bayani game da su - duba ƙasa). Shell-operator kanta yana biyan kuɗi zuwa wasu abubuwan da suka faru a cikin Kubernetes API, akan abin da ya faru wanda ya ƙaddamar da ƙugiya masu dacewa.

Ta yaya ma'aikacin harsashi ya san waɗanne ƙugiya don kiran waɗanne abubuwan da suka faru? Ana watsa wannan bayanin zuwa ma'aikacin harsashi ta hanyar ƙugiya da kansu, kuma suna yin hakan cikin sauƙi.

ƙugiya rubutun Bash ne ko kowane fayil ɗin da za a iya aiwatarwa wanda ya karɓi hujja guda ɗaya --config kuma ya amsa da JSON. Na ƙarshe ya ƙayyade waɗanne abubuwa ne ke sha'awar shi da kuma waɗanne abubuwan da suka faru (na waɗannan abubuwan) yakamata a amsa su:

Zan kwatanta aiwatarwa akan mai sarrafa harsashi na ɗaya daga cikin misalan mu - ɓarna asirin don samun damar yin rajista mai zaman kansa tare da hotunan aikace-aikacen. Ya ƙunshi matakai biyu.

Ayyuka: 1. Rubuta ƙugiya

Da farko, a cikin ƙugiya za mu aiwatar --config, yana nuna cewa muna sha'awar wuraren suna, kuma musamman, lokacin halittar su:

[[ $1 == "--config" ]] ; then
  cat << EOF
{
  "onKubernetesEvent": [
    {
      "kind": "namespace",
      "event": ["add"]
    }
  ]
}
EOF
…

Yaya dabara zata yi kama? Hakanan mai sauqi qwarai:

…
else
  createdNamespace=$(jq -r '.[0].resourceName' $BINDING_CONTEXT_PATH)
  kubectl create -n ${createdNamespace} -f - << EOF
Kind: Secret
...
EOF
fi

Mataki na farko shine gano ko wane suna ne aka ƙirƙira, na biyu kuma shine ƙirƙirar shi ta amfani da shi kubectl sirrin wannan sunan.

Ayyuka: 2. Haɗa hoton

Duk abin da ya rage shi ne wuce ƙugiya da aka ƙirƙira zuwa ma'aikacin harsashi - yadda za a yi haka? Ma'aikacin harsashi da kansa ya zo a matsayin hoton Docker, don haka aikinmu shine ƙara ƙugiya zuwa kundin adireshi na musamman a cikin wannan hoton:

FROM flant/shell-operator:v1.0.0-beta.1
ADD my-handler.sh /hooks

Abin da ya rage shi ne a hada shi a tura shi.

$ docker build -t registry.example.com/my-operator:v1 .
$ docker push registry.example.com/my-operator:v1

Taɓawar ƙarshe shine a tura hoton zuwa gungu. Don yin wannan, bari mu rubuta girke:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-operator
spec:
  template:
    spec:
      containers:
      - name: my-operator
        image: registry.example.com/my-operator:v1 # 1
      serviceAccountName: my-operator              # 2

Akwai abubuwa biyu da ya kamata a kula da su:

nuni na sabon hoton da aka halicce;
Wannan ɓangaren tsarin ne wanda (aƙaƙƙarfan) yana buƙatar haƙƙoƙi don biyan kuɗi zuwa abubuwan da ke faruwa a cikin Kubernetes da kuma ware asirin ga wuraren suna, don haka muna ƙirƙirar AccountAccount (da saitin dokoki) don ƙugiya.

Sakamakon - mun magance matsalar mu dangi don Kubernetes ta hanyar da ke haifar da ma'aikaci don lalata asirin.

Wasu fasalulluka na harsashi

Don iyakance abubuwan nau'in da kuka zaɓa waɗanda ƙugiya za ta yi aiki da su, ana iya tace su, zaɓi bisa ga wasu alamomi (ko amfani da matchExpressions):

"onKubernetesEvent": [
  {
    "selector": {
      "matchLabels": {
        "foo": "bar",
       },
       "matchExpressions": [
         {
           "key": "allow",
           "operation": "In",
           "values": ["wan", "warehouse"],
         },
       ],
     }
     …
  }
]

An bayar tsarin cirewa, wanda - ta amfani da tace jq - yana ba ku damar canza manyan abubuwan JSON zuwa ƙananan, inda kawai waɗannan sigogi suka rage waɗanda muke son saka idanu don canje-canje.

Lokacin da aka kira ƙugiya, mai sarrafa harsashi ya wuce shi bayanan abu, wanda za a iya amfani dashi ga kowace bukata.

Abubuwan da ke haifar da ƙugiya ba su iyakance ga abubuwan Kubernetes ba: mai sarrafa harsashi yana ba da tallafi ga kiran ƙugiya ta lokaci (mai kama da crontab a cikin tsarin al'ada), da kuma wani taron na musamman onStartup. Duk waɗannan abubuwan ana iya haɗa su kuma sanya su zuwa ƙugiya ɗaya.

Da kuma ƙarin fasali guda biyu na mai aikin harsashi:

Yana aiki asynchronously. Tun lokacin da aka karɓi taron Kubernetes (kamar wani abu da ake ƙirƙira), wasu abubuwan da suka faru (kamar abu ɗaya da ake sharewa) na iya faruwa a cikin gungu, kuma ƙugiya suna buƙatar lissafin wannan. Idan an kashe ƙugiya tare da kuskure, to ta tsohuwa zai kasance sake kira har sai an yi nasara (wannan hali za a iya canza).
Yana fitarwa awo don Prometheus, wanda zaku iya fahimtar ko mai sarrafa harsashi yana aiki, gano adadin kurakurai ga kowane ƙugiya da girman layin yanzu.

Don taƙaita wannan ɓangaren rahoton:

Ana shigar da add-ons

Don jin daɗin aiki tare da Kubernetes, an kuma ambaci buƙatar shigar da ƙari. Zan gaya muku game da shi ta amfani da misalin hanyar kamfaninmu zuwa yadda muke yi yanzu.

Mun fara aiki tare da Kubernetes tare da gungu da yawa, ƙari kawai wanda shine Ingress. Ana buƙatar shigar da shi daban a cikin kowane gungu, kuma mun yi saitunan YAML da yawa don mahalli daban-daban: ƙarfe mara ƙarfi, AWS ...

Da yake akwai ƙarin gungu, an sami ƙarin daidaitawa. Bugu da kari, mun inganta wadannan jeri da kansu, a sakamakon abin da suka zama quite iri-iri:

Don tsara komai, mun fara da rubutun (install-ingress.sh), wanda ya ɗauki a matsayin hujja nau'in gungu wanda za mu tura shi, ya haifar da daidaitaccen tsarin YAML kuma ya mirgine shi zuwa Kubernetes.

A taqaice dai tafarkinmu na gaba da dalilan da ke tattare da shi sun kasance kamar haka;

don yin aiki tare da saitunan YAML, ana buƙatar injin samfuri (a farkon matakan wannan sed mai sauƙi);
tare da karuwar adadin gungu, buƙatar sabuntawa ta atomatik ta zo (mafilar farko ita ce sanya rubutun a cikin Git, sabunta shi ta amfani da cron da gudanar da shi);
an buƙaci irin wannan rubutun don Prometheus (install-prometheus.sh), duk da haka, ya zama sananne saboda gaskiyar cewa yana buƙatar ƙarin bayanan shigarwa, da kuma ajiyar su (ta hanya mai kyau - a tsakiya da kuma a cikin gungu), kuma ana iya samar da wasu bayanai (kalmomin sirri) ta atomatik:
Haɗarin fitar da wani abu ba daidai ba ga yawan tarin gungu yana ƙaruwa koyaushe, don haka mun gane cewa masu sakawa (watau rubutun biyu: na Ingress da Prometheus) ana buƙatar tsari (rassoshi da yawa a Git, cron da yawa don sabunta su a cikin daidaitattun: barga ko gungu na gwaji);
с kubectl apply ya zama da wuya a yi aiki tare da shi saboda ba sanarwa ba ne kuma yana iya ƙirƙirar abubuwa kawai, amma ba yanke shawara game da matsayin su / share su ba;
Mun rasa wasu ayyuka da ba mu aiwatar da su ba a wancan lokacin:
- cikakken iko akan sakamakon sabunta tari,
- Ƙayyadaddun bayanai ta atomatik na wasu sigogi (shigarwa don rubutun shigarwa) dangane da bayanan da za a iya samu daga gungu (ganowa),
- haɓakar basirarsa ta hanyar ci gaba da ganowa.

Mun aiwatar da duk wannan tarin gogewa a cikin tsarin sauran aikin mu - adon-operator.

Addon-operator

Ya dogara ne akan mai aikin harsashi da aka ambata. Dukkan tsarin yayi kama da haka:

Ana ƙara mai zuwa zuwa ƙugiya mai aiki da harsashi:

adana dabi'u,
Tsarin Helm,
bangaren cewa yana lura da ma'ajiyar dabi'u kuma - idan akwai wani canje-canje - ya nemi Helm ya sake mirgine ginshiƙi.

Don haka, za mu iya amsa wani taron a Kubernetes, ƙaddamar da ƙugiya, kuma daga wannan ƙugiya za mu iya yin canje-canje ga ajiya, bayan haka za a sake sauke ginshiƙi. A cikin zanen da aka samu, muna raba saitin ƙugiya da ginshiƙi zuwa kashi ɗaya, wanda muke kira module:

Za a iya samun nau'o'i da yawa, kuma a gare su muna ƙara ƙugiya na duniya, kantin sayar da kimar duniya, da kuma ɓangaren da ke kula da wannan kantin sayar da duniya.

Yanzu, lokacin da wani abu ya faru a Kubernetes, za mu iya amsawa da shi ta amfani da ƙugiya ta duniya kuma mu canza wani abu a cikin kantin sayar da duniya. Za a lura da wannan canjin kuma zai sa a fitar da duk samfuran da ke cikin gungu:

Wannan makirci ya cika duk buƙatun don shigar da add-ons waɗanda aka bayyana a sama:

Helm yana da alhakin yin samfuri da bayyanawa.
An warware batun sabunta ta atomatik ta amfani da ƙugiya na duniya, wanda ke zuwa wurin yin rajista a kan jadawalin kuma, idan ya ga sabon hoton tsarin a can, ya fitar da shi (watau "kansa").
Ana aiwatar da saitunan adanawa a cikin tari ta amfani da ConfigMap, wanda ya ƙunshi bayanan farko don ma'ajiyar (a lokacin farawa ana loda su cikin ma'ajiyar).
Matsaloli tare da samar da kalmar sirri, ganowa da ci gaba da ganowa an magance su ta amfani da ƙugiya.
Ana samun ci gaba godiya ga alamun, wanda Docker ke tallafawa daga cikin akwatin.
Ana lura da sakamakon ta amfani da ma'auni wanda za mu iya fahimtar matsayi.

Ana aiwatar da wannan gabaɗayan tsarin ta hanyar binary guda ɗaya a cikin Go, wanda ake kira addon-operator. Wannan yana sa zane ya zama mai sauƙi:

Babban abin da ke cikin wannan zane shine saitin kayayyaki (an haskaka da launin toka a ƙasa). Yanzu za mu iya rubuta wani tsari don ƙarawa da ake buƙata tare da ɗan ƙoƙari kuma tabbatar da cewa za a shigar da shi a cikin kowane gungu, za a sabunta shi kuma ya amsa abubuwan da yake bukata a cikin gungu.

Ana amfani da "Flant". adon-operator akan 70+ Kubernetes gungu. Halin halin yanzu - sigar alfa. Yanzu muna shirya takaddun don sakin beta, amma a yanzu a cikin ma'ajiyar misalai akwai, akan abin da zaku iya ƙirƙirar addon ku.

A ina zan iya samun kayan aikin addon-operator? Buga ɗakin karatun mu shine mataki na gaba a gare mu; muna shirin yin hakan a lokacin rani.

Bidiyo da nunin faifai

Bidiyo daga wasan kwaikwayon (~ mintuna 50):

Gabatar da rahoton:

PS

Sauran rahotanni a kan shafinmu:

«Databases da Kubernetes"; (Dmitry Stolyarov; Nuwamba 8, 2018 akan HighLoad++);
«Kulawa da Kubernetes"; (Dmitry Stolyarov; Mayu 28, 2018 a RootConf);
«Mafi kyawun Ayyuka na CI/CD tare da Kubernetes da GitLab"; (Dmitry Stolyarov; Nuwamba 7, 2017 akan HighLoad++);
«Kwarewarmu tare da Kubernetes a cikin ƙananan ayyuka"; (Dmitry Stolyarov; Yuni 6, 2017 a RootConf).

Hakanan kuna iya sha'awar littattafai masu zuwa:

«Gabatar da mai sarrafa harsashi: ƙirƙirar masu aiki don Kubernetes kawai ya sami sauƙi".

source: www.habr.com

Fadadawa da haɓaka Kubernetes (rahoton bita da bidiyo)