Afrilu 27 a taron , A matsayin wani ɓangare na sashin "DevOps", an ba da rahoton "Aikin kai-kai da sarrafa kayan aiki a Kubernetes". Yana magana game da yadda zaku iya amfani da K8s don tabbatar da samun wadatar aikace-aikacenku da tabbatar da kololuwar aiki.
Bisa al'ada, mun ji daɗin gabatarwa (minti 44, ƙarin bayani fiye da labarin) da babban taƙaice a cikin sigar rubutu. Tafi!
Bari mu bincika batun rahoton kalma da kalma kuma mu fara daga ƙarshe.
Kubernetes
Bari mu ce muna da kwantena Docker akan mai masaukinmu. Don me? Don tabbatar da maimaitawa da keɓewa, wanda hakan yana ba da damar sauƙi da ingantaccen aiki, CI/CD. Muna da irin waɗannan motocin da yawa da kwantena.
Menene Kubernetes ke bayarwa a wannan yanayin?
- Mun daina tunani game da waɗannan inji kuma fara aiki tare da "girgije" tari na kwantena ko kwasfa (rukunin kwantena).
- Bugu da ƙari, ba ma yin tunani game da kwasfa ɗaya ɗaya, amma sarrafa ƙarinоmanyan kungiyoyi. Irin wannan babban matakin farko bari mu ce akwai samfuri don gudanar da wani aikin aiki, kuma ga adadin lokuta da ake buƙata don gudanar da shi. Idan muka canza samfuri daga baya, duk abubuwan zasu canza.
- Tare da taimakon API ɗin bayyanawa Maimakon aiwatar da jerin takamaiman umarni, muna bayyana "tsarin duniya" (a cikin YAML), wanda Kubernetes ya ƙirƙira. Kuma kuma: lokacin da bayanin ya canza, ainihin nunin sa shima zai canza.
Gudanar da albarkatun
CPU
Bari mu gudu nginx, php-fpm da mysql akan sabar. Waɗannan sabis ɗin za su sami ƙarin matakai da ke gudana, kowannensu yana buƙatar albarkatun kwamfuta:
(lambobin da ke kan nunin su ne "parrots", buƙatun kowane tsari don ikon sarrafa kwamfuta)
Don sauƙaƙe yin aiki tare da wannan, yana da ma'ana don haɗa matakai cikin ƙungiyoyi (misali, duk hanyoyin nginx zuwa rukuni ɗaya "nginx"). Hanya mai sauƙi kuma bayyananne don yin haka ita ce sanya kowace ƙungiya a cikin akwati:
Don ci gaba, kuna buƙatar tuna menene akwati (a cikin Linux). Bayyanar su ya yiwu godiya ga mahimman fasali guda uku a cikin kwaya, wanda aka aiwatar da shi da daɗewa: , и . Kuma ƙarin haɓakawa ya sami sauƙi ta hanyar wasu fasaha (ciki har da "harsashi" masu dacewa kamar Docker):
A cikin mahallin rahoton, muna sha'awar kawai rukuni-rukuni, saboda ƙungiyoyin sarrafawa sune ɓangare na ayyuka na kwantena (Docker, da dai sauransu) wanda ke aiwatar da sarrafa albarkatun. Tsarin da aka haɗa zuwa ƙungiyoyi, kamar yadda muke so, ƙungiyoyin sarrafawa ne.
Bari mu koma ga buƙatun CPU don waɗannan hanyoyin, kuma yanzu don ƙungiyoyin tsari:
(Na sake maimaita cewa duk lambobi bayyananne ne na buƙatun albarkatun)
A lokaci guda, CPU kanta yana da takamaiman albarkatu masu iyaka (a cikin misalin wannan shine 1000), wanda kowa zai iya rasa (jimilar buƙatun dukkan ƙungiyoyi shine 150+850+460=1460). Menene zai faru a wannan yanayin?
Kwayar tana fara rarraba albarkatu kuma tana yin shi “ainihin”, yana ba da adadin albarkatun iri ɗaya ga kowane rukuni. Amma a yanayin farko, akwai su fiye da yadda ake buƙata (333>150), don haka abin da ya wuce (333-150=183) ya kasance a ajiye, wanda kuma ana rarraba shi tsakanin wasu kwantena guda biyu:
A sakamakon haka: kwantena na farko yana da isasshen albarkatu, na biyu - ba shi da isasshen albarkatun, na uku - ba shi da isasshen kayan aiki. Wannan sakamakon ayyuka ne "Mai gaskiya" mai tsarawa a cikin Linux - . Ana iya daidaita aikinta ta amfani da aikin kaya masu nauyi kowanne daga cikin kwantena. Misali, kamar haka:
Bari mu dubi lamarin rashin albarkatun a cikin akwati na biyu (php-fpm). Ana rarraba duk albarkatun kwantena daidai tsakanin matakai. A sakamakon haka, aikin maigida yana aiki da kyau, amma duk ma'aikata suna raguwa, suna karɓar ƙasa da rabin abin da suke bukata:
Wannan shine yadda mai tsara tsarin CFS ke aiki. Za mu ƙara kiran ma'aunin nauyi da muka sanya wa kwantena buƙatun. Me yasa wannan yake haka - duba ƙarin.
Bari mu kalli yanayin gaba daya daga wancan bangaren. Kamar yadda ka sani, duk hanyoyi suna kaiwa zuwa Roma, kuma a cikin yanayin kwamfuta, zuwa CPU. CPU ɗaya, ayyuka da yawa - kuna buƙatar hasken zirga-zirga. Hanya mafi sauƙi don sarrafa albarkatu ita ce "hasken zirga-zirga": sun ba da tsari ɗaya ƙayyadadden lokacin isa ga CPU, sannan na gaba, da sauransu.
Ana kiran wannan hanya mai wuyar ƙima (Hard iyaka). Bari mu tuna da shi kawai kamar yadda iyaka. Koyaya, idan kun rarraba iyaka ga duk kwantena, matsala ta taso: mysql yana tuƙi a hanya kuma a wani lokaci buƙatar CPU ta ƙare, amma duk sauran hanyoyin ana tilasta su jira har sai CPU. zaman banza.
Bari mu koma ga Linux kernel da mu'amalarsa da CPU - cikakken hoton shine kamar haka:
cgroup yana da saituna guda biyu - ainihin waɗannan su ne "ƙarƙasa" guda biyu masu sauƙi waɗanda ke ba ku damar ƙayyade:
- nauyi ga akwati (buƙatun) shine hannun jari;
- kashi na jimlar lokacin CPU don aiki akan ayyukan kwantena (iyaka) shine quota.
Yadda za a auna CPU?
Akwai hanyoyi daban-daban:
- Abin da aku, Babu wanda ya sani - kuna buƙatar yin shawarwari kowane lokaci.
- Abin sha'awa bayyananne, amma dangi: 50% na uwar garken da 4 cores kuma tare da 20 cores ne gaba daya daban-daban abubuwa.
- Kuna iya amfani da waɗanda aka riga aka ambata kaya masu nauyi, wanda Linux ya sani, amma kuma dangi ne.
- Mafi dacewa zaɓi shine auna albarkatun kwamfuta a ciki seconds. Wadancan. a cikin daƙiƙa na lokacin sarrafawa dangane da daƙiƙa na ainihin lokacin: 1 seconds na lokacin sarrafawa an ba da shi a kowane sakan 1 na gaske - wannan shine tushen CPU guda ɗaya.
Don a sami sauƙin magana, suka fara auna kai tsaye a ciki kwaya, ma'ana da su lokaci guda CPU dangane da ainihin. Tun da Linux yana fahimtar ma'auni, amma ba lokacin CPU/cores sosai ba, ana buƙatar wata hanya don fassara daga ɗayan zuwa wancan.
Bari mu yi la'akari da misali mai sauƙi tare da uwar garken tare da 3 CPU cores, inda za a ba da ma'auni guda uku ma'auni (500, 1000 da 1500) waɗanda aka sauƙaƙe su canza zuwa sassan da aka ba su daidai (0,5, 1 da 1,5).
Idan ka ɗauki uwar garken na biyu, inda za a sami ninki biyu (6), kuma ka sanya kwasfa iri ɗaya a wurin, za a iya ƙididdige rabon muryoyin cikin sauƙi ta hanyar ninka ta 2 (1, 2 da 3, bi da bi). Amma wani lokaci mai mahimmanci yana faruwa lokacin da kwasfa na huɗu ya bayyana akan wannan uwar garken, wanda nauyinsa, don dacewa, zai zama 3000. Yana ɗaukar wani ɓangare na albarkatun CPU (rabin cores), kuma ga sauran kwas ɗin ana sake ƙididdige su (rabi):
Kubernetes da albarkatun CPU
A cikin Kubernetes, yawancin albarkatun CPU ana auna su milliadrax, i.e. 0,001 cores ana ɗaukar su azaman ma'aunin tushe. (Irin abu ɗaya a cikin kalmomin Linux / ƙungiyoyi ana kiransa rabon CPU, kodayake, ƙari daidai, 1000 millicores = 1024 CPU hannun jari.) K8s yana tabbatar da cewa baya sanya kwasfan fayiloli akan uwar garken fiye da yadda ake samun albarkatun CPU don jimlar ma'aunin kowane kwasfa.
Ta yaya hakan ke faruwa? Lokacin da kuka ƙara uwar garken zuwa gungu na Kubernetes, ana ba da rahoton adadin adadin CPU ɗin da yake da shi. Kuma lokacin ƙirƙirar sabon kwafsa, mai tsara tsarin Kubernetes ya san adadin adadin wannan kwaf ɗin zai buƙaci. Don haka, za a sanya kwas ɗin zuwa uwar garken inda akwai isassun muryoyi.
Me zai faru idan ba An ƙayyadadden buƙatun (watau kwaf ɗin ba shi da ƙayyadadden adadin muryoyin da yake buƙata)? Bari mu gano yadda Kubernetes gabaɗaya ke ƙirga albarkatun.
Don kwafsa zaka iya ƙayyade buƙatun biyu (mai tsara tsarin CFS) da iyaka (tuna da hasken zirga-zirga?):
- Idan an ayyana su daidai, to, an sanya kwaf ɗin ajin QoS tabbace. Wannan adadin muryoyin da ake samu koyaushe yana da garanti.
- Idan buƙatar ta kasance ƙasa da iyaka - QoS class fashewa. Wadancan. Muna tsammanin kwafsa, alal misali, koyaushe yana amfani da 1 core, amma wannan ƙimar ba ta iyakancewa ba ce: wani lokaci pod zai iya amfani da ƙarin (lokacin da uwar garken yana da albarkatun kyauta don wannan).
- Akwai kuma QoS aji mafi kyau kokarin - ya haɗa da ƙwanƙwasa waɗanda ba a ƙayyade buƙatar su ba. Ana ba su albarkatu na ƙarshe.
Waƙwalwa
Tare da ƙwaƙwalwar ajiya, halin da ake ciki yana kama, amma dan kadan daban-daban - bayan haka, yanayin waɗannan albarkatun ya bambanta. Gabaɗaya, kwatankwacinsu shine kamar haka:
Bari mu ga yadda ake aiwatar da buƙatun a ƙwaƙwalwar ajiya. Bari kwas ɗin su rayu akan uwar garken, suna canza amfani da ƙwaƙwalwar ajiya, har sai ɗayansu ya yi girma har ya ƙare da ƙwaƙwalwar ajiya. A wannan yanayin, mai kashe OOM ya bayyana kuma ya kashe mafi girman tsari:
Wannan ba koyaushe ya dace da mu ba, don haka yana yiwuwa a tsara hanyoyin da suke da mahimmanci a gare mu kuma bai kamata a kashe su ba. Don yin wannan, yi amfani da siga oom_score_adj.
Bari mu koma azuzuwan QoS na CPU kuma mu zana kwatance tare da ƙimar oom_score_adj waɗanda ke ƙayyade fifikon amfani da ƙwaƙwalwar ajiya don kwasfa:
- Mafi ƙarancin ƙimar oom_score_adj don kwafsa - -998 - yana nufin cewa yakamata a kashe irin wannan kwaf ɗin a ƙarshe, wannan. tabbace.
- Mafi girma - 1000 - shine mafi kyau kokarin, irin wannan kwas ɗin ana fara kashe su.
- Don lissafta ragowar ƙimar (fashewa) akwai wata dabara, wanda asalinsa ya taso zuwa ga cewa yawan albarkatun da kwafsa ya nema, ba za a iya kashe shi ba.
Na biyu "karkacewa" - iyaka_in_bytes - don iyaka. Tare da shi, komai ya fi sauƙi: kawai muna sanya matsakaicin adadin ƙwaƙwalwar da aka bayar, kuma a nan (ba kamar CPU ba) babu batun yadda ake auna shi (ƙwaƙwalwar ajiya).
Jimlar
Ana ba da kowane kwasfa a Kubernetes requests и limits - duka sigogi don CPU da ƙwaƙwalwar ajiya:
- bisa ga buƙatun, mai tsara tsarin Kubernetes yana aiki, wanda ke rarraba kwasfan fayiloli tsakanin sabobin;
- bisa ga dukkan sigogi, an ƙaddara ajin QoS na kwafsa;
- Ana ƙididdige ma'auni na dangi bisa buƙatun CPU;
- an saita mai tsara tsarin CFS bisa buƙatun CPU;
- An saita killer OOM bisa ga buƙatun ƙwaƙwalwar ajiya;
- an saita "hasken zirga-zirga" bisa iyakokin CPU;
- Dangane da iyakokin ƙwaƙwalwar ajiya, ana saita iyaka don ƙungiyar.
Gabaɗaya, wannan hoton yana amsa duk tambayoyin game da yadda yawancin sarrafa albarkatun ke faruwa a Kubernetes.
Gyaran atomatik
K8s cluster-autoscaler
Bari mu yi tunanin cewa an riga an shagaltar da duka rukunin kuma ana buƙatar ƙirƙirar sabon kwafsa. Yayin da kwaf ɗin ba zai iya bayyana ba, yana rataye a matsayi a lokacin. Domin ya bayyana, za mu iya haɗa sabon uwar garken zuwa gungu ko... shigar da cluster-autoscaler, wanda zai yi mana: odar na'ura mai mahimmanci daga mai samar da girgije (ta amfani da buƙatar API) kuma haɗa shi zuwa gungu. , bayan haka za a ƙara kwafsa.
Wannan shine autoscaling na Kubernetes cluster, wanda ke aiki mai girma (a cikin kwarewarmu). Koyaya, kamar sauran wurare, akwai wasu nuances a nan ...
Muddin mun ƙara girman gungu, komai yana da kyau, amma abin da ke faruwa lokacin tari ya fara 'yantar da kansa? Matsalar ita ce ƙwanƙwasa ƙaura (don 'yantar da runduna) yana da wahala ta fasaha da tsada ta fuskar albarkatu. Kubernetes yana amfani da wata hanya ta daban.
Yi la'akari da gungu na sabobin 3 waɗanda ke da Ƙaddamarwa. Yana da pods 6: yanzu akwai 2 ga kowane uwar garken. Don wasu dalilai mun so mu kashe ɗaya daga cikin sabar. Don yin wannan za mu yi amfani da umarnin kubectl drainwanda:
- zai haramta aika sabbin kwasfan fayiloli zuwa wannan uwar garken;
- zai share kwasfan fayiloli a kan uwar garke.
Tunda Kubernetes ke da alhakin kiyaye adadin kwasfa (6), shi kawai zai sake halitta su a wasu nodes, amma ba akan wanda aka kashe ba, tunda an riga an yi masa alama a matsayin babu shi don ɗaukar sabbin kwasfa. Wannan babban makaniki ne ga Kubernetes.
Duk da haka, akwai kuma nuance a nan ma. A cikin irin wannan yanayin, don StatefulSet (maimakon Aiki), ayyukan zasu bambanta. Yanzu mun riga mun sami ingantaccen aikace-aikacen - alal misali, kwasfan fayiloli guda uku tare da MongoDB, ɗayan ɗayan yana da wasu nau'ikan matsala (bayanan sun lalace ko wani kuskuren da ke hana kwaf ɗin farawa daidai). Kuma mun sake yanke shawarar kashe uwar garken guda ɗaya. Me zai faru?
MongoDB iya mutu saboda yana buƙatar ƙididdiga: don gungu na shigarwa uku, aƙalla biyu dole ne suyi aiki. Duk da haka, wannan ba faruwa - godiya ga Kudiddigar Kuɗi. Wannan siga yana ƙayyade mafi ƙarancin adadin da ake buƙata na kwas ɗin aiki. Sanin cewa ɗaya daga cikin kwas ɗin MongoDB baya aiki, da ganin cewa PodDisruptionBudget an saita don MongoDB minAvailable: 2, Kubernetes ba zai ƙyale ku share kwasfa ba.
Layin ƙasa: domin motsi (kuma a zahiri, sake ƙirƙirar) na kwasfan fayiloli suyi aiki daidai lokacin da aka fitar da gungu, kuna buƙatar saita PodDisruptionBudget.
Ƙimar kai tsaye
Bari mu yi la’akari da wani yanayi. Akwai aikace-aikacen da ke gudana azaman Ƙaddamarwa a Kubernetes. Traffic zirga-zirga zuwa ga kwasfansa (misali, akwai uku daga cikinsu), kuma muna auna wani ma'ana a cikinsu (ce, CPU load). Lokacin da nauyin ya ƙaru, muna yin rikodin wannan akan jadawalin kuma muna ƙara adadin kwasfa don rarraba buƙatun.
A yau a cikin Kubernetes wannan baya buƙatar yin da hannu: haɓaka / raguwa ta atomatik a cikin adadin kwas ɗin an saita shi gwargwadon ƙimar ma'aunin nauyi.
Manyan tambayoyin anan su ne: me daidai gwargwado и yadda ake fassara samu dabi'u (don yanke shawara kan canza adadin kwasfa). Kuna iya aunawa da yawa:
Yadda ake yin wannan ta hanyar fasaha - tattara ma'auni, da sauransu. - Na yi magana daki-daki a cikin rahoton game da . Kuma babban shawara don zaɓar mafi kyawun sigogi shine gwaji!
Akwai (Saturation na Amfani da Kurakurai), ma'anarsa kamar haka. A kan wane tushe yake da ma'ana don ma'auni, misali, php-fpm? Dangane da gaskiyar cewa ma'aikata suna kurewa, wannan shine amfani. Kuma idan ma'aikatan sun ƙare kuma ba a karɓi sabon haɗin gwiwa ba, wannan ya riga ya kasance jikewa. Duk waɗannan sigogi dole ne a auna su, kuma dangane da ƙimar, dole ne a aiwatar da sikelin.
Maimakon a ƙarshe
Rahoton yana da ci gaba: game da ma'auni a tsaye da kuma yadda za a zaɓi albarkatun da suka dace. Zan yi magana game da wannan a cikin bidiyo na gaba - yi subscribing domin kada ku yi kuskure!
Bidiyo da nunin faifai
Bidiyo daga wasan kwaikwayon (minti 44):
Gabatar da rahoton:
PS
Sauran rahotanni game da Kubernetes akan shafinmu:
- «» (Andrey Polovov; Afrilu 8, 2019 akan Saint HighLoad++);
- «» (Dmitry Stolyarov; Nuwamba 8, 2018 akan HighLoad++);
- «» (Dmitry Stolyarov; Mayu 28, 2018 a RootConf);
- «» (Dmitry Stolyarov; Nuwamba 7, 2017 akan HighLoad++);
- «» (Dmitry Stolyarov; Yuni 6, 2017 a RootConf).
source: www.habr.com