Zomwe takumana nazo pakupanga dalaivala wa CSI ku Kubernetes wa Yandex.Cloud

Ndife okondwa kulengeza kuti Flant ikukulitsa zopereka zake ku Open Source zida za Kubernetes potulutsa mtundu wa alpha wa driver wa CSI (Chiyankhulo Chosungira Chosungira) cha Yandex.Cloud.

Koma tisanapitirire tsatanetsatane wa kukhazikitsa, tiyeni tiyankhe funso la chifukwa chake izi ndizofunikira pamene Yandex ili kale ndi ntchito. Ntchito Yoyang'anira Kubernetes.

Mau oyamba

Chifukwa chiyani?

Mkati mwa kampani yathu, kuyambira pachiyambi chogwiritsa ntchito Kubernetes popanga (ie kwa zaka zingapo tsopano), takhala tikupanga chida chathu (deckhouse), chomwe, mwa njira, tikukonzekeranso kuti posachedwapa tipezeke ngati polojekiti ya Open Source. . Ndi chithandizo chake, timakonza mofananamo ndikukonza magulu athu onse, ndipo pakali pano pali oposa 100 a iwo, pamitundu yosiyanasiyana ya hardware ndi mautumiki onse amtambo omwe alipo.

Magulu omwe amagwiritsa ntchito deckhouse ali ndi zigawo zonse zofunika kuti azigwira ntchito: owerengera, kuyang'anira ndi ma chart osavuta, ma metrics ndi zidziwitso, kutsimikizika kwa ogwiritsa ntchito kudzera mwa othandizira akunja kuti athe kupeza ma dashboard onse, ndi zina zotero. Palibe chifukwa chokhazikitsa gulu la "pumped up" mu njira yoyendetsedwa, chifukwa izi nthawi zambiri zimakhala zosatheka kapena zimabweretsa kufunikira koletsa theka la zigawozo.

NB: Izi ndi zomwe takumana nazo, ndipo ndi zachindunji. Sitikunena kuti aliyense agwiritse ntchito magulu a Kubernetes pawokha m'malo mogwiritsa ntchito mayankho okonzeka. Mwa njira, tilibe chidziwitso chenicheni chogwiritsira ntchito Kubernetes kuchokera ku Yandex ndipo sitidzapereka kuwunika kulikonse kwa ntchitoyi m'nkhaniyi.

Ndi chiyani ndipo cha ndani?

Chifukwa chake, talankhula kale za njira yamakono yosungira ku Kubernetes: Kodi CSI imagwira ntchito bwanji? и mmene anthu ammudzi anadza ku njira iyi.

Pakadali pano, ambiri opereka chithandizo chamtambo apanga madalaivala ogwiritsira ntchito ma disks awo amtambo monga Persistent Volume ku Kubernetes. Ngati wothandizira alibe dalaivala wotere, koma ntchito zonse zofunika zimaperekedwa kudzera pa API, ndiye kuti palibe chomwe chimakulepheretsani kukhazikitsa dalaivala nokha. Izi ndi zomwe zidachitika ndi Yandex.Cloud.

Tinatenga ngati maziko a chitukuko CSI woyendetsa wa DigitalOcean mtambo ndi malingaliro angapo kuchokera madalaivala a GCP, popeza kuyanjana ndi API ya mitambo iyi (Google ndi Yandex) ili ndi zofanana. Makamaka, API ndi Mtengo wa GCP, ndi Yandex bwezerani chinthu Operation kuti muwone momwe ntchito zanthawi yayitali (mwachitsanzo, kupanga disk yatsopano). Kuti muyanjane ndi Yandex.Cloud API, gwiritsani ntchito Yandex.Cloud Go SDK.

Zotsatira za ntchito yochitidwa lofalitsidwa pa GitHub ndipo zingakhale zothandiza kwa iwo omwe, pazifukwa zina, amagwiritsa ntchito Kubernetes kuyika kwawo pa Yandex.Cloud makina enieni (koma osati gulu lokonzekera lokonzekera) ndipo akufuna kugwiritsa ntchito (kuitanitsa) ma disks kupyolera mu CSI.

Реализация

Zofunikira zazikulu

Pakadali pano dalaivala amathandizira ntchito zotsatirazi:

• Kuyitanitsa ma disks m'madera onse a masango malinga ndi topology ya node mu masango;
• Kuchotsa ma disks omwe adalamulidwa kale;
• Kusintha kwapaintaneti kwa ma disks (Yandex.Cloud osachirikiza kuwonjezera ma disks omwe amayikidwa ku makina enieni). Kuti mumve zambiri zamomwe dalaivala adayenera kusinthidwa kuti musamve ululu momwe mungathere, onani pansipa.

M'tsogolomu, tikukonzekera kukhazikitsa zothandizira pakupanga ndi kuchotsa zithunzithunzi za disk.

Vuto lalikulu ndi momwe mungagonjetsere

Kulephera kukulitsa ma disks mu nthawi yeniyeni mu Yandex.Cloud API ndi malire omwe amalepheretsa kusintha kwa PV (Persistent Volume): pamenepa, m'pofunika kuti pulojekiti yomwe imagwiritsa ntchito disk iyimitsidwe, ndipo izi zingayambitse kutsitsa ntchito.

Malingana ndi Zotsatira za CSI, ngati wolamulira wa CSI anena kuti akhoza kusintha ma disks okha "opanda intaneti" (VolumeExpansion.OFFLINE), ndiye njira yowonjezera disk iyenera kupita motere:

Ngati pulogalamu yowonjezera ili yokha VolumeExpansion.OFFLINE Kuthekera kokulitsa ndi kuchuluka kwake kumasindikizidwa kapena kupezeka pa node ndiye ControllerExpandVolume IYENERA kuyitanidwa POKHALA:

• Pulogalamu yowonjezera ili ndi chowongolera PUBLISH_UNPUBLISH_VOLUME luso ndi ControllerUnpublishVolume yayitanidwa bwino.

KAPENA ENA

• Pulogalamu yowonjezera ilibe chowongolera PUBLISH_UNPUBLISH_VOLUME kuthekera, plugin ili ndi node STAGE_UNSTAGE_VOLUME luso, ndi NodeUnstageVolume yatsirizidwa bwino.

KAPENA ENA

• Pulogalamu yowonjezera ilibe chowongolera PUBLISH_UNPUBLISH_VOLUME luso, kapena node STAGE_UNSTAGE_VOLUME luso, ndi NodeUnpublishVolume wamaliza bwino.

Izi zikutanthauza kuti muyenera kuchotsa diskiyo pamakina enieni musanayikulitse.

Komabe, mwatsoka kukhazikitsa Mafotokozedwe a CSI kudzera pa sidecars samakwaniritsa izi:

• Mu chidebe cham'mbali csi-attacher, yomwe imayenera kukhala ndi udindo wa kukhalapo kwa kusiyana kofunikira pakati pa ma mounts, izi sizimagwiritsidwa ntchito pakusintha kwapaintaneti. Kukambitsirana pankhaniyi kunayambika apa.
• Kodi chotengera cham'mbali ndi chiyani kwenikweni munkhaniyi? Pulagi ya CSI palokha simalumikizana ndi Kubernetes API, koma imangoyankha mafoni a gRPC omwe amatumizidwa ndi zotengera zam'mbali. Zaposachedwa akukonzedwa ndi gulu la Kubernetes.

Kwa ife (CSI plugin), ntchito yowonjezera disk ikuwoneka motere:

1. Timalandila foni ya gRPC ControllerExpandVolume;
2. Tikuyesera kuonjezera disk mu API, koma timalandira cholakwika ponena za zosatheka kuchita ntchitoyi chifukwa disk imayikidwa;
3. Timasunga chizindikiritso cha disk pamapu, omwe ali ndi ma disks omwe ntchito yowonjezera iyenera kuchitidwa. Pansipa, mwachidule, tidzatcha mapu awa ngati volumeResizeRequired;
4. Chotsani pamanja poto yomwe ikugwiritsa ntchito diski. Kubernetes ayambitsanso. Chifukwa chake disk ilibe nthawi yoyika (ControllerPublishVolume) tisanamalize ntchito yowonjezereka poyesa kukwera, timayang'ana kuti disk yomwe tapatsidwa idakalipo volumeResizeRequired ndi kubwezera cholakwa;
5. Dalaivala wa CSI amayesa kuyambiranso ntchitoyo. Ngati ntchitoyo idachita bwino, chotsani diskyo volumeResizeRequired;
6. Chifukwa Disk ID ikusowa volumeResizeRequired, ControllerPublishVolume imadutsa bwino, disk imayikidwa, pod imayamba.

Chilichonse chikuwoneka chophweka mokwanira, koma monga nthawi zonse pali misampha. Kukulitsa ma disks kunja-resizer, zomwe zikachitika cholakwika panthawi ya opareshoni amagwiritsa ntchito mzere ndi kuchulukitsidwa kwanthawi yayitali mpaka masekondi 1000:

func DefaultControllerRateLimiter() RateLimiter {
  return NewMaxOfRateLimiter(
  NewItemExponentialFailureRateLimiter(5*time.Millisecond, 1000*time.Second),
  // 10 qps, 100 bucket size.  This is only for retry speed and its only the overall factor (not per item)
  &BucketRateLimiter{Limiter: rate.NewLimiter(rate.Limit(10), 100)},
  )
}

Izi zitha kuchititsa kuti nthawi ndi nthawi ntchito yokulitsa disk ikulitsidwe kwa mphindi 15+ ndipo, motero, pod yofananirayo sikupezeka.

Njira yokhayo yomwe idatilola kuti tichepetse nthawi yochepetsera mosavuta komanso mosavutikira inali kugwiritsa ntchito mtundu wathu wa resizer wakunja wokhala ndi nthawi yayitali yotha. mu 5 masekondi:

workqueue.NewItemExponentialFailureRateLimiter(5*time.Millisecond, 5*time.Second)

Sitinawone kuti ndikofunikira kuyambitsa zokambirana mwachangu ndikuyika chigamba chakunja, chifukwa kusanja kwa ma disks osalumikizidwa ndi intaneti ndikubweza komwe kudzatha posachedwa kwa onse opereka mitambo.

Kodi kuyamba ntchito?

Dalaivala amathandizidwa ndi Kubernetes mtundu 1.15 ndi kupitilira apo. Kuti dalaivala agwire ntchito, zofunika izi ziyenera kukwaniritsidwa:

• Sakanizani --allow-privileged khazikitsani mtengo true kwa seva ya API ndi kubelet;
• Kuphatikizidwa --feature-gates=VolumeSnapshotDataSource=true,KubeletPluginsWatcher=true,CSINodeInfo=true,CSIDriverRegistry=true kwa seva ya API ndi kubelet;
• Kufalikira kwa phiri (kufalikira kwa phiri) ziyenera kuyatsidwa pamagulu. Mukamagwiritsa ntchito Docker, daemon iyenera kukonzedwa kuti ilole zokwera zogawana.

Njira zonse zofunika kukhazikitsa palokha zafotokozedwa mu README. Kuyika kumaphatikizapo kupanga zinthu ku Kubernetes kuchokera pazowonetsera.

Kuti driver agwire ntchito mufunika izi:

• Tchulani chizindikiritso cha chikwatu mu manifesto (folder-idYandex.Cloud (onani zolembedwa);
• Kuti muyanjane ndi Yandex.Cloud API, woyendetsa CSI amagwiritsa ntchito akaunti yautumiki. Mu Chinsinsi chowonekera, muyenera kudutsa makiyi ololedwa kuchokera ku akaunti ya utumiki. Mu zolembedwa anafotokoza, momwe mungapangire akaunti yautumiki ndikupeza makiyi.

Komabe mwazonse - yesani, ndipo tidzakhala okondwa kulandira ndemanga ndi nkhani zatsopanongati mukukumana ndi mavuto!

Thandizo lina

Zotsatira zake, tikufuna kuzindikira kuti takhazikitsa dalaivala wa CSI uyu osati chifukwa chofuna kusangalala ndi zolemba mu Go, koma chifukwa chosowa mwachangu mkati mwa kampani. Zikuwoneka kuti sizothandiza kwa ife kuti tisungebe kukhazikitsa kwathu, kotero ngati Yandex ikuwonetsa chidwi ndikusankha kupitiliza kuthandizira dalaivala, tidzakhala okondwa kusamutsa malo osungiramo zinthu kwa iwo.

Kuphatikiza apo, Yandex mwina ili ndi kukhazikitsa kwake kwa dalaivala wa CSI mugulu lake loyendetsedwa la Kubernetes, lomwe litha kutulutsidwa mu Open Source. Timawonanso njira yachitukukoyi ngati yabwino - anthu ammudzi azitha kugwiritsa ntchito dalaivala wotsimikiziridwa kuchokera kwa wothandizira, osati kuchokera ku kampani yachitatu.

PS

Werenganinso pa blog yathu:

• «Mapulagini amtundu wa Kubernetes yosungirako: kuchokera ku Flexvolume kupita ku CSI";
• «Timamvetsetsa Container Storage Interface (ku Kubernetes osati kokha)";
• «Kodi ndizosavuta komanso zosavuta kukonzekera gulu la Kubernetes? Kulengeza addon-operator";
• «Kukulitsa ndi kukwaniritsa Kubernetes (ndemanga ndi lipoti lamavidiyo)".

Source: www.habr.com