{"id":40721,"date":"2020-02-03T14:41:59","date_gmt":"2020-02-03T11:41:59","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka"},"modified":"2020-02-03T14:41:59","modified_gmt":"2020-02-03T11:41:59","slug":"nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sl\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka","title":{"rendered":"Na\u0161e iskustvo razvoja CSI drajvera u Kubernetesu za Yandex.Cloud","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Na\u0161e iskustvo razvoja CSI drajvera u Kubernetesu za Yandex.Cloud\" src=\"\/wp-content\/uploads\/2020\/02\/86adf77b2ee08c426bf79c5b9b2d3d1e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nSa rado\u0161\u0107u objavljujemo da kompanija \u201eFlant\u201c nastavlja svoj doprinos Open Source alatima za Kubernetes, objavljuju\u0107i <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver\">alfa verziju CSI drajvera<\/a><\/noindex> (Container Storage Interface) za Yandex.Cloud.<\/p>\n<p>Ali pre nego \u0161to pre\u0111emo na detalje implementacije, odgovori\u0107emo na pitanje za\u0161to je to uop\u0161te potrebno, kada Yandex ve\u0107 ima uslugu <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/managed-kubernetes\/\">Managed Service for Kubernetes<\/a><\/noindex>.<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2>Uvod<\/h2>\n<p><\/p>\n<h3>Zakaj to?<\/h3>\n<p>\nUnutar na\u0161e kompanije, jo\u0161 od samog po\u010detka kori\u0161\u0107enja Kubernetes-a u produkciji (tj. ve\u0107 nekoliko godina), razvija se na\u0161 vlastiti alat (deckhouse), koji, usput, planiramo uskoro u\u010diniti dostupnim kao Open Source projekt. Njime jedinstveno konfiguri\u0161emo i pode\u0161avamo sve na\u0161e klastere, a trenutno ih ve\u0107 imamo vi\u0161e od 100, sa najrazli\u010ditijim konfiguracijama hardvera i na svim dostupnim cloud servisima.<\/p>\n<p>Klasteri koji koriste deckhouse imaju sve potrebne komponente za rad: balansere, monitoring sa prakti\u010dnim grafikonima, metrike i alerte, autentifikaciju korisnika putem eksternih provajdera za pristup svim dashboard-ovima itd. Takav \u201enapredni\u201c klaster nema smisla postavljati u managed re\u0161enja, jer to \u010desto ili nije mogu\u0107e, ili \u0107e dovesti do potrebe za isklju\u010divanjem pola komponenti.<\/p>\n<p><i><b>NB<\/b>: Ovo je na\u0161e iskustvo, i ono je prili\u010dno specifi\u010dno. Ni u kom slu\u010daju ne tvrdimo da bi svako trebao samostalno da se bavi postavljanjem Kubernetes klastera umesto da koristi gotova re\u0161enja. Usput, nemamo stvarno iskustvo kori\u0161\u0107enja Kubernetes-a od Yandex-a i ne\u0107emo davati bilo kakvu ocenu ove usluge u ovom \u010dlanku.<\/i><\/p>\n<h3>\u0160ta je to i za koga?<\/h3>\n<p>\nDakle, ve\u0107 smo govorili o savremenom pristupu skladi\u0161tima u Kubernetes-u: <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/424211\/\">kako funkcioni\u0161e CSI<\/a><\/noindex> in <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/465417\/\">kako je zajednica do\u0161la<\/a><\/noindex> do takvog pristupa.<\/p>\n<p>Trenutno su mnogi veliki pru\u017eaoci cloud usluga razvili drajvere za kori\u0161\u0107enje svojih \u201ecloud\u201c diskova kao Persistent Volume u Kubernetes-u. Ako takvog drajvera nema kod provajdera, ali su sve potrebne funkcije dostupne putem API-ja, ni\u0161ta ne spre\u010dava da se drajver implementira sopstvenim snagama. Tako je i nama uspelo sa Yandex.Cloud.<\/p>\n<p>Osnova za razvoj je bila <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/digitalocean\/csi-digitalocean\">CSI drajver za cloud DigitalOcean<\/a><\/noindex> i nekoliko ideja iz <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/kubernetes-sigs\/gcp-compute-persistent-disk-csi-driver\">drajvera za GCP<\/a><\/noindex>, ker je interakcija z API-ji teh oblakov (Google in Yandex) podobna. Zlasti imajo API-ji oba <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/compute\/docs\/api\/how-tos\/api-requests-responses#handling_api_responses\">GCP<\/a><\/noindex>, kot tudi <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/api-design-guide\/concepts\/about-async\">Yandex<\/a><\/noindex> , vrnejo objekt <code>Operacija<\/code> , ki sledi statusu dolgih operacij (na primer, ustvarjanje novega diska). Za interakcijo z API-jem Yandex.Oblaka se uporablja <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/yandex-cloud\/go-sdk\">Yandex.Cloud Go SDK<\/a><\/noindex>.<\/p>\n<p>Rezultat opravljenega dela <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver\">je objavljen na GitHubu<\/a><\/noindex> in lahko koristi tistim, ki iz kakr\u0161nega koli razloga uporabljajo lastno namestitev Kubernetes na virtualnih strojih Yandex.\u041e\u0431\u043b\u0430\u043a\u0430 (ne pripravljenega upravljanega grozda) in bi \u017eeleli uporabljati (naro\u010dati) diske prek CSI.<\/p>\n<h2>Implementacija<\/h2>\n<p><\/p>\n<h3>Main Features<\/h3>\n<p>\nTrenutno gonilnik podpira naslednje funkcije:<\/p>\n<ul>\n<li> Naro\u010danje diskov v vseh obmo\u010djih grozda v skladu z topologijo obstoje\u010dih vozli\u0161\u010d v grozdu;<\/li>\n<li> Odstranjevanje prej naro\u010denih diskov;<\/li>\n<li> Offline raz\u0161iritev diskov (Yandex.\u041e\u0431\u043b\u0430\u043a\u043e <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/compute\/operations\/disk-control\/update#change-disk-size\">ne podpira<\/a><\/noindex> pove\u010danja diskov, ki so priklju\u010deni na virtualni stroj). O tem, kako je bilo treba prilagoditi gonilnik, da bi kar najbolj brez te\u017eav izvedli raz\u0161iritev, glej spodaj.<\/li>\n<\/ul>\n<p>\nV prihodnosti je na\u010drtovana podpora za ustvarjanje in odstranjevanje posnetkov diskov.<\/p>\n<h3>Glavna te\u017eava in njeno re\u0161evanje<\/h3>\n<p>\nPomanjkanje mo\u017enosti v API-ju Yandex.\u041e\u0431\u043b\u0430\u043a\u0430 za pove\u010danje diskov v realnem \u010dasu \u2014 omejitev, ki ote\u017euje operacijo raz\u0161iritve za PV (Persistent Volume): saj je v tem primeru potrebno, da je pod aplikacije, ki uporablja disk, ustavljen, kar lahko povzro\u010di izpad aplikacije.<\/p>\n<p>Po <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/container-storage-interface\/spec\">specifikacije CSI<\/a><\/noindex>, \u010de CSI-nadzornik poro\u010da, da lahko izvaja raz\u0161iritev diskov le \"v offline\" (<code>VolumeExpansion.OFFLINE<\/code>), mora proces pove\u010danja diska potekati tako:<\/p>\n<blockquote><p>\u010ce ima vti\u010dnik le <code>VolumeExpansion.OFFLINE<\/code> sposobnost raz\u0161iritve in je volumen trenutno objavljen ali na voljo na vozli\u0161\u010du, potem <code>ControllerExpandVolume<\/code> MORA biti poklican LE po tem, ko je bilo izpolnjeno:<\/p>\n<ul>\n<li> Vti\u010dnik ima nadzor <code>PUBLISH_UNPUBLISH_VOLUME<\/code> sposobnost in <code>ControllerUnpublishVolume<\/code> je bil uspe\u0161no poklican.<\/li>\n<\/ul>\n<p>\nALI PA<\/p>\n<ul>\n<li> Vti\u010dnik NIMA nadzorne <code>PUBLISH_UNPUBLISH_VOLUME<\/code> sposobnosti, vti\u010dnik ima vozli\u0161\u010dno <code>STAGE_UNSTAGE_VOLUME<\/code> sposobnost in <code>NodeUnstageVolume<\/code> je bil uspe\u0161no zaklju\u010den.<\/li>\n<\/ul>\n<p>\nALI PA<\/p>\n<ul>\n<li>Vti\u010dnik NIMA nadzorne <code>PUBLISH_UNPUBLISH_VOLUME<\/code> sposobnost, niti vozli\u0161\u010de <code>STAGE_UNSTAGE_VOLUME<\/code> sposobnost in <code>NodeUnpublishVolume<\/code> je bilo uspe\u0161no zaklju\u010deno.<\/li>\n<\/ul>\n<\/blockquote>\n<p>\nV bistvu to pomeni, da je potrebno odklopiti disk od virtualnega stroja, preden ga pove\u010damo.<\/p>\n<p>\u017dal pa <b>izvedba<\/b> specifikacije CSI preko sidecar-ov ne izpolnjuje teh zahtev:<\/p>\n<ul>\n<li> V sidecar-kontejnerju <code>csi-attacher<\/code>, ki bi moral biti odgovoren za ohranjanje potrebnega intervala med montiranji, pri offline-raz\u0161iritvi funkcionalnost preprosto ni izvedena. Razpravo o tem so spro\u017eili <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/kubernetes-csi\/external-attacher\/issues\/207\">ovde<\/a><\/noindex>.<\/li>\n<li> Kaj sploh je sidecar kontejner v tem kontekstu? Sam CSI vti\u010dnik ne deluje z Kubernetes API-jem, temve\u010d se le odziva na gRPC klice, ki mu jih po\u0161iljajo sidecar kontejnerji. Zadnji <noindex><a rel=\"nofollow\" href=\"https:\/\/kubernetes-csi.github.io\/docs\/sidecar-containers.html\">se razvijajo<\/a><\/noindex> kubernetes skupnosti.<\/li>\n<\/ul>\n<p>\nV na\u0161em primeru (CSI vti\u010dnik) postopek pove\u010danja diska izgleda tako:<\/p>\n<ol>\n<li> Prejmemo gRPC klic <code>ControllerExpandVolume<\/code>;<\/li>\n<li> Poskusimo pove\u010dati disk v API-ju, vendar dobimo napako o tem, da operacije ni mogo\u010de izvesti, ker je disk priklju\u010den;<\/li>\n<li> Shranjujemo identifikator diska v mapo, ki vsebuje diske, za katere je potrebno izvesti operacijo pove\u010danja. Nadalje bomo iz riznice za kraj\u0161o obliko klicali to mapo <code>volumeResizeRequired<\/code>;<\/li>\n<li> Ro\u010dno odstranimo pod, ki uporablja disk. Kubernetes ga bo pri tem ponovno zagnal. Da disk ne bi bil priklju\u010den (<code>ControllerPublishVolume<\/code>) pred zaklju\u010dkom operacije pove\u010danja ob poskusu priklju\u010ditve preverimo, da je ta disk \u0161e vedno v <code>volumeResizeRequired<\/code> in vrnemo napako;<\/li>\n<li> CSI gonilnik posku\u0161a ponovno izvesti operacijo pove\u010danja. \u010ce je operacija uspe\u0161na, disk odstranimo iz <code>volumeResizeRequired<\/code>;<\/li>\n<li> Ker identifikator diska ni prisoten v <code>volumeResizeRequired<\/code>, <code>ControllerPublishVolume<\/code> operacija uspe\u0161no poteka, disk se prika\u017ee, pod se za\u017eene.<\/li>\n<\/ol>\n<p>\nVideti je dovolj preprosto, a kot vedno obstajajo skrite te\u017eave. Pove\u010danje diskov opravlja <noindex><a rel=\"nofollow\" href=\"https:\/\/kubernetes-csi.github.io\/docs\/external-resizer.html\">external-resizer<\/a><\/noindex>, ki v primeru napake pri izvajanju operacije <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/kubernetes-csi\/external-resizer\/blob\/master\/vendor\/k8s.io\/client-go\/util\/workqueue\/default_rate_limiters.go#L41\">uporablja vrsto<\/a><\/noindex> z eksponentialnim pove\u010danjem \u010dasovnega izteka do 1000 sekund:<\/p>\n<pre><code class=\"go\">func DefaultControllerRateLimiter() RateLimiter {\n  return NewMaxOfRateLimiter(\n  NewItemExponentialFailureRateLimiter(5*time.Millisecond, 1000*time.Second),\n  \/\/ 10 qps, 100 velikost vedra. To je samo za hitrost ponovnega poskusa in je samo splo\u0161en faktor (ne na element)\n  &amp;BucketRateLimiter{Limiter: rate.NewLimiter(rate.Limit(10), 100)},\n  )\n}<\/code><\/pre>\n<p>\nTo lahko ob\u010dasno privede do tega, da se operacija pove\u010danja diska raztegne na 15+ minut in tako povzro\u010di nedostopnost ustreznega poda.<\/p>\n<p>Edina mo\u017enost, ki nam je dovolj enostavno in brez bole\u010din omogo\u010dila zmanj\u0161anje potencialnega \u010dasa nedosegljivosti, je bila uporaba na\u0161e razli\u010dice external-resizer z najve\u010djim \u010dasovnim omejitvijo <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/external-resizer\/blob\/faster-workqueue\/pkg\/controller\/controller.go#L81\">v 5 sekundah<\/a><\/noindex>:<\/p>\n<pre><code class=\"go\">workqueue.NewItemExponentialFailureRateLimiter(5*time.Millisecond, 5*time.Second)<\/code><\/pre>\n<p>\nNismo menili, da je potrebno nujno za\u010deti razpravo in popraviti external-resizer, ker offline pove\u010danje diskov predstavlja arhaizem, ki bo kmalu izginil pri vseh obla\u010dnih ponudnikih.<\/p>\n<h2>Kako za\u010deti uporabljati?<\/h2>\n<p>\nGonilnik je podprt v Kubernetes razli\u010dici 1.15 in vi\u0161je. Za delovanje gonilnika morajo biti izpolnjene naslednje zahteve:<\/p>\n<ul>\n<li> Zastava <code>--allow-privileged<\/code> postavljen na <code>true<\/code> za API server i kubelet;<\/li>\n<li> Omogu\u0107eno <code>--feature-gates=VolumeSnapshotDataSource=true,KubeletPluginsWatcher=true,CSINodeInfo=true,CSIDriverRegistry=true<\/code> za API server i kubelet;<\/li>\n<li> Distribucija montiranja (<noindex><a rel=\"nofollow\" href=\"https:\/\/kubernetes.io\/docs\/concepts\/storage\/volumes\/#mount-propagation\">mount propagation<\/a><\/noindex>) mora biti omogu\u0107ena u klasteru. Kada koristite Docker, demonu treba omogu\u0107iti sharing montiranja (shared mounts).<\/li>\n<\/ul>\n<p>\nSvi potrebni koraci za instalaciju <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver#installing-driver\">opisani su u README<\/a><\/noindex>. Instalacija se sastoji od kreiranja objekata u Kubernetesu iz manifestacija.<\/p>\n<p>Za rad drajvera potrebno je slede\u0107e:<\/p>\n<ul>\n<li> Navesti u manifestu identifikator direktorijuma (<code>folder-id<\/code>) Yandex.Clouda (<noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/resource-manager\/operations\/folder\/get-id\">vidi dokumentaciju<\/a><\/noindex>);<\/li>\n<li> Za interakciju sa API-jem Yandex.Clouda u CSI drajveru koristi se servisni nalog. U manifestu Secret potrebno je proslediti <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.com\/docs\/iam\/concepts\/authorization\/key\">ovla\u0161\u0107ene klju\u010deve<\/a><\/noindex> sa servisnog naloga. U dokumentaciji <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/iam\/quickstart-sa\">opisano je<\/a><\/noindex>, kako napraviti servisni nalog i dobiti klju\u010deve.<\/li>\n<\/ul>\n<p>\nUkratko \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver\">probajte<\/a><\/noindex>, a mi \u0107emo biti zahvalni za povratne informacije i <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver\/issues\">novim pitanjima<\/a><\/noindex>, ako se suo\u010dite s nekim problemima!<\/p>\n<h2>Dalja podr\u0161ka<\/h2>\n<p>\nNa kraju, \u017eelimo napomenuti da smo ovaj CSI drajver razvijali ne iz velike \u017eelje za igrom s pisanjem aplikacija u Go-u, ve\u0107 zbog hitne potrebe unutar kompanije. Odr\u017eavanje vlastite implementacije nam se ne \u010dini razumnim, pa ako Yandex poka\u017ee interes i odlu\u010di nastaviti s podr\u0161kom drajvera, rado \u0107emo preneti repozitorijum na njihovo raspolaganje.<\/p>\n<p>Pored toga, verovatno, Yandex u managed klasteru Kubernetes ima vlastitu implementaciju CSI drajvera koju bi mogli objaviti kao Open Source. Takva opcija se tako\u0111e \u010dini povoljnom za nas \u2014 zajednica bi mogla koristiti provereni drajver od provajdera usluga, a ne od tre\u0107ih strana.<\/p>\n<h2>P.S.<\/h2>\n<p>\nPreberite tudi na na\u0161em blogu:<\/p>\n<ul>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/465417\/\">Plugin za volumene za skladi\u0161ta u Kubernetesu: od Flexvolume do CSI<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/424211\/\">Razumemo Container Storage Interface (u Kubernetesu i ne samo)<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/455543\/\">Je priprava Kubernetes klasterja preprosta in priro\u010dna? Najavljamo addon-operator<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/455543\/\">Raz\u0161irjamo in dopolnjujemo Kubernetes (pregled in video poro\u010dila)<\/a><\/noindex>\u00bb.<\/li>\n<\/ul>\n<p>Izvor: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/486190\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0420\u0430\u0434\u044b \u043e\u0431\u044a\u044f\u0432\u0438\u0442\u044c, \u0447\u0442\u043e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f \u00ab\u0424\u043b\u0430\u043d\u0442\u00bb \u043f\u043e\u043f\u043e\u043b\u043d\u044f\u0435\u0442 \u0441\u0432\u043e\u0439 \u0432\u043a\u043b\u0430\u0434 \u0432 Open Source-\u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u044b \u0434\u043b\u044f Kubernetes, \u0432\u044b\u043f\u0443\u0441\u0442\u0438\u0432 \u0430\u043b\u044c\u0444\u0430-\u0432\u0435\u0440\u0441\u0438\u044e \u0434\u0440\u0430\u0439\u0432\u0435\u0440\u0430 CSI (Container Storage Interface) \u0434\u043b\u044f \u042f\u043d\u0434\u0435\u043a\u0441.\u041e\u0431\u043b\u0430\u043a\u0430. \u041d\u043e \u043f\u0435\u0440\u0435\u0434 \u0442\u0435\u043c, \u043a\u0430\u043a \u043f\u0435\u0440\u0435\u0439\u0442\u0438 \u043a \u0434\u0435\u0442\u0430\u043b\u044f\u043c \u0440\u0435\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u0438, \u043e\u0442\u0432\u0435\u0442\u0438\u043c \u043d\u0430 \u0432\u043e\u043f\u0440\u043e\u0441, \u0437\u0430\u0447\u0435\u043c \u044d\u0442\u043e \u0432\u043e\u043e\u0431\u0449\u0435 \u043d\u0443\u0436\u043d\u043e, \u043a\u043e\u0433\u0434\u0430 \u0443 \u042f\u043d\u0434\u0435\u043a\u0441\u0430 \u0443\u0436\u0435 \u0435\u0441\u0442\u044c \u0443\u0441\u043b\u0443\u0433\u0430 Managed Service for Kubernetes. \u0412\u0432\u0435\u0434\u0435\u043d\u0438\u0435 \u0417\u0430\u0447\u0435\u043c \u044d\u0442\u043e? \u0412\u043d\u0443\u0442\u0440\u0438 \u043d\u0430\u0448\u0435\u0439 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438, \u0435\u0449\u0451 \u0441 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":40722,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-40721","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.5.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0420\u0430\u0434\u044b \u043e\u0431\u044a\u044f\u0432\u0438\u0442\u044c, \u0447\u0442\u043e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sl\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka\" \/>\n\t\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.5.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"sl_SI\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041d\u0430\u0448 \u043e\u043f\u044b\u0442 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0438 CSI-\u0434\u0440\u0430\u0439\u0432\u0435\u0440\u0430 \u0432 Kubernetes \u0434\u043b\u044f \u042f\u043d\u0434\u0435\u043a\u0441.\u041e\u0431\u043b\u0430\u043a\u0430 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0420\u0430\u0434\u044b \u043e\u0431\u044a\u044f\u0432\u0438\u0442\u044c, \u0447\u0442\u043e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sl\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-02-03T11:41:59+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-02-03T11:41:59+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Na\u0161e iskustvo u razvoju CSI drajvera u Kubernetesu za Yandex.Cloud | ProHoster","description":"Radosni smo \u0161to kompanija.","canonical_url":"https:\/\/prohoster.info\/sl\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sl_SI","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041d\u0430\u0448 \u043e\u043f\u044b\u0442 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0438 CSI-\u0434\u0440\u0430\u0439\u0432\u0435\u0440\u0430 \u0432 Kubernetes \u0434\u043b\u044f \u042f\u043d\u0434\u0435\u043a\u0441.\u041e\u0431\u043b\u0430\u043a\u0430 | ProHoster","og:description":"\u0420\u0430\u0434\u044b \u043e\u0431\u044a\u044f\u0432\u0438\u0442\u044c, \u0447\u0442\u043e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f.","og:url":"https:\/\/prohoster.info\/sl\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-02-03T11:41:59+00:00","article:modified_time":"2020-02-03T11:41:59+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"40721","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 00:32:40","updated":"2022-09-30 15:42:14","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/posts\/40721","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/comments?post=40721"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/posts\/40721\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/media\/40722"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/media?parent=40721"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/categories?post=40721"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/tags?post=40721"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}