{"id":40721,"date":"2020-02-03T14:41:59","date_gmt":"2020-02-03T11:41:59","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka"},"modified":"2020-02-03T14:41:59","modified_gmt":"2020-02-03T11:41:59","slug":"nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka","status":"publish","type":"post","link":"https:\/\/prohoster.info\/et\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka","title":{"rendered":"Meie kogemus CSI-draiveri v\u00e4ljat\u00f6\u00f6tamisel Kuberneteses Yandex.Cloudile","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Meie kogemus CSI-draiveri v\u00e4ljat\u00f6\u00f6tamisel Kuberneteses Yandex.Cloudile\" src=\"\/wp-content\/uploads\/2020\/02\/86adf77b2ee08c426bf79c5b9b2d3d1e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nMeil on hea meel teatada, et ettev\u00f5te \"Flant\" suurendab oma panust avatud l\u00e4htekoodiga t\u00f6\u00f6riistadesse Kubernetes'ile, v\u00e4lja andes <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver\">alpha-versiooni CSI-draiverist<\/a><\/noindex> (Container Storage Interface) Yandex Cloudile.<\/p>\n<p>Kuid enne, kui l\u00e4heme rakenduse \u00fcksikasjade juurde, vastame k\u00fcsimusele, miks see \u00fcldse vajalik on, kui Yandex'il on juba teenus <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/managed-kubernetes\/\">Managed Service for Kubernetes<\/a><\/noindex>.<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2>Sissejuhatus<\/h2>\n<p><\/p>\n<h3>Miks see vajalik on?<\/h3>\n<p>\nMeie ettev\u00f5ttes on juba algusest peale, kui Kubernetes'it hakatakse tootmises kasutama (st juba mitu aastat), arendatud oma t\u00f6\u00f6riista (deckhouse), mille me kavatseme varsti muuta kergesti k\u00e4ttesaadavaks kui avatud l\u00e4htekoodiga projekti. Selle abil konfigureerime ja seadistame k\u00f5iki oma klustreid \u00fchtlaselt ning hetkel on neid juba \u00fcle 100, erinevates riistvara konfiguratsioonides ja k\u00f5ikides saadaolevates pilveteenustes.<\/p>\n<p>Klusterid, milles kasutatakse deckhouse'i, sisaldavad k\u00f5iki vajalikke t\u00f6\u00f6komponente: laadijad, mugava graafiku, m\u00f5\u00f5dikute ja h\u00e4iretega j\u00e4lgimine, kasutajate autentimine v\u00e4liste teenusepakkujate kaudu juurdep\u00e4\u00e4suks k\u00f5ikidele armatuurlaudadele ja nii edasi. Sellist \"t\u00e4iustatud\" klustrit pole m\u00f5tet paigaldada hallatud lahendusse, kuna see on sageli kas v\u00f5imatu v\u00f5i viib vajaduseni v\u00e4lja \u043e\u0442\u043a\u043b\u044e\u0447\u0438\u0442\u044c poole komponente.<\/p>\n<p><i><b>NB<\/b>: See on meie kogemus ja see on \u00fcsna spetsiifiline. Me ei v\u00e4ida mingil juhul, et k\u00f5igil tasuks iseseisvalt Kubernetes'i klustreid \u00fcles seada, selle asemel et kasutada valmis lahendusi. \u00dctleme nii, et meil ei ole Yandex'i tegelikest Kubernetes'i kasutuse kaalutlustest ja me ei anna selle teenuse kohta hinnangut k\u00e4esolevas artiklis.<\/i><\/p>\n<h3>Mis see on ja kellele?<\/h3>\n<p>\nNii et me oleme juba r\u00e4\u00e4kinud kaasaegsest l\u00e4henemisest andmete salvestamisele Kubernetes'is: <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/424211\/\">kuidas CSI t\u00f6\u00f6tab<\/a><\/noindex> ja <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/465417\/\">kuidas kogukond j\u00f5udis<\/a><\/noindex> sellise l\u00e4henemiseni.<\/p>\n<p>Praegu on paljud suured pilveteenuse pakkujad v\u00e4lja t\u00f6\u00f6tanud draiverid, et kasutada oma \"pilve\" kettaid kui Persistent Volume Kubernetes'es. Kui pakkujal sellist draiverit ei ole, kuid k\u00f5ik vajalikud funktsioonid on API kaudu saadaval, siis ei ole midagi, mis takistaks draiveri ise ellu viimist. Meie puhul ongi nii l\u00e4inud Yandex Cloudiga.<\/p>\n<p>Arenduse aluseks v\u00f5tsime <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/digitalocean\/csi-digitalocean\">CSI-draiveri DigitalOceani pilve jaoks<\/a><\/noindex> ja m\u00f5ned ideed <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/kubernetes-sigs\/gcp-compute-persistent-disk-csi-driver\">GCP-draiverist<\/a><\/noindex>, kuna interaktsioon API-dega nendes pilvedes (Google ja Yandex) on mitmeid sarnaseid jooni. Eelk\u00f5ige on API nii <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/compute\/docs\/api\/how-tos\/api-requests-responses#handling_api_responses\">GCP<\/a><\/noindex>, kui ka <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/api-design-guide\/concepts\/about-async\">Yandex<\/a><\/noindex> tagastab objekti <code>Toiming<\/code> pikaajaliste toimingute (nt uue ketta loomise) staatuse j\u00e4lgimiseks. API-ga suhtlemiseks Yandex.Cloudis kasutatakse <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/yandex-cloud\/go-sdk\">Yandex.Cloud Go SDK-d<\/a><\/noindex>.<\/p>\n<p>Tehtud t\u00f6\u00f6 tulemus <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver\">on avaldatud GitHubis<\/a><\/noindex> ja v\u00f5ib olla kasulik neile, kes mingil p\u00f5hjusel kasutavad oma isiklikku Kubernetes'i installatsiooni Yandex.Cloudi virtuaalmasinatel (aga mitte hallatud klastrit) ja soovivad kettaid kasutada (tellida) kaudu CSI.<\/p>\n<h2>Rakendamine<\/h2>\n<p><\/p>\n<h3>P\u00f5hiomadused<\/h3>\n<p>\nPraegu toetab draiver j\u00e4rgmisi funktsioone:<\/p>\n<ul>\n<li> Kettad tellitakse igas klastrite tsoonis vastavalt klastris olemasolevate s\u00f5lmede topoloogiale;<\/li>\n<li> Varem tellitud ketaste kustutamine;<\/li>\n<li> Offline-ketaste muutmine (Yandex.Cloud <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/compute\/operations\/disk-control\/update#change-disk-size\">ei toeta<\/a><\/noindex> kettad, mis on \u00fchendatud virtuaalmasinaga). Kuidas draiverit tuli kohandada, et v\u00f5imalikult valutult teostada muutmist, vt allpool.<\/li>\n<\/ul>\n<p>\nTulevikus plaanitakse rakendada toe loomist ja kustutamist kettaste jaoks.<\/p>\n<h3>Peamine keerukus ja selle \u00fcletamine<\/h3>\n<p>\nYandex.Cloud API-l puudub v\u00f5imalus suurendada kettaid reaalajas \u2014 piirang, mis raskendab PV (p\u00fcsiv maht) suurendamise operatsiooni: sellisel juhul peab rakenduse pod, mis kasutab ketast, olema peatatud, mis v\u00f5ib p\u00f5hjustada rakenduse seiskumist.<\/p>\n<p>Vastavalt <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/container-storage-interface\/spec\">specifikatsioonide CSI<\/a><\/noindex>, kui CSI-kontroller teatab, et suudab suurendada kettaid ainult \u00aboffline\u00bb (<code>VolumeExpansion.OFFLINE<\/code>), peab ketta suurendamise protsess toimuma j\u00e4rgmiselt:<\/p>\n<blockquote><p>Kui pistikprogrammil on ainult <code>VolumeExpansion.OFFLINE<\/code> suurendamise v\u00f5imekus ja maht on hetkel avaldatud v\u00f5i s\u00f5lmes saadaval, siis <code>ControllerExpandVolume<\/code> PEAB olema kutsutud AINULT p\u00e4rast \u00fchte j\u00e4rgmist:<\/p>\n<ul>\n<li> Pistikprogrammil on kontroller <code>PUBLISH_UNPUBLISH_VOLUME<\/code> v\u00f5imekus ja <code>ControllerUnpublishVolume<\/code> on edukalt kutsutud.<\/li>\n<\/ul>\n<p>\nV\u00d5I MUUD<\/p>\n<ul>\n<li> Pistikprogrammil ei ole kontrolleri <code>PUBLISH_UNPUBLISH_VOLUME<\/code> v\u00f5imekust, pistikprogrammil on s\u00f5lme <code>STAGE_UNSTAGE_VOLUME<\/code> v\u00f5imekust ja <code>NodeUnstageVolume<\/code> on edukalt l\u00f5pule viidud.<\/li>\n<\/ul>\n<p>\nV\u00d5I MUUD<\/p>\n<ul>\n<li>Pistikprogrammil ei ole kontrolleri <code>PUBLISH_UNPUBLISH_VOLUME<\/code> v\u00f5imekust, ega s\u00f5lme <code>STAGE_UNSTAGE_VOLUME<\/code> v\u00f5imekust ja <code>NodeUnpublishVolume<\/code> ei ole edukalt l\u00f5pule viidud.<\/li>\n<\/ul>\n<\/blockquote>\n<p>\nTegelikult t\u00e4hendab see, et enne ketta suurendamist tuleb see virtuaalmasinast lahti \u00fchendada.<\/p>\n<p>Kahjuks, <b>rakendus<\/b> CSI spetsifikatsioonid ei vasta suunistele sidecar'ide kaudu:<\/p>\n<ul>\n<li> Sidecar konteineris <code>csi-attacher<\/code>, mis peaks vastutama vajaliku vahemaa olemasolu eest mountide vahel, offline-resize'i puhul see funktsioon lihtsalt ei ole rakendatud. Arutelu selle \u00fcle algatasid <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/kubernetes-csi\/external-attacher\/issues\/207\">siin<\/a><\/noindex>.<\/li>\n<li> Mis on sidecar-konteiner antud kontekstis? CSI-plugin ise ei suhtle Kubernetes API-ga, vaid reageerib vaid gRPC-k\u00f5nedele, mida saadavad sidecar-konteinerid. Viimased <noindex><a rel=\"nofollow\" href=\"https:\/\/kubernetes-csi.github.io\/docs\/sidecar-containers.html\">arendatakse<\/a><\/noindex> Kubernetes kogukonna.<\/li>\n<\/ul>\n<p>\nMeie juhul (CSI-plugin) disk space'i suurendamise protsess n\u00e4eb v\u00e4lja j\u00e4rgmine:<\/p>\n<ol>\n<li> Saame gRPC-k\u00f5ne <code>ControllerExpandVolume<\/code>;<\/li>\n<li> P\u00fc\u00fcame suurendada ketast API-s, kuid saame vea, et operatsiooni ei saa teostada, kuna ketas on \u00fchendatud;<\/li>\n<li> Salvestame ketta ID kaardile, mis sisaldab kettasid, mille puhul tuleb teostada suurendamisoperatsioon. Edasi kutsume seda kaarti <code>volumeResizeRequired<\/code>;<\/li>\n<li> K\u00e4sitsi kustutame pod\u2019i, mis kasutab ketast. Kubernetes taask\u00e4ivitab selle. Et ketas ei j\u00f5uaks enne operatsiooni l\u00f5ppu \u00fchenduda (<code>ControllerPublishVolume<\/code>) suurendamisoperatsiooni katsetamisel kontrollime, et ketas on endiselt <code>volumeResizeRequired<\/code> ja tagastame vea;<\/li>\n<li> CSI-draiver p\u00fc\u00fcab uuesti suurendamisoperatsiooni teostada. Kui operatsioon on edukas, eemaldame ketta <code>volumeResizeRequired<\/code>;<\/li>\n<li> Kuna ketta ID puudub <code>volumeResizeRequired<\/code>, <code>ControllerPublishVolume<\/code> j\u00f5uab edukalt l\u00f5pule, ketas mountitakse, pod k\u00e4ivitatakse.<\/li>\n<\/ol>\n<p>\nK\u00f5ik n\u00e4eb v\u00e4lja piisavalt lihtne, kuid nagu ikka, on peidetud l\u00f5kse. Diskide suurendamisega tegeleb <noindex><a rel=\"nofollow\" href=\"https:\/\/kubernetes-csi.github.io\/docs\/external-resizer.html\">external-resizer<\/a><\/noindex>, mis juhul, kui operatsioon eba\u00f5nnestub, <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/kubernetes-csi\/external-resizer\/blob\/master\/vendor\/k8s.io\/client-go\/util\/workqueue\/default_rate_limiters.go#L41\">kasutab j\u00e4rjekorda<\/a><\/noindex> mille n\u00e4ol on tegu eksponentsiaalselt kasvava ajavahemikuga kuni 1000 sekundini:<\/p>\n<pre><code class=\"go\">func DefaultControllerRateLimiter() RateLimiter {\n  return NewMaxOfRateLimiter(\n  NewItemExponentialFailureRateLimiter(5*time.Millisecond, 1000*time.Second),\n  \\\/\\\/ 10 qps, 100 bucket size.  This is only for retry speed and its only the overall factor (not per item)\n  &amp;BucketRateLimiter{Limiter: rate.NewLimiter(rate.Limit(10), 100)},\n  )\n}<\/code><\/pre>\n<p>\nSee v\u00f5ib aeg-ajalt p\u00f5hjustada, et disk space'i suurendamise operatsioon venib 15+ minutiks ja seega vastava pod'i mitte k\u00e4ttesaadavuseks.<\/p>\n<p>Ainukeseks variandiks, mis v\u00f5imaldas meil piisavalt lihtsalt ja valutult v\u00e4hendada potentsiaalset seisaku aega, oli oma versiooni external-resizer kasutamine maksimaalse ajapiiranguga <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/external-resizer\/blob\/faster-workqueue\/pkg\/controller\/controller.go#L81\">5 sekundile<\/a><\/noindex>:<\/p>\n<pre><code class=\"go\">workqueue.NewItemExponentialFailureRateLimiter(5*time.Millisecond, 5*time.Second)<\/code><\/pre>\n<p>\nMe ei pidanud vajalikuks \u00e4kitselt arutelusid alustada ja external-resizer'i pat\u0161i teha, kuna offline diskide suurendamine on anahronism, mis peagi kaob k\u00f5igi pilveteenuste pakkujate juurest.<\/p>\n<h2>Kuidas alustada kasutamist?<\/h2>\n<p>\nDraiver on toetatud Kubernetes versioonis 1.15 ja uuemates. Draiveri t\u00f6\u00f6ks peavad olema t\u00e4idetud j\u00e4rgmised n\u00f5uded:<\/p>\n<ul>\n<li> Lipp <code>--allow-privileged<\/code> seatud v\u00e4\u00e4rtuseks <code>true<\/code> API-serveri ja kubeleti jaoks;<\/li>\n<li> Aktiveeritud <code>--feature-gates=VolumeSnapshotDataSource=true,KubeletPluginsWatcher=true,CSINodeInfo=true,CSIDriverRegistry=true<\/code> API-serveri ja kubeleti jaoks;<\/li>\n<li> Mountimise levitamine (<noindex><a rel=\"nofollow\" href=\"https:\/\/kubernetes.io\/docs\/concepts\/storage\/volumes\/#mount-propagation\">mount propagation<\/a><\/noindex>) peab olema klastris lubatud. Docker'i kasutamisel peab deemon olema konfigureeritud nii, et lubatud on jagatud mount'id.<\/li>\n<\/ul>\n<p>\nK\u00f5ik vajalikud installimisprotsessi sammud <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver#installing-driver\">on kirjas README's.<\/a><\/noindex>Installeerimine h\u00f5lmab objektide loomist Kuberneteses manifestide p\u00f5hjal.<\/p>\n<p>Draiveri t\u00f6\u00f6ks on vajalik j\u00e4rgmine:<\/p>\n<ul>\n<li> Manifestis tuleb n\u00e4idata katalooge ID-d (<code>folder-id<\/code>) Yandex.Cloudis (<noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/resource-manager\/operations\/folder\/get-id\">vt dokumentatsiooni<\/a><\/noindex>);<\/li>\n<li> Yandex.Cloud API-ga suhtlemiseks kasutatakse CSI-draiveris teenusekontot. Manifestis Secret tuleb edastada <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.com\/docs\/iam\/concepts\/authorization\/key\">volitatud v\u00f5tmed<\/a><\/noindex> teenusekontolt. Dokumentatsioonis <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/iam\/quickstart-sa\">kirjeldatud<\/a><\/noindex>, kuidas luua teenusekonto ja saada v\u00f5tmed.<\/li>\n<\/ul>\n<p>\n\u00dcldiselt \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver\">proovige<\/a><\/noindex>, ja me oleksime r\u00f5\u00f5msad tagasiside ja <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver\/issues\">uute probleemide teemade \u00fcle<\/a><\/noindex>, kui satute mingitesse probleemidesse!<\/p>\n<h2>Edasine tugi<\/h2>\n<p>\nKokkuv\u00f5tteks tahame r\u00f5hutada, et me arendasime selle CSI-draiveri mitte mitte niiv\u00f5rd soovist nautida Go keeles rakenduste loomist, vaid ettev\u00f5tte sees esinenud \u00e4geda vajaduse t\u00f5ttu. Oma rakenduse toetamine ei tundu meile m\u00f5ttekas, seega kui Yandex n\u00e4itab huvi ja otsustab draiveri toetamise j\u00e4tkata, oleme huvitatud reposi neile \u00fcleandmisest.<\/p>\n<p>Lisaks, v\u00f5ib-olla on Yandex'l oma rakendus CSI-draiver t\u00f5statatud Kubernetes'is oma hallatud klastris, mille nad v\u00f5iksid v\u00e4lja anda avatud l\u00e4htekoodiga. See variant n\u00e4ib meile samuti soodne \u2014 kogukond saab kasutada teenusepakkuja poolt testitud draiverit, mitte kolmanda osapoole oma.<\/p>\n<h2>P.S.<\/h2>\n<p>\nLugege ka meie blogist:<\/p>\n<ul>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/465417\/\">Kubernetesi salvestusmahtude pluginad: Flexvolume'ist CSI-ni<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/424211\/\">M\u00f5istame Container Storage Interface'i (nii Kubernetes'es kui ka mujal)<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/455543\/\">Kubernetes klastrit on lihtne ja mugav seadistada? Tutvustame addon-operatorit<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/455543\/\">T\u00e4iendame ja t\u00e4iustame Kubernetes\u2019t (\u00fclevaade ja videodoklad)<\/a><\/noindex>\u00bb.<\/li>\n<\/ul>\n<p>Allikas: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/486190\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0420\u0430\u0434\u044b \u043e\u0431\u044a\u044f\u0432\u0438\u0442\u044c, \u0447\u0442\u043e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f \u00ab\u0424\u043b\u0430\u043d\u0442\u00bb \u043f\u043e\u043f\u043e\u043b\u043d\u044f\u0435\u0442 \u0441\u0432\u043e\u0439 \u0432\u043a\u043b\u0430\u0434 \u0432 Open Source-\u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u044b \u0434\u043b\u044f Kubernetes, \u0432\u044b\u043f\u0443\u0441\u0442\u0438\u0432 \u0430\u043b\u044c\u0444\u0430-\u0432\u0435\u0440\u0441\u0438\u044e \u0434\u0440\u0430\u0439\u0432\u0435\u0440\u0430 CSI (Container Storage Interface) \u0434\u043b\u044f \u042f\u043d\u0434\u0435\u043a\u0441.\u041e\u0431\u043b\u0430\u043a\u0430. \u041d\u043e \u043f\u0435\u0440\u0435\u0434 \u0442\u0435\u043c, \u043a\u0430\u043a \u043f\u0435\u0440\u0435\u0439\u0442\u0438 \u043a \u0434\u0435\u0442\u0430\u043b\u044f\u043c \u0440\u0435\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u0438, \u043e\u0442\u0432\u0435\u0442\u0438\u043c \u043d\u0430 \u0432\u043e\u043f\u0440\u043e\u0441, \u0437\u0430\u0447\u0435\u043c \u044d\u0442\u043e \u0432\u043e\u043e\u0431\u0449\u0435 \u043d\u0443\u0436\u043d\u043e, \u043a\u043e\u0433\u0434\u0430 \u0443 \u042f\u043d\u0434\u0435\u043a\u0441\u0430 \u0443\u0436\u0435 \u0435\u0441\u0442\u044c \u0443\u0441\u043b\u0443\u0433\u0430 Managed Service for Kubernetes. \u0412\u0432\u0435\u0434\u0435\u043d\u0438\u0435 \u0417\u0430\u0447\u0435\u043c \u044d\u0442\u043e? \u0412\u043d\u0443\u0442\u0440\u0438 \u043d\u0430\u0448\u0435\u0439 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438, \u0435\u0449\u0451 \u0441 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":40722,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-40721","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.1.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0420\u0430\u0434\u044b \u043e\u0431\u044a\u044f\u0432\u0438\u0442\u044c, \u0447\u0442\u043e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/et\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.1.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"et_EE\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041d\u0430\u0448 \u043e\u043f\u044b\u0442 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0438 CSI-\u0434\u0440\u0430\u0439\u0432\u0435\u0440\u0430 \u0432 Kubernetes \u0434\u043b\u044f \u042f\u043d\u0434\u0435\u043a\u0441.\u041e\u0431\u043b\u0430\u043a\u0430 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0420\u0430\u0434\u044b \u043e\u0431\u044a\u044f\u0432\u0438\u0442\u044c, \u0447\u0442\u043e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/et\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-02-03T11:41:59+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-02-03T11:41:59+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Meie kogemus CSI-draiveri arendamisel Kuberneteses Yandex.Cloudi jaoks | ProHoster","description":"Meil on hea meel teatada, et firma.","canonical_url":"https:\/\/prohoster.info\/et\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"et_EE","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041d\u0430\u0448 \u043e\u043f\u044b\u0442 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0438 CSI-\u0434\u0440\u0430\u0439\u0432\u0435\u0440\u0430 \u0432 Kubernetes \u0434\u043b\u044f \u042f\u043d\u0434\u0435\u043a\u0441.\u041e\u0431\u043b\u0430\u043a\u0430 | ProHoster","og:description":"\u0420\u0430\u0434\u044b \u043e\u0431\u044a\u044f\u0432\u0438\u0442\u044c, \u0447\u0442\u043e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f.","og:url":"https:\/\/prohoster.info\/et\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-02-03T11:41:59+00:00","article:modified_time":"2020-02-03T11:41:59+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"40721","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 00:32:40","updated":"2022-09-30 15:42:14","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/40721","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/comments?post=40721"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/40721\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media\/40722"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media?parent=40721"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/categories?post=40721"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/tags?post=40721"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}