{"id":40721,"date":"2020-02-03T14:41:59","date_gmt":"2020-02-03T11:41:59","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka"},"modified":"2020-02-03T14:41:59","modified_gmt":"2020-02-03T11:41:59","slug":"nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka","status":"publish","type":"post","link":"https:\/\/prohoster.info\/ro\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka","title":{"rendered":"Experien\u021ba noastr\u0103 \u00een dezvoltarea driverului CSI pentru Yandex.Cloud","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Experien\u021ba noastr\u0103 \u00een dezvoltarea driverului CSI pentru Yandex.Cloud\" src=\"\/wp-content\/uploads\/2020\/02\/86adf77b2ee08c426bf79c5b9b2d3d1e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nSuntem \u00eenc\u00e2nta\u021bi s\u0103 anun\u021b\u0103m c\u0103 compania \u201eFlant\u201d \u00ee\u0219i aduce contribu\u021bia la instrumentele Open Source pentru Kubernetes, lans\u00e2nd <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver\">versiunea alfa a driverului CSI<\/a><\/noindex> (Container Storage Interface) pentru Yandex.Cloud.<\/p>\n<p>Dar \u00eenainte de a intra \u00een detalii despre implementare, s\u0103 r\u0103spundem la \u00eentrebarea: de ce este necesar acest lucru, av\u00e2nd \u00een vedere c\u0103 Yandex ofer\u0103 deja serviciul <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/managed-kubernetes\/\">Managed Service for Kubernetes<\/a><\/noindex>.<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2>Introducere<\/h2>\n<p><\/p>\n<h3>De ce este necesar?<\/h3>\n<p>\n\u00cen cadrul companiei noastre, \u00eenc\u0103 de la \u00eenceputul utiliz\u0103rii Kubernetes \u00een produc\u021bie (deci, de c\u00e2\u021biva ani), s-a dezvoltat un instrument propriu (deckhouse), pe care, de fapt, inten\u021bion\u0103m \u00een cur\u00e2nd s\u0103-l facem disponibil ca proiect Open Source. Cu ajutorul s\u0103u, configur\u0103m \u0219i set\u0103m uniform toate clusterele noastre, iar \u00een prezent avem deja peste 100, pe cele mai variate configura\u021bii hardware \u0219i \u00een toate serviciile cloud disponibile.<\/p>\n<p>Clusterele care folosesc deckhouse au toate componentele necesare pentru func\u021bionare: balansatoare de \u00eenc\u0103rcare, monitorizare cu grafice, metrici \u0219i alerte convenabile, autentificare a utilizatorilor prin furnizori externi pentru acces la toate dashboard-urile etc. Un astfel de cluster \u201eavansat\u201d nu are rost s\u0103 fie instalat \u00eentr-o solu\u021bie gestionat\u0103, deoarece de cele mai multe ori fie este imposibil, fie va necesita dezactivarea jum\u0103tate din componente.<\/p>\n<p><i><b>NB<\/b>: Aceasta este experien\u021ba noastr\u0103, \u0219i este destul de specific\u0103. Nu afirm\u0103m c\u0103 toat\u0103 lumea ar trebui s\u0103 se ocupe de desf\u0103\u0219urarea clustrelor Kubernetes \u00een loc s\u0103 foloseasc\u0103 solu\u021bii gata f\u0103cute. De altfel, nu avem experien\u021b\u0103 real\u0103 \u00een utilizarea Kubernetes de la Yandex \u0219i nu vom face nicio evaluare a acestui serviciu \u00een acest articol.<\/i><\/p>\n<h3>Ce este \u0219i pentru cine?<\/h3>\n<p>\nA\u0219adar, am vorbit deja despre abordarea modern\u0103 a stoc\u0103rii \u00een Kubernetes: <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/424211\/\">cum func\u021bioneaz\u0103 CSI<\/a><\/noindex> \u0219i <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/465417\/\">cum a ajuns comunitatea<\/a><\/noindex> la aceast\u0103 abordare.<\/p>\n<p>\u00cen prezent, mul\u021bi furnizori mari de servicii cloud au dezvoltat drivere pentru a utiliza disquele lor \u201ecloud\u201d ca Volume Persistent \u00een Kubernetes. Dac\u0103 \u00eens\u0103 un astfel de driver nu exist\u0103 la furnizor, dar toate func\u021biile necesare sunt oferite prin API, nimic nu \u00eempiedic\u0103 implementarea unui driver propriu. A\u0219a a fost \u0219i cazul nostru cu Yandex.Cloud.<\/p>\n<p>Ca baz\u0103 pentru dezvoltare am luat <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/digitalocean\/csi-digitalocean\">driverul CSI pentru cloud-ul DigitalOcean<\/a><\/noindex> \u0219i c\u00e2teva idei din <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/kubernetes-sigs\/gcp-compute-persistent-disk-csi-driver\">driverul pentru GCP<\/a><\/noindex>, deoarece interac\u021biunea cu API-urile acestor cloud-uri (Google \u0219i Yandex) are anumite similitudini. \u00cen special, at\u00e2t API-ul c\u00e2t \u0219i <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/compute\/docs\/api\/how-tos\/api-requests-responses#handling_api_responses\">GCP<\/a><\/noindex>, \u0219i <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/api-design-guide\/concepts\/about-async\">Yandex<\/a><\/noindex> returneaz\u0103 un obiect <code>Opera\u021biune<\/code> pentru a urm\u0103ri starea opera\u021biunilor de lung\u0103 durat\u0103 (de exemplu, crearea unui nou disco). Pentru interac\u021biunea cu API-ul Yandex.Cloud se utilizeaz\u0103 <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/yandex-cloud\/go-sdk\">Yandex.Cloud Go SDK<\/a><\/noindex>.<\/p>\n<p>Rezultatul muncii efectuate <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver\">a fost publicat pe GitHub<\/a><\/noindex> \u0219i poate fi util celor care, din diverse motive, folosesc propriile instal\u0103ri Kubernetes pe ma\u0219ini virtuale \u00een Yandex.Cloud (dar nu un cluster gestionat gata) \u0219i ar dori s\u0103 foloseasc\u0103 (s\u0103 comande) discuri prin CSI.<\/p>\n<h2>Implementarea<\/h2>\n<p><\/p>\n<h3>Func\u021bionalit\u0103\u021bi principale<\/h3>\n<p>\n\u00cen prezent, driverul suport\u0103 urm\u0103toarele func\u021bii:<\/p>\n<ul>\n<li> Comandarea de discuri \u00een toate zonele cluster-ului conform topologiei nodurilor existente \u00een cluster;<\/li>\n<li> \u0218tergerea discurilor comandate anterior;<\/li>\n<li> Redimensionare offline pentru discuri (Yandex.Cloud <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/compute\/operations\/disk-control\/update#change-disk-size\">nu suport\u0103<\/a><\/noindex> cre\u0219terea dimensiunii discurilor care sunt montate pe ma\u0219ina virtual\u0103). Despre cum a fost necesar s\u0103 se modifice driverul pentru a efectua redimensionarea c\u00e2t mai f\u0103r\u0103 probleme, vede\u021bi mai jos.<\/li>\n<\/ul>\n<p>\n\u00cen viitor se preconizeaz\u0103 implementarea suportului pentru crearea \u0219i \u0219tergerea instantaneelor de discuri.<\/p>\n<h3>Dificultatea principal\u0103 \u0219i dep\u0103\u0219irea acesteia<\/h3>\n<p>\nLipsa \u00een API-ul Yandex.Cloud a posibilit\u0103\u021bii de a cre\u0219te dimensiunea discurilor \u00een timp real \u2014 o restric\u021bie care complic\u0103 opera\u021bia de redimensionare pentru PV (Persistent Volume): deoarece, \u00een acest caz, trebuie s\u0103 se opreasc\u0103 podul aplica\u021biei care utilizeaz\u0103 discul, ceea ce poate provoca oprirea aplica\u021biei.<\/p>\n<p>Conform <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/container-storage-interface\/spec\">specifica\u021biei CSI<\/a><\/noindex>, dac\u0103 controllerul CSI comunic\u0103 c\u0103 poate efectua redimensionarea discurilor doar \u201e\u00een offline\u201d (<code>VolumeExpansion.OFFLINE<\/code>), atunci procesul de cre\u0219tere a discului trebuie s\u0103 decurg\u0103 astfel:<\/p>\n<blockquote><p>Dac\u0103 pluginul are doar <code>VolumeExpansion.OFFLINE<\/code> capabilitatea de expansiune \u0219i volumul este \u00een prezent publicat sau disponibil pe un nod, atunci <code>ControllerExpandVolume<\/code> TREBUIE s\u0103 fie apelat NUMAI dup\u0103 ce fie:<\/p>\n<ul>\n<li> Pluginul are controller <code>PUBLISH_UNPUBLISH_VOLUME<\/code> capabilitate \u0219i <code>ControllerUnpublishVolume<\/code> a fost invocat cu succes.<\/li>\n<\/ul>\n<p>\nSAU ALTFEL<\/p>\n<ul>\n<li> Pluginul nu are capacitate de controller, pluginul are nod <code>PUBLISH_UNPUBLISH_VOLUME<\/code> STAGE_UNSTAGE_VOLUME <code>capabilitate \u0219i<\/code> NodeUnstageVolume <code>a fost finalizat cu succes.<\/code> capabilitate, nici nodul<\/li>\n<\/ul>\n<p>\nSAU ALTFEL<\/p>\n<ul>\n<li>Pluginul nu are capacitate de controller, pluginul are nod <code>PUBLISH_UNPUBLISH_VOLUME<\/code> NodeUnpublishVolume <code>capabilitate \u0219i<\/code> NodeUnstageVolume <code>nu a fost finalizat cu succes.<\/code> Practic, aceasta \u00eenseamn\u0103 c\u0103 este necesar s\u0103 deconecta\u021bi discul de la ma\u0219ina virtual\u0103 \u00eenainte de a-l cre\u0219te.<\/li>\n<\/ul>\n<\/blockquote>\n<p>\nDin p\u0103cate,<\/p>\n<p>implementarea <b>specifica\u021biei CSI prin sidecar-uri nu corespunde acestor cerin\u021be:<\/b> \u00cen containerul sidecar<\/p>\n<ul>\n<li> csi-attacher <code>csi-attacher<\/code>, care ar trebui s\u0103 fie responsabil pentru prezen\u021ba intervalului necesar \u00eentre mont\u0103ri, deoarece aceast\u0103 funcionalitate pur \u0219i simplu nu este implementat\u0103 \u00een cazul redimension\u0103rii offline. Discu\u021bia despre acest subiect a fost ini\u021biat\u0103 de <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/kubernetes-csi\/external-attacher\/issues\/207\">aici<\/a><\/noindex>.<\/li>\n<li> Ce este, de fapt, un container sidecar \u00een acest context? \u00cen sine, pluginul CSI nu interac\u021bioneaz\u0103 cu API-ul Kubernetes, ci doar r\u0103spunde la apelurile gRPC pe care le trimite containerul sidecar. Acestea <noindex><a rel=\"nofollow\" href=\"https:\/\/kubernetes-csi.github.io\/docs\/sidecar-containers.html\">sunt dezvoltate<\/a><\/noindex> de comunitatea Kubernetes.<\/li>\n<\/ul>\n<p>\n\u00cen cazul nostru (pluginul CSI), opera\u021biunea de cre\u0219tere a discului arat\u0103 astfel:<\/p>\n<ol>\n<li> Primim apelul gRPC <code>ControllerExpandVolume<\/code>;<\/li>\n<li> \u00cencerc\u0103m s\u0103 cre\u0219tem discul \u00een API, dar primim o eroare cu privire la imposibilitatea de a efectua opera\u021bia, deoarece discul este montat;<\/li>\n<li> Salv\u0103m identificatorul discului \u00eentr-o map\u0103 care con\u021bine discurile pentru care trebuie s\u0103 efectu\u0103m opera\u021bia de cre\u0219tere. Vom denumi aceast\u0103 map\u0103 pentru scurt <code>volumeResizeRequired<\/code>;<\/li>\n<li> \u0218tergem manual podul care utilizeaz\u0103 discul. Kubernetes \u00eel va reporni. Pentru a ne asigura c\u0103 discul nu este montat (<code>ControllerPublishVolume<\/code>) \u00eenainte de finalizarea opera\u021biei de cre\u0219tere, verific\u0103m c\u0103 acest disc se afl\u0103 \u00een continuare \u00een <code>volumeResizeRequired<\/code> \u0219i return\u0103m o eroare;<\/li>\n<li> Driverul CSI \u00eencearc\u0103 s\u0103 reia opera\u021bia de redimensionare. Dac\u0103 opera\u021bia a fost efectuat\u0103 cu succes, atunci \u0219tergem discul din <code>volumeResizeRequired<\/code>;<\/li>\n<li> Fiindc\u0103 identificatorul discului este absent \u00een <code>volumeResizeRequired<\/code>, <code>ControllerPublishVolume<\/code> opera\u021bia se desf\u0103\u0219oar\u0103 cu succes, discul este montat, podul este repornit.<\/li>\n<\/ol>\n<p>\nTotul pare destul de simplu, dar ca de obicei, exist\u0103 capcane. Cre\u0219terea discurilor este responsabilitatea <noindex><a rel=\"nofollow\" href=\"https:\/\/kubernetes-csi.github.io\/docs\/external-resizer.html\">external-resizer<\/a><\/noindex>, care \u00een cazul unei erori \u00een timpul execu\u021biei opera\u021biei <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/kubernetes-csi\/external-resizer\/blob\/master\/vendor\/k8s.io\/client-go\/util\/workqueue\/default_rate_limiters.go#L41\">folose\u0219te o coad\u0103<\/a><\/noindex> cu o cre\u0219tere exponen\u021bial\u0103 a timpului de a\u0219teptare p\u00e2n\u0103 la 1000 de secunde:<\/p>\n<pre><code class=\"go\">func DefaultControllerRateLimiter() RateLimiter {\n  return NewMaxOfRateLimiter(\n  NewItemExponentialFailureRateLimiter(5*time.Millisecond, 1000*time.Second),\n  \t\/\/ 10 qps, 100 bucket size. Aceasta este doar pentru viteza de retry \u0219i este doar factorul general (nu pe element)\n  &amp;BucketRateLimiter{Limiter: rate.NewLimiter(rate.Limit(10), 100)},\n  )\n}<\/code><\/pre>\n<p>\nAceasta poate duce periodic la o \u00eentindere a opera\u021biei de cre\u0219tere a discului pe 15+ minute \u0219i, prin urmare, la indisponibilitatea podului corespunz\u0103tor.<\/p>\n<p>Singura op\u021biune care ne-a permis s\u0103 reduc\u0103 destul de u\u0219or \u0219i f\u0103r\u0103 durere timpul poten\u021bial de nefunc\u021bionare a fost utilizarea propriei versiuni a external-resizer cu o limitare maxim\u0103 a timpului de a\u0219teptare <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/external-resizer\/blob\/faster-workqueue\/pkg\/controller\/controller.go#L81\">de 5 secunde<\/a><\/noindex>:<\/p>\n<pre><code class=\"go\">workqueue.NewItemExponentialFailureRateLimiter(5*time.Millisecond, 5*time.Second)<\/code><\/pre>\n<p>\nNu am considerat necesar s\u0103 ini\u021biem o discu\u021bie de urgen\u021b\u0103 \u0219i s\u0103 patch-uim external-resizer, deoarece redimensionarea offline a discurilor este un atavism care \u00een cur\u00e2nd va disp\u0103rea din toate serviciile cloud.<\/p>\n<h2>Cum s\u0103 \u00eencepi s\u0103 folose\u0219ti?<\/h2>\n<p>\nDriverul este compatibil cu Kubernetes versiunea 1.15 \u0219i mai sus. Pentru a func\u021biona, driverul trebuie s\u0103 respecte urm\u0103toarele cerin\u021be:<\/p>\n<ul>\n<li> Steagul <code>--allow-privileged<\/code> este setat la <code>true<\/code> pentru API-server \u0219i kubelet;<\/li>\n<li> Inclusiv <code>--feature-gates=VolumeSnapshotDataSource=true,KubeletPluginsWatcher=true,CSINodeInfo=true,CSIDriverRegistry=true<\/code> pentru API-server \u0219i kubelet;<\/li>\n<li> Propagarea mont\u0103rii (<noindex><a rel=\"nofollow\" href=\"https:\/\/kubernetes.io\/docs\/concepts\/storage\/volumes\/#mount-propagation\">mount propagation<\/a><\/noindex>) trebuie s\u0103 fie activat\u0103 \u00een cluster. Atunci c\u00e2nd utiliza\u021bi Docker, demonul trebuie s\u0103 fie configurat pentru a permite mont\u0103rile partajate.<\/li>\n<\/ul>\n<p>\nTo\u021bi pa\u0219ii necesari pentru instalare <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver#installing-driver\">sunt descri\u0219i \u00een README<\/a><\/noindex>. Instalarea const\u0103 \u00een crearea de obiecte \u00een Kubernetes din manifesturi.<\/p>\n<p>Pentru ca driverul s\u0103 func\u021bioneze, ave\u021bi nevoie de urm\u0103toarele:<\/p>\n<ul>\n<li> Specifica\u021bi \u00een manifest ID-ul folderului (<code>folder-id<\/code>) din Yandex.Cloud (<noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/resource-manager\/operations\/folder\/get-id\">vezi documenta\u021bia<\/a><\/noindex>);<\/li>\n<li> Pentru a interac\u021biona cu API-ul Yandex.Cloud \u00een driverul CSI, este utilizat un cont de serviciu. \u00cen manifestul Secret, trebuie s\u0103 transmite\u021bi <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.com\/docs\/iam\/concepts\/authorization\/key\">cheile autorizate<\/a><\/noindex> de la contul de serviciu. \u00cen documenta\u021bie <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.yandex.ru\/docs\/iam\/quickstart-sa\">este descris<\/a><\/noindex>, cum s\u0103 crea\u021bi un cont de serviciu \u0219i s\u0103 ob\u021bine\u021bi cheile.<\/li>\n<\/ul>\n<p>\n\u00cen general, <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver\">\u00eencerca\u021bi<\/a><\/noindex>, iar noi vom fi bucuro\u0219i s\u0103 primim feedback \u0219i <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/flant\/yandex-csi-driver\/issues\">noi probleme<\/a><\/noindex>, dac\u0103 v\u0103 confrunta\u021bi cu probleme!<\/p>\n<h2>Suportul ulterior<\/h2>\n<p>\n\u00cen concluzie, am dori s\u0103 subliniem c\u0103 acest driver CSI a fost realizat nu din dorin\u021ba de a ne distra scriind aplica\u021bii \u00een Go, ci din necesitatea acut\u0103 din interiorul companiei. A p\u0103stra propria noastr\u0103 implementare nu ni se pare rentabil, a\u0219a c\u0103, dac\u0103 Yandex ar manifesta interesul \u0219i ar decide s\u0103 continue suportul pentru driver, ne vom bucura s\u0103 transfer\u0103m repository-ul \u00een grija lor.<\/p>\n<p>\u00cen plus, probabil c\u0103 Yandex are \u00een clusterul Kubernetes gestionat propria implementare a driverului CSI, pe care ar putea s\u0103 o elibereze \u00een Open Source. Aceast\u0103 variant\u0103 ne-ar p\u0103rea, de asemenea, favorabil\u0103 \u2014 comunitatea va putea beneficia de un driver verificat de la furnizorul de servicii, \u0219i nu de la o companie ter\u021b\u0103.<\/p>\n<h2>P.S.<\/h2>\n<p>\nCiti\u021bi \u0219i \u00een blogul nostru:<\/p>\n<ul>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/465417\/\">Pluginuri pentru volume de stocare \u00een Kubernetes: de la Flexvolume la CSI<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/424211\/\">\u00cen\u021belegem Container Storage Interface (\u00een Kubernetes \u0219i nu numai)<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/455543\/\">Este simplu \u0219i convenabil s\u0103 preg\u0103te\u0219ti un cluster Kubernetes? Anun\u021b\u0103m addon-operator<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/455543\/\">Extindem \u0219i complet\u0103m Kubernetes (prezentare general\u0103 \u0219i video al expunerii)<\/a><\/noindex>\u00bb.<\/li>\n<\/ul>\n<p>Sursa: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/486190\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0420\u0430\u0434\u044b \u043e\u0431\u044a\u044f\u0432\u0438\u0442\u044c, \u0447\u0442\u043e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f \u00ab\u0424\u043b\u0430\u043d\u0442\u00bb \u043f\u043e\u043f\u043e\u043b\u043d\u044f\u0435\u0442 \u0441\u0432\u043e\u0439 \u0432\u043a\u043b\u0430\u0434 \u0432 Open Source-\u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u044b \u0434\u043b\u044f Kubernetes, \u0432\u044b\u043f\u0443\u0441\u0442\u0438\u0432 \u0430\u043b\u044c\u0444\u0430-\u0432\u0435\u0440\u0441\u0438\u044e \u0434\u0440\u0430\u0439\u0432\u0435\u0440\u0430 CSI (Container Storage Interface) \u0434\u043b\u044f \u042f\u043d\u0434\u0435\u043a\u0441.\u041e\u0431\u043b\u0430\u043a\u0430. \u041d\u043e \u043f\u0435\u0440\u0435\u0434 \u0442\u0435\u043c, \u043a\u0430\u043a \u043f\u0435\u0440\u0435\u0439\u0442\u0438 \u043a \u0434\u0435\u0442\u0430\u043b\u044f\u043c \u0440\u0435\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u0438, \u043e\u0442\u0432\u0435\u0442\u0438\u043c \u043d\u0430 \u0432\u043e\u043f\u0440\u043e\u0441, \u0437\u0430\u0447\u0435\u043c \u044d\u0442\u043e \u0432\u043e\u043e\u0431\u0449\u0435 \u043d\u0443\u0436\u043d\u043e, \u043a\u043e\u0433\u0434\u0430 \u0443 \u042f\u043d\u0434\u0435\u043a\u0441\u0430 \u0443\u0436\u0435 \u0435\u0441\u0442\u044c \u0443\u0441\u043b\u0443\u0433\u0430 Managed Service for Kubernetes. \u0412\u0432\u0435\u0434\u0435\u043d\u0438\u0435 \u0417\u0430\u0447\u0435\u043c \u044d\u0442\u043e? \u0412\u043d\u0443\u0442\u0440\u0438 \u043d\u0430\u0448\u0435\u0439 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438, \u0435\u0449\u0451 \u0441 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":40722,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-40721","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0420\u0430\u0434\u044b \u043e\u0431\u044a\u044f\u0432\u0438\u0442\u044c, \u0447\u0442\u043e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/ro\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"ro_RO\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041d\u0430\u0448 \u043e\u043f\u044b\u0442 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0438 CSI-\u0434\u0440\u0430\u0439\u0432\u0435\u0440\u0430 \u0432 Kubernetes \u0434\u043b\u044f \u042f\u043d\u0434\u0435\u043a\u0441.\u041e\u0431\u043b\u0430\u043a\u0430 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0420\u0430\u0434\u044b \u043e\u0431\u044a\u044f\u0432\u0438\u0442\u044c, \u0447\u0442\u043e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/ro\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-02-03T11:41:59+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-02-03T11:41:59+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Experien\u021ba noastr\u0103 \u00een dezvoltarea driverului CSI \u00een Kubernetes pentru Yandex.Cloud | ProHoster","description":"Suntem \u00eenc\u00e2nta\u021bi s\u0103 anun\u021b\u0103m c\u0103 compania.","canonical_url":"https:\/\/prohoster.info\/ro\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"ro_RO","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041d\u0430\u0448 \u043e\u043f\u044b\u0442 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0438 CSI-\u0434\u0440\u0430\u0439\u0432\u0435\u0440\u0430 \u0432 Kubernetes \u0434\u043b\u044f \u042f\u043d\u0434\u0435\u043a\u0441.\u041e\u0431\u043b\u0430\u043a\u0430 | ProHoster","og:description":"\u0420\u0430\u0434\u044b \u043e\u0431\u044a\u044f\u0432\u0438\u0442\u044c, \u0447\u0442\u043e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044f.","og:url":"https:\/\/prohoster.info\/ro\/blog\/nash-opyt-razrabotki-csi-drajvera-v-kubernetes-dlya-yandeks-oblaka","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-02-03T11:41:59+00:00","article:modified_time":"2020-02-03T11:41:59+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"40721","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 00:32:40","updated":"2022-09-30 15:42:14","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/40721","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/comments?post=40721"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/40721\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media\/40722"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media?parent=40721"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/categories?post=40721"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/tags?post=40721"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}