{"id":94739,"date":"2020-09-20T19:42:27","date_gmt":"2020-09-20T17:42:27","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes"},"modified":"2021-01-02T13:05:08","modified_gmt":"2021-01-02T11:05:08","slug":"prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes","status":"publish","type":"post","link":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes","title":{"rendered":"Exemplu practic de conectare a unui stocare bazat\u0103 pe Ceph \u00eentr-un cluster Kubernetes","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>Container Storage Interface (CSI) este o interfa\u021b\u0103 unificat\u0103 de interac\u021biune \u00eentre Kubernetes \u0219i sistemele de stocare de date. Am discutat pe scurt despre ea anterior, <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/southbridge\/blog\/519130\/\">despre<\/a><\/noindex>, iar ast\u0103zi ne vom concentra mai \u00een detaliu asupra leg\u0103turii dintre CSI \u0219i Ceph: vom ar\u0103ta cum <noindex><a rel=\"nofollow\" href=\"http:\/\/to.slurm.io\/WkM2Eg\">s\u0103 conect\u0103m stocarea Ceph<\/a><\/noindex> la clusterul Kubernetes.<br \/>\nArticolul ofer\u0103 exemple reale, de\u0219i pu\u021bin simplificate pentru a fi mai u\u0219or de \u00een\u021beles. Nu vom discuta despre instalarea \u0219i configurarea clusterelor Ceph \u0219i Kubernetes.<\/p>\n<p>E\u0219ti interesat s\u0103 afli cum func\u021bioneaz\u0103?<\/p>\n<p><img decoding=\"async\" alt=\"Exemplu practic de conectare a unui stocare bazat\u0103 pe Ceph \u00eentr-un cluster Kubernetes\" src=\"\/wp-content\/uploads\/2020\/09\/db0a1d1e4c38c693f591057a219c141c.jpg\" style=\"display:block;margin: 0 auto;\"><\/p>\n<p><noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<p>Deci, ai un cluster Kubernetes la dispozi\u021bie, de exemplu, <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/kubernetes-sigs\/kubespray\">kubespray<\/a><\/noindex>. Al\u0103turi, exist\u0103 un cluster Ceph \u2014 acesta poate fi de asemenea instalat, de exemplu, folosind acest <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/ceph\/ceph-ansible\">set de playbook-uri<\/a><\/noindex>. Sper c\u0103 nu e nevoie s\u0103 men\u021bionez c\u0103 pentru produc\u021bie ar trebui s\u0103 existe o re\u021bea \u00eentre ele cu o l\u0103\u021bime de band\u0103 de cel pu\u021bin 10 Gbit\/s.<\/p>\n<p>Dac\u0103 toate acestea le ai, s\u0103 \u00eencepem!<\/p>\n<p>Mai \u00eent\u00e2i, s\u0103 acces\u0103m unul dintre nodurile clusterului Ceph \u0219i s\u0103 verific\u0103m c\u0103 totul este \u00een ordine:<\/p>\n<pre><code class=\"plaintext\">ceph health\nceph -s<\/code><\/pre>\n<p>Apoi, vom crea un pool pentru discurile RBD:<\/p>\n<pre><code class=\"plaintext\">ceph osd pool create kube 32\nceph osd pool application enable kube rbd<\/code><\/pre>\n<p>Trecem la clusterul Kubernetes. Acolo, primul pas este s\u0103 instal\u0103m driverul Ceph CSI pentru RBD. \u00cel vom instala, a\u0219a cum trebuie, prin Helm.<br \/>\nAd\u0103ug\u0103m repository-ul cu chart-ul, ob\u021binem setul de variabile pentru chart-ul ceph-csi-rbd:<\/p>\n<pre><code class=\"plaintext\">helm repo add ceph-csi https:\/\/ceph.github.io\/csi-charts\nhelm inspect values ceph-csi\/ceph-csi-rbd &gt; cephrbd.yml<\/code><\/pre>\n<p>Acum trebuie s\u0103 complet\u0103m fi\u0219ierul cephrbd.yml. Pentru aceasta, vom afla ID-ul clusterului \u0219i adresele IP ale monitorilor din Ceph:<\/p>\n<pre><code class=\"plaintext\">ceph fsid  # astfel afl\u0103m clusterID\nceph mon dump  # iar a\u0219a vom ob\u021bine adresele IP ale monitorilor<\/code><\/pre>\n<p>Valorile ob\u021binute se vor introduce \u00een fi\u0219ierul cephrbd.yml. \u00centre timp, activ\u0103m crearea politicilor PSP (Pod Security Policies). Op\u021biunile din sec\u021biunile <strong>nodeplugin<\/strong> \u0219i <strong>provisioner<\/strong> sunt deja \u00een fi\u0219ier, le putem modifica a\u0219a cum este ar\u0103tat mai jos:<\/p>\n<pre><code class=\"plaintext\">csiConfig:\n  - clusterID: \"bcd0d202-fba8-4352-b25d-75c89258d5ab\"\n    monitors:\n      - \"v2:172.18.8.5:3300\/0,v1:172.18.8.5:6789\/0\"\n      - \"v2:172.18.8.6:3300\/0,v1:172.18.8.6:6789\/0\"\n      - \"v2:172.18.8.7:3300\/0,v1:172.18.8.7:6789\/0\"\n\nnodeplugin:\n  podSecurityPolicy:\n    enabled: true\n\nprovisioner:\n  podSecurityPolicy:\n    enabled: true<\/code><\/pre>\n<p>Apoi, tot ce ne r\u0103m\u00e2ne de f\u0103cut este s\u0103 instal\u0103m chart-ul \u00een clusterul Kubernetes.<\/p>\n<pre><code class=\"plaintext\">helm upgrade -i ceph-csi-rbd ceph-csi\/ceph-csi-rbd -f cephrbd.yml -n ceph-csi-rbd --create-namespace<\/code><\/pre>\n<p>Excelent, driverul RBD func\u021bioneaz\u0103!<br \/>\nS\u0103 cre\u0103m un nou StorageClass \u00een Kubernetes. Pentru aceasta, va fi necesar s\u0103 lucr\u0103m pu\u021bin cu Ceph.<\/p>\n<p>Cre\u0103m un nou utilizator \u00een Ceph \u0219i \u00eei d\u0103m permisiuni de scriere \u00een pool-ul <strong>kube<\/strong>:<\/p>\n<pre><code class=\"plaintext\">ceph auth get-or-create client.rbdkube mon 'profile rbd' osd 'profile rbd pool=kube'<\/code><\/pre>\n<p>\u0218i acum, s\u0103 vedem cheia de acces acolo unde am l\u0103sat-o:<\/p>\n<pre><code class=\"plaintext\">ceph auth get-key client.rbdkube<\/code><\/pre>\n<p>Comanda va returna ceva de genul:<\/p>\n<pre><code class=\"plaintext\">AQCO9NJbhYipKRAAMqZsnqqS\/T8OYQX20xIa9A==<\/code><\/pre>\n<p>Vom transmitem aceast\u0103 valoare \u00een Secretul din clusterul Kubernetes \u2014 acolo unde este necesar\u0103 <strong>userKey<\/strong>:<\/p>\n<pre><code class=\"plaintext\">---\napiVersion: v1\nkind: Secret\nmetadata:\n  name: csi-rbd-secret\n  namespace: ceph-csi-rbd\nstringData:\n  # Valorile cheilor corespund numelui de utilizator \u0219i cheii acestuia, a\u0219a cum este indicat \u00een\n  # clusterul Ceph. ID-ul utilizatorului trebuie s\u0103 aib\u0103 acces la pool-ul,\n  # specificat \u00een clasa de stocare\n  userID: rbdkube\n  userKey:<\/code><\/pre>\n<p>\u0218i cre\u0103m secretul nostru:<\/p>\n<pre><code class=\"plaintext\">kubectl apply -f secret.yaml<\/code><\/pre>\n<p>Apoi avem nevoie de un manifest StorageClass de genul:<\/p>\n<pre><code class=\"plaintext\">---\napiVersion: storage.k8s.io\/v1\nkind: StorageClass\nmetadata:\n   name: csi-rbd-sc\nprovisioner: rbd.csi.ceph.com\nparameters:\n   clusterID: \n   pool: kube\n\n   imageFeatures: layering\n\n   # Aceste secrete trebuie s\u0103 con\u021bin\u0103 date pentru autorizare\n   # \u00een pool-ul dvs.\n   csi.storage.k8s.io\/provisioner-secret-name: csi-rbd-secret\n   csi.storage.k8s.io\/provisioner-secret-namespace: ceph-csi-rbd\n   csi.storage.k8s.io\/controller-expand-secret-name: csi-rbd-secret\n   csi.storage.k8s.io\/controller-expand-secret-namespace: ceph-csi-rbd\n   csi.storage.k8s.io\/node-stage-secret-name: csi-rbd-secret\n   csi.storage.k8s.io\/node-stage-secret-namespace: ceph-csi-rbd\n\n   csi.storage.k8s.io\/fstype: ext4\n\nreclaimPolicy: Delete\nallowVolumeExpansion: true\nmountOptions:\n  - discard<\/code><\/pre>\n<p>Trebuie completat <strong>clusterID<\/strong>, pe care l-am aflat deja cu comanda <em>ceph fsid<\/em>, \u0219i aplic\u0103m acest manifest \u00een clusterul Kubernetes:<\/p>\n<pre><code class=\"plaintext\">kubectl apply -f storageclass.yaml<\/code><\/pre>\n<p>Pentru a verifica func\u021bionarea clusterelor \u00eempreun\u0103, voi crea un astfel de PVC (Persistent Volume Claim):<\/p>\n<pre><code class=\"plaintext\">apiVersion: v1\nkind: PersistentVolumeClaim\nmetadata:\n  name: rbd-pvc\nspec:\n  accessModes:\n  - ReadWriteOnce\n  resources:\n    requests:\n      storage: 1Gi\n  storageClassName: csi-rbd-sc<\/code><\/pre>\n<p>S\u0103 vedem imediat cum a creat Kubernetes volumul solicitat \u00een Ceph:<\/p>\n<pre><code class=\"plaintext\">kubectl get pvc\nkubectl get pv<\/code><\/pre>\n<p>Se pare c\u0103 totul este \u00een regul\u0103! Cum arat\u0103 din partea Ceph?<br \/>\nOb\u021binem lista volumelor din pool \u0219i vizualiz\u0103m informa\u021biile despre volumul nostru:<\/p>\n<pre><code class=\"plaintext\">rbd ls -p kube\nrbd -p kube info csi-vol-eb3d257d-8c6c-11ea-bff5-6235e7640653  # aici, evident, va fi un alt ID al volumului, generat de comanda anterioar\u0103<\/code><\/pre>\n<p>Acum s\u0103 vedem cum func\u021bioneaz\u0103 modificarea dimensiunii volumului RBD.<br \/>\nSchimb\u0103m dimensiunea volumului \u00een manifestul pvc.yaml la 2Gi \u0219i \u00eel aplic\u0103m:<\/p>\n<pre><code class=\"plaintext\">kubectl apply -f pvc.yaml<\/code><\/pre>\n<p>S\u0103 a\u0219tept\u0103m s\u0103 intre \u00een vigoare modific\u0103rile \u0219i s\u0103 verific\u0103m din nou dimensiunea volumului.<\/p>\n<pre><code class=\"plaintext\">rbd -p kube info csi-vol-eb3d257d-8c6c-11ea-bff5-6235e7640653\n\nkubectl get pv\nkubectl get pvc<\/code><\/pre>\n<p>Vedem c\u0103 dimensiunea PVC-ului nu s-a modificat. Pentru a afla motivul, putem solicita descrierea PVC-ului de la Kubernetes \u00een format YAML:<\/p>\n<pre><code class=\"plaintext\">kubectl get pvc rbd-pvc -o yaml<\/code><\/pre>\n<p>Iat\u0103 \u0219i problema:<\/p>\n<p><em>message: Waiting for user to (re-)start a pod to finish file system resize of volume on node. type: FileSystemResizePending<\/em><\/p>\n<p>A\u0219a c\u0103 discul s-a m\u0103rit, dar sistemul de fi\u0219iere de pe acesta nu.<br \/>\nPentru a extinde sistemul de fi\u0219iere, trebuie s\u0103 mont\u0103m volumul. PVC-ul\/PV-ul nostru creat nu este utilizat acum \u00een niciun fel.<\/p>\n<p>Putem crea un Pod de test, de exemplu astfel:<\/p>\n<pre><code class=\"plaintext\">---\napiVersion: v1\nkind: Pod\nmetadata:\n  name: csi-rbd-demo-pod\nspec:\n  containers:\n    - name: web-server\n      image: nginx:1.17.6\n      volumeMounts:\n        - name: mypvc\n          mountPath: \/data\n  volumes:\n    - name: mypvc\n      persistentVolumeClaim:\n        claimName: rbd-pvc\n        readOnly: false<\/code><\/pre>\n<p>\u0218i acum s\u0103 ne uit\u0103m la PVC:<\/p>\n<pre><code class=\"plaintext\">kubectl get pvc<\/code><\/pre>\n<p>Dimensiunea s-a modificat, totul este \u00een regul\u0103.<\/p>\n<p>\u00cen prima parte am lucrat cu un dispozitiv de blocare RBD (care se refer\u0103 la Rados Block Device), dar nu este bine s\u0103 proced\u0103m astfel dac\u0103 se dore\u0219te acces simultan la acest disc din partea diferitelor microservicii. Pentru a lucra cu fi\u0219iere, mai degrab\u0103 dec\u00e2t cu o imagine de disc, CephFS este mult mai potrivit.<br \/>\nPe baza clusterelor Ceph \u0219i Kubernetes, vom configura CSI \u0219i celelalte entit\u0103\u021bi necesare pentru a lucra cu CephFS.<\/p>\n<p>Vom ob\u021bine valorile din noul Helm chart de care avem nevoie:<\/p>\n<pre><code class=\"plaintext\">helm inspect values ceph-csi\/ceph-csi-cephfs &gt; cephfs.yml<\/code><\/pre>\n<p>Din nou, trebuie s\u0103 complet\u0103m fi\u0219ierul cephfs.yml. A\u0219a cum am mai v\u0103zut, comenzile Ceph ne vor ajuta:<\/p>\n<pre><code class=\"plaintext\">ceph fsid\nceph mon dump<\/code><\/pre>\n<p>Complet\u0103m fi\u0219ierul cu valorile aproximativ astfel:<\/p>\n<pre><code class=\"plaintext\">csiConfig:\n  - clusterID: \"bcd0d202-fba8-4352-b25d-75c89258d5ab\"\n    monitors:\n      - \"172.18.8.5:6789\"\n      - \"172.18.8.6:6789\"\n      - \"172.18.8.7:6789\"\n\nnodeplugin:\n  httpMetrics:\n    enabled: true\n    containerPort: 8091\n  podSecurityPolicy:\n    enabled: true\n\nprovisioner:\n  replicaCount: 1\n  podSecurityPolicy:\n    enabled: true<\/code><\/pre>\n<p>Re\u021bine\u021bi c\u0103 adresele monitorilor sunt specificate \u00eentr-o form\u0103 simpl\u0103 address:port. Pentru a monta cephfs pe nod, aceste adrese sunt transmise modulului kernel, care \u00eenc\u0103 nu poate lucra cu protocolul monitorilor v2.<br \/>\nPortul pentru httpMetrics (unde Prometheus va ob\u021bine metricile pentru monitorizare) este schimbat pentru a nu intra \u00een conflict cu nginx-proxy, care este instalat de Kubespray. Aceasta, poate, nu va fi necesar pentru dvs.<\/p>\n<p>Instal\u0103m Helm chart-ul \u00een clusterul Kubernetes:<\/p>\n<pre><code class=\"plaintext\">helm upgrade -i ceph-csi-cephfs ceph-csi\/ceph-csi-cephfs -f cephfs.yml -n ceph-csi-cephfs --create-namespace<\/code><\/pre>\n<p>Trecem la stocarea datelor Ceph pentru a crea un utilizator separat. \u00cen documenta\u021bie se men\u021bioneaz\u0103 c\u0103 provisioner-ului CephFS \u00eei sunt necesare privilegii de acces de administrator al clusterului. Dar vom crea un utilizator separat <em>fs<\/em> cu drepturi limitate:<\/p>\n<pre><code class=\"plaintext\">ceph auth get-or-create client.fs mon 'allow r' mgr 'allow rw' mds 'allow rws' osd 'allow rw pool=cephfs_data, allow rw pool=cephfs_metadata'<\/code><\/pre>\n<p>\u0218i imediat vom verifica cheia de acces, de care ne va fi nevoie mai departe:<\/p>\n<pre><code class=\"plaintext\">ceph auth get-key client.fs<\/code><\/pre>\n<p>Vom crea un Secret \u0219i un StorageClass separate.<br \/>\nNimic nou, am v\u0103zut deja acest lucru pe baza RBD:<\/p>\n<pre><code class=\"plaintext\">---\napiVersion: v1\nkind: Secret\nmetadata:\n  name: csi-cephfs-secret\n  namespace: ceph-csi-cephfs\nstringData:\n  # Necesitate pentru volumele create dinamic\n  adminID: fs\n  adminKey:<\/code><\/pre>\n<p>Aplic\u0103m manifestul:<\/p>\n<pre><code class=\"plaintext\">kubectl apply -f secret.yaml<\/code><\/pre>\n<p>Iar acum \u2013 un StorageClass separat:<\/p>\n<pre><code class=\"plaintext\">---\napiVersion: storage.k8s.io\/v1\nkind: StorageClass\nmetadata:\n  name: csi-cephfs-sc\nprovisioner: cephfs.csi.ceph.com\nparameters:\n  clusterID: \n\n  # Numele sistemului de fi\u0219iere CephFS \u00een care va fi creat volum\n  fsName: cephfs\n\n  # (op\u021bional) Pool-ul Ceph \u00een care vor fi stocate datele volumului\n  # pool: cephfs_data\n\n  # (op\u021bional) Op\u021biuni de montare separate prin virgul\u0103 pentru Ceph-fuse\n  # de exemplu:\n  # fuseMountOptions: debug\n\n  # (op\u021bional) Op\u021biuni de montare CephFS pentru kernel, separate prin virgul\u0103\n  # Consulta\u021bi man mount.ceph pentru lista acestor op\u021biuni. De exemplu:\n  # kernelMountOptions: readdir_max_bytes=1048576,norbytes\n\n  # Secretele trebuie s\u0103 con\u021bin\u0103 acces pentru administrator \u0219i\/sau utilizatorul Ceph.\n  csi.storage.k8s.io\/provisioner-secret-name: csi-cephfs-secret\n  csi.storage.k8s.io\/provisioner-secret-namespace: ceph-csi-cephfs\n  csi.storage.k8s.io\/controller-expand-secret-name: csi-cephfs-secret\n  csi.storage.k8s.io\/controller-expand-secret-namespace: ceph-csi-cephfs\n  csi.storage.k8s.io\/node-stage-secret-name: csi-cephfs-secret\n  csi.storage.k8s.io\/node-stage-secret-namespace: ceph-csi-cephfs\n\n  # (op\u021bional) Driverul poate folosi fie ceph-fuse (fuse), \n  # fie ceph kernelclient (kernel).\n  # Dac\u0103 nu este specificat, se va utiliza montarea volumelor implicit,\n  # aceasta fiind definit\u0103 prin c\u0103utarea ceph-fuse \u0219i mount.ceph\n  # mounter: kernel\nreclaimPolicy: Delete\nallowVolumeExpansion: true\nmountOptions:\n  - debug<\/code><\/pre>\n<p>Vom completa aici <strong>clusterID<\/strong> \u0219i vom aplica \u00een Kubernetes:<\/p>\n<pre><code class=\"plaintext\">kubectl apply -f storageclass.yaml<\/code><\/pre>\n<h2 id=\"proverka\">Verificare<\/h2>\n<p>Pentru a verifica, la fel ca \u00een exemplul anterior, vom crea un PVC:<\/p>\n<pre><code class=\"plaintext\">---\napiVersion: v1\nkind: PersistentVolumeClaim\nmetadata:\n  name: csi-cephfs-pvc\nspec:\n  accessModes:\n    - ReadWriteMany\n  resources:\n    requests:\n      storage: 5Gi\n  storageClassName: csi-cephfs-sc<\/code><\/pre>\n<p>\u0218i vom verifica prezen\u021ba PVC\/PV:<\/p>\n<pre><code class=\"plaintext\">kubectl get pvc\nkubectl get pv<\/code><\/pre>\n<p>Dac\u0103 dori\u021bi s\u0103 vede\u021bi fi\u0219ierele \u0219i directoarele din CephFS, pute\u021bi monta acest sistem de fi\u0219iere undeva. De exemplu, a\u0219a cum este prezentat mai jos.<\/p>\n<p>S\u0103 ne ducem pe unul dintre nodurile clusterului Ceph \u0219i s\u0103 execut\u0103m aceste ac\u021biuni:<\/p>\n<pre><code class=\"plaintext\"># \u0422\u043e\u0447\u043a\u0430 \u043c\u043e\u043d\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f\nmkdir -p \/mnt\/cephfs\n\n# \u0421\u043e\u0437\u0434\u0430\u0451\u043c \u0444\u0430\u0439\u043b \u0441 \u043a\u043b\u044e\u0447\u043e\u043c \u0430\u0434\u043c\u0438\u043d\u0438\u0441\u0442\u0440\u0430\u0442\u043e\u0440\u0430\nceph auth get-key client.admin &gt;\/etc\/ceph\/secret.key\n\n# \u0414\u043e\u0431\u0430\u0432\u043b\u044f\u0435\u043c \u0437\u0430\u043f\u0438\u0441\u044c \u0432 \/etc\/fstab\n# !! \u0418\u0437\u043c\u0435\u043d\u044f\u0435\u043c ip \u0430\u0434\u0440\u0435\u0441 \u043d\u0430 \u0430\u0434\u0440\u0435\u0441 \u043d\u0430\u0448\u0435\u0433\u043e \u0443\u0437\u043b\u0430\necho \"172.18.8.6:6789:\/ \/mnt\/cephfs ceph name=admin,secretfile=\/etc\/ceph\/secret.key,noatime,_netdev    0       2\" &gt;&gt; \/etc\/fstab\n\nmount \/mnt\/cephfs<\/code><\/pre>\n<p>Desigur, o astfel de montare FS pe un nod Ceph este exclusiv pentru scopuri de \u00eenv\u0103\u021bare, ceea ce facem la cursurile noastre <noindex><a rel=\"nofollow\" href=\"http:\/\/slurm.io\">Slurm<\/a><\/noindex>. Nu cred c\u0103 cineva ar face asta \u00een produc\u021bie, riscul de a \u0219terge accidental fi\u0219iere importante este mare.<\/p>\n<p>\u0218i, \u00een cele din urm\u0103, s\u0103 verific\u0103m cum stau lucrurile cu schimbarea dimensiunii volumului \u00een cazul CephFS. Ne \u00eentoarcem \u00een Kubernetes \u0219i edit\u0103m manifestul nostru pentru PVC \u2014 vom cre\u0219te dimensiunea, de exemplu, la 7Gi.<\/p>\n<p>Vom aplica fi\u0219ierul editat:<\/p>\n<pre><code class=\"plaintext\">kubectl apply -f pvc.yaml<\/code><\/pre>\n<p>S\u0103 vedem \u00een directorul montat cum s-a schimbat limita:<\/p>\n<pre><code class=\"plaintext\">getfattr -n ceph.quota.max_bytes<\/code><\/pre>\n<p>Pentru a rula aceast\u0103 comand\u0103, este posibil s\u0103 trebuiasc\u0103 s\u0103 instala\u021bi \u00een sistem pachetul <strong>attr<\/strong>.<\/p>\n<h2 id=\"glaza-boyatsya-a-ruki-delayut\">Ochiul se teme, dar m\u00e2inile fac<\/h2>\n<p>La prima vedere, toate aceste incanta\u021bii \u0219i lungi manifeste YAML par complicate, dar \u00een practic\u0103, studen\u021bii de la Slyrm se descurc\u0103 destul de repede cu ele.<br \/>\n\u00cen acest articol nu ne-am aprofundat \u00een detalii \u2014 pentru asta exist\u0103 documenta\u021bia oficial\u0103. Dac\u0103 sunte\u021bi interesat de detaliile configur\u0103rii stoc\u0103rii Ceph \u00eempreun\u0103 cu clusterul Kubernetes, aceste linkuri v\u0103 vor ajuta:<\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/kubernetes.io\/docs\/concepts\/storage\/volumes\/\">Principiile generale de func\u021bionare ale Kubernetes cu volume<\/a><\/noindex><br \/>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/ceph.readthedocs.io\/en\/latest\/rbd\/\">Documenta\u021bia RBD<\/a><\/noindex><br \/>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/docs.ceph.com\/docs\/master\/rbd\/rbd-kubernetes\/\">Integrarea RBD \u0219i Kubernetes din perspectiva Ceph<\/a><\/noindex><br \/>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/ceph\/ceph-csi\/blob\/master\/docs\/deploy-rbd.md\">Integrarea RBD \u0219i Kubernetes din perspectiva CSI<\/a><\/noindex><br \/>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/ceph.readthedocs.io\/en\/latest\/cephfs\/\">Documenta\u021bia general\u0103 pentru CephFS<\/a><\/noindex><br \/>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/ceph\/ceph-csi\/blob\/master\/docs\/deploy-cephfs.md\">Integrarea CephFS \u0219i Kubernetes din perspectiva CSI<\/a><\/noindex><\/p>\n<p>\u00cen cursul Slyrm <noindex><a rel=\"nofollow\" href=\"http:\/\/to.slurm.io\/6p9lng\">Kubernetes Baza<\/a><\/noindex> pute\u021bi merge pu\u021bin mai departe \u0219i desf\u0103\u0219ura o aplica\u021bie real\u0103 \u00een Kubernetes care va folosi CephFS ca stocare pentru fi\u0219iere. Prin intermediul cererilor GET\/POST, ve\u021bi putea trimite \u0219i primi fi\u0219iere din Ceph.<\/p>\n<p>\u0218i dac\u0103 sunte\u021bi mai interesat de stocarea datelor, atunci \u00eenscrie\u021bi-v\u0103 la <noindex><a rel=\"nofollow\" href=\"http:\/\/to.slurm.io\/WkM2Eg\">cursul nou despre Ceph<\/a><\/noindex>. \u00cen timp ce cursul este \u00een testare beta, acesta poate fi ob\u021binut cu o reducere \u0219i ave\u021bi ocazia s\u0103 influen\u021ba\u021bi con\u021binutul s\u0103u.<\/p>\n<p><em>Autorul articolului: Alexander Shvalov, inginer practician <noindex><a rel=\"nofollow\" href=\"http:\/\/to.slurm.io\/ophLww\">Southbridge<\/a><\/noindex>, Administrator Kubernetes Certificat, autor \u0219i dezvoltator de cursuri Slyrm.<\/em><\/p>\n<p>Sursa: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/southbridge\/blog\/519642\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>Container Storage Interface (CSI) \u2014 \u044d\u0442\u043e \u0443\u043d\u0438\u0444\u0438\u0446\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0439 \u0438\u043d\u0442\u0435\u0440\u0444\u0435\u0439\u0441 \u0432\u0437\u0430\u0438\u043c\u043e\u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f Kubernetes \u0438 \u0441\u0438\u0441\u0442\u0435\u043c \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445. \u0412\u043a\u0440\u0430\u0442\u0446\u0435 \u043e \u043d\u0451\u043c \u043c\u044b \u0443\u0436\u0435 \u0440\u0430\u0441\u0441\u043a\u0430\u0437\u044b\u0432\u0430\u043b\u0438, \u0430 \u0441\u0435\u0433\u043e\u0434\u043d\u044f \u043f\u043e\u0434\u0440\u043e\u0431\u043d\u0435\u0435 \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0441\u0432\u044f\u0437\u043a\u0443 CSI \u0438 Ceph: \u043f\u043e\u043a\u0430\u0436\u0435\u043c, \u043a\u0430\u043a \u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0438\u0442\u044c \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0435 Ceph \u043a \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0443 Kubernetes. \u0412 \u0441\u0442\u0430\u0442\u044c\u0435 \u043f\u0440\u0438\u0432\u0435\u0434\u0435\u043d\u044b \u0440\u0435\u0430\u043b\u044c\u043d\u044b\u0435, \u0445\u043e\u0442\u044f \u0438 \u043d\u0435\u043c\u043d\u043e\u0433\u043e \u0443\u043f\u0440\u043e\u0449\u0451\u043d\u043d\u044b\u0435 \u0434\u043b\u044f \u0443\u0434\u043e\u0431\u0441\u0442\u0432\u0430 \u0432\u043e\u0441\u043f\u0440\u0438\u044f\u0442\u0438\u044f \u043f\u0440\u0438\u043c\u0435\u0440\u044b. \u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0443 \u0438 \u043d\u0430\u0441\u0442\u0440\u043e\u0439\u043a\u0443 \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u043e\u0432 Ceph \u0438 Kubernetes [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":94740,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-94739","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"Container Storage Interface (CSI) \u2014 \u044d\u0442\u043e \u0443\u043d\u0438\u0444\u0438\u0446\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0439 \u0438\u043d\u0442\u0435\u0440\u0444\u0435\u0439\u0441 \u0432\u0437\u0430\u0438\u043c\u043e\u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f Kubernetes \u0438 \u0441\u0438\u0441\u0442\u0435\u043c \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"ro_RO\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041f\u0440\u0430\u043a\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u043f\u0440\u0438\u043c\u0435\u0440 \u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0435\u043d\u0438\u044f \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0430 \u043d\u0430 \u0431\u0430\u0437\u0435 Ceph \u0432 \u043a\u043b\u0430\u0441\u0442\u0435\u0440 Kubernetes | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"Container Storage Interface (CSI) \u2014 \u044d\u0442\u043e \u0443\u043d\u0438\u0444\u0438\u0446\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0439 \u0438\u043d\u0442\u0435\u0440\u0444\u0435\u0439\u0441 \u0432\u0437\u0430\u0438\u043c\u043e\u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f Kubernetes \u0438 \u0441\u0438\u0441\u0442\u0435\u043c \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-09-20T17:42:27+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2021-01-02T11:05:08+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Exemplu practic de conectare a stoc\u0103rii bazate pe Ceph \u00een clusterul Kubernetes | ProHoster","description":"Container Storage Interface (CSI) este o interfa\u021b\u0103 unificat\u0103 de interac\u021biune \u00eentre Kubernetes \u0219i sistemele de stocare a datelor.","canonical_url":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"ro_RO","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041f\u0440\u0430\u043a\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u043f\u0440\u0438\u043c\u0435\u0440 \u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0435\u043d\u0438\u044f \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0430 \u043d\u0430 \u0431\u0430\u0437\u0435 Ceph \u0432 \u043a\u043b\u0430\u0441\u0442\u0435\u0440 Kubernetes | ProHoster","og:description":"Container Storage Interface (CSI) \u2014 \u044d\u0442\u043e \u0443\u043d\u0438\u0444\u0438\u0446\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0439 \u0438\u043d\u0442\u0435\u0440\u0444\u0435\u0439\u0441 \u0432\u0437\u0430\u0438\u043c\u043e\u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f Kubernetes \u0438 \u0441\u0438\u0441\u0442\u0435\u043c \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445.","og:url":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-09-20T17:42:27+00:00","article:modified_time":"2021-01-02T11:05:08+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"94739","title":null,"description":"","keywords":"","keyphrases":null,"primary_term":null,"canonical_url":"","og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 11:19:21","updated":"2022-09-30 10:03:10","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/94739","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/comments?post=94739"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/94739\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media\/94740"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media?parent=94739"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/categories?post=94739"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/tags?post=94739"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}