{"id":94739,"date":"2020-09-20T19:42:27","date_gmt":"2020-09-20T17:42:27","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes"},"modified":"2021-01-02T13:05:08","modified_gmt":"2021-01-02T11:05:08","slug":"prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sl\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes","title":{"rendered":"Prakti\u010den primer povezovanja shranjevanja na osnovi Ceph v Kubernetes klaster","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Container Storage Interface (CSI) je enotna vmesna plast za interakcijo med Kubernetesom in sistemi za shranjevanje podatkov. Kratko o njem smo \u017ee su pri\u010dali<\/a><\/noindex>, danes pa bomo podrobneje obravnavali povezavo CSI in Ceph: prikazali bomo, kako povezati shrambo Ceph<\/a><\/noindex> s Kubernetes klastrom.
\nV \u010dlanku so predstavljeni resni\u010dni, \u010deprav nekoliko poenostavljeni primeri za la\u017eje razumevanje. Namestitve in konfiguracije Ceph in Kubernetes klastrov ne obravnavamo.<\/p>\n

Vas zanima, kako to deluje?<\/p>\n

\"Prakti\u010den<\/p>\n

<\/a><\/noindex><\/p>\n

Torej, imate na voljo Kubernetes klaster, razporejen, na primer, kubespray<\/a><\/noindex>. Ob njem deluje klaster Ceph \u2014 ta se lahko prav tako namesti, na primer, s tem naborom igralk<\/a><\/noindex>. Upam, da ni treba omenjati, da mora biti za produkcijo med njima omre\u017eje s pasovno \u0161irino vsaj 10 Gb\/s.<\/p>\n

\u010ce imate vse to, gremo naprej!<\/p>\n

Najprej se prijavimo na eno od vozli\u0161\u010d klasterja Ceph in preverimo, ali je vse v redu:<\/p>\n

ceph health\nceph -s<\/code><\/pre>\n
Nato takoj ustvarimo bazen za RBD diske:<\/p>\n
ceph osd pool create kube 32\nceph osd pool application enable kube rbd<\/code><\/pre>\n
Preidemo v Kubernetes klster. Tam bomo najprej namestili Ceph CSI gonilnik za RBD. Namestitev bomo izvedli, kot je potrebno, preko Helma.
\nDodajamo repozitorij s chartom in pridobimo niz spremenljivk charta ceph-csi-rbd:<\/p>\n
helm repo add ceph-csi https:\/\/ceph.github.io\/csi-charts\nhelm inspect values ceph-csi\/ceph-csi-rbd > cephrbd.yml<\/code><\/pre>\n
Zdaj je treba izpolniti datoteko cephrbd.yml. Za to izvedemo ID klasterja in IP-naslove monitorjev v Ceph:<\/p>\n
ceph fsid  # tako ugotovimo clusterID\nceph mon dump  # in tako bomo videli IP-naslove monitorjev<\/code><\/pre>\n
Pridobljene vrednosti vnesemo v datoteko cephrbd.yml. Hkrati vklopimo ustvarjanje politik PSP (Pod Security Policies). Mo\u017enosti v razdelkih nodeplugin<\/strong> in provisioner<\/strong> so \u017ee v datoteki, jih lahko popravimo tako, kot je razlo\u017eeno spodaj:<\/p>\n
csiConfig:\n  - clusterID: \"bcd0d202-fba8-4352-b25d-75c89258d5ab\"\n    monitors:\n      - \"v2:172.18.8.5:3300\/0,v1:172.18.8.5:6789\/0\"\n      - \"v2:172.18.8.6:3300\/0,v1:172.18.8.6:6789\/0\"\n      - \"v2:172.18.8.7:3300\/0,v1:172.18.8.7:6789\/0\"\n\nnodeplugin:\n  podSecurityPolicy:\n    enabled: true\n\nprovisioner:\n  podSecurityPolicy:\n    enabled: true<\/code><\/pre>\n
Nato je vse, kar nam ostane, namestiti chart v Kubernetes kluster.<\/p>\n
helm upgrade -i ceph-csi-rbd ceph-csi\/ceph-csi-rbd -f cephrbd.yml -n ceph-csi-rbd --create-namespace<\/code><\/pre>\n
Odli\u010dno, RBD gonilnik deluje!
\nUstvarimo v Kubernetes nov StorageClass. Za to bo treba spet malo delati s Ceph.<\/p>\n
Ustvarjamo novega uporabnika v Ceph in mu dodelimo pravice za pisanje v bazen kube<\/strong>:<\/p>\n
ceph auth get-or-create client.rbdkube mon 'profile rbd' osd 'profile rbd pool=kube'<\/code><\/pre>\n
Zdaj pa si oglejmo klju\u010de dostopa vse tam:<\/p>\n
ceph auth get-key client.rbdkube<\/code><\/pre>\n
Ukaz bo prikazal nekaj takega:<\/p>\n
AQCO9NJbhYipKRAAMqZsnqqS\/T8OYQX20xIa9A==<\/code><\/pre>\n
Zabele\u017eimo to vrednost v Secret v klastru Kubernetes \u2014 tja, kjer jo potrebujemo userKey<\/strong>:<\/p>\n
---\napiVersion: v1\nkind: Secret\nmetadata:\n  name: csi-rbd-secret\n  namespace: ceph-csi-rbd\nstringData:\n  # Klju\u010dne vrijednosti odgovaraju korisni\u010dkom imenu i njegovom klju\u010du, kako je navedeno u\n  # Ceph klasteru. ID korisnika mora imati pristup bazenu,\n  # navedenom u storage class\n  userID: rbdkube\n  userKey:<\/code><\/pre>\n
I stvaramo na\u0161 tajni klju\u010d:<\/p>\n
kubectl apply -f secret.yaml<\/code><\/pre>\n
Zatim nam treba otprilike ovakav manifest StorageClass:<\/p>\n
---\napiVersion: storage.k8s.io\/v1\nkind: StorageClass\nmetadata:\n   name: csi-rbd-sc\nprovisioner: rbd.csi.ceph.com\nparameters:\n   clusterID: \n   pool: kube\n\n   imageFeatures: layering\n\n   # Ovi tajni podaci trebaju sadr\u017eavati informacije za autentifikaciju\n   # u va\u0161 bazen.\n   csi.storage.k8s.io\/provisioner-secret-name: csi-rbd-secret\n   csi.storage.k8s.io\/provisioner-secret-namespace: ceph-csi-rbd\n   csi.storage.k8s.io\/controller-expand-secret-name: csi-rbd-secret\n   csi.storage.k8s.io\/controller-expand-secret-namespace: ceph-csi-rbd\n   csi.storage.k8s.io\/node-stage-secret-name: csi-rbd-secret\n   csi.storage.k8s.io\/node-stage-secret-namespace: ceph-csi-rbd\n\n   csi.storage.k8s.io\/fstype: ext4\n\nreclaimPolicy: Delete\nallowVolumeExpansion: true\nmountOptions:\n  - discard<\/code><\/pre>\n
Treba popuniti clusterID<\/strong>, kojeg smo ve\u0107 saznali naredbom ceph fsid<\/em>, i primijeniti ovaj manifest u Kubernetes klasteru:<\/p>\n
kubectl apply -f storageclass.yaml<\/code><\/pre>\n
Da bismo provjerili rad klastera u suradnji, stvorimo ovakav PVC (Persistent Volume Claim):<\/p>\n
apiVersion: v1\nkind: PersistentVolumeClaim\nmetadata:\n  name: rbd-pvc\nspec:\n  accessModes:\n  - ReadWriteOnce\n  resources:\n    requests:\n      storage: 1Gi\n  storageClassName: csi-rbd-sc<\/code><\/pre>\n
Odmah \u0107emo provjeriti kako je Kubernetes stvorio tra\u017eeni volumen u Ceph-u:<\/p>\n
kubectl get pvc\nkubectl get pv<\/code><\/pre>\n
Izgleda odli\u010dno! Kako to izgleda s Ceph strane?
\nDobivamo popis volumena u bazenu i pregledavamo informacije o na\u0161em volumenu:<\/p>\n
rbd ls -p kube\nrbd -p kube info csi-vol-eb3d257d-8c6c-11ea-bff5-6235e7640653  # ovdje \u0107e, naravno, biti druga\u010diji ID volumena koji je prethodna naredba izdala<\/code><\/pre>\n
Sada pogledajmo kako funkcionira promjena veli\u010dine RBD volumena.
\nPromijenimo veli\u010dinu volumena u manifestu pvc.yaml na 2Gi i primijenimo ga:<\/p>\n
kubectl apply -f pvc.yaml<\/code><\/pre>\n
Sa\u010dekajmo da promjene stupe na snagu i jo\u0161 jednom provjerimo veli\u010dinu volumena.<\/p>\n
rbd -p kube info csi-vol-eb3d257d-8c6c-11ea-bff5-6235e7640653\n\nkubectl get pv\nkubectl get pvc<\/code><\/pre>\n
Vidimo da veli\u010dina PVC nije promijenjena. Da bismo saznali razlog, mo\u017eemo zatra\u017eiti od Kubernetesa opisati PVC u YAML formatu:<\/p>\n
kubectl get pvc rbd-pvc -o yaml<\/code><\/pre>\n
A evo i problema:<\/p>\n
message: \u010cekanje da korisnik (re-)pokrene pod za zavr\u0161etak promjene veli\u010dine datote\u010dnog sustava volumena na \u010dvoru. type: FileSystemResizePending<\/em><\/p>\n
Dakle, disk se pove\u0107ao, ali datote\u010dni sustav na njemu \u2014 nije.
\nDa bismo pove\u0107ali datote\u010dni sustav, moramo montirati volumen. Na\u0161 PVC\/PV trenutno nije u upotrebi.<\/p>\n
Mo\u017eemo stvoriti testni Pod, na primjer ovako:<\/p>\n
---\napiVersion: v1\nkind: Pod\nmetadata:\n  name: csi-rbd-demo-pod\nspec:\n  containers:\n    - name: web-server\n      image: nginx:1.17.6\n      volumeMounts:\n        - name: mypvc\n          mountPath: \/data\n  volumes:\n    - name: mypvc\n      persistentVolumeClaim:\n        claimName: rbd-pvc\n        readOnly: false<\/code><\/pre>\n
In now let's take a look at the PVC:<\/p>\n
kubectl get pvc<\/code><\/pre>\n
The size has changed, everything is fine.<\/p>\n
In the first part we worked with the RBD block device (which stands for Rados Block Device), but this approach is not suitable if different microservices need to access this disk concurrently. For working with files rather than a disk image, CephFS is a much better choice.
\nWe will set up CSI and the other necessary entities for working with CephFS using Ceph and Kubernetes clusters as an example.<\/p>\n
We will obtain values from the relevant new Helm chart:<\/p>\n
helm inspect values ceph-csi\/ceph-csi-cephfs > cephfs.yml<\/code><\/pre>\n
Once again, we need to fill in the cephfs.yml file. As before, Ceph commands will assist us:<\/p>\n
ceph fsid\nceph mon dump<\/code><\/pre>\n
We fill in the values file approximately like this:<\/p>\n
csiConfig:\n  - clusterID: \"bcd0d202-fba8-4352-b25d-75c89258d5ab\"\n    monitors:\n      - \"172.18.8.5:6789\"\n      - \"172.18.8.6:6789\"\n      - \"172.18.8.7:6789\"\n\nnodeplugin:\n  httpMetrics:\n    enabled: true\n    containerPort: 8091\n  podSecurityPolicy:\n    enabled: true\n\nprovisioner:\n  replicaCount: 1\n  podSecurityPolicy:\n    enabled: true<\/code><\/pre>\n
Note that the addresses of the monitors are specified in a simple form address:port. To mount cephfs on the node, these addresses are passed to the kernel module, which does not yet support the v2 monitor protocol.
\nWe change the port for httpMetrics (where Prometheus will fetch metrics for monitoring) so that it does not conflict with the nginx-proxy set up by Kubespray. You might not need this.<\/p>\n
We install the Helm chart in the Kubernetes cluster:<\/p>\n
helm upgrade -i ceph-csi-cephfs ceph-csi\/ceph-csi-cephfs -f cephfs.yml -n ceph-csi-cephfs --create-namespace<\/code><\/pre>\n
We go to the Ceph data storage to create a separate user there. The documentation states that the CephFS provisioner needs admin privileges to the cluster. However, we will create a separate user fs<\/em> with limited permissions:<\/p>\n
ceph auth get-or-create client.fs mon 'allow r' mgr 'allow rw' mds 'allow rws' osd 'allow rw pool=cephfs_data, allow rw pool=cephfs_metadata'<\/code><\/pre>\n
And now let's check its access key, which will be useful later:<\/p>\n
ceph auth get-key client.fs<\/code><\/pre>\n
We will create separate Secret and StorageClass.
\nNothing new, we've already seen this with RBD:<\/p>\n
---\napiVersion: v1\nkind: Secret\nmetadata:\n  name: csi-cephfs-secret\n  namespace: ceph-csi-cephfs\nstringData:\n  # Required for dynamically created volumes\n  adminID: fs\n  adminKey:<\/code><\/pre>\n
We apply the manifest:<\/p>\n
kubectl apply -f secret.yaml<\/code><\/pre>\n
And now \u2013 a separate StorageClass:<\/p>\n
---\napiVersion: storage.k8s.io\/v1\nkind: StorageClass\nmetadata:\n  name: csi-cephfs-sc\nprovisioner: cephfs.csi.ceph.com\nparameters:\n  clusterID: \n\n  # The name of the CephFS file system where the volume will be created\n  fsName: cephfs\n\n  # (optional) Ceph pool where the volume data will be stored\n  # pool: cephfs_data\n\n  # (optional) Comma-separated mount options for Ceph-fuse\n  # e.g.:\n  # fuseMountOptions: debug\n\n  # (optional) Comma-separated mount options for CephFS for kernel\n  # See man mount.ceph for a list of these options. For example:\n  # kernelMountOptions: readdir_max_bytes=1048576,norbytes\n\n  # Secrets must contain access for admin and\/or user of Ceph.\n  csi.storage.k8s.io\/provisioner-secret-name: csi-cephfs-secret\n  csi.storage.k8s.io\/provisioner-secret-namespace: ceph-csi-cephfs\n  csi.storage.k8s.io\/controller-expand-secret-name: csi-cephfs-secret\n  csi.storage.k8s.io\/controller-expand-secret-namespace: ceph-csi-cephfs\n  csi.storage.k8s.io\/node-stage-secret-name: csi-cephfs-secret\n  csi.storage.k8s.io\/node-stage-secret-namespace: ceph-csi-cephfs\n\n  # (optional) The driver can use either ceph-fuse (fuse), \n  # or ceph kernelclient (kernel).\n  # If not specified, the default volume mounting will be used,\n  # determined by searching for ceph-fuse and mount.ceph\n  # mounter: kernel\nreclaimPolicy: Delete\nallowVolumeExpansion: true\nmountOptions:\n  - debug<\/code><\/pre>\n
Let's fill this in here clusterID<\/strong> and apply it in Kubernetes:<\/p>\n
kubectl apply -f storageclass.yaml<\/code><\/pre>\n
\u041f\u0440\u043e\u0432\u0435\u0440\u043a\u0430<\/h2>\n
To check, as in the previous example, we will create a PVC:<\/p>\n
---\napiVersion: v1\nkind: PersistentVolumeClaim\nmetadata:\n  name: csi-cephfs-pvc\nspec:\n  accessModes:\n    - ReadWriteMany\n  resources:\n    requests:\n      storage: 5Gi\n  storageClassName: csi-cephfs-sc<\/code><\/pre>\n
And check for the presence of PVC\/PV:<\/p>\n
kubectl get pvc\nkubectl get pv<\/code><\/pre>\n
If you want to look at the files and directories in CephFS, you can mount this file system somewhere. For example, as shown below.<\/p>\n
Let's go to one of the nodes in the Ceph cluster and perform the following actions:<\/p>\n
# \u0422\u043e\u0447\u043a\u0430 \u043c\u043e\u043d\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f\nmkdir -p \/mnt\/cephfs\n\n# \u0421\u043e\u0437\u0434\u0430\u0451\u043c \u0444\u0430\u0439\u043b \u0441 \u043a\u043b\u044e\u0447\u043e\u043c \u0430\u0434\u043c\u0438\u043d\u0438\u0441\u0442\u0440\u0430\u0442\u043e\u0440\u0430\nceph auth get-key client.admin >\/etc\/ceph\/secret.key\n\n# \u0414\u043e\u0431\u0430\u0432\u043b\u044f\u0435\u043c \u0437\u0430\u043f\u0438\u0441\u044c \u0432 \/etc\/fstab\n# !! \u0418\u0437\u043c\u0435\u043d\u044f\u0435\u043c ip \u0430\u0434\u0440\u0435\u0441 \u043d\u0430 \u0430\u0434\u0440\u0435\u0441 \u043d\u0430\u0448\u0435\u0433\u043e \u0443\u0437\u043b\u0430\necho \"172.18.8.6:6789:\/ \/mnt\/cephfs ceph name=admin,secretfile=\/etc\/ceph\/secret.key,noatime,_netdev    0       2\" >> \/etc\/fstab\n\nmount \/mnt\/cephfs<\/code><\/pre>\n
Of course, this kind of FS mounting on a Ceph node is suitable only for educational purposes, which we are engaged in on our Slurm courses<\/a><\/noindex>. I don't think anyone would do this in production, as there is a high risk of accidentally overwriting important files.<\/p>\n
And lastly, let's check how the volume resizing works with CephFS. We return to Kubernetes and edit our PVC manifest \u2014 let's increase the size, for example, to 7Gi.<\/p>\n
We will apply the edited file:<\/p>\n
kubectl apply -f pvc.yaml<\/code><\/pre>\n
Let's check the mounted directory to see how the quota has changed:<\/p>\n
getfattr -n ceph.quota.max_bytes<\/code><\/pre>\n
For this command to work, you may need to install the package in your system attr<\/strong>.<\/p>\n
The eyes are afraid, but the hands are doing<\/h2>\n
Na prvi pogled, svi ovi alati i dugi YAML manifesti izgledaju slo\u017eeno, ali u praksi studenti Sl\u00ebrma se brzo snalaze sa njima.
\nU ovom \u010dlanku se ne\u0107emo zate\u0107i u dubinama \u2014 za to postoji zvani\u010dna dokumentacija. Ako vas zanimaju detalji konfiguracije Ceph skladi\u0161ta u vezi sa Kubernetes klasterom, pomo\u0107i \u0107e vam sljede\u0107e veze:<\/p>\n
Op\u0161ti principi rada Kubernetes-a sa volumenima<\/a><\/noindex>
\nDokumentacija o RBD-u<\/a><\/noindex>
\nIntegracija RBD-a i Kubernetes-a iz perspektive Ceph-a<\/a><\/noindex>
\nIntegracija RBD-a i Kubernetes-a iz perspektive CSI-a<\/a><\/noindex>
\nOp\u0161ta dokumentacija o CephFS-u<\/a><\/noindex>
\nIntegracija CephFS-a i Kubernetes-a iz perspektive CSI-a<\/a><\/noindex><\/p>\n
Na kursu Sl\u00ebrm Kubernetes Osnova<\/a><\/noindex> mo\u017eete da odete jo\u0161 malo dalje i implementirate stvarnu aplikaciju u Kubernetes-u koja \u0107e koristiti CephFS kao skladi\u0161te za datoteke. Putem GET\/POST zahteva mo\u0107i \u0107ete da \u0161aljete datoteke i preuzimate ih iz Ceph-a.<\/p>\n
A ako vas vi\u0161e zanimaju podaci za skladi\u0161tenje, prijavite se na novi kurs o Ceph-u<\/a><\/noindex>. Dok traje beta test, kurs mo\u017eete dobiti sa popustom i uticati na njegov sadr\u017eaj.<\/p>\n
Autor \u010dlanka: Aleksandar \u0160valov, prakti\u010dni in\u017eenjer Southbridge<\/a><\/noindex>, Certified Kubernetes Administrator, autor i developer kurseva Sl\u00ebrm.<\/em><\/p>\n
Izvor: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"
Container Storage Interface (CSI) \u2014 \u044d\u0442\u043e \u0443\u043d\u0438\u0444\u0438\u0446\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0439 \u0438\u043d\u0442\u0435\u0440\u0444\u0435\u0439\u0441 \u0432\u0437\u0430\u0438\u043c\u043e\u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f Kubernetes \u0438 \u0441\u0438\u0441\u0442\u0435\u043c \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445. \u0412\u043a\u0440\u0430\u0442\u0446\u0435 \u043e \u043d\u0451\u043c \u043c\u044b \u0443\u0436\u0435 \u0440\u0430\u0441\u0441\u043a\u0430\u0437\u044b\u0432\u0430\u043b\u0438, \u0430 \u0441\u0435\u0433\u043e\u0434\u043d\u044f \u043f\u043e\u0434\u0440\u043e\u0431\u043d\u0435\u0435 \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0441\u0432\u044f\u0437\u043a\u0443 CSI \u0438 Ceph: \u043f\u043e\u043a\u0430\u0436\u0435\u043c, \u043a\u0430\u043a \u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0438\u0442\u044c \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0435 Ceph \u043a \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0443 Kubernetes. \u0412 \u0441\u0442\u0430\u0442\u044c\u0435 \u043f\u0440\u0438\u0432\u0435\u0434\u0435\u043d\u044b \u0440\u0435\u0430\u043b\u044c\u043d\u044b\u0435, \u0445\u043e\u0442\u044f \u0438 \u043d\u0435\u043c\u043d\u043e\u0433\u043e \u0443\u043f\u0440\u043e\u0449\u0451\u043d\u043d\u044b\u0435 \u0434\u043b\u044f \u0443\u0434\u043e\u0431\u0441\u0442\u0432\u0430 \u0432\u043e\u0441\u043f\u0440\u0438\u044f\u0442\u0438\u044f \u043f\u0440\u0438\u043c\u0435\u0440\u044b. \u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0443 \u0438 \u043d\u0430\u0441\u0442\u0440\u043e\u0439\u043a\u0443 \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u043e\u0432 Ceph \u0438 Kubernetes […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":94740,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-94739","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Prakti\u010dan primer povezivanja skladi\u0161ta zasnovanog na Ceph-u u Kubernetes klaster | ProHoster","description":"Container Storage Interface (CSI) je ujedna\u010den interfejs za interakciju Kubernetes-a i sistema za skladi\u0161tenje podataka.","canonical_url":"https:\/\/prohoster.info\/sl\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sl_SI","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041f\u0440\u0430\u043a\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u043f\u0440\u0438\u043c\u0435\u0440 \u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0435\u043d\u0438\u044f \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0430 \u043d\u0430 \u0431\u0430\u0437\u0435 Ceph \u0432 \u043a\u043b\u0430\u0441\u0442\u0435\u0440 Kubernetes | ProHoster","og:description":"Container Storage Interface (CSI) \u2014 \u044d\u0442\u043e \u0443\u043d\u0438\u0444\u0438\u0446\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0439 \u0438\u043d\u0442\u0435\u0440\u0444\u0435\u0439\u0441 \u0432\u0437\u0430\u0438\u043c\u043e\u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f Kubernetes \u0438 \u0441\u0438\u0441\u0442\u0435\u043c \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445.","og:url":"https:\/\/prohoster.info\/sl\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-09-20T17:42:27+00:00","article:modified_time":"2021-01-02T11:05:08+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"94739","title":null,"description":"","keywords":"","keyphrases":null,"primary_term":null,"canonical_url":"","og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 11:19:21","updated":"2022-09-30 10:03:10","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/posts\/94739","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/comments?post=94739"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/posts\/94739\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/media\/94740"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/media?parent=94739"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/categories?post=94739"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sl\/wp-json\/wp\/v2\/tags?post=94739"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}