{"id":94739,"date":"2020-09-20T19:42:27","date_gmt":"2020-09-20T17:42:27","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes"},"modified":"2021-01-02T13:05:08","modified_gmt":"2021-01-02T11:05:08","slug":"prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes","status":"publish","type":"post","link":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes","title":{"rendered":"Praktiline n\u00e4ide Ceph-p\u00f5hise salvestuse \u00fchendamisest Kubernetes'i klastri juurde","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Container Storage Interface (CSI) on \u00fchtne liides Kubernetes'e ja andmesalvestuss\u00fcsteemide vahel. L\u00fchidalt oleme juba r\u00e4\u00e4kinud selle toimimisest, r\u00e4\u00e4kinud<\/a><\/noindex>, ja t\u00e4na vaatame l\u00e4hemalt CSI ja Ceph'i seost: n\u00e4itame, kuidas \u00fchendada Ceph'i salvestust<\/a><\/noindex> Kubernetes'i klastriga.
\nArtiklis on esitatud reaalsed, kuigi veidi lihtsustatud n\u00e4ited mugavamaks arusaamiseks. Ceph'i ja Kubernetes'i klastrite paigaldamist ja seadistamist me ei k\u00e4sitle.<\/p>\n

Kas teil on huvi, kuidas see toimib?<\/p>\n

\"Praktiline<\/p>\n

<\/a><\/noindex><\/p>\n

Oletame, et teil on k\u00e4ep\u00e4rast Kubernetes'i klaster, mis on seadistatud n\u00e4iteks kubespray<\/a><\/noindex>. K\u00f5rval t\u00f6\u00f6tab Ceph'i klaster \u2013 selle saab samuti paigaldada n\u00e4iteks selle playbookide kogumi<\/a><\/noindex>. Loodetavasti ei ole vaja mainida, et tootmis\u00fclesannetes peab nende vahel olema v\u00f5rk, mille ribalaius on v\u00e4hemalt 10 Gbit\/s.<\/p>\n

Kui see k\u00f5ik on teil olemas, siis asume asja kallale!<\/p>\n

Esmalt logime sisse \u00fchte Ceph'i klastri s\u00f5lme ja kontrollime, et k\u00f5ik on korras:<\/p>\n

ceph health\nceph -s<\/code><\/pre>\n
Seej\u00e4rel loome kohe RBD-kettaste jaoks basseini:<\/p>\n
ceph osd pool create kube 32\nceph osd pool application enable kube rbd<\/code><\/pre>\n
Liigume Kubernetes'i klastrisse. Esmalt paigaldame Ceph CSI draiveri RBD jaoks. Paigaldame selle nagu ikka l\u00e4bi Helm'i.
\nLisame chart'i reposti ning saame ceph-csi-rbd chart'i muutuja komplekti:<\/p>\n
helm repo add ceph-csi https:\/\/ceph.github.io\/csi-charts\nhelm inspect values ceph-csi\/ceph-csi-rbd > cephrbd.yml<\/code><\/pre>\n
N\u00fc\u00fcd tuleb t\u00e4ita fail cephrbd.yml. Selleks saame teada klastri ID ja Ceph'i monitoride IP-aadressid:<\/p>\n
ceph fsid  # nii saame teada clusterID\nceph mon dump  # ja nii n\u00e4eme monitoride IP-aadresse<\/code><\/pre>\n
Saadud v\u00e4\u00e4rtused paneme faili cephrbd.yml. Samuti l\u00fclitame sisse PSP (Pod Security Policies) loomise. Valikud jaotistes nodeplugin<\/strong> ja provisioner<\/strong> on juba failis olemas, neid saab muuta nagu allpool n\u00e4idatud:<\/p>\n
csiConfig:\n  - clusterID: \"bcd0d202-fba8-4352-b25d-75c89258d5ab\"\n    monitors:\n      - \"v2:172.18.8.5:3300\/0,v1:172.18.8.5:6789\/0\"\n      - \"v2:172.18.8.6:3300\/0,v1:172.18.8.6:6789\/0\"\n      - \"v2:172.18.8.7:3300\/0,v1:172.18.8.7:6789\/0\"\n\nnodeplugin:\n  podSecurityPolicy:\n    enabled: true\n\nprovisioner:\n  podSecurityPolicy:\n    enabled: true<\/code><\/pre>\n
Seej\u00e4rel j\u00e4\u00e4b alles vaid paigaldada chart Kubernetes'i klastrisse.<\/p>\n
helm upgrade -i ceph-csi-rbd ceph-csi\/ceph-csi-rbd -f cephrbd.yml -n ceph-csi-rbd --create-namespace<\/code><\/pre>\n
Suurep\u00e4rane, RBD draiver t\u00f6\u00f6tab!
\nLoodame Kubernetes'is uut StorageClass'i. Selleks on j\u00e4lle vaja natuke t\u00f6\u00f6d Ceph'iga.<\/p>\n
Loome Ceph'is uue kasutaja ja anname talle \u00f5iguse kirjutada basseini kube<\/strong>:<\/p>\n
ceph auth get-or-create client.rbdkube mon 'profile rbd' osd 'profile rbd pool=kube'<\/code><\/pre>\n
Ja n\u00fc\u00fcd vaatame p\u00e4\u00e4sukoodi k\u00f5ik ikka seal:<\/p>\n
ceph auth get-key client.rbdkube<\/code><\/pre>\n
K\u00e4sk v\u00e4ljastab midagi sellist:<\/p>\n
AQCO9NJbhYipKRAAMqZsnqqS\/T8OYQX20xIa9A==<\/code><\/pre>\n
Kandke see v\u00e4\u00e4rtus Kubernetes'i klastris Secretisse - sinna, kus see vajalik on userKey<\/strong>:<\/p>\n
---\napiVersion: v1\nkind: Secret\nmetadata:\n  name: csi-rbd-secret\n  namespace: ceph-csi-rbd\nstringData:\n  # V\u00f5tme v\u00e4\u00e4rtused vastavad kasutajanimele ja tema v\u00f5ti, nagu on m\u00e4\u00e4ratud\n  # Ceph klastris. Kasutaja ID-l peab olema ligip\u00e4\u00e4s basseinile,\n  # mis on m\u00e4\u00e4ratud salvestusklassis\n  userID: rbdkube\n  userKey:<\/code><\/pre>\n
Ja loome meie salajase v\u00f5tme:<\/p>\n
kubectl apply -f secret.yaml<\/code><\/pre>\n
Edasi vajame umbes sellist StorageClass'i manifesti:<\/p>\n
---\napiVersion: storage.k8s.io\/v1\nkind: StorageClass\nmetadata:\n   name: csi-rbd-sc\nprovisioner: rbd.csi.ceph.com\nparameters:\n   clusterID: \n   pool: kube\n\n   imageFeatures: layering\n\n   # Need saladused peavad sisaldama andmeid autentimiseks\n   # teie basseiniga.\n   csi.storage.k8s.io\/provisioner-secret-name: csi-rbd-secret\n   csi.storage.k8s.io\/provisioner-secret-namespace: ceph-csi-rbd\n   csi.storage.k8s.io\/controller-expand-secret-name: csi-rbd-secret\n   csi.storage.k8s.io\/controller-expand-secret-namespace: ceph-csi-rbd\n   csi.storage.k8s.io\/node-stage-secret-name: csi-rbd-secret\n   csi.storage.k8s.io\/node-stage-secret-namespace: ceph-csi-rbd\n\n   csi.storage.k8s.io\/fstype: ext4\n\nreclaimPolicy: Delete\nallowVolumeExpansion: true\nmountOptions:\n  - discard<\/code><\/pre>\n
Peame t\u00e4itma clusterID<\/strong>, mille me juba teadsime k\u00e4su ceph fsid<\/em>, ja rakendama selle manifesti Kubernetes'i klastris:<\/p>\n
kubectl apply -f storageclass.yaml<\/code><\/pre>\n
Klastrite t\u00f6\u00f6 kontrollimiseks loome sellise PVC (Persistent Volume Claim):<\/p>\n
apiVersion: v1\nkind: PersistentVolumeClaim\nmetadata:\n  name: rbd-pvc\nspec:\n  accessModes:\n  - ReadWriteOnce\n  resources:\n    requests:\n      storage: 1Gi\n  storageClassName: csi-rbd-sc<\/code><\/pre>\n
Vaadakem kohe, kuidas Kubernetes loob Ceph'is taotletud mahtu:<\/p>\n
kubectl get pvc\nkubectl get pv<\/code><\/pre>\n
Tundub, et k\u00f5ik on suurep\u00e4rane! Kuidas see paistab Ceph'i k\u00fcljest?
\nSaame nimekirja basseinist ja vaatame meie mahu kohta teavet:<\/p>\n
rbd ls -p kube\nrbd -p kube info csi-vol-eb3d257d-8c6c-11ea-bff5-6235e7640653  # siin on muidugi teine mahu ID, mille eelmine k\u00e4sk tagastas<\/code><\/pre>\n
N\u00fc\u00fcd vaadakem, kuidas RBD mahu suuruse muutmine t\u00f6\u00f6tab.
\nMuudame mahu suuruse manifestis pvc.yaml 2Gi-ks ja rakendame selle:<\/p>\n
kubectl apply -f pvc.yaml<\/code><\/pre>\n
Ootame, kuni muutused j\u00f5ustuvad, ja vaatame veel kord mahu suurust.<\/p>\n
rbd -p kube info csi-vol-eb3d257d-8c6c-11ea-bff5-6235e7640653\n\nkubectl get pv\nkubectl get pvc<\/code><\/pre>\n
N\u00e4eme, et PVC suurus ei muutunud. Probleemi v\u00e4ljaselgitamiseks v\u00f5ime k\u00fcsida Kubernetes'e YAML formaadis PVC kirjeldust:<\/p>\n
kubectl get pvc rbd-pvc -o yaml<\/code><\/pre>\n
Ja siin on probleem:<\/p>\n
message: Ootame kasutajat, et (uuesti) k\u00e4ivitada pod, et l\u00f5petada mahu failis\u00fcsteemi suuruse muutmine s\u00f5lmel. type: FileSystemResizePending<\/em><\/p>\n
Ehkki ketas suurenes, ei suurenenud selles failis\u00fcsteem.
\nFailis\u00fcsteemi suurendamiseks tuleb maht mountida. Meil on loodud PVC\/PV, mida hetkel ei kasutata.<\/p>\n
Saame luua katse Pod'i, n\u00e4iteks nii:<\/p>\n
---\napiVersion: v1\nkind: Pod\nmetadata:\n  name: csi-rbd-demo-pod\nspec:\n  containers:\n    - name: web-server\n      image: nginx:1.17.6\n      volumeMounts:\n        - name: mypvc\n          mountPath: \/data\n  volumes:\n    - name: mypvc\n      persistentVolumeClaim:\n        claimName: rbd-pvc\n        readOnly: false<\/code><\/pre>\n
Ja vaatame PVC-d:<\/p>\n
kubectl get pvc<\/code><\/pre>\n
Suurus on muutunud, k\u00f5ik on korras.<\/p>\n
Esimeses osas t\u00f6\u00f6tasime RBD (Rados Block Device) plokiseadmestikuga, kuid see ei sobi, kui erinevad mikroteenused peavad sama kettaga samaaegselt t\u00f6\u00f6tama. Failide t\u00f6\u00f6tlemiseks sobib CephFS oluliselt paremini kui kettapilt.
\nSeet\u00f5ttu seadistame Ceph ja Kubernetes klastrite n\u00e4itel CSI ja muud vajalikud \u00fcksused CephFS-i kasutamiseks.<\/p>\n
Saame vajalikud v\u00e4\u00e4rtused meie uue Helm-chart'i jaoks:<\/p>\n
helm inspect values ceph-csi\/ceph-csi-cephfs > cephfs.yml<\/code><\/pre>\n
Peame uuesti t\u00e4itma faili cephfs.yml. Nagu varem, aitavad Ceph'i k\u00e4sud:<\/p>\n
ceph fsid\nceph mon dump<\/code><\/pre>\n
T\u00e4idame faili v\u00e4\u00e4rtustega umbes nii:<\/p>\n
csiConfig:\n  - clusterID: \"bcd0d202-fba8-4352-b25d-75c89258d5ab\"\n    monitors:\n      - \"172.18.8.5:6789\"\n      - \"172.18.8.6:6789\"\n      - \"172.18.8.7:6789\"\n\nnodeplugin:\n  httpMetrics:\n    enabled: true\n    containerPort: 8091\n  podSecurityPolicy:\n    enabled: true\n\nprovisioner:\n  replicaCount: 1\n  podSecurityPolicy:\n    enabled: true<\/code><\/pre>\n
Pange t\u00e4hele, et monitooringu aadresse esitatakse lihtsas vormis aadress:port. CephFS-i mountimiseks s\u00f5lmes edastatakse need aadressid tuumamoodulile, mis ei oska veel t\u00f6\u00f6tada v2 monitooringu protokolliga.
\nMuutame httpMetrics'i porti (sinna saadab Prometheus j\u00e4lgimiseks m\u00f5\u00f5dikud), et v\u00e4ltida konflikti nginx-proxyga, mille installeerib Kubespray. Teie jaoks see v\u00f5ib-olla vajalik ei ole.<\/p>\n
Paigaldame Helm-chart'i Kubernetes klastrisse:<\/p>\n
helm upgrade -i ceph-csi-cephfs ceph-csi\/ceph-csi-cephfs -f cephfs.yml -n ceph-csi-cephfs --create-namespace<\/code><\/pre>\n
L\u00e4heme Ceph andmetehoidu, et luua seal eraldi kasutaja. Dokumentatsioonis on \u00f6eldud, et CephFS-i provisjonimiseks on vaja klastrihalduri \u00f5igusi. Kuid loome eraldi kasutaja fs<\/em> piiratud \u00f5igustega:<\/p>\n
ceph auth get-or-create client.fs mon 'allow r' mgr 'allow rw' mds 'allow rws' osd 'allow rw pool=cephfs_data, allow rw pool=cephfs_metadata'<\/code><\/pre>\n
Ja vaatame kohe tema juurdep\u00e4\u00e4suv\u00f5tit, see on meile edaspidi kasulik:<\/p>\n
ceph auth get-key client.fs<\/code><\/pre>\n
Loome eraldi Secret ja StorageClass.
\nMidagi uut, oleme seda juba n\u00e4inud RBD n\u00e4itel:<\/p>\n
---\napiVersion: v1\nkind: Secret\nmetadata:\n  name: csi-cephfs-secret\n  namespace: ceph-csi-cephfs\nstringData:\n  # Vajalik d\u00fcnaamiliselt loodud mahutite jaoks\n  adminID: fs\n  adminKey:<\/code><\/pre>\n
Rakendame manifesti:<\/p>\n
kubectl apply -f secret.yaml<\/code><\/pre>\n
Ja n\u00fc\u00fcd - eraldi StorageClass:<\/p>\n
---\napiVersion: storage.k8s.io\/v1\nkind: StorageClass\nmetadata:\n  name: csi-cephfs-sc\nprovisioner: cephfs.csi.ceph.com\nparameters:\n  clusterID: \n\n  # CephFS faili s\u00fcsteemi nimi, milles maht luuakse\n  fsName: cephfs\n\n  # (valikuline) Ceph bassein, kus vol\u00fc\u00fcmide andmed hoiustatakse\n  # pool: cephfs_data\n\n  # (valikuline) Comma eraldatud Ceph-fuse mountimise valikud\n  # n\u00e4iteks:\n  # fuseMountOptions: debug\n\n  # (valikuline) Comma eraldatud CephFS mountimise valikud kernelile\n  # Vaata man mount.ceph nende valikute loendi jaoks. N\u00e4iteks:\n  # kernelMountOptions: readdir_max_bytes=1048576,norbytes\n\n  # Salajastes peavad olema administraatori ja\/v\u00f5i Ceph kasutaja juurdep\u00e4\u00e4sud.\n  csi.storage.k8s.io\/provisioner-secret-name: csi-cephfs-secret\n  csi.storage.k8s.io\/provisioner-secret-namespace: ceph-csi-cephfs\n  csi.storage.k8s.io\/controller-expand-secret-name: csi-cephfs-secret\n  csi.storage.k8s.io\/controller-expand-secret-namespace: ceph-csi-cephfs\n  csi.storage.k8s.io\/node-stage-secret-name: csi-cephfs-secret\n  csi.storage.k8s.io\/node-stage-secret-namespace: ceph-csi-cephfs\n\n  # (valikuline) Draiver v\u00f5ib kasutada kas ceph-fuse (fuse) v\u00f5i\n  # ceph kernelclient (kernel).\n  # Kui ei ole m\u00e4\u00e4ratud, kasutatakse vaikimisi mahtude mountimist,\n  # seda m\u00e4\u00e4rab ceph-fuse ja mount.ceph otsing\n  # mounter: kernel\nreclaimPolicy: Delete\nallowVolumeExpansion: true\nmountOptions:\n  - debug<\/code><\/pre>\n
T\u00e4idame siia clusterID<\/strong> ja rakendame Kubernetesis:<\/p>\n
kubectl apply -f storageclass.yaml<\/code><\/pre>\n
Kontrollimine<\/h2>\n
Kontrollimiseks, nagu eelmises n\u00e4ites, loome PVC:<\/p>\n
---\napiVersion: v1\nkind: PersistentVolumeClaim\nmetadata:\n  name: csi-cephfs-pvc\nspec:\n  accessModes:\n    - ReadWriteMany\n  resources:\n    requests:\n      storage: 5Gi\n  storageClassName: csi-cephfs-sc<\/code><\/pre>\n
Ja kontrollime PVC\/PV olemasolu:<\/p>\n
kubectl get pvc\nkubectl get pv<\/code><\/pre>\n
Kui soovite n\u00e4ha faile ja katalooge CephFS-is, saate selle faili s\u00fcsteemi kuhugi monteerida. N\u00e4iteks nagu allpool n\u00e4idatud.<\/p>\n
L\u00e4heme \u00fchele Ceph klastrite nodest ja teeme sellised toimingud:<\/p>\n
# \u0422\u043e\u0447\u043a\u0430 \u043c\u043e\u043d\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f\nmkdir -p \/mnt\/cephfs\n\n# \u0421\u043e\u0437\u0434\u0430\u0451\u043c \u0444\u0430\u0439\u043b \u0441 \u043a\u043b\u044e\u0447\u043e\u043c \u0430\u0434\u043c\u0438\u043d\u0438\u0441\u0442\u0440\u0430\u0442\u043e\u0440\u0430\nceph auth get-key client.admin >\/etc\/ceph\/secret.key\n\n# \u0414\u043e\u0431\u0430\u0432\u043b\u044f\u0435\u043c \u0437\u0430\u043f\u0438\u0441\u044c \u0432 \/etc\/fstab\n# !! \u0418\u0437\u043c\u0435\u043d\u044f\u0435\u043c ip \u0430\u0434\u0440\u0435\u0441 \u043d\u0430 \u0430\u0434\u0440\u0435\u0441 \u043d\u0430\u0448\u0435\u0433\u043e \u0443\u0437\u043b\u0430\necho \"172.18.8.6:6789:\/ \/mnt\/cephfs ceph name=admin,secretfile=\/etc\/ceph\/secret.key,noatime,_netdev    0       2\" >> \/etc\/fstab\n\nmount \/mnt\/cephfs<\/code><\/pre>\n
Muidugi, selline FS montaa\u017e Ceph nodul sobib ainult \u00f5ppe eesm\u00e4rkidel, millega me tegeleme meie Slurm kursustel<\/a><\/noindex>. Ei usu, et keegi seda tootmises teeks, suur oht on olulisi faile kogemata \u00fcle kirjutada.<\/p>\n
Ja viimasena kontrollime, kuidas on CephFS-i mahutite muutmisega. Tagasi Kubernetesisse ja redigeerime meie PVC manifesti - suurendame seal suurust, n\u00e4iteks 7Gi-ks.<\/p>\n
Rakendame redigeeritud faili:<\/p>\n
kubectl apply -f pvc.yaml<\/code><\/pre>\n
Vaata monteeritud kataloogis, kuidas kvoot on muutunud:<\/p>\n
getfattr -n ceph.quota.max_bytes<\/code><\/pre>\n
Selle k\u00e4su t\u00f6\u00f6ks v\u00f5ib olla vajalik installida s\u00fcsteemi pakett attr<\/strong>.<\/p>\n
Silmad kardavad, aga k\u00e4ed teevad<\/h2>\n
Esialgu n\u00e4ivad k\u00f5ik need loitsud ja pikad YAML manifeste keerulised, kuid praktikas saavad Sl\u00f8rmi \u00fcli\u00f5pilased nendega \u00fcsna kiiresti hakkama.
\nSelles artiklis me ei s\u00fcvenenud s\u00fcgavale \u2014 selleks on ametlik dokumentatsioon. Kui teid huvitavad Cephi salvestusruumi seadistamise \u00fcksikasjad koos Kubernetes'i klastriga, aitavad need lingid:<\/p>\n
Kubernetes'i ja mahtude \u00fcldp\u00f5him\u00f5tted<\/a><\/noindex>
\nRBD dokumentatsioon<\/a><\/noindex>
\nRBD ja Kubernetes'i integreerimine Cephi vaatenurgast<\/a><\/noindex>
\nRBD ja Kubernetes'i integreerimine CSI vaatenurgast<\/a><\/noindex>
\nCephFS \u00fclddokumendid<\/a><\/noindex>
\nCephFS ja Kubernetes'i integreerimine CSI vaatenurgast<\/a><\/noindex><\/p>\n
Sl\u00f8rmi kursusel Kubernetes P\u00f5hi<\/a><\/noindex> saate minna veelgi kaugemale ja k\u00e4ivitada Kubernetes'is reaalne rakendus, mis kasutab CephFS-i failide salvestamiseks. GET\/POST p\u00e4ringute kaudu saate faili edastada ja neid Ceph'st saada.<\/p>\n
Kui teid rohkem huvitab andmete salvestamine, siis registreeruge uus kursus Cephi kohta<\/a><\/noindex>. Seni, kuni beta-testimine on k\u00e4imas, on kursus saadaval soodushinnaga ja saate m\u00f5jutada selle sisu.<\/p>\n
Artikli autor: Aleksandr \u0160valov, praktikantinsener Southbridge<\/a><\/noindex>, Certified Kubernetes Administrator, Sl\u00f8rmi kursuste autor ja arendaja.<\/em><\/p>\n
Allikas: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"
Container Storage Interface (CSI) \u2014 \u044d\u0442\u043e \u0443\u043d\u0438\u0444\u0438\u0446\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0439 \u0438\u043d\u0442\u0435\u0440\u0444\u0435\u0439\u0441 \u0432\u0437\u0430\u0438\u043c\u043e\u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f Kubernetes \u0438 \u0441\u0438\u0441\u0442\u0435\u043c \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445. \u0412\u043a\u0440\u0430\u0442\u0446\u0435 \u043e \u043d\u0451\u043c \u043c\u044b \u0443\u0436\u0435 \u0440\u0430\u0441\u0441\u043a\u0430\u0437\u044b\u0432\u0430\u043b\u0438, \u0430 \u0441\u0435\u0433\u043e\u0434\u043d\u044f \u043f\u043e\u0434\u0440\u043e\u0431\u043d\u0435\u0435 \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0441\u0432\u044f\u0437\u043a\u0443 CSI \u0438 Ceph: \u043f\u043e\u043a\u0430\u0436\u0435\u043c, \u043a\u0430\u043a \u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0438\u0442\u044c \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0435 Ceph \u043a \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0443 Kubernetes. \u0412 \u0441\u0442\u0430\u0442\u044c\u0435 \u043f\u0440\u0438\u0432\u0435\u0434\u0435\u043d\u044b \u0440\u0435\u0430\u043b\u044c\u043d\u044b\u0435, \u0445\u043e\u0442\u044f \u0438 \u043d\u0435\u043c\u043d\u043e\u0433\u043e \u0443\u043f\u0440\u043e\u0449\u0451\u043d\u043d\u044b\u0435 \u0434\u043b\u044f \u0443\u0434\u043e\u0431\u0441\u0442\u0432\u0430 \u0432\u043e\u0441\u043f\u0440\u0438\u044f\u0442\u0438\u044f \u043f\u0440\u0438\u043c\u0435\u0440\u044b. \u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0443 \u0438 \u043d\u0430\u0441\u0442\u0440\u043e\u0439\u043a\u0443 \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u043e\u0432 Ceph \u0438 Kubernetes […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":94740,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-94739","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Praktiline n\u00e4ide Cephi baasil oleva salvestuse \u00fchendamisest Kubernetes'i klastriga | ProHoster","description":"Container Storage Interface (CSI) on \u00fchtne liides Kubernetes'i ja andmesalvestuss\u00fcsteemide vahel.","canonical_url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"et_EE","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041f\u0440\u0430\u043a\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u043f\u0440\u0438\u043c\u0435\u0440 \u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0435\u043d\u0438\u044f \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0430 \u043d\u0430 \u0431\u0430\u0437\u0435 Ceph \u0432 \u043a\u043b\u0430\u0441\u0442\u0435\u0440 Kubernetes | ProHoster","og:description":"Container Storage Interface (CSI) \u2014 \u044d\u0442\u043e \u0443\u043d\u0438\u0444\u0438\u0446\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0439 \u0438\u043d\u0442\u0435\u0440\u0444\u0435\u0439\u0441 \u0432\u0437\u0430\u0438\u043c\u043e\u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f Kubernetes \u0438 \u0441\u0438\u0441\u0442\u0435\u043c \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445.","og:url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/prakticheskij-primer-podklyucheniya-hranilishha-na-baze-ceph-v-klaster-kubernetes","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-09-20T17:42:27+00:00","article:modified_time":"2021-01-02T11:05:08+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"94739","title":null,"description":"","keywords":"","keyphrases":null,"primary_term":null,"canonical_url":"","og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 11:19:21","updated":"2022-09-30 10:03:10","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/94739","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/comments?post=94739"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/94739\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media\/94740"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media?parent=94739"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/categories?post=94739"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/tags?post=94739"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}