🥇Rook or not Rook - ndiwo mubvunzo

Mukutanga kwemwedzi uno, muna Chivabvu 3, kuburitswa kukuru kwe "management system yekugovera kuchengetedza data muKubernetes" yakaziviswa - Rook 1.0.0. Anopfuura gore rapfuura isu tatova rakabudiswa general overview yeRook. Takabva tanzi titaure zvakaitika kwaari shandisa mukuita - uye iko zvino, munguva chaiyo yechiitiko chakakosha kudaro munhoroondo yeprojekti, tinofara kugovera zvatinowana.

Muchidimbu, Rook isethi vashandisi yeKubernetes, iyo inotora kutonga kwakazara kwekutumira, manejimendi, otomatiki kudzoreredza kwekuchengetedza data mhinduro dzakadai seCeph, EdgeFS, Minio, Cassandra, CockroachDB.

Parizvino iyo yakagadziridzwa zvakanyanya (uye mumwe chete в stable stage) mhinduro iri rook-ceph-operator.

taura pfungwa: Pakati pekuchinja kukuru muRook 1.0.0 kusunungurwa kwakabatana neCeph, tinogona kucherechedza kutsigirwa kweCeph Nautilus uye kukwanisa kushandisa NFS yeCephFS kana RGW mabhakiti. Chinomira pachena pakati pevamwe ndiko kukura kweEdgeFS rutsigiro kusvika padanho rebeta.

Saka, mune ino chinyorwa isu:

Ngatipindurei mubvunzo pamusoro pezvakanakira zvatinoona mukushandisa Rook kuendesa Ceph muboka reKubernetes;
Tichagovera ruzivo rwedu uye maonerwo ekushandisa Rook mukugadzira;
Ngatikuudzei chikonzero nei tichiti “Hongu!” kuna Rook, uye nezvezvirongwa zvedu kwaari.

Ngatitangei ne general concepts ne theory.

"Ndine mukana weimwe Rook!" (asingazivikanwe chess mutambi)

Imwe yemabhenefiti makuru eRook ndeyekuti kudyidzana nezvitoro zvedata kunoitwa kuburikidza neKubernetes nzira. Izvi zvinoreva kuti iwe hauchadi kutevedzera mirairo kugadzirisa Ceph kubva pashizha kupinda mukoni.

- Iwe unoda kuendesa CephFS musumbu? Ingonyora YAML faira!
- Chii? Iwe unoda zvakare kuendesa chitoro chechinhu neS3 API? Ingonyora yechipiri YAML faira!

Rook inogadzirwa zvinoenderana nemitemo yese yeanojairwa opareta. Kudyidzana naye kunoitika kushandisa CRD (Custom Resource Definitions), umo tinotsanangura maitiro eCeph masangano atinoda (sezvo iyi ndiyo yega yekuitwa yakagadzikana, nekusarudzika chinyorwa ichi chichataura nezveCeph, kunze kwekunge zvataurwa neimwe nzira). Zvinoenderana neakatsanangurwa ma paramita, mushandisi anozoita otomatiki mirairo inodiwa pakugadzirisa.

Ngatitarisei kune chaiwo tichishandisa muenzaniso wekugadzira Chitoro cheChinhu, kana kuti - CephObjectStoreUser.

apiVersion: ceph.rook.io/v1
kind: CephObjectStore
metadata:
  name: {{ .Values.s3.crdName }}
  namespace: kube-rook
spec:
  metadataPool:
    failureDomain: host
    replicated:
      size: 3
  dataPool:
    failureDomain: host
    erasureCoded:
      dataChunks: 2
      codingChunks: 1
  gateway:
    type: s3
    sslCertificateRef:
    port: 80
    securePort:
    instances: 1
    allNodes: false
---
apiVersion: ceph.rook.io/v1
kind: CephObjectStoreUser
metadata:
  name: {{ .Values.s3.crdName }}
  namespace: kube-rook
spec:
  store: {{ .Values.s3.crdName }}
  displayName: {{ .Values.s3.username }}

Iwo maparamendi anoratidzwa mune rondedzero akajairwa uye haatomboda makomendi, asi zvakakosha kutarisisa kune iyo yakagoverwa kune template machinjiro.

Hurongwa hwese hwebasa hunouya kunyaya yekuti isu "tinoodha" zviwanikwa kuburikidza neYAML faira, iyo mushandisi anoita mirairo inodiwa uye anotidzosera "kwete-chaiyo-chaiyo" chakavanzika chatinogona kuwedzera kushanda nacho. (ona pazasi). Uye kubva pane zvakasiyana-siyana zvakanyorwa pamusoro, murairo uye zita rakavanzika richaumbwa.

Ichi chikwata cherudzii? Paunenge uchigadzira mushandisi wekuchengetedza chinhu, iyo Rook opareta mukati mepodhi achaita zvinotevera:

radosgw-admin user create --uid="rook-user" --display-name="{{ .Values.s3.username }}"

Mhedzisiro yekuita uyu murairo ichave JSON chimiro:

{
    "user_id": "rook-user",
    "display_name": "{{ .Values.s3.username }}",
    "keys": [
        {
           "user": "rook-user",
           "access_key": "NRWGT19TWMYOB1YDBV1Y",
           "secret_key": "gr1VEGIV7rxcP3xvXDFCo4UDwwl2YoNrmtRlIAty"
        }
    ],
    ...
}

Keys - ndeapi maapplication achazoda kuwana chinhu chekuchengetedza kuburikidza neS3 API. Mushandi weRook anovasarudza nemutsa uye anoisa munzvimbo yake yezita nenzira yechakavanzika ine zita rook-ceph-object-user-{{ $.Values.s3.crdName }}-{{ $.Values.s3.username }}.

Kuti ushandise iyo data kubva kune ichi chakavanzika, ingoiwedzera kune mudziyo seyakasiyana siyana. Semuenzaniso, ini ndichapa template yeJobho, umo isu tinongogadzira mabhakiti ega ega mushandisi nharaunda:

{{- range $bucket := $.Values.s3.bucketNames }}
apiVersion: batch/v1
kind: Job
metadata:
  name: create-{{ $bucket }}-bucket-job
  annotations:
    "helm.sh/hook": post-install
    "helm.sh/hook-weight": "2"
spec:
  template:
    metadata:
      name: create-{{ $bucket }}-bucket-job
    spec:
      restartPolicy: Never
      initContainers:
      - name: waitdns
        image: alpine:3.6
        command: ["/bin/sh", "-c", "while ! getent ahostsv4 rook-ceph-rgw-{{ $.Values.s3.crdName }}; do sleep 1; done" ]
      - name: config
        image: rook/ceph:v1.0.0
        command: ["/bin/sh", "-c"]
        args: ["s3cmd --configure --access_key=$(ACCESS-KEY) --secret_key=$(SECRET-KEY) -s --no-ssl --dump-config | tee /config/.s3cfg"]
        volumeMounts:
        - name: config
          mountPath: /config
        env:
        - name: ACCESS-KEY
          valueFrom:
            secretKeyRef:
              name: rook-ceph-object-user-{{ $.Values.s3.crdName }}-{{ $.Values.s3.username }}
              key: AccessKey
        - name: SECRET-KEY
          valueFrom:
            secretKeyRef:
              name: rook-ceph-object-user-{{ $.Values.s3.crdName }}-{{ $.Values.s3.username }}
              key: SecretKey
      containers:
      - name: create-bucket
        image: rook/ceph:v1.0.0
        command: 
        - "s3cmd"
        - "mb"
        - "--host=rook-ceph-rgw-{{ $.Values.s3.crdName }}"
        - "--host-bucket= "
        - "s3://{{ $bucket }}"
        ports:
        - name: s3-no-sll
          containerPort: 80
        volumeMounts:
        - name: config
          mountPath: /root
      volumes:
      - name: config
        emptyDir: {}
---
{{- end }}

Zvese zviito zvakanyorwa mune ino Jobho zvakaitwa mukati meiyo Kubernetes. Izvo zvimiro zvinotsanangurwa mumafaira eYAML zvinochengetwa muGit repository uye zvinoshandiswazve kakawanda. Isu tinoona izvi sekuwedzera kukuru kune DevOps mainjiniya uye iyo CI/CD maitiro ese.

Ndinofara naRook naRados

Kushandisa iyo Ceph + RBD musanganiswa inoisa zvimwe zvirambidzo pakukwira mavhoriyamu kumapodhi.

Kunyanya, iyo nzvimbo yezita inofanira kunge iine chakavanzika chekuwana Ceph kuitira kuti zvikumbiro zvine musoro zvishande. Zvakanaka kana uine 2-3 nharaunda munzvimbo dzavo dzemazita: unogona kuenda uye kukopa chakavanzika pamaoko. Asi ko kana pane chimwe nechimwe nharaunda yakaparadzana ine zita rayo pachayo yakagadzirwa kune vanogadzira?

Isu takagadzirisa dambudziko iri isu tichishandisa shell-operator, iyo yakatora otomatiki zvakavanzika kune mitsva yemazita (muenzaniso wehoko yakadaro inotsanangurwa mukati ichi chinyorwa).

#! /bin/bash

if [[ $1 == “--config” ]]; then
   cat <<EOF
{"onKubernetesEvent":[
 {"name": "OnNewNamespace",
  "kind": "namespace",
  "event": ["add"]
  }
]}
EOF
else
    NAMESPACE=$(kubectl get namespace -o json | jq '.items | max_by( .metadata.creationTimestamp ) | .metadata.name')
    kubectl -n ${CEPH_SECRET_NAMESPACE} get secret ${CEPH_SECRET_NAME} -o json | jq ".metadata.namespace="${NAMESPACE}"" | kubectl apply -f -
fi

Nekudaro, kana uchishandisa Rook dambudziko iri harisipo. Iyo yekumisikidza maitiro inoitika uchishandisa madhiraivha ayo zvichienderana Flexvolume kana CSI (ichiri mubeta nhanho) uye saka haidi zvakavanzika.

Rook inogadzirisa otomatiki matambudziko mazhinji, ayo anotikurudzira kuishandisa mumapurojekiti matsva.

Kukombwa kweRook

Ngatipedzei chikamu chinoshanda nekuisa Rook neCeph kuitira kuti isu tiite zvedu zviedzo. Kuita kuti zvive nyore kuputira shongwe iyi isingapindike, vagadziri vakagadzirira Helm package. Ngatitorei

$ helm fetch rook-master/rook-ceph --untar --version 1.0.0

Mufaira rook-ceph/values.yaml unogona kuwana akawanda akasiyana marongero. Chinhu chinonyanya kukosha ndechekutsanangura kushivirira kwevamiririri uye kutsvaga. Isu takatsanangura zvakadzama izvo zvinosvibisa / kushivirira nzira inogona kushandiswa mairi ichi chinyorwa.

Muchidimbu, isu hatidi kuti mutengi application mapodhi ave panzvimbo dzakafanana nemadhisiki ekuchengetedza data. Chikonzero chiri nyore: nenzira iyi basa reRook agents harizokanganisa mashandisirwo acho.

Saka, vhura faira rook-ceph/values.yaml neyako yaunofarira mupepeti uye wedzera inotevera block kumagumo:

discover:
  toleration: NoExecute
  tolerationKey: node-role/storage
agent:
  toleration: NoExecute
  tolerationKey: node-role/storage
  mountSecurityMode: Any

Kune imwe neimwe node yakachengeterwa kuchengetedza data, wedzera inoenderana taint:

$ kubectl taint node ${NODE_NAME} node-role/storage="":NoExecute

Wobva waisa Helm chati nemurairo:

$ helm install --namespace ${ROOK_NAMESPACE} ./rook-ceph

Iye zvino unoda kugadzira sumbu uye tsanangura nzvimbo OSD:

apiVersion: ceph.rook.io/v1
kind: CephCluster
metadata:
  clusterName: "ceph"
  finalizers:
  - cephcluster.ceph.rook.io
  generation: 1
  name: rook-ceph
spec:
  cephVersion:
    image: ceph/ceph:v13
  dashboard:
    enabled: true
  dataDirHostPath: /var/lib/rook/osd
  mon:
    allowMultiplePerNode: false
    count: 3
  network:
    hostNetwork: true
  rbdMirroring:
    workers: 1
  placement:
    all:
      tolerations:
      - key: node-role/storage
        operator: Exists
  storage:
    useAllNodes: false
    useAllDevices: false
    config:
      osdsPerDevice: "1"
      storeType: filestore
    resources:
      limits:
        memory: "1024Mi"
      requests:
        memory: "1024Mi"
    nodes:
    - name: host-1
      directories:
      - path: "/mnt/osd"
    - name: host-2
      directories:
      - path: "/mnt/osd"
    - name: host-3
      directories:
      - path: "/mnt/osd"

Kutarisa chimiro cheCeph - tarisira kuona HEALTH_OK:

$ kubectl -n ${ROOK_NAMESPACE} exec $(kubectl -n ${ROOK_NAMESPACE} get pod -l app=rook-ceph-operator -o name -o jsonpath='{.items[0].metadata.name}') -- ceph -s

Panguva imwecheteyo, ngatitarisei kuti mapods ane mutengi application haapere pane node dzakachengeterwa Ceph:

$ kubectl -n ${APPLICATION_NAMESPACE} get pods -o custom-columns=NAME:.metadata.name,NODE:.spec.nodeName

Uyezve, zvimwe zvikamu zvinogona kugadzirwa sezvaunoda. Mamwe mashoko pamusoro pavo anoratidzwa mu zvinyorwa. Zvekutonga, isu tinokurudzira zvakasimba kuisa dashboard uye bhokisi rekushandisa.

Rook uye zvikorekedzo: Rook inokwana kune zvese here?

Sezvauri kuona, kuvandudzwa kweRook kuri kuzara. Asi kuchine matambudziko asingatibvumidze kusiya zvachose manyorero eCeph:

Hapana Rook Driver handikwanise ekisipoti metrics pakushandiswa kweakaiswa mabhuroki, ayo anotitadzisa kutarisa.
Flexvolume uye CSI handizivi kuti sei shandura saizi yemavhoriyamu (kusiyana neiyo RBD), saka Rook inonyimwa inobatsira (uye dzimwe nguva inodiwa zvakanyanya!)
Rook haisati yave kuchinjika seyakajairwa Ceph. Kana isu tichida kugadzirisa dziva reCephFS metadata kuti ichengetwe paSSD, uye iyo data pachayo kuti ichengetwe paHDD, isu tichada kunyoresa mapoka akasiyana emidziyo mumamepu eCRUSH nemaoko.
Kunyangwe ichokwadi chekuti rook-ceph-operator inoonekwa seyakagadzikana, pane parizvino mamwe matambudziko pakuvandudza Ceph kubva mushanduro 13 kusvika 14.

zvakawanikwa

"Parizvino Rook akavharirwa kunze kwenyika nemapawn, asi isu tinotenda kuti rimwe zuva achatora chikamu chakakosha mumutambo!" (Mashoko akagadzirwa zvakanangana nechinyorwa chino)

Chirongwa cheRook pasina mubvunzo chakunda moyo yedu - tinotenda kuti [nezvese zvayakanakira nezvayakaipira] inofanirwa kutariswa nemi.

Zvirongwa zvedu zveramangwana zvinofashamira kugadzira rook-ceph module ye addon-operator, iyo ichaita kuti kushandiswa kwayo mumasumbu edu akawanda eKubernetes kuve nyore uye nyore.

PS

Verenga zvakare pablog yedu:

Source: www.habr.com

Rook kana kwete Rook, ndiwo mubvunzo

"Ndine mukana weimwe Rook!" (asingazivikanwe chess mutambi)

Ndinofara naRook naRados

Kukombwa kweRook

Rook uye zvikorekedzo: Rook inokwana kune zvese here?

zvakawanikwa

PS

Voeg kukanzura mhinduro