To Rook iyo in kale - taasi waa su'aasha

Bilowgii bishaan, Maajo 3, siideynta weyn ee "nidaamka maamulka ee kaydinta xogta la qaybiyey ee Kubernetes" ayaa lagu dhawaaqay - Rook 1.0.0. In ka badan hal sano ka hor waxaan hore u la daabacay dulmar guud ee Rook. Kadibna waxa nala waydiiyay in aanu ka hadalno khibradiisa u isticmaal ficil ahaan - iyo hadda, kaliya waqtiga sida muhiimka ah ee taariikhda mashruuca, waxaan ku faraxsanahay in aan wadaagno riyooyinka urursan.

Marka la soo koobo, Rook waa go'an hawlwadeennada loogu talagalay Kubernetes, kuwaas oo si buuxda u xakameynaya geynta, maaraynta, soo kabashada tooska ah ee xalalka kaydinta xogta sida Ceph, EdgeFS, Minio, Cassandra, CockroachDB.

Waqtigan xaadirka ah kuwa ugu horumarsan (iyo ka kaliya в deggan marxalad) xalku waa rook-ceph-operator.

tacliiq: Waxaa ka mid ah isbeddelada muhiimka ah ee Rook 1.0.0 siideynta ee la xiriirta Ceph, waxaan ogaan karnaa taageerada Ceph Nautilus iyo awoodda loo isticmaalo NFS ee baaldiyada CephFS ama RGW. Waxa ka dhex muuqda kuwa kale waa korriinka taageerada EdgeFS ilaa heerka beta.

Haddaba, maqaalkan waxaan:

• Aan ka jawaabno su'aasha ku saabsan faa'iidooyinka aan ku aragno isticmaalka Rook si loo geeyo Ceph ee kutlada Kubernetes;
• Waxaan wadaagi doonaa waayo-aragnimadayada iyo aragtidayada isticmaalka Rook ee wax soo saarka;
• Aan kuu sheegno sababta aan u nidhaahno "Haa!" Rook, iyo waxa ku saabsan qorshayaashayada isaga.

Aan ku bilowno fikradaha guud iyo aragtida.

"Waxaan faa'iido u leeyahay hal Rook!" (ciyaaryahanka chess-ka aan la aqoon)

Mid ka mid ah faa'iidooyinka ugu muhiimsan ee Rook waa in la dhexgalka dukaamada xogta lagu fuliyo hababka Kubernetes. Tani waxay ka dhigan tahay inaadan u baahnayn inaad nuqul ka sameyso amarada si aad Ceph uga dejiso xaashida console-ka.

- Ma doonaysaa inaad kooxda CephFS geyso koox? Kaliya qor faylka YAML!
- Waa maxay? Ma waxaad sidoo kale dooneysaa inaad geyso dukaan shayga leh S3 API? Kaliya qor faylka YAML labaad!

Rook waxaa loo abuuray si waafaqsan dhammaan xeerarka hawlwadeenka caadiga ah. Is dhexgalka isaga ayaa ku dhaca isticmaalka CRD (Qeexitaannada Kheyraadka Gaarka ah), kaas oo aan ku sifeyno sifada hay'adaha Ceph ee aan u baahanahay (maadaama kani yahay fulinta kaliya ee xasiloon, asal ahaan maqaalkani wuxuu ka hadli doonaa Ceph, ilaa si cad loo sheego mooyee). Marka loo eego xuduudaha la cayimay, hawlwadeenku wuxuu si toos ah u fulin doonaa amarrada lagama maarmaanka u ah qaabeynta.

Aynu eegno waxyaabaha gaarka ah anagoo tusaale u ah abuurista Dukaanka Shayga, ama taa beddelkeeda - CephObjectStoreUser.

apiVersion: ceph.rook.io/v1
kind: CephObjectStore
metadata:
  name: {{ .Values.s3.crdName }}
  namespace: kube-rook
spec:
  metadataPool:
    failureDomain: host
    replicated:
      size: 3
  dataPool:
    failureDomain: host
    erasureCoded:
      dataChunks: 2
      codingChunks: 1
  gateway:
    type: s3
    sslCertificateRef:
    port: 80
    securePort:
    instances: 1
    allNodes: false
---
apiVersion: ceph.rook.io/v1
kind: CephObjectStoreUser
metadata:
  name: {{ .Values.s3.crdName }}
  namespace: kube-rook
spec:
  store: {{ .Values.s3.crdName }}
  displayName: {{ .Values.s3.username }}

Halbeegyada lagu tilmaamay liisku waa heer caadi ah oo aan u baahnayn faallooyin, laakiin waxa habboon in fiiro gaar ah loo yeesho kuwa loo qoondeeyey doorsoomayaasha template.

Nidaamka guud ee shaqadu wuxuu hoos ugu soo dhacayaa xaqiiqda ah in aan "dalbano" kheyraadka iyada oo loo marayo faylka YAML, kaas oo hawlwadeenku uu fuliyo amarrada lagama maarmaanka ah oo uu nagu soo celiyo sirta "aan-dhab ah" oo aan ku sii shaqeyn karno (hoos eeg). Iyo doorsoomayaasha kor ku taxan, amarka iyo magaca sirta ah ayaa la ururin doonaa.

Waa koox noocee ah? Marka la abuurayo isticmaale kaydinta shayga, hawl wadeenka Rook ee gudaha podka ayaa samayn doona waxa soo socda:

radosgw-admin user create --uid="rook-user" --display-name="{{ .Values.s3.username }}"

Natiijada fulinta amarkan waxay noqon doontaa qaab-dhismeedka JSON:

{
    "user_id": "rook-user",
    "display_name": "{{ .Values.s3.username }}",
    "keys": [
        {
           "user": "rook-user",
           "access_key": "NRWGT19TWMYOB1YDBV1Y",
           "secret_key": "gr1VEGIV7rxcP3xvXDFCo4UDwwl2YoNrmtRlIAty"
        }
    ],
    ...
}

Keys - waa maxay codsiyada mustaqbalka u baahan doonaan si ay u galaan kaydinta shayga iyada oo loo marayo S3 API. Hawl-wadeenka Rook si naxariis leh ayuu u xushaa oo u geliyaa meel magaciisa si qarsoodi ah magaca rook-ceph-object-user-{{ $.Values.s3.crdName }}-{{ $.Values.s3.username }}.

Si aad u isticmaasho xogta sirtan, kaliya ku dar weelka sidii doorsoomayaasha deegaanka. Tusaale ahaan, waxaan siin doonaa jaantus shaqo, kaas oo aan si toos ah ugu abuurno baaldiyada jawiga isticmaalaha:

{{- range $bucket := $.Values.s3.bucketNames }}
apiVersion: batch/v1
kind: Job
metadata:
  name: create-{{ $bucket }}-bucket-job
  annotations:
    "helm.sh/hook": post-install
    "helm.sh/hook-weight": "2"
spec:
  template:
    metadata:
      name: create-{{ $bucket }}-bucket-job
    spec:
      restartPolicy: Never
      initContainers:
      - name: waitdns
        image: alpine:3.6
        command: ["/bin/sh", "-c", "while ! getent ahostsv4 rook-ceph-rgw-{{ $.Values.s3.crdName }}; do sleep 1; done" ]
      - name: config
        image: rook/ceph:v1.0.0
        command: ["/bin/sh", "-c"]
        args: ["s3cmd --configure --access_key=$(ACCESS-KEY) --secret_key=$(SECRET-KEY) -s --no-ssl --dump-config | tee /config/.s3cfg"]
        volumeMounts:
        - name: config
          mountPath: /config
        env:
        - name: ACCESS-KEY
          valueFrom:
            secretKeyRef:
              name: rook-ceph-object-user-{{ $.Values.s3.crdName }}-{{ $.Values.s3.username }}
              key: AccessKey
        - name: SECRET-KEY
          valueFrom:
            secretKeyRef:
              name: rook-ceph-object-user-{{ $.Values.s3.crdName }}-{{ $.Values.s3.username }}
              key: SecretKey
      containers:
      - name: create-bucket
        image: rook/ceph:v1.0.0
        command: 
        - "s3cmd"
        - "mb"
        - "--host=rook-ceph-rgw-{{ $.Values.s3.crdName }}"
        - "--host-bucket= "
        - "s3://{{ $bucket }}"
        ports:
        - name: s3-no-sll
          containerPort: 80
        volumeMounts:
        - name: config
          mountPath: /root
      volumes:
      - name: config
        emptyDir: {}
---
{{- end }}

Dhammaan ficillada ku taxan Shaqadan waxaa lagu sameeyay qaabka Kubernetes. Qaab dhismeedka lagu sifeeyay faylasha YAML waxa lagu kaydiyaa kaydka Git oo marar badan dib loo isticmaalay. Waxaan u aragnaa tan mid aad u weyn oo loogu talagalay injineerada DevOps iyo habka CI/CD guud ahaan.

Ku faraxsan Rook iyo Rados

Isticmaalka isku darka Ceph + RBD wuxuu ku soo rogayaa xaddidaadyo gaar ah kor u qaadista mugga galalka.

Gaar ahaan, goobta magacu waa in ay ka kooban tahay sir gelitaanka Ceph si codsiyo dawladeed ay u shaqeeyaan. Way fiicantahay haddii aad ku leedahay 2-3 deegaan magacyadooda: waxaad u tagi kartaa oo koobi kartaa sirta gacanta. Laakiin ka waran haddii sifo kasta deegaan gaar ah oo leh magac u gaar ah loo sameeyo horumariyeyaasha?

Waxaan xallinay dhibaatadan nafteena anagoo adeegsanayna qolof-shaqaale, kaas oo si toos ah u koobiyeeyay siraha meelo magacyo cusub ah (tusaale ahaan jillaab noocan oo kale ah ayaa lagu sharaxay maqaalkani).

#! /bin/bash

if [[ $1 == “--config” ]]; then
   cat <<EOF
{"onKubernetesEvent":[
 {"name": "OnNewNamespace",
  "kind": "namespace",
  "event": ["add"]
  }
]}
EOF
else
    NAMESPACE=$(kubectl get namespace -o json | jq '.items | max_by( .metadata.creationTimestamp ) | .metadata.name')
    kubectl -n ${CEPH_SECRET_NAMESPACE} get secret ${CEPH_SECRET_NAME} -o json | jq ".metadata.namespace="${NAMESPACE}"" | kubectl apply -f -
fi

Si kastaba ha ahaatee, marka la isticmaalayo Rook dhibaatadan si fudud ma jiraan. Habka dhejinta wuxuu ku dhacaa iyadoo la adeegsanayo darawalkeeda iyada oo ku saleysan Flexvolume ama CSI (wali marxaladda beta) oo sidaas darteed uma baahna siraha.

Rook wuxuu si toos ah u xalliyaa dhibaatooyin badan, taas oo nagu dhiirigelisa inaan u isticmaalno mashaariicda cusub.

Go'doominta Rook

Aynu dhamaystirno qaybta la taaban karo anagoo geynayna Rook iyo Ceph si aan u samayno tijaabooyin noo gaar ah. Si loo fududeeyo in la weeraro munaaraddan aan la dafiri karin, horumariyayaashu waxay diyaariyeen xirmo Helm ah. Aan soo dejino:

$ helm fetch rook-master/rook-ceph --untar --version 1.0.0

Faylka ku jira rook-ceph/values.yaml Waxaad ka heli kartaa goobo badan oo kala duwan. Waxa ugu muhiimsan waa in la qeexo dulqaadka wakiilada iyo raadinta. Waxaan si faahfaahsan u sharaxnay habka wasakhda/dulqaadashada loo isticmaali karo maqaalkani.

Marka la soo koobo, ma doonayno in sanduuqyada codsiga macmiilku ay ku yaalliin qanjidhada kaydinta xogta. Sababtu waa sahlan tahay: habkan shaqada wakiilada Rook ma saameyn doonto codsiga laftiisa.

Markaa, fur faylka rook-ceph/values.yaml leh tifaftiraha aad jeceshahay oo ku dar qaybtan soo socota dhamaadka:

discover:
  toleration: NoExecute
  tolerationKey: node-role/storage
agent:
  toleration: NoExecute
  tolerationKey: node-role/storage
  mountSecurityMode: Any

Nod kasta oo loo qoondeeyay kaydinta xogta, ku dar taint u dhiganta:

$ kubectl taint node ${NODE_NAME} node-role/storage="":NoExecute

Ka dibna ku dheji jaantuska Helm oo leh amarka:

$ helm install --namespace ${ROOK_NAMESPACE} ./rook-ceph

Hadda waxaad u baahan tahay inaad abuurto koox oo qeex meesha OSD:

apiVersion: ceph.rook.io/v1
kind: CephCluster
metadata:
  clusterName: "ceph"
  finalizers:
  - cephcluster.ceph.rook.io
  generation: 1
  name: rook-ceph
spec:
  cephVersion:
    image: ceph/ceph:v13
  dashboard:
    enabled: true
  dataDirHostPath: /var/lib/rook/osd
  mon:
    allowMultiplePerNode: false
    count: 3
  network:
    hostNetwork: true
  rbdMirroring:
    workers: 1
  placement:
    all:
      tolerations:
      - key: node-role/storage
        operator: Exists
  storage:
    useAllNodes: false
    useAllDevices: false
    config:
      osdsPerDevice: "1"
      storeType: filestore
    resources:
      limits:
        memory: "1024Mi"
      requests:
        memory: "1024Mi"
    nodes:
    - name: host-1
      directories:
      - path: "/mnt/osd"
    - name: host-2
      directories:
      - path: "/mnt/osd"
    - name: host-3
      directories:
      - path: "/mnt/osd"

Hubinta heerka Ceph - filo inaad aragto HEALTH_OK:

$ kubectl -n ${ROOK_NAMESPACE} exec $(kubectl -n ${ROOK_NAMESPACE} get pod -l app=rook-ceph-operator -o name -o jsonpath='{.items[0].metadata.name}') -- ceph -s

Isla mar ahaantaana, aynu eegno in galalka codsiga macmiilku aanay ku dhammaanayn noodhka loo qoondeeyay Ceph:

$ kubectl -n ${APPLICATION_NAMESPACE} get pods -o custom-columns=NAME:.metadata.name,NODE:.spec.nodeName

Dheeraad ah, qaybo dheeraad ah ayaa loo habeyn karaa sida la rabo. Faahfaahin dheeraad ah oo ku saabsan iyaga ayaa lagu tilmaamay dukumentiyo. Maamulka, waxaanu si adag ugu talinaynaa in la rakibo dashboard-ka iyo sanduuqa.

Rook iyo jillaab: Rook ma ku filan yahay wax walba?

Sida aad arki karto, horumarka Rook ayaa si xawli ah ku socda. Laakiin weli waxaa jira dhibaatooyin aan noo oggolaan in aan gebi ahaanba ka tagno qaabeynta gacanta ee Ceph:

• Ma jiro Rook Driver ma awoodo qiyaasaha dhoofinta ee isticmaalka baloogyada rakiban, taas oo naga diidaysa la socodka.
• Flexvolume iyo CSI ma garanayo sida beddel cabbirka mugga (oo ka soo horjeeda isla RBD), sidaa darteed Rook waa laga reebay aalad waxtar leh (oo mararka qaarkood si daran loogu baahan yahay!)
• Rook weli uma dabacsana sida Ceph caadiga ah. Haddii aan rabno in aan u habeyno barkadda xogta badan ee CephFS si loogu keydiyo SSD, iyo xogta lafteeda lagu keydiyo HDD, waxaan u baahan doonaa inaan iska diiwaan geliyo kooxo kala duwan oo qalab ah khariidadaha CRUSH.
• In kasta oo xaqiiqda ah in rook-ceph-operator loo arko mid deggan, hadda waxaa jira dhibaatooyin qaar marka laga soo cusboonaysiinayo Ceph nooca 13 ilaa 14.

natiijooyinka

"Hadda Rook waxaa laga xiray adduunka dibadda by pawns, laakiin waxaan aaminsanahay in maalin ay ciyaari doonto door muhiim ah ciyaarta!" (xigasho si gaar ah loogu hindisay maqaalkan)

Mashruuca Rook wuxuu shaki la'aan kasbaday qalbiyadeenna - waxaan aaminsanahay in [oo leh dhammaan faa'iidooyinka iyo khasaarooyinka] ay hubaal tahay inay mudan tahay dareenkaaga.

Qorshayaashayada mustaqbalka waxay isku dubbaridaan samaynta rook-ceph module ah addon-operator, kaas oo ka dhigi doona adeegsigiisa kutlaasheena tirada badan ee Kubernetes xitaa ka fudud oo ka sahlan.

PS

Sidoo kale ka akhri boggayaga:

• «Rook - bakhaarka xogta "is-adeegga" ee Kubernetes";
• «Abuuritaanka kaydinta joogtada ah ee bixinta Kubernetes ee ku salaysan Ceph";
• «Databases iyo Kubernetes (dib u eegis iyo warbixinta muuqaalka)";
• «Soo bandhigida hawl-wadeenka qolofka: abuurista hawl-wadeennada Kubernetes way fududaatay";
• «Hawl-wadeennada Kubernetes: sida loo socodsiiyo codsiyada xaaladdaha leh".

Source: www.habr.com