Rook kapena ayi Rook, limenelo ndi funso

Kumayambiriro kwa mwezi uno, pa Meyi 3, kutulutsidwa kwakukulu kwa "dongosolo loyang'anira zosungirako zogawidwa ku Kubernetes" kudalengezedwa - Mtundu 1.0.0. Zoposa chaka chapitacho ife kale zosindikizidwa mwachidule mwachidule Rook. Kenako tinapemphedwa kuti tikambirane zimene zinamuchitikira kugwiritsa ntchito - ndipo tsopano, mu nthawi yake yofunikira kwambiri m'mbiri ya polojekitiyi, ndife okondwa kugawana zomwe tapeza.

Mwachidule, Rook ndi seti ogwira ntchito kwa Kubernetes, yomwe imayang'anira kuyika, kuyang'anira, kubwezeretsanso njira zosungiramo deta monga Ceph, EdgeFS, Minio, Cassandra, CockroachDB.

Pakadali pano otukuka kwambiri (ndi yekhayo в khola stage) yankho ndilo rook-ceph-operator.

ndemanga: Pakati pa kusintha kwakukulu kwa Rook 1.0.0 kumasulidwa kokhudzana ndi Ceph, tikhoza kuzindikira chithandizo cha Ceph Nautilus ndi kuthekera kogwiritsa ntchito NFS kwa CephFS kapena RGW ndowa. Chomwe chimadziwika pakati pa ena ndikukula kwa chithandizo cha EdgeFS pamlingo wa beta.

Choncho, m'nkhaniyi ife:

• Tiyeni tiyankhe funso lokhudza zabwino zomwe timawona pogwiritsa ntchito Rook kutumiza Ceph mu gulu la Kubernetes;
• Tigawana zomwe takumana nazo komanso zomwe tikuwona pogwiritsa ntchito Rook popanga;
• Tiye tikuuzeni chifukwa chake timati “Inde!” kwa Rook, ndi za mapulani athu kwa iye.

Tiyeni tiyambe ndi mfundo wamba ndi chiphunzitso.

"Ndili ndi mwayi wa Rook m'modzi!" (wosewerera chess wosadziwika)

Chimodzi mwazabwino zazikulu za Rook ndikuti kulumikizana ndi masitolo a data kumachitika kudzera mu njira za Kubernetes. Izi zikutanthauza kuti simukufunikanso kukopera malamulo kuti mukonze Ceph kuchokera papepala kupita ku console.

- Kodi mukufuna kutumiza CephFS mgulu? Ingolembani fayilo ya YAML!
- Chani? Kodi mukufunanso kutumiza sitolo ya zinthu ndi S3 API? Ingolembani fayilo yachiwiri ya YAML!

Rook amapangidwa motsatira malamulo onse a woyendetsa. Kuyanjana ndi iye kumachitika pogwiritsa ntchito CRD (Matanthauzo Azinthu Zachikhalidwe), momwe timafotokozera zamagulu a Ceph omwe timafunikira (popeza uku ndiye kukhazikitsa kokhazikika, mwachisawawa nkhaniyi ilankhula za Ceph, pokhapokha zitanenedwa momveka bwino). Malinga ndi magawo omwe atchulidwa, wogwiritsa ntchitoyo adzipangira okha malamulo ofunikira kuti asinthidwe.

Tiyeni tiwone zenizeni pogwiritsa ntchito chitsanzo chopanga Sitolo Yazinthu, kapena m'malo mwake - CephObjectStoreUser.

apiVersion: ceph.rook.io/v1
kind: CephObjectStore
metadata:
  name: {{ .Values.s3.crdName }}
  namespace: kube-rook
spec:
  metadataPool:
    failureDomain: host
    replicated:
      size: 3
  dataPool:
    failureDomain: host
    erasureCoded:
      dataChunks: 2
      codingChunks: 1
  gateway:
    type: s3
    sslCertificateRef:
    port: 80
    securePort:
    instances: 1
    allNodes: false
---
apiVersion: ceph.rook.io/v1
kind: CephObjectStoreUser
metadata:
  name: {{ .Values.s3.crdName }}
  namespace: kube-rook
spec:
  store: {{ .Values.s3.crdName }}
  displayName: {{ .Values.s3.username }}

Magawo omwe asonyezedwa pamndandandawo ndi okhazikika ndipo safuna ndemanga, koma ndikofunikira kulabadira kwambiri zomwe zaperekedwa kumitundu yosiyanasiyana ya ma template.

Dongosolo lonse lantchito limatsikira ku mfundo yakuti "timayitanitsa" zothandizira kudzera pa fayilo ya YAML, yomwe wogwiritsa ntchitoyo amatsatira malamulo ofunikira ndikutibweretsera chinsinsi "chosakhala chenicheni" chomwe titha kugwirirapo ntchito. (Onani pansipa). Ndipo kuchokera pazosintha zomwe zalembedwa pamwambapa, lamulo ndi dzina lachinsinsi lidzapangidwa.

Ndi timu yanji iyi? Popanga wogwiritsa ntchito posungira zinthu, wogwiritsa ntchito Rook mkati mwa pod adzachita izi:

radosgw-admin user create --uid="rook-user" --display-name="{{ .Values.s3.username }}"

Chotsatira chotsatira lamuloli chidzakhala mawonekedwe a JSON:

{
    "user_id": "rook-user",
    "display_name": "{{ .Values.s3.username }}",
    "keys": [
        {
           "user": "rook-user",
           "access_key": "NRWGT19TWMYOB1YDBV1Y",
           "secret_key": "gr1VEGIV7rxcP3xvXDFCo4UDwwl2YoNrmtRlIAty"
        }
    ],
    ...
}

Keys - Ndi mapulogalamu ati amtsogolo omwe adzafunikire kupeza zosungirako zinthu kudzera pa S3 API. Wogwiritsa ntchito Rook amawasankha mokoma mtima ndikuwayika m'malo ake a mayina mwachinsinsi chokhala ndi dzinalo rook-ceph-object-user-{{ $.Values.s3.crdName }}-{{ $.Values.s3.username }}.

Kuti mugwiritse ntchito zomwe zili muchinsinsichi, ingowonjezerani ku chidebe ngati zosintha za chilengedwe. Mwachitsanzo, ndipereka template ya Job, momwe timapangira zidebe zamtundu uliwonse wa ogwiritsa ntchito:

{{- range $bucket := $.Values.s3.bucketNames }}
apiVersion: batch/v1
kind: Job
metadata:
  name: create-{{ $bucket }}-bucket-job
  annotations:
    "helm.sh/hook": post-install
    "helm.sh/hook-weight": "2"
spec:
  template:
    metadata:
      name: create-{{ $bucket }}-bucket-job
    spec:
      restartPolicy: Never
      initContainers:
      - name: waitdns
        image: alpine:3.6
        command: ["/bin/sh", "-c", "while ! getent ahostsv4 rook-ceph-rgw-{{ $.Values.s3.crdName }}; do sleep 1; done" ]
      - name: config
        image: rook/ceph:v1.0.0
        command: ["/bin/sh", "-c"]
        args: ["s3cmd --configure --access_key=$(ACCESS-KEY) --secret_key=$(SECRET-KEY) -s --no-ssl --dump-config | tee /config/.s3cfg"]
        volumeMounts:
        - name: config
          mountPath: /config
        env:
        - name: ACCESS-KEY
          valueFrom:
            secretKeyRef:
              name: rook-ceph-object-user-{{ $.Values.s3.crdName }}-{{ $.Values.s3.username }}
              key: AccessKey
        - name: SECRET-KEY
          valueFrom:
            secretKeyRef:
              name: rook-ceph-object-user-{{ $.Values.s3.crdName }}-{{ $.Values.s3.username }}
              key: SecretKey
      containers:
      - name: create-bucket
        image: rook/ceph:v1.0.0
        command: 
        - "s3cmd"
        - "mb"
        - "--host=rook-ceph-rgw-{{ $.Values.s3.crdName }}"
        - "--host-bucket= "
        - "s3://{{ $bucket }}"
        ports:
        - name: s3-no-sll
          containerPort: 80
        volumeMounts:
        - name: config
          mountPath: /root
      volumes:
      - name: config
        emptyDir: {}
---
{{- end }}

Zochita zonse zomwe zalembedwa mu Job iyi zidachitika mkati mwa Kubernetes. Zomwe zafotokozedwa m'mafayilo a YAML zimasungidwa munkhokwe ya Git ndipo zimagwiritsidwanso ntchito nthawi zambiri. Tikuwona izi ngati kuphatikiza kwakukulu kwa mainjiniya a DevOps ndi njira ya CI/CD yonse.

Wokondwa ndi Rook ndi Rados

Kugwiritsa ntchito kuphatikiza kwa Ceph + RBD kumayika zoletsa zina pakukweza ma voliyumu ku ma pod.

Makamaka, malowa ayenera kukhala ndi chinsinsi chofikira ku Ceph kuti mapulogalamu ovomerezeka agwire ntchito. Ndibwino ngati muli ndi malo 2-3 m'malo awo: mutha kupita ndikukopera chinsinsi pamanja. Koma bwanji ngati pachinthu chilichonse malo osiyana okhala ndi dzina lake apangidwira opanga?

Tinathetsa vutoli tokha pogwiritsa ntchito chipolopolo-woyendetsa, zomwe zimakopera zinsinsi ku malo atsopano (chitsanzo cha mbedza yoteroyo chafotokozedwa mu nkhaniyi).

#! /bin/bash

if [[ $1 == “--config” ]]; then
   cat <<EOF
{"onKubernetesEvent":[
 {"name": "OnNewNamespace",
  "kind": "namespace",
  "event": ["add"]
  }
]}
EOF
else
    NAMESPACE=$(kubectl get namespace -o json | jq '.items | max_by( .metadata.creationTimestamp ) | .metadata.name')
    kubectl -n ${CEPH_SECRET_NAMESPACE} get secret ${CEPH_SECRET_NAME} -o json | jq ".metadata.namespace="${NAMESPACE}"" | kubectl apply -f -
fi

Komabe, mukamagwiritsa ntchito Rook vutoli kulibe. Njira yowonjezera imachitika pogwiritsa ntchito madalaivala ake kutengera Flex volume kapena CSI (akadali mu siteji ya beta) choncho sichifuna zinsinsi.

Rook amathetsa mavuto ambiri, zomwe zimatilimbikitsa kuti tizigwiritse ntchito m'mapulojekiti atsopano.

Kuzingidwa kwa Rook

Tiyeni timalize gawo lothandizira potumiza Rook ndi Ceph kuti titha kuchita zoyeserera zathu. Kuti zikhale zosavuta kuwononga nsanja yosagonjetsekayi, opanga akonza phukusi la Helm. Tiyeni tipange dawunilodi:

$ helm fetch rook-master/rook-ceph --untar --version 1.0.0

Mu fayilo rook-ceph/values.yaml mutha kupeza zokonda zosiyanasiyana. Chofunikira kwambiri ndikutanthauzira kulekerera kwa othandizira ndi kusaka. Tidafotokozera mwatsatanetsatane zomwe njira zotayirira / zolekerera zitha kugwiritsidwa ntchito nkhaniyi.

Mwachidule, sitikufuna kuti ma pods ogwiritsira ntchito kasitomala azikhala pamalo omwewo ngati ma disks osungira. Chifukwa chake ndi chosavuta: motere ntchito ya othandizira a Rook sidzakhudza kugwiritsa ntchito komweko.

Kenako, tsegulani fayilo rook-ceph/values.yaml ndi mkonzi wanu womwe mumakonda ndikuwonjezera chipika chotsatira kumapeto:

discover:
  toleration: NoExecute
  tolerationKey: node-role/storage
agent:
  toleration: NoExecute
  tolerationKey: node-role/storage
  mountSecurityMode: Any

Pa node iliyonse yomwe yasungidwa kuti isungidwe deta, onjezani chodetsa chofananira:

$ kubectl taint node ${NODE_NAME} node-role/storage="":NoExecute

Kenako ikani tchati cha Helm ndi lamulo:

$ helm install --namespace ${ROOK_NAMESPACE} ./rook-ceph

Tsopano muyenera kupanga masango ndikulongosola malo OSD:

apiVersion: ceph.rook.io/v1
kind: CephCluster
metadata:
  clusterName: "ceph"
  finalizers:
  - cephcluster.ceph.rook.io
  generation: 1
  name: rook-ceph
spec:
  cephVersion:
    image: ceph/ceph:v13
  dashboard:
    enabled: true
  dataDirHostPath: /var/lib/rook/osd
  mon:
    allowMultiplePerNode: false
    count: 3
  network:
    hostNetwork: true
  rbdMirroring:
    workers: 1
  placement:
    all:
      tolerations:
      - key: node-role/storage
        operator: Exists
  storage:
    useAllNodes: false
    useAllDevices: false
    config:
      osdsPerDevice: "1"
      storeType: filestore
    resources:
      limits:
        memory: "1024Mi"
      requests:
        memory: "1024Mi"
    nodes:
    - name: host-1
      directories:
      - path: "/mnt/osd"
    - name: host-2
      directories:
      - path: "/mnt/osd"
    - name: host-3
      directories:
      - path: "/mnt/osd"

Kuyang'ana mawonekedwe a Ceph - yembekezerani kuwona HEALTH_OK:

$ kubectl -n ${ROOK_NAMESPACE} exec $(kubectl -n ${ROOK_NAMESPACE} get pod -l app=rook-ceph-operator -o name -o jsonpath='{.items[0].metadata.name}') -- ceph -s

Nthawi yomweyo, tiyeni tiwone kuti ma pod omwe ali ndi kasitomala samatha kukhala pazida zosungidwa kwa Ceph:

$ kubectl -n ${APPLICATION_NAMESPACE} get pods -o custom-columns=NAME:.metadata.name,NODE:.spec.nodeName

Kuphatikiza apo, zida zowonjezera zitha kukhazikitsidwa momwe mukufunira. Zambiri za iwo zikuwonetsedwa mu zolemba. Kwa oyang'anira, timalimbikitsa kwambiri kukhazikitsa dashboard ndi bokosi la zida.

Rook ndi mbedza: kodi Rook amakwanira chilichonse?

Monga mukuonera, kukula kwa Rook kuli pachimake. Koma pali mavuto omwe satilola kuti tisiye kusinthika kwa Ceph:

• Palibe Woyendetsa Rook sindingathe ma metrics otumiza kunja pakugwiritsa ntchito midadada yokwera, zomwe zimatilepheretsa kuyang'anira.
• Flexvolume ndi CSI sindikudziwa momwe sinthani kukula kwa voliyumu (mosiyana ndi RBD yomweyo), kotero Rook amachotsedwa chida chothandiza (ndipo nthawi zina chofunikira kwambiri!).
• Rook akadali wosasinthika ngati Ceph wamba. Ngati tikufuna kukonza dziwe la CephFS metadata kuti lisungidwe pa SSD, ndipo deta yokha kuti isungidwe pa HDD, tidzafunika kulembetsa magulu osiyana a zipangizo mumapu a CRUSH pamanja.
• Ngakhale kuti rook-ceph-operator imatengedwa kuti ndi yokhazikika, pakali pano pali zovuta zina pamene mukukweza Ceph kuchokera ku 13 mpaka 14.

anapezazo

"Pakadali pano Rook watsekedwa ndi anthu akunja, koma tikukhulupirira kuti tsiku lina adzachita mbali yofunika kwambiri pamasewerawa!" (mawu omwe adapangidwa makamaka pankhaniyi)

Ntchito ya Rook mosakayikira yatipindulira mitima yathu - tikukhulupirira kuti [ndi zabwino zonse ndi zoyipa zake] ikuyeneradi chidwi chanu.

Zolinga zathu zam'tsogolo zimabwera ndikupanga rook-ceph kukhala gawo la addon-wothandizira, zomwe zipangitsa kugwiritsa ntchito kwake m'magulu athu ambiri a Kubernetes kukhala kosavuta komanso kosavuta.

PS

Werenganinso pa blog yathu:

• «Rook - "self-service" yosungiramo deta ya Kubernetes";
• «Kupanga kusungirako kosalekeza ndikupereka ku Kubernetes kutengera Ceph";
• «Databases ndi Kubernetes (review and video report)";
• «Kuyambitsa zipolopolo: kupanga ogwiritsa ntchito Kubernetes kwakhala kosavuta";
• «Othandizira a Kubernetes: momwe mungayendetsere mapulogalamu apamwamba".

Source: www.habr.com