Ukuba Rook okanye hayi Rook - ngumbuzo lowo

Ekuqaleni kwale nyanga, ngoMeyi 3, ukukhululwa okukhulu "kwenkqubo yokulawula ukuhanjiswa kwedatha kwi-Kubernetes" yabhengezwa - Rook 1.0.0. Ngaphezu konyaka ophelileyo sele ipapashiwe amagqabantshintshi jikelele Rook. Emva koko sacelwa ukuba sithethe ngamava akhe sebenzisa ekusebenzeni - kwaye ngoku, kanye ngexesha lembali ebaluleke kangaka kwimbali yeprojekthi, siyakuvuyela ukwabelana ngezimvo zethu eziqokelelweyo.

Ngamafutshane, iRook iseti abaqhubi I-Kubernetes, ethatha ulawulo olupheleleyo lokuthunyelwa, ulawulo, ukubuyiswa ngokuzenzekelayo kwezixazululo zokugcina idatha ezifana neCeph, EdgeFS, Minio, Cassandra, CockroachDB.

Okwangoku eyona iphuhliswe kakhulu (kunye omnye kuphela в ezinzile isigaba) isisombululo i-rook-ceph-opharetha.

Qaphela:: Phakathi kweenguqu eziphawulekayo kwi-Rook 1.0.0 yokukhululwa ehambelana neCeph, sinokuqaphela inkxaso yeCeph Nautilus kunye nokukwazi ukusebenzisa i-NFS ye-CephFS okanye i-RGW ibhakethi. Yintoni ebonakalayo phakathi kwabanye kukuvuthwa kwenkxaso ye-EdgeFS kwinqanaba le-beta.

Ke, kweli nqaku si:

• Masiphendule umbuzo malunga nokuba zeziphi iingenelo esizibonayo ekusebenziseni iRook ukuhambisa iCeph kwiqela leKubernetes;
• Siza kwabelana ngamava ethu kunye nemibono yokusebenzisa iRook kwimveliso;
• Makhe sikuxelele ukuba kutheni sisithi “Ewe!” kuRook, nangezicwangciso zethu ngaye.

Masiqale ngeekhonsepthi ngokubanzi kunye nethiyori.

"Ndine-advanteji yeRook enye!" (umdlali wechess ongaziwayo)

Enye yeenzuzo eziphambili zeRook kukuba ukusebenzisana kunye neevenkile zedatha kwenziwa ngeendlela zeKubernetes. Oku kuthetha ukuba akusekho mfuneko yokuba ukhuphele imiyalelo yokuqwalasela iCeph ukusuka kwiphepha ukuya kwiconsole.

— Ngaba uyafuna ukubeka i-CephFS kwiqela? Bhala nje ifayile yeYAML!
- Intoni? Ngaba uyafuna kwakhona ukubeka ivenkile yezinto nge-S3 API? Bhala nje ifayile yesibini yeYAML!

I-Rook idalwe ngokwemigaqo yonke yomqhubi oqhelekileyo. Ukusebenzisana naye kwenzeka ngokusebenzisa I-CRD (IiNkcazelo zeSibonelelo sesiKhokelo), apho sichaza iimpawu zamaqumrhu eCeph esiwadingayo (ekubeni oku kuphela kokuphunyezwa okuzinzileyo, ngokungagqibekanga eli nqaku liza kuthetha ngeCeph, ngaphandle kokuba kuchazwe ngokucacileyo ngenye indlela). Ngokweeparamitha ezichaziweyo, umqhubi uya kwenza ngokuzenzekelayo imiyalelo eyimfuneko yoqwalaselo.

Makhe sijonge izinto ezithile sisebenzisa umzekelo wokudala iVenkile yeNto, okanye kunoko - CephObjectStoreUser.

apiVersion: ceph.rook.io/v1
kind: CephObjectStore
metadata:
  name: {{ .Values.s3.crdName }}
  namespace: kube-rook
spec:
  metadataPool:
    failureDomain: host
    replicated:
      size: 3
  dataPool:
    failureDomain: host
    erasureCoded:
      dataChunks: 2
      codingChunks: 1
  gateway:
    type: s3
    sslCertificateRef:
    port: 80
    securePort:
    instances: 1
    allNodes: false
---
apiVersion: ceph.rook.io/v1
kind: CephObjectStoreUser
metadata:
  name: {{ .Values.s3.crdName }}
  namespace: kube-rook
spec:
  store: {{ .Values.s3.crdName }}
  displayName: {{ .Values.s3.username }}

Iiparamitha ezibonisiweyo kuluhlu zisemgangathweni kwaye azifuni zimvo, kodwa kufanelekile ukunikela ingqalelo ekhethekileyo kwezo zabelwe ukuguquguquka kwetemplate.

Iskimu esiqhelekileyo somsebenzi sehla kwinto yokuba "siyaodola" izixhobo ngefayile ye-YAML, apho umqhubi wenza imiyalelo eyimfuneko kwaye asibuyisele imfihlo "engeyiyo-yokwenyani" esinokuthi siqhubeke sisebenza ngayo. (bona ngezantsi). Kwaye ukusuka kwiinguqu ezidweliswe ngasentla, umyalelo kunye negama eliyimfihlo liya kuqulunqwa.

Liqela elinjani eli? Xa udala umsebenzisi wokugcina into, umqhubi weRook ngaphakathi kwepod uya kwenza oku kulandelayo:

radosgw-admin user create --uid="rook-user" --display-name="{{ .Values.s3.username }}"

Isiphumo sokuphumeza lo myalelo siya kuba sisakhiwo se-JSON:

{
    "user_id": "rook-user",
    "display_name": "{{ .Values.s3.username }}",
    "keys": [
        {
           "user": "rook-user",
           "access_key": "NRWGT19TWMYOB1YDBV1Y",
           "secret_key": "gr1VEGIV7rxcP3xvXDFCo4UDwwl2YoNrmtRlIAty"
        }
    ],
    ...
}

Keys - zeziphi izicelo ezizayo eziza kufuna ukufikelela kwindawo yokugcina into nge-S3 API. Umsebenzisi weRook uzikhetha ngobubele aze azibeke kwindawo yakhe yamagama ngendlela eyimfihlo enegama rook-ceph-object-user-{{ $.Values.s3.crdName }}-{{ $.Values.s3.username }}.

Ukusebenzisa idatha evela kule mfihlelo, yongeza nje kwisikhongozeli njengezinto eziguquguqukayo zokusingqongileyo. Njengomzekelo, ndiza kunika ithempleyithi yoMsebenzi, apho sizenzela khona iibhakethi kwindawo nganye yomsebenzisi:

{{- range $bucket := $.Values.s3.bucketNames }}
apiVersion: batch/v1
kind: Job
metadata:
  name: create-{{ $bucket }}-bucket-job
  annotations:
    "helm.sh/hook": post-install
    "helm.sh/hook-weight": "2"
spec:
  template:
    metadata:
      name: create-{{ $bucket }}-bucket-job
    spec:
      restartPolicy: Never
      initContainers:
      - name: waitdns
        image: alpine:3.6
        command: ["/bin/sh", "-c", "while ! getent ahostsv4 rook-ceph-rgw-{{ $.Values.s3.crdName }}; do sleep 1; done" ]
      - name: config
        image: rook/ceph:v1.0.0
        command: ["/bin/sh", "-c"]
        args: ["s3cmd --configure --access_key=$(ACCESS-KEY) --secret_key=$(SECRET-KEY) -s --no-ssl --dump-config | tee /config/.s3cfg"]
        volumeMounts:
        - name: config
          mountPath: /config
        env:
        - name: ACCESS-KEY
          valueFrom:
            secretKeyRef:
              name: rook-ceph-object-user-{{ $.Values.s3.crdName }}-{{ $.Values.s3.username }}
              key: AccessKey
        - name: SECRET-KEY
          valueFrom:
            secretKeyRef:
              name: rook-ceph-object-user-{{ $.Values.s3.crdName }}-{{ $.Values.s3.username }}
              key: SecretKey
      containers:
      - name: create-bucket
        image: rook/ceph:v1.0.0
        command: 
        - "s3cmd"
        - "mb"
        - "--host=rook-ceph-rgw-{{ $.Values.s3.crdName }}"
        - "--host-bucket= "
        - "s3://{{ $bucket }}"
        ports:
        - name: s3-no-sll
          containerPort: 80
        volumeMounts:
        - name: config
          mountPath: /root
      volumes:
      - name: config
        emptyDir: {}
---
{{- end }}

Zonke izenzo ezidweliswe kulo msebenzi zenziwa kwisakhelo seKubernetes. Izakhiwo ezichazwe kwiifayile ze-YAML zigcinwa kwindawo yokugcina yeGit kwaye ziphinda zisetyenziswe amaxesha amaninzi. Sibona oku njengento enkulu yeenjineli ze-DevOps kunye nenkqubo ye-CI / CD iyonke.

Ndonwabile noRook kunye noRados

Ukusebenzisa iCeph + RBD indibaniselwano ibeka izithintelo ezithile ekunyuseni imiqulu kwiipods.

Ngokukodwa, indawo yegama kufuneka iqulathe imfihlo yokufikelela kwiCeph ukuze izicelo ezisemthethweni zisebenze. Kulungile ukuba une-2-3 yemimandla kwiindawo zabo zamagama: ungaya kwaye ukope imfihlo ngesandla. Kodwa kuthekani ukuba kwinqaku ngalinye indawo engqongileyo eyahlukileyo enendawo yayo yegama yenzelwe abaphuhlisi?

Le ngxaki siyisombulule ngokwethu sisebenzisa iqokobhe-umqhubi, ekhuphela ngokuzenzekelayo iimfihlo kwiindawo ezintsha zamagama (umzekelo wekhonkco elinjalo lichazwe kuyo eli nqaku).

#! /bin/bash

if [[ $1 == “--config” ]]; then
   cat <<EOF
{"onKubernetesEvent":[
 {"name": "OnNewNamespace",
  "kind": "namespace",
  "event": ["add"]
  }
]}
EOF
else
    NAMESPACE=$(kubectl get namespace -o json | jq '.items | max_by( .metadata.creationTimestamp ) | .metadata.name')
    kubectl -n ${CEPH_SECRET_NAMESPACE} get secret ${CEPH_SECRET_NAME} -o json | jq ".metadata.namespace="${NAMESPACE}"" | kubectl apply -f -
fi

Nangona kunjalo, xa usebenzisa iRook le ngxaki ayikho nje. Inkqubo yokunyuswa kwenzeka usebenzisa abaqhubi bayo ngokusekelwe I-Flexvolume okanye CSI (esekwinqanaba le-beta) kwaye ke ayifuni zimfihlo.

URook usombulula ngokuzenzekelayo iingxaki ezininzi, ezisikhuthaza ukuba sisebenzise kwiiprojekthi ezintsha.

Ukungqingwa kweRook

Masigqibezele icandelo elisebenzayo ngokuthumela iRook kunye neCeph ukuze sikwazi ukwenza eyethu imifuniselo. Ukwenza kube lula ukuvuthuza le nqaba ingenakungena, abaphuhlisi balungiselele iphakheji yeHelm. Masiyikhuphele:

$ helm fetch rook-master/rook-ceph --untar --version 1.0.0

Kwifayile rook-ceph/values.yaml ungafumana izicwangciso ezininzi ezahlukeneyo. Into ebaluleke kakhulu kukucacisa ukunyamezela kwee-arhente kunye nokukhangela. Sichaze ngokweenkcukacha ukuba ithini indlela yokungcolisa/yokunyamezela ingasetyenziselwa yona eli nqaku.

Ngokufutshane, asifuni ukuba iipods zesicelo somthengi zibekwe kwiindawo ezifanayo njengediski yokugcina idatha. Isizathu silula: ngale ndlela umsebenzi we-Rook agents awuyi kuchaphazela isicelo ngokwawo.

Ngoko, vula ifayile rook-ceph/values.yaml ngomhleli wakho owuthandayo kwaye wongeze ibhloko elandelayo ekugqibeleni:

discover:
  toleration: NoExecute
  tolerationKey: node-role/storage
agent:
  toleration: NoExecute
  tolerationKey: node-role/storage
  mountSecurityMode: Any

Kwindawo nganye egcinelwe ugcino lwedatha, yongeza i-taint ehambelanayo:

$ kubectl taint node ${NODE_NAME} node-role/storage="":NoExecute

Emva koko faka itshathi yeHelm ngomyalelo:

$ helm install --namespace ${ROOK_NAMESPACE} ./rook-ceph

Ngoku kufuneka udale i-cluster kwaye ucacise indawo OSD:

apiVersion: ceph.rook.io/v1
kind: CephCluster
metadata:
  clusterName: "ceph"
  finalizers:
  - cephcluster.ceph.rook.io
  generation: 1
  name: rook-ceph
spec:
  cephVersion:
    image: ceph/ceph:v13
  dashboard:
    enabled: true
  dataDirHostPath: /var/lib/rook/osd
  mon:
    allowMultiplePerNode: false
    count: 3
  network:
    hostNetwork: true
  rbdMirroring:
    workers: 1
  placement:
    all:
      tolerations:
      - key: node-role/storage
        operator: Exists
  storage:
    useAllNodes: false
    useAllDevices: false
    config:
      osdsPerDevice: "1"
      storeType: filestore
    resources:
      limits:
        memory: "1024Mi"
      requests:
        memory: "1024Mi"
    nodes:
    - name: host-1
      directories:
      - path: "/mnt/osd"
    - name: host-2
      directories:
      - path: "/mnt/osd"
    - name: host-3
      directories:
      - path: "/mnt/osd"

Ukujonga imeko yeCeph - lindela ukubona HEALTH_OK:

$ kubectl -n ${ROOK_NAMESPACE} exec $(kubectl -n ${ROOK_NAMESPACE} get pod -l app=rook-ceph-operator -o name -o jsonpath='{.items[0].metadata.name}') -- ceph -s

Kwangelo xesha, masijonge ukuba iipods ezinesicelo somthengi azipheli kwiindawo ezigcinelwe iCeph:

$ kubectl -n ${APPLICATION_NAMESPACE} get pods -o custom-columns=NAME:.metadata.name,NODE:.spec.nodeName

Ngaphaya koko, iinxalenye ezongezelelweyo zinokumiselwa njengoko zifunwa. Iinkcukacha ezithe kratya malunga nazo zibonisiwe kwi amaxwebhu. Kulawulo, sicebisa kakhulu ukuba kufakwe ideshibhodi kunye nebhokisi yezixhobo.

I-Rook kunye neekhonkco: Ngaba iRook yanele yonke into?

Njengoko ubona, uphuhliso lweRook luhamba ngokupheleleyo. Kodwa kusekho iingxaki ezingasivumeli ukuba silulahle ngokupheleleyo uqwalaselo lwencwadi lweCeph:

• Akukho Rook Driver ngekhe i-metrics yokuthumela ngaphandle kusetyenziso lweebhloko ezixhonyiweyo, ezisivalelayo ukubeka iliso.
• I-Flexvolume kunye ne-CSI andazi ukuba njani tshintsha ubungakanani bemiqulu (ngokuchasene ne-RBD efanayo), ngoko i-Rook ivinjwe i-eluncedo (kwaye ngamanye amaxesha ifunekayo kakhulu!) Isixhobo.
• I-Rook ayikabi bhetyebhetye njengeCeph eqhelekileyo. Ukuba sifuna ukuqwalasela i-pool ye-metadata ye-CephFS ukuba igcinwe kwi-SSD, kwaye idatha ngokwayo igcinwe kwi-HDD, kuya kufuneka sibhalise amaqela ahlukeneyo ezixhobo kwiimephu ze-CRUSH ngesandla.
• Ngaphandle kwento yokuba i-rook-ceph-operator ithathwa njengezinzile, kukho iingxaki ngoku xa uphucula iCeph ukusuka kwinguqulo ye-13 ukuya kwi-14.

ezifunyanisiweyo

“Okwangoku uRook uvaliwe kwihlabathi langaphandle, kodwa sikholelwa ukuba ngenye imini uya kudlala indima ebalulekileyo emdlalweni!” (isicatshulwa senzelwe eli nqaku)

Iprojekthi yeRook ngokungathandabuzekiyo iziphumelele iintliziyo zethu - sikholelwa ukuba [kunye nazo zonke iingenelo kunye neengxaki zayo] ngokuqinisekileyo ifanelwe ingqalelo yakho.

Izicwangciso zethu zexesha elizayo ziphelela ekwenzeni i-rook-ceph imodyuli ye i-addon-operator, eya kwenza ukusetyenziswa kwayo kumaqela ethu amaninzi e-Kubernetes kube lula kwaye kulula ngakumbi.

PS

Funda nakwibhlog yethu:

• «URook - "inkonzo yokuzenzela" indawo yokugcina idatha yeKubernetes";
• «Ukudala ukugcinwa okuqhubekayo kunye nokubonelela kwi-Kubernetes esekelwe kwi-Ceph";
• «Iidatabase kunye neKubernetes (uphononongo kunye nengxelo yevidiyo)";
• «Ukwazisa i-shell-operator: ukudala abasebenzi be-Kubernetes kube lula";
• «AbaSebenzi be-Kubernetes: indlela yokuqhuba izicelo ezifanelekileyo».

umthombo: www.habr.com