🥇Ama-plugin wevolumu yokugcina ku-Kubernetes: ukusuka ku-Flexvolume kuya ku-CSI

Emuva ngenkathi i-Kubernetes isengu-v1.0.0, kwakunama-plugin evolumu. Bezidingeka ukuze kuxhunywe amasistimu ku-Kubernetes ukuze kugcinwe idatha yesiqukathi eqhubekayo (ingunaphakade). Inombolo yabo yayincane, futhi phakathi kwabokuqala kwakukhona abahlinzeki besitoreji abanjengoGCE PD, Ceph, AWS EBS nabanye.

Ama-plugin alethwe kanye ne-Kubernetes, yingakho athola igama lawo - esihlahleni. Kodwa-ke, kwabaningi, isethi ekhona yama-plugin anjalo alanele. Izingcweti zengeze ama-plugin alula kumongo we-Kubernetes zisebenzisa ama-patches, ngemva kwalokho zahlanganisa ama-Kubernetes azo futhi zawafaka kumaseva abo. Kodwa ngokuhamba kwesikhathi, abathuthukisi be-Kubernetes bakubona lokho inhlanzi inkinga ayikwazi ukuxazululeka. Abantu bayakudinga induku yokudoba. Futhi ekukhishweni kwe-Kubernetes v1.2.0 kuvele...

I-plugin ye-Flexvolume: induku encane yokudoba

Abathuthukisi be-Kubernetes badale i-plugin ye-FlexVolume, okwakuwuhlaka olunengqondo lwezinto eziguquguqukayo nezindlela zokusebenza nezishayeli ze-Flexvolume ezisetshenziswa onjiniyela bezinkampani zangaphandle.

Ake sime sibhekisise kahle ukuthi yini umshayeli weFlexVolume. Lokhu kuqinisekile ifayela elisebenzisekayo (ifayela kanambambili, iskripthi se-Python, iskripthi se-Bash, njll.), okuthi, lapho sisetshenziswa, sithathe izimpikiswano zomugqa womyalo njengokufakwayo futhi sibuyisele umlayezo onezinkambu ezaziwayo ngefomethi ye-JSON. Ngokwesimiso, impikiswano yokuqala yomugqa womyalo ihlale iyindlela, futhi izimpikiswano ezisele ziyimingcele yayo.

Umdwebo wokuxhuma wamasheya e-CIFS ku-OpenShift. I-Flexvolume Driver - Khona kanye Esikhungweni

Isethi encane yezindlela kubukeka kanjena:

flexvolume_driver mount # отвечает за присоединение тома к pod'у
# Формат возвращаемого сообщения:
{
  "status": "Success"/"Failure"/"Not supported",
  "message": "По какой причине был возвращен именно такой статус",
}

flexvolume_driver unmount # отвечает за отсоединение тома от pod'а
# Формат возвращаемого сообщения:
{
  "status": "Success"/"Failure"/"Not supported",
  "message": "По какой причине был возвращен именно такой статус",
}

flexvolume_driver init # отвечает за инициализацию плагина
# Формат возвращаемого сообщения:
{
  "status": "Success"/"Failure"/"Not supported",
  "message": "По какой причине был возвращен именно такой статус",
  // Определяет, использует ли драйвер методы attach/deatach
  "capabilities":{"attach": True/False}
}

Ukusebenzisa Izindlela attach и detach izochaza isimo lapho i-kubelet izosebenza khona esikhathini esizayo lapho ishayela umshayeli. Kukhona nezindlela ezikhethekile expandvolume и expandfs, abanomthwalo wemfanelo wokushintsha usayizi wevolumu ngamandla.

Njengesibonelo sezinguquko indlela eyengeza ngayo expandvolume, futhi ngayo ikhono lokushintsha usayizi wamavolumu ngesikhathi sangempela, ungazijwayeza isicelo sethu sokudonsa ku-Rook Ceph Operator.

Futhi nasi isibonelo sokuqaliswa komshayeli we-Flexvolume ekusebenzeni ne-NFS:

usage() {
    err "Invalid usage. Usage: "
    err "t$0 init"
    err "t$0 mount <mount dir> <json params>"
    err "t$0 unmount <mount dir>"
    exit 1
}

err() {
    echo -ne $* 1>&2
}

log() {
    echo -ne $* >&1
}

ismounted() {
    MOUNT=`findmnt -n ${MNTPATH} 2>/dev/null | cut -d' ' -f1`
    if [ "${MOUNT}" == "${MNTPATH}" ]; then
        echo "1"
    else
        echo "0"
    fi
}

domount() {
    MNTPATH=$1

    NFS_SERVER=$(echo $2 | jq -r '.server')
    SHARE=$(echo $2 | jq -r '.share')

    if [ $(ismounted) -eq 1 ] ; then
        log '{"status": "Success"}'
        exit 0
    fi

    mkdir -p ${MNTPATH} &> /dev/null

    mount -t nfs ${NFS_SERVER}:/${SHARE} ${MNTPATH} &> /dev/null
    if [ $? -ne 0 ]; then
        err "{ "status": "Failure", "message": "Failed to mount ${NFS_SERVER}:${SHARE} at ${MNTPATH}"}"
        exit 1
    fi
    log '{"status": "Success"}'
    exit 0
}

unmount() {
    MNTPATH=$1
    if [ $(ismounted) -eq 0 ] ; then
        log '{"status": "Success"}'
        exit 0
    fi

    umount ${MNTPATH} &> /dev/null
    if [ $? -ne 0 ]; then
        err "{ "status": "Failed", "message": "Failed to unmount volume at ${MNTPATH}"}"
        exit 1
    fi

    log '{"status": "Success"}'
    exit 0
}

op=$1

if [ "$op" = "init" ]; then
    log '{"status": "Success", "capabilities": {"attach": false}}'
    exit 0
fi

if [ $# -lt 2 ]; then
    usage
fi

shift

case "$op" in
    mount)
        domount $*
        ;;
    unmount)
        unmount $*
        ;;
    *)
        log '{"status": "Not supported"}'
        exit 0
esac

exit 1

Ngakho-ke, ngemuva kokulungiselela ifayela langempela elisebenzisekayo, udinga ukwenza layisha umshayeli kuqoqo le-Kubernetes. Umshayeli kufanele abekwe endaweni ngayinye yeqoqo ngokuvumelana nendlela enqunywe kusengaphambili. Ngokuzenzakalelayo ikhethiwe:

/usr/libexec/kubernetes/kubelet-plugins/volume/exec/имя_поставщика_хранилища~имя_драйвера/

... kodwa uma usebenzisa ukusabalalisa okuhlukile kwe-Kubernetes (OpenShift, Rancher...) indlela ingase yehluke.

Izinkinga ze-Flexvolume: indlela yokuphonsa induku yokudoba ngendlela efanele?

Ukulayisha umshayeli we-Flexvolume kuma-cluster node kuvele kwaba umsebenzi ongewona omncane. Ngemva kokwenza umsebenzi ngesandla kanye, kulula ukubhekana nesimo lapho ama-node amasha avela ku-cluster: ngenxa yokwengezwa kwe-node entsha, ukukala okuzenzakalelayo okuvundlile, noma - okubi nakakhulu - ukushintshwa kwe-node ngenxa yokungasebenzi kahle. Kulokhu, umsebenzi wokulondoloza kulawa ma-node kufanele wenziwe akunakwenzeka, uze ungeze ngokwakho umshayeli we-Flexvolume kubo.

Isixazululo sale nkinga kwakungesinye seziqalo zaseKubernetes - DaemonSet. Lapho i-node entsha ivela eqoqweni, ngokuzenzakalelayo iqukethe i-pod evela ku-DaemonSet yethu, lapho ivolumu yendawo ixhunywa khona endleleni ukuze kutholwe abashayeli be-Flexvolume. Ekudalweni okuphumelelayo, i-pod ikopisha amafayela adingekayo ukuze umshayeli asebenze kudiski.

Nasi isibonelo se-DaemonSet enjalo yokubeka i-plugin ye-Flexvolume:

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: flex-set
spec:
  template:
    metadata:
      name: flex-deploy
      labels:
        app: flex-deploy
    spec:
      containers:
        - image: <deployment_image>
          name: flex-deploy
          securityContext:
              privileged: true
          volumeMounts:
            - mountPath: /flexmnt
              name: flexvolume-mount
      volumes:
        - name: flexvolume-mount
          hostPath:
            path: <host_driver_directory>

... kanye nesibonelo seskripthi se-Bash sokubeka umshayeli we-Flexvolume:

#!/bin/sh

set -o errexit
set -o pipefail

VENDOR=k8s.io
DRIVER=nfs

driver_dir=$VENDOR${VENDOR:+"~"}${DRIVER}
if [ ! -d "/flexmnt/$driver_dir" ]; then
  mkdir "/flexmnt/$driver_dir"
fi

cp "/$DRIVER" "/flexmnt/$driver_dir/.$DRIVER"
mv -f "/flexmnt/$driver_dir/.$DRIVER" "/flexmnt/$driver_dir/$DRIVER"

while : ; do
  sleep 3600
done

Kubalulekile ukuthi ungakhohlwa ukuthi umsebenzi wokukopisha akuyona i-athomu. Maningi amathuba okuthi i-kubelet iqale ukusebenzisa umshayeli ngaphambi kokuthi kuqedwe inqubo yayo yokuhlinzeka, okubangele ukuthi uhlelo luphahlazeke. Indlela efanele iwukuba uqale ukopishe amafayela omshayeli ngaphansi kwegama elihlukile, bese usebenzisa umsebenzi wokuqamba kabusha i-athomu.

Umdwebo wokusebenza noCeph ku-opharetha we-Rook: umshayeli we-Flexvolume kumdwebo utholakala ngaphakathi kwe-ejenti ye-Rook.

Inkinga elandelayo uma usebenzisa abashayeli be-Flexvolume ukuthi isitoreji esiningi ku-cluster node isofthiwe edingekayo yalokhu kufanele ifakwe (ngokwesibonelo, iphakheji elivamile le-ceph). Ekuqaleni, i-plugin ye-Flexvolume yayingaklanyelwe ukusebenzisa lezi zinhlelo eziyinkimbinkimbi.

Isixazululo sangempela sale nkinga singabonakala ekusetshenzisweni komshayeli we-Flexvolume we-Rook operator:

Umshayeli ngokwawo uklanywe njengeklayenti le-RPC. Isokhethi ye-IPC yokuxhumana itholakala kumkhombandlela ofanayo nomshayeli ngokwakhe. Sikhumbula ukuthi ukukopisha amafayela omshayeli kungaba kuhle ukusebenzisa i-DaemonSet, exhuma uhla lwemibhalo nomshayeli njengevolumu. Ngemva kokukopisha amafayela adingekayo omshayeli we-rook, le pod ayifi, kodwa ixhuma kusokhethi ye-IPC ngevolumu enamathiselwe njengeseva ye-RPC egcwele. Iphakheji ye-ceph-common isivele ifakiwe ngaphakathi kwesiqukathi se-pod. Isokhethi ye-IPC iqinisekisa ukuthi i-kubelet izoxhumana ngqo ne-pod etholakala endaweni efanayo. Konke okuhlakaniphile kulula! ..

Sala kahle, ama-plugin wethu athandekayo... esihlahleni!

Abathuthukisi be-Kubernetes bathole ukuthi inani lama-plugin okugcinwa ngaphakathi komgogodla lingamashumi amabili. Futhi ushintsho kuzo zonke, ngandlela thize, ludlula emjikelezweni wokukhululwa we-Kubernetes ogcwele.

Kuvele ukuthi ukusebenzisa inguqulo entsha ye-plugin yesitoreji, udinga ukubuyekeza lonke iqoqo. Ngaphezu kwalokhu, ungase umangale ukuthi inguqulo entsha ye-Kubernetes izobe ingahambelani ngokuzumayo ne-Linux kernel oyisebenzisayo... Ngakho-ke usula izinyembezi zakho futhi, ubambe amazinyo akho, uxhumanise nabaphathi bakho kanye nabasebenzisi isikhathi buyekeza i-Linux kernel kanye neqoqo le-Kubernetes. Ngokusebenza kwesikhathi sokuphumula okungenzeka ekuhlinzekweni kwezinsizakalo.

Isimo singaphezu kokuhlekisa, awucabangi? Kucacele umphakathi wonke ukuthi le ndlela ayisebenzi. Ngesinqumo samabomu, abathuthukisi be-Kubernetes bamemezela ukuthi ama-plugin amasha okusebenza ngesitoreji ngeke esamukelwa ku-kernel. Ngaphezu kwalokho, njengoba sesivele sazi, kutholwe iziphambeko eziningi ekusetshenzisweni kwe-plugin ye-Flexvolume...

I-plugin yakamuva eyengeziwe yamavolumu ku-Kubernetes, CSI, yacelwa ukuthi ivale udaba ngokuqhubekayo kokugcinwa kwedatha kanye. Inguqulo yayo ye-alpha, ebizwa ngokugcwele ngokuthi i-Out-of-Tree CSI Volume Plugins, imenyezelwe ekukhululweni. Ama-Kubernetes 1.9.

I-Container Storage Interface, noma induku yokuphotha ye-CSI 3000!

Okokuqala, ngithanda ukuqaphela ukuthi i-CSI ayiyona nje i-plugin yevolumu, kodwa ingokoqobo standard ekudaleni izingxenye zangokwezifiso zokusebenza nezinqolobane zedatha. Izinhlelo zokucula iziqukathi ezifana ne-Kubernetes ne-Mesos bekufanele “zifunde” ukuthi zisebenza kanjani ngezinto ezisetshenziswa ngokwaleli zinga. Futhi manje sengiyifundile kakade i-Kubernetes.

Siyini isakhiwo se-plugin ye-CSI ku-Kubernetes? I-plugin ye-CSI isebenza nabashayeli abakhethekile (Abashayeli be-CSI) ebhalwe ngabathuthukisi bezinkampani zangaphandle. Umshayeli we-CSI e-Kubernetes kufanele okungenani abe nezingxenye ezimbili (ama-pods):

isilawuli - iphatha isitoreji sangaphandle esiqhubekayo. Isetshenziswa njengeseva ye-gRPC, lapho okokuqala kusetshenziselwa khona StatefulSet.
I-Node - inesibopho sokukhuphula isitoreji esiqhubekayo kuma-cluster node. Iphinde isetshenziswe njengeseva ye-gRPC, kodwa isebenzisa okwakudala DaemonSet.

Isebenza kanjani i-plugin ye-CSI ku-Kubernetes

Ungafunda ngeminye imininingwane yomsebenzi we-CSI, ngokwesibonelo, esihlokweni esithi “Ukuqonda i-C.S.I." ukuhunyushwa kwayo sashicilela ngonyaka odlule.

Izinzuzo zokuqaliswa okunjalo

Ezintweni eziyisisekelo njengokubhalisa umshayeli we-node, abathuthukisi be-Kubernetes basebenzise isethi yeziqukathi. Awusadingi ukukhiqiza impendulo ye-JSON ngamakhono ngokwakho, njengoba kwenziwa ku-plugin ye-Flexvolume.
Esikhundleni sokuthi "sishelele" amafayela asebenzisekayo kumanodi, manje sesilayisha ama-pods kuqoqo. Yilokhu esikulindele ekuqaleni kwa-Kubernetes: zonke izinqubo zenzeka ngaphakathi kweziqukathi ezisetshenziswe kusetshenziswa i-Kubernetes primitives.
Awusadingi ukuthuthukisa iseva ye-RPC kanye neklayenti le-RPC ukuze usebenzise izishayeli eziyinkimbinkimbi. Iklayenti lenzelwa thina ngabathuthukisi be-Kubernetes.
Ukudlulisa ama-agumenti ukuze kusebenze phezu kwephrothokholi ye-gRPC kulula kakhulu, kuyavumelana nezimo futhi kunokwethenjelwa kunokudlulisa izimpikiswano zomugqa womyalo. Ukuze uqonde ukuthi ungakwengeza kanjani usekelo lwamamethrikhi okusetshenziswa kwevolumu ku-CSI ngokungeza indlela ye-gRPC esezingeni, ungafunda: isicelo sethu sokudonsa yomshayeli we-vsphere-csi.
Ukuxhumana kwenzeka ngamasokhethi e-IPC, ukuze ungadideki ukuthi i-kubelet ithumele isicelo ku-pod efanele.

Ingabe lolu hlu likukhumbuza okuthile? Izinzuzo ze-CSI yilezi ukuxazulula lezo zinkinga ezifanayo, ezingazange zinakwe lapho kwakhiwa i-plugin ye-Flexvolume.

okutholakele

I-CSI njengendinganiso yokusebenzisa ama-plugin wangokwezifiso wokusebenzelana nezindawo zokugcina idatha yamukelwe ngokufudumele kakhulu umphakathi. Ngaphezu kwalokho, ngenxa yezinzuzo zabo nokuguquguquka, abashayeli be-CSI badalelwa ngisho nezinhlelo zokugcina ezifana ne-Ceph noma i-AWS EBS, ama-plugin okusebenza nawo angezwe kunguqulo yokuqala ye-Kubernetes.

Ekuqaleni kuka-2019, ama-plugin angaphakathi kwesihlahla kuthiwa ayisasebenzi. Sihlela ukuqhubeka nokusekela i-plugin ye-Flexvolume, kodwa ngeke sithuthukise ukusebenza kwayo okusha.

Thina ngokwethu sesivele sinolwazi lokusebenzisa i-ceph-csi, vsphere-csi futhi silungele ukungeza kulolu hlu! Kuze kube manje, i-CSI ibhekene nemisebenzi eyabelwe yona ngokugqama, kodwa sizolinda futhi sibone.

Ungakhohlwa ukuthi konke okusha kuwukucatshangelwa kabusha okuhle kwakudala!

PS

Funda futhi kubhulogi yethu:

Source: www.habr.com

Ama-plugin wevolumu okugcinwa kwe-Kubernetes: ukusuka ku-Flexvolume kuya ku-CSI