Iiplagi zevolumu zokugcinwa kweKubernetes: ukusuka kwiFlexvolume ukuya kwiCSI

Emuva xa i-Kubernetes isekho v1.0.0, kwakukho iiplagi zevolumu. Babefuneka ukudibanisa iinkqubo kwi-Kubernetes yokugcina idatha yesikhongozeli esiqhubekayo (esisigxina). Inani labo lalincinci, kwaye phakathi kwezokuqala kwakukho ababoneleli abanjalo be-GCE PD, i-Ceph, i-AWS EBS kunye nabanye.

Iiplagi zahanjiswa kunye neKubernetes, yiyo loo nto bafumana igama labo - emthini. Nangona kunjalo, kwabaninzi, iseti ekhoyo yezo plugins ibonakale ingonelanga. Abachwephesha bongeza iiplagi ezilula kumbindi we-Kubernetes besebenzisa iipatches, emva koko bahlanganisa i-Kubernetes yabo kwaye bayifaka kwiiseva zabo. Kodwa ekuhambeni kwexesha, abaphuhlisi beKubernetes baqonda ukuba intlanzi ingxaki ayinakusonjululwa. Abantu bayayidinga intonga yokuloba. Kwaye ekukhutshweni kwe-Kubernetes v1.2.0 kwavela...

I-plugin ye-Flexvolume: intonga encinci yokuloba

Abaphuhlisi be-Kubernetes benza i-plugin ye-FlexVolume, eyayiyisikhokelo esinengqiqo sezinto eziguquguqukayo kunye neendlela zokusebenza kunye nabaqhubi be-Flexvolume abaphunyezwe ngabaphuhlisi beqela lesithathu.

Masime sijonge ngakumbi ukuba yintoni umqhubi weFlexVolume. Oku kuqinisekileyo ifayile ephunyeziweyo (ifayile yokubini, iskripthi sePython, iskripthi seBash, njl.njl.), ethi, xa yenziwe, ithathe iimpikiswano zelayini yomyalelo njengegalelo kwaye ibuyisela umyalezo onemihlaba eyaziwayo ngaphambili kwifomati ye-JSON. Ngokwengqungquthela, ingxabano yomgca wokuqala womyalelo isoloko iyindlela, kwaye iingxoxo eziseleyo ziiparameters zayo.

Umzobo woQhagamshelwano lwezabelo ze-CIFS kwi-OpenShift. Umqhubi weFlexvolume-Ekunene kwiZiko

Ubuncinci beendlela zeendlela libukeka kanje:

flexvolume_driver mount # отвечает за присоединение тома к pod'у
# Формат возвращаемого сообщения:
{
  "status": "Success"/"Failure"/"Not supported",
  "message": "По какой причине был возвращен именно такой статус",
}

flexvolume_driver unmount # отвечает за отсоединение тома от pod'а
# Формат возвращаемого сообщения:
{
  "status": "Success"/"Failure"/"Not supported",
  "message": "По какой причине был возвращен именно такой статус",
}

flexvolume_driver init # отвечает за инициализацию плагина
# Формат возвращаемого сообщения:
{
  "status": "Success"/"Failure"/"Not supported",
  "message": "По какой причине был возвращен именно такой статус",
  // Определяет, использует ли драйвер методы attach/deatach
  "capabilities":{"attach": True/False}
}

Ukusebenzisa Iindlela attach и detach izakuchaza imeko apho ikubelet iyakwenza kwixesha elizayo xa ifowunela umqhubi. Kukho neendlela ezikhethekileyo expandvolume и expandfs, abanoxanduva lokutshintsha ubungakanani bevolumu.

Njengomzekelo utshintsho ukuba indlela yongeza expandvolume, kunye nokukwazi ukwenza ubungakanani bobungakanani kwakhona ngexesha lokwenyani, unokuziqhelanisa nayo isicelo sethu sokutsala kwiRook Ceph Operator.

Kwaye nanku umzekelo wokuphunyezwa komqhubi we-Flexvolume ekusebenzeni ne-NFS:

usage() {
    err "Invalid usage. Usage: "
    err "t$0 init"
    err "t$0 mount <mount dir> <json params>"
    err "t$0 unmount <mount dir>"
    exit 1
}

err() {
    echo -ne $* 1>&2
}

log() {
    echo -ne $* >&1
}

ismounted() {
    MOUNT=`findmnt -n ${MNTPATH} 2>/dev/null | cut -d' ' -f1`
    if [ "${MOUNT}" == "${MNTPATH}" ]; then
        echo "1"
    else
        echo "0"
    fi
}

domount() {
    MNTPATH=$1

    NFS_SERVER=$(echo $2 | jq -r '.server')
    SHARE=$(echo $2 | jq -r '.share')

    if [ $(ismounted) -eq 1 ] ; then
        log '{"status": "Success"}'
        exit 0
    fi

    mkdir -p ${MNTPATH} &> /dev/null

    mount -t nfs ${NFS_SERVER}:/${SHARE} ${MNTPATH} &> /dev/null
    if [ $? -ne 0 ]; then
        err "{ "status": "Failure", "message": "Failed to mount ${NFS_SERVER}:${SHARE} at ${MNTPATH}"}"
        exit 1
    fi
    log '{"status": "Success"}'
    exit 0
}

unmount() {
    MNTPATH=$1
    if [ $(ismounted) -eq 0 ] ; then
        log '{"status": "Success"}'
        exit 0
    fi

    umount ${MNTPATH} &> /dev/null
    if [ $? -ne 0 ]; then
        err "{ "status": "Failed", "message": "Failed to unmount volume at ${MNTPATH}"}"
        exit 1
    fi

    log '{"status": "Success"}'
    exit 0
}

op=$1

if [ "$op" = "init" ]; then
    log '{"status": "Success", "capabilities": {"attach": false}}'
    exit 0
fi

if [ $# -lt 2 ]; then
    usage
fi

shift

case "$op" in
    mount)
        domount $*
        ;;
    unmount)
        unmount $*
        ;;
    *)
        log '{"status": "Not supported"}'
        exit 0
esac

exit 1

Ke, emva kokulungiselela eyona fayile ephunyezwayo, kufuneka wenze njalo layisha umqhubi kwiqela leKubernetes. Umqhubi kufuneka abekwe kwindawo yeqela ngalinye ngokwendlela emiselwe kwangaphambili. Ngokungagqibekanga ikhethiwe:

/usr/libexec/kubernetes/kubelet-plugins/volume/exec/имя_поставщика_хранилища~имя_драйвера/

... kodwa xa usebenzisa unikezelo lweKubernetes olwahlukileyo (OpenShift, Rancher...) indlela inokwahluka.

Iingxaki zeFlexvolume: indlela yokuphosa intonga yokuloba ngokuchanekileyo?

Ukulayisha umqhubi we-Flexvolume kwii-cluster nodes kuye kwaba ngumsebenzi ongeyonto encinci. Emva kokuba wenze umsebenzi ngesandla kanye, kulula ukudibana nemeko apho iinqununu ezintsha zivela kwiqela: ngenxa yokongezwa kwendawo entsha, ukulinganisa okuzenzekelayo okuthe tye, okanye-okubi nakakhulu - ukutshintshwa kwe-node ngenxa yokungasebenzi kakuhle. Kule meko, umsebenzi kunye nokugcinwa kwezi nodi kufuneka kwenziwe akunakwenzeka, ude wongeze ngesandla umqhubi weFlexvolume kubo.

Isisombululo kule ngxaki sasisinye se-Kubernetes primitives - DaemonSet. Xa i-node entsha ibonakala kwiqela, iqulethe ngokuzenzekelayo i-pod evela kwi-DaemonSet yethu, apho umthamo wendawo uncanyathiselwe endleleni yokufumana abaqhubi be-Flexvolume. Ekudalweni okuyimpumelelo, i-pod ikopisha iifayile eziyimfuneko ukuze umqhubi asebenze kwidiski.

Nanku umzekelo weDaemonSet yokubeka iplagi yeFlexvolume:

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: flex-set
spec:
  template:
    metadata:
      name: flex-deploy
      labels:
        app: flex-deploy
    spec:
      containers:
        - image: <deployment_image>
          name: flex-deploy
          securityContext:
              privileged: true
          volumeMounts:
            - mountPath: /flexmnt
              name: flexvolume-mount
      volumes:
        - name: flexvolume-mount
          hostPath:
            path: <host_driver_directory>

... kunye nomzekelo weskripthi seBash sokubeka umqhubi weFlexvolume:

#!/bin/sh

set -o errexit
set -o pipefail

VENDOR=k8s.io
DRIVER=nfs

driver_dir=$VENDOR${VENDOR:+"~"}${DRIVER}
if [ ! -d "/flexmnt/$driver_dir" ]; then
  mkdir "/flexmnt/$driver_dir"
fi

cp "/$DRIVER" "/flexmnt/$driver_dir/.$DRIVER"
mv -f "/flexmnt/$driver_dir/.$DRIVER" "/flexmnt/$driver_dir/$DRIVER"

while : ; do
  sleep 3600
done

Kubalulekile ukuba ungalibali ukuba umsebenzi wokukhuphela ayiyo athom. Kukho ithuba elikhulu lokuba i-kubelet iqale ukusebenzisa umqhubi ngaphambi kokuba inkqubo yonikezelo igqitywe, ibangele ukuba inkqubo iphazamiseke. Indlela echanekileyo kukukhuphela kuqala iifayile zomqhubi phantsi kwegama elahlukileyo, kwaye emva koko usebenzise i-atomic rename operation.

Umzobo wokusebenza noCeph kumqhubi weRook: umqhubi weFlexvolume kumzobo ungaphakathi kwiarhente yeRook.

Ingxaki elandelayo xa usebenzisa abaqhubi beFlexvolume kukuba uninzi lokugcinwa kwindawo yeqela isoftware efunekayo kule nto kufuneka ifakwe (umzekelo, iphakheji ye-ceph-common ye-Ceph). Ekuqaleni, i-plugin ye-Flexvolume ayizange yenzelwe ukuphumeza iinkqubo ezinzima.

Isisombululo sokuqala kule ngxaki sinokubonwa ekuphunyezweni komqhubi weFlexvolume weRook operator:

Umqhubi ngokwawo uyilwe njengomthengi we-RPC. I-IPC socket yonxibelelwano ibekwe kulawulo olufanayo nomqhubi ngokwakhe. Siyakhumbula ukuba ukukopa iifayile zomqhubi kuya kuba kuhle ukusebenzisa i-DaemonSet, edibanisa ulawulo kunye nomqhubi njengomthamo. Emva kokukopisha iifayile eziyimfuneko zomqhubi we-rook, le pod ayifi, kodwa idibanisa kwi-IPC socket ngokusebenzisa umthamo oqhotyoshelweyo njengomncedisi we-RPC opheleleyo. Iphakheji ye-ceph-eqhelekileyo sele ifakwe ngaphakathi kwisitya se-pod. I-IPC socket iqinisekisa ukuba i-kubelet iyakunxibelelana ngqo ne-pod ebekwe kwindawo enye. Yonke into enobuchule ilula! ..

Sala kakuhle, uthando lwethu... iiplagi ezingaphakathi emthini!

Abaphuhlisi be-Kubernetes bafumanisa ukuba inani leeplagi zokugcinwa ngaphakathi kwengundoqo ngamashumi amabini. Kwaye utshintsho kuzo zonke, ngendlela enye okanye enye, luhamba ngomjikelo opheleleyo wokukhululwa we-Kubernetes.

Kuyavela ukuba ukusebenzisa inguqulelo entsha yeplagin yokugcina, kufuneka uhlaziye lonke iqela. Ukongeza koku, unokumangaliswa kukuba inguqu entsha ye-Kubernetes iya kuba ngokukhawuleza ingahambelani ne-Linux kernel oyisebenzisayo ... Ngoko usula iinyembezi zakho kwaye, ubambe amazinyo akho, ulungelelanise kunye nolawulo lwakho kunye nabasebenzisi ixesha hlaziya i-Linux kernel kunye neqela leKubernetes. Ngexesha elinokubakho ekuboneleleni ngeenkonzo.

Imeko ingaphezulu kokuhlekisa, awucingi? Kwacaca kuluntu luphela ukuba le ndlela ayisebenzi. Ngesigqibo sabom, abaphuhlisi be-Kubernetes babhengeza ukuba iiplagi ezintsha zokusebenza kunye nokugcinwa aziyi kuphinda zamkelwe kwi-kernel. Ukongezelela, njengoko sele sisazi, inani leentsilelo liye lachongwa ekuphunyezweni kwe-plugin yeFlexvolume...

Iplagi eyongeziweyo yamva nje yemithamo eKubernetes, CSI, ibizelwe ukuba ivale umba ngokuzingisa kokugcinwa kwedatha kube kanye. Uguqulelo lwayo lwe-alpha, olubizwa ngokupheleleyo njenge-Out-of-Tree-CSI Volume Plugins, yabhengezwa kukhupho. Iibubernetes 1.9.

IsiBonelelo sokuGcina isikhongozeli, okanye i-CSI 3000 intonga yokusonta!

Okokuqala, ndingathanda ukuqaphela ukuba i-CSI ayisiyiyo nje iplagi yevolumu, kodwa iyinyani umgangatho ekudaleni amacandelo angokwezifiso zokusebenza kunye neendawo zokugcina idatha. Iinkqubo ze-orchestration ye-Container ezifana ne-Kubernetes kunye ne-Mesos bekufanele "ifunde" indlela yokusebenza kunye namacandelo aphunyezwe ngokwalo mgangatho. Kwaye ngoku sele ndiyifundile iKubernetes.

Yintoni isakhiwo seplagin yeCSI eKubernetes? Iplagi yeCSI isebenza nabaqhubi abakhethekileyo (Abaqhubi beCSI) ibhalwe ngabaphuhlisi beqela lesithathu. Umqhubi we-CSI e-Kubernetes kufuneka abe nezinto ezimbini (iipod):

• Controller - ilawula ukugcinwa kwangaphandle okuqhubekayo. Iphunyezwa njengeseva ye-gRPC, apho i-primitive isetyenziselwa khona StatefulSet.
• INode - inoxanduva lokunyusela ugcino oluzingileyo kwiindawo zeqela. Ikwaphunyezwa njengeseva ye-gRPC, kodwa isebenzisa i-primitive DaemonSet.

Isebenza njani iplagin yeCSI kwiKubernetes

Unokufunda malunga nezinye iinkcukacha zomsebenzi weCSI, umzekelo, kwinqaku elithi "Ukuqonda iC.S.I.», inguqulelo yayo sapapasha kunyaka ophelileyo.

Iinzuzo zokuphunyezwa okunjalo

• Kwizinto ezisisiseko ezifana nokubhalisa umqhubi we-node, abaphuhlisi be-Kubernetes baphumeze isethi yezikhongozeli. Akusekho mfuneko yokuba uvelise impendulo ye-JSON ngezakhono ngokwakho, njengoko kwenziwa kwiplagi yeFlexvolume.
• Endaweni "yokutyibilika" iifayile ezisebenzisekayo kwiindawo, ngoku silayisha iipods kwiqela. Yile nto ebesiyilindele ekuqaleni kwi-Kubernetes: zonke iinkqubo zenzeka ngaphakathi kwezikhongozeli ezifakwe kusetyenziswa i-Kubernetes primitives.
• Akusekho mfuneko yokuba uphuhlise iseva ye-RPC kunye nomxhasi we-RPC ukuphumeza abaqhubi abanzima. Umxhasi waphunyelelwa kuthi ngabaphuhlisi beKubernetes.
• Ukudlulisa iingxoxo ukusebenza ngaphezulu kweprotocol ye-gRPC kulula kakhulu, kuyabhetyebhetye kwaye kuthembekile kunokudlula kwiingxoxo zomgca womyalelo. Ukuqonda indlela yokongeza inkxaso yokusetyenziswa kweemetrics zevolumu kwi-CSI ngokongeza indlela esemgangathweni ye-gRPC, unokufunda: isicelo sethu sokutsala kuba vsphere-csi umqhubi.
• Unxibelelwano lwenzeka ngeesokethi ze-IPC, ukuze ungabhideki nokuba i-kubelet ithumele isicelo kwi-pod echanekileyo.

Ngaba olu luhlu lukukhumbuza nantoni na? Iinzuzo zeCSI zi ukusombulula kwa ezo ngxaki, ezingazange zithathelwe ingqalelo xa kuphuhliswa iplagi yeFlexvolume.

ezifunyanisiweyo

I-CSI njengomgangatho wokuphumeza iiplagi zesiqhelo zokusebenzisana neendawo zokugcina idatha yamkelwa ngokufudumeleyo luluntu. Ngaphezu koko, ngenxa yeenzuzo zabo kunye nokuguquguquka, abaqhubi be-CSI benzelwe iinkqubo zokugcina ezifana ne-Ceph okanye i-AWS EBS, iiplagi zokusebenzisana nazo zongezwa kwinguqulelo yokuqala ye-Kubernetes.

Ekuqaleni kuka-2019, iiplagi ezingaphakathi emthini zichazwe njengeziphelelwe lixesha. Siceba ukuqhubeka nokuxhasa iplagi yeFlexvolume, kodwa ayiyi kuphuhlisa umsebenzi omtsha kuyo.

Thina ngokwethu sele sinamava sisebenzisa i-ceph-csi, vsphere-csi kwaye sikulungele ukongeza kolu luhlu! Ukuza kuthi ga ngoku, i-CSI ijongene nemisebenzi eyabelwe yona nge-bang, kodwa siya kulinda kwaye sibone.

Ungalibali ukuba yonke into entsha kukucinga ngokutsha kwakudala!

PS

Funda nakwibhlog yethu:

• «Siyasiqonda isiNxibelelwano sokuGcina isikhongozeli (kwiKubernetes hayi kuphela)";
• «Ukuba Rook okanye hayi Rook - ngumbuzo lowo";
• «Ukwazisa inguqulelo yealpha yevolumu yesnapshots kwi-Kubernetes».

umthombo: www.habr.com