Mapulagini amtundu wa Kubernetes yosungirako: kuchokera ku Flexvolume kupita ku CSI

Kalelo Kubernetes akadali v1.0.0, panali mapulagini a voliyumu. Amafunikira kuti alumikizane ndi Kubernetes kuti asungire deta yokhazikika (yokhazikika) yachidebe. Chiwerengero chawo chinali chaching'ono, ndipo mwa oyamba anali osungira zinthu monga GCE PD, Ceph, AWS EBS ndi ena.

Mapulagini adaperekedwa limodzi ndi Kubernetes, ndichifukwa chake adapeza dzina lawo - mumtengo. Komabe, kwa ambiri, seti yomwe ilipo ya mapulagini oterowo idakhala yosakwanira. Amisiri anawonjezera mapulagini osavuta pachimake cha Kubernetes pogwiritsa ntchito zigamba, pambuyo pake adasonkhanitsa Kubernetes yawo ndikuyiyika pa maseva awo. Koma patapita nthawi, opanga Kubernetes adazindikira izi nsomba vuto silingatheke. Anthu amafunikira ndodo ya usodzi. Ndipo pakutulutsidwa kwa Kubernetes v1.2.0 zidawoneka ...

Flexvolume plugin: ndodo yaying'ono yosodza

Madivelopa a Kubernetes adapanga pulogalamu yowonjezera ya FlexVolume, yomwe inali njira yolongosoka yosinthika ndi njira zogwirira ntchito ndi madalaivala a Flexvolume omwe amakhazikitsidwa ndi opanga gulu lachitatu.

Tiyeni tiyime ndikuyang'anitsitsa zomwe dalaivala wa FlexVolume ali. Izi ndi zoona executable file (fayilo ya binary, Python script, Bash script, ndi zina zotero), zomwe, zikachitidwa, zimatengera mfundo za mzere wa malamulo monga zolowetsa ndikubweza uthenga wokhala ndi magawo odziwika kale mumtundu wa JSON. Mwachigwirizano, kutsutsana kwa mzere woyamba nthawi zonse kumakhala njira, ndipo zotsalira zotsalira ndizo magawo ake.

Chithunzi cholumikizira cha Magawo a CIFS mu OpenShift. Flexvolume Driver - Pakatikati

Njira zochepa zikuwoneka ngati izi:

flexvolume_driver mount # отвечает за присоединение тома к pod'у
# Формат возвращаемого сообщения:
{
  "status": "Success"/"Failure"/"Not supported",
  "message": "По какой причине был возвращен именно такой статус",
}

flexvolume_driver unmount # отвечает за отсоединение тома от pod'а
# Формат возвращаемого сообщения:
{
  "status": "Success"/"Failure"/"Not supported",
  "message": "По какой причине был возвращен именно такой статус",
}

flexvolume_driver init # отвечает за инициализацию плагина
# Формат возвращаемого сообщения:
{
  "status": "Success"/"Failure"/"Not supported",
  "message": "По какой причине был возвращен именно такой статус",
  // Определяет, использует ли драйвер методы attach/deatach
  "capabilities":{"attach": True/False}
}

Kugwiritsa Ntchito Njira attach и detach idzafotokozera momwe kubelet idzachitira mtsogolo poyitana dalaivala. Palinso njira zapadera expandvolume и expandfs, omwe ali ndi udindo wosintha kukula kwa voliyumu.

Monga chitsanzo cha zosintha zomwe njira ikuwonjezera expandvolume, ndipo ndi kuthekera kosinthira ma voliyumu munthawi yeniyeni, mutha kuzidziwa bwino pempho lathu kukoka mu Rook Ceph Operator.

Ndipo nachi chitsanzo cha kukhazikitsa dalaivala Flexvolume ntchito ndi NFS:

usage() {
    err "Invalid usage. Usage: "
    err "t$0 init"
    err "t$0 mount <mount dir> <json params>"
    err "t$0 unmount <mount dir>"
    exit 1
}

err() {
    echo -ne $* 1>&2
}

log() {
    echo -ne $* >&1
}

ismounted() {
    MOUNT=`findmnt -n ${MNTPATH} 2>/dev/null | cut -d' ' -f1`
    if [ "${MOUNT}" == "${MNTPATH}" ]; then
        echo "1"
    else
        echo "0"
    fi
}

domount() {
    MNTPATH=$1

    NFS_SERVER=$(echo $2 | jq -r '.server')
    SHARE=$(echo $2 | jq -r '.share')

    if [ $(ismounted) -eq 1 ] ; then
        log '{"status": "Success"}'
        exit 0
    fi

    mkdir -p ${MNTPATH} &> /dev/null

    mount -t nfs ${NFS_SERVER}:/${SHARE} ${MNTPATH} &> /dev/null
    if [ $? -ne 0 ]; then
        err "{ "status": "Failure", "message": "Failed to mount ${NFS_SERVER}:${SHARE} at ${MNTPATH}"}"
        exit 1
    fi
    log '{"status": "Success"}'
    exit 0
}

unmount() {
    MNTPATH=$1
    if [ $(ismounted) -eq 0 ] ; then
        log '{"status": "Success"}'
        exit 0
    fi

    umount ${MNTPATH} &> /dev/null
    if [ $? -ne 0 ]; then
        err "{ "status": "Failed", "message": "Failed to unmount volume at ${MNTPATH}"}"
        exit 1
    fi

    log '{"status": "Success"}'
    exit 0
}

op=$1

if [ "$op" = "init" ]; then
    log '{"status": "Success", "capabilities": {"attach": false}}'
    exit 0
fi

if [ $# -lt 2 ]; then
    usage
fi

shift

case "$op" in
    mount)
        domount $*
        ;;
    unmount)
        unmount $*
        ;;
    *)
        log '{"status": "Not supported"}'
        exit 0
esac

exit 1

Chifukwa chake, mutatha kukonza fayilo yomwe ingathe kuchitika, muyenera kutero kwezani dalaivala ku gulu la Kubernetes. Dalaivala ayenera kukhala pamagulu onse amagulu malinga ndi njira yokonzedweratu. Mwachisawawa adasankhidwa:

/usr/libexec/kubernetes/kubelet-plugins/volume/exec/имя_поставщика_хранилища~имя_драйвера/

... koma mukamagwiritsa ntchito magawo osiyanasiyana a Kubernetes (OpenShift, Rancher ...) njirayo ikhoza kukhala yosiyana.

Mavuto a Flexvolume: momwe mungapangire ndodo yosodza moyenera?

Kuyika dalaivala wa Flexvolume kumagulu am'magulu kunakhala ntchito yosachepera. Mukachita ntchitoyi pamanja kamodzi, ndizosavuta kukumana ndi malo pomwe ma node atsopano amawonekera m'gululi: chifukwa chowonjezera node yatsopano, kuwongolera kozungulira, kapena - choyipa kwambiri - m'malo mwa mfundo chifukwa chakusokonekera. Pankhaniyi, ntchito ndi kusungira pa mfundo izi ziyenera kuchitidwa zosatheka, mpaka mutawonjezera pamanja dalaivala wa Flexvolume kwa iwo.

Yankho la vutoli linali limodzi mwazinthu zoyambirira za Kubernetes - DaemonSet. Node yatsopano ikawoneka mgululi, imakhala ndi pod kuchokera ku DaemonSet yathu, pomwe voliyumu yakumaloko imamangidwira panjira kuti mupeze madalaivala a Flexvolume. Pakupanga bwino, pod imakopera mafayilo ofunikira kuti dalaivala agwire ntchito ku diski.

Nachi chitsanzo cha DaemonSet yoyika pulogalamu yowonjezera ya Flexvolume:

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: flex-set
spec:
  template:
    metadata:
      name: flex-deploy
      labels:
        app: flex-deploy
    spec:
      containers:
        - image: <deployment_image>
          name: flex-deploy
          securityContext:
              privileged: true
          volumeMounts:
            - mountPath: /flexmnt
              name: flexvolume-mount
      volumes:
        - name: flexvolume-mount
          hostPath:
            path: <host_driver_directory>

... ndi chitsanzo cha Bash script pakuyika dalaivala wa Flexvolume:

#!/bin/sh

set -o errexit
set -o pipefail

VENDOR=k8s.io
DRIVER=nfs

driver_dir=$VENDOR${VENDOR:+"~"}${DRIVER}
if [ ! -d "/flexmnt/$driver_dir" ]; then
  mkdir "/flexmnt/$driver_dir"
fi

cp "/$DRIVER" "/flexmnt/$driver_dir/.$DRIVER"
mv -f "/flexmnt/$driver_dir/.$DRIVER" "/flexmnt/$driver_dir/$DRIVER"

while : ; do
  sleep 3600
done

M'pofunika kuti musaiwale kuti kukopera ntchito si atomiki. Pali mwayi waukulu woti kubelet ayambe kugwiritsa ntchito dalaivala isanathe, zomwe zimapangitsa kuti dongosololi liwonongeke. Njira yolondola ndiyoyamba kukopera mafayilo oyendetsa pansi pa dzina lina, ndiyeno gwiritsani ntchito dzina la atomiki.

Chithunzi chogwira ntchito ndi Ceph mu Rook operator: Woyendetsa Flexvolume pazithunzi ali mkati mwa Rook agent.

Vuto lotsatira mukamagwiritsa ntchito madalaivala a Flexvolume ndikuti posungira zambiri pamagulu amagulu pulogalamu yofunikira ya izi iyenera kukhazikitsidwa (mwachitsanzo, phukusi la ceph-common la Ceph). Poyambirira, pulogalamu yowonjezera ya Flexvolume sinapangidwe kuti igwiritse ntchito machitidwe ovuta.

Yankho loyambirira la vutoli litha kuwoneka pakukhazikitsa kwa oyendetsa Flexvolume kwa oyendetsa Rook:

Dalaivala yekha adapangidwa ngati kasitomala wa RPC. Soketi ya IPC yolumikizirana ili mu bukhu lofanana ndi dalaivala yemweyo. Timakumbukira kuti kukopera mafayilo oyendetsa kungakhale bwino kugwiritsa ntchito DaemonSet, yomwe imagwirizanitsa chikwatu ndi dalaivala ngati voliyumu. Pambuyo kukopera mafayilo ofunikira oyendetsa rook, pod iyi siifa, koma imagwirizanitsa ndi socket ya IPC kupyolera mu voliyumu yolumikizidwa ngati seva ya RPC yodzaza. Phukusi la ceph-common layikidwa kale mkati mwa chidebe cha pod. Soketi ya IPC imawonetsetsa kuti kubelet ilumikizana ndi ndendende pod yomwe ili pamalo omwewo. Chilichonse chanzeru ndi chosavuta! ..

Chabwino, okondedwa athu ... mumtengo mapulagini!

Opanga Kubernetes adapeza kuti kuchuluka kwa mapulagini osungira mkati mwapakati ndi makumi awiri. Ndipo kusintha kwa aliyense wa iwo, mwanjira ina, kumadutsa Kubernetes kumasulidwa kwathunthu.

Zikutheka kuti kugwiritsa ntchito mtundu watsopano wa pulogalamu yowonjezera yosungirako, muyenera kusintha gulu lonse. Kuphatikiza pa izi, mungadabwe kuti mtundu watsopano wa Kubernetes mwadzidzidzi udzakhala wosagwirizana ndi Linux kernel yomwe mukugwiritsa ntchito ... Kotero mumapukuta misozi yanu ndipo, mukukuta mano, gwirizanitsani ndi oyang'anira anu ndi ogwiritsa ntchito nthawi sinthani Linux kernel ndi gulu la Kubernetes. Ndi nthawi yocheperapo pakuperekera ntchito.

Mkhalidwewu ndi wopitilira nthabwala, sichoncho? Zinadziwika kwa anthu onse ammudzi kuti njirayo sikugwira ntchito. Mwa kusankha mwadala, opanga Kubernetes alengeza kuti mapulagini atsopano ogwiritsira ntchito posungira sadzalandiridwanso mu kernel. Kuphatikiza apo, monga tikudziwira kale, zolakwika zingapo zidadziwika pakukhazikitsa pulogalamu yowonjezera ya Flexvolume...

Pulogalamu yowonjezera yaposachedwa yamavoliyumu ku Kubernetes, CSI, idapemphedwa kuti atseke nkhaniyi ndi kusungirako kosalekeza kwa data kamodzi kokha. Mtundu wake wa alpha, womwe umatchedwa Out-of-Tree CSI Volume Plugins, udalengezedwa pakutulutsidwa. Kubernetes 1.9.

Container Storage Interface, kapena CSI 3000 ndodo yozungulira!

Choyamba, ndikufuna kudziwa kuti CSI si pulogalamu yowonjezera, koma yeniyeni muyezo pakupanga zida zachikhalidwe zogwirira ntchito ndi malo osungiramo data. Makina oyimba ma Container monga Kubernetes ndi Mesos amayenera "kuphunzira" momwe angagwiritsire ntchito zida zomwe zakhazikitsidwa molingana ndi muyezowu. Ndipo tsopano ndaphunzira kale Kubernetes.

Kodi mawonekedwe a CSI plugin ku Kubernetes ndi chiyani? Pulogalamu yowonjezera ya CSI imagwira ntchito ndi madalaivala apadera (Madalaivala a CSI) yolembedwa ndi opanga chipani chachitatu. Dalaivala wa CSI ku Kubernetes ayenera kukhala ndi zigawo ziwiri (pods):

• Mtsogoleri - Imayang'anira zosunga zokhazikika zakunja. Imakhazikitsidwa ngati seva ya gRPC, yomwe yoyambira imagwiritsidwa ntchito StatefulSet.
• Node - ali ndi udindo wokweza kusungirako kosalekeza ku ma cluster node. Imakhazikitsidwanso ngati seva ya gRPC, koma yachikale imagwiritsidwa ntchito DaemonSet.

Momwe plugin ya CSI imagwirira ntchito ku Kubernetes

Mutha kudziwa zambiri za ntchito ya CSI, mwachitsanzo, kuchokera m'nkhani yakuti "Kumvetsetsa C.S.I.», kumasulira kwake tinasindikiza chaka chapitacho.

Ubwino wa kukhazikitsa koteroko

• Pazinthu zofunika monga kulembetsa dalaivala wa node, opanga Kubernetes adakhazikitsa zida zingapo. Simufunikanso kupanga yankho la JSON ndi kuthekera kwanu, monga zidachitikira pulogalamu yowonjezera ya Flexvolume.
• M'malo mwa "kutsetsereka" mafayilo omwe angathe kuchitidwa pa node, tsopano timayika ma pods pagulu. Izi ndi zomwe timayembekezera poyambira Kubernetes: njira zonse zimachitika mkati mwa zotengera zomwe zimagwiritsidwa ntchito ndi Kubernetes primitives.
• Simufunikanso kupanga seva ya RPC ndi kasitomala wa RPC kuti mugwiritse ntchito madalaivala ovuta. Makasitomala adakhazikitsidwa kwa ife ndi opanga Kubernetes.
• Kupereka mikangano kuti mugwire ntchito pa protocol ya gRPC ndikosavuta, kusinthika komanso kodalirika kuposa kuwadutsa pamakangano amzere. Kuti mumvetsetse momwe mungawonjezere thandizo la ma metric ogwiritsira ntchito voliyumu ku CSI powonjezera njira yokhazikika ya gRPC, mutha kuwerenga: pempho lathu kukoka kwa vsphere-csi driver.
• Kulankhulana kumachitika kudzera pama socket a IPC, kuti musasokonezeke ngati kubelet idatumiza pempholi polondola.

Kodi mndandandawu ukukukumbutsani chilichonse? Ubwino wa CSI ndi kuthetsa mavuto omwewo, zomwe sizinaganizidwe popanga pulogalamu yowonjezera ya Flexvolume.

anapezazo

CSI ngati mulingo wokhazikitsa mapulagini achikhalidwe polumikizana ndi malo osungiramo data idalandiridwa mwachikondi kwambiri ndi anthu ammudzi. Komanso, chifukwa cha ubwino wawo ndi kusinthasintha, madalaivala a CSI amapangidwira ngakhale makina osungiramo zinthu monga Ceph kapena AWS EBS, mapulagini ogwirira ntchito omwe anawonjezedwa mu mtundu woyamba wa Kubernetes.

Kumayambiriro kwa 2019, mapulagini amtengo zanenedwa kuti ndi zachikale. Tikukonzekera kupitiliza kuthandizira pulogalamu yowonjezera ya Flexvolume, koma sitipanga magwiridwe antchito atsopano.

Ife tokha tadziwa kale kugwiritsa ntchito ceph-csi, vsphere-csi ndipo takonzeka kuwonjezera pamndandandawu! Pakadali pano, CSI ikulimbana ndi ntchito zomwe idapatsidwa movutikira, koma tidikirira ndikuwona.

Musaiwale kuti chilichonse chatsopano ndikuganiziranso zakale!

PS

Werenganinso pa blog yathu:

• «Timamvetsetsa Container Storage Interface (ku Kubernetes osati kokha)";
• «Rook kapena ayi Rook, limenelo ndi funso";
• «Kuyambitsa mtundu wa alpha wazithunzithunzi za voliyumu ku Kubernetes".

Source: www.habr.com