🥇Volume plugins rau khaws cia hauv Kubernetes: los ntawm Flexvolume rau CSI

Rov qab thaum Kubernetes tseem yog v1.0.0, muaj ntim plugins. Lawv xav tau los txuas cov tshuab rau Kubernetes rau khaws cov ntaub ntawv tsis tu ncua (ntev) thawv. Lawv cov naj npawb tsawg, thiab cov thawj yog cov chaw muab khoom xws li GCE PD, Ceph, AWS EBS thiab lwm yam.

Cov plugins tau xa nrog Kubernetes, uas yog vim li cas lawv tau txais lawv lub npe - hauv tsob ntoo. Txawm li cas los xij, rau ntau tus, cov txheej txheem uas twb muaj lawm ntawm xws li plugins tau dhau los ua tsis txaus. Craftsmen ntxiv cov plugins yooj yim rau Kubernetes core siv thaj ua rau thaj, tom qab ntawd lawv sib sau lawv tus kheej Kubernetes thiab nruab rau ntawm lawv cov servers. Tab sis dhau sij hawm, Kubernetes developers pom tau hais tias ntses qhov teeb meem daws tsis tau. Cov neeg xav tau pas nuv ntses. Thiab hauv kev tso tawm Kubernetes v1.2.0 nws tshwm sim ...

Flexvolume plugin: tus pas nuv ntses tsawg heev

Cov neeg tsim khoom Kubernetes tau tsim lub FlexVolume plugin, uas yog lub hauv paus ntsiab lus ntawm kev hloov pauv thiab cov txheej txheem rau kev ua haujlwm nrog Flexvolume tsav tsheb siv los ntawm cov neeg tsim tawm thib peb.

Cia peb nres thiab ua tib zoo saib seb tus tsav tsheb FlexVolume yog dab tsi. Qhov no yog qhov tseeb executable ntaub ntawv (cov ntaub ntawv binary, Python tsab ntawv, Bash tsab ntawv, thiab lwm yam), uas, thaum ua tiav, siv cov kab lus hais kom ua raws li cov lus nkag thiab xa rov qab cov lus nrog cov lus paub ua ntej hauv JSON hom. Los ntawm lub rooj sib tham, thawj kab lus sib cav yog ib txoj hauv kev, thiab cov lus sib cav ntxiv yog nws qhov tsis muaj.

Daim duab sib txuas rau CIFS Shares hauv OpenShift. Flexvolume Tsav Tsheb - Txoj cai nyob rau hauv Center

Yam tsawg kawg ntawm cov txheej txheem zoo li no:

flexvolume_driver mount # отвечает за присоединение тома к pod'у
# Формат возвращаемого сообщения:
{
  "status": "Success"/"Failure"/"Not supported",
  "message": "По какой причине был возвращен именно такой статус",
}

flexvolume_driver unmount # отвечает за отсоединение тома от pod'а
# Формат возвращаемого сообщения:
{
  "status": "Success"/"Failure"/"Not supported",
  "message": "По какой причине был возвращен именно такой статус",
}

flexvolume_driver init # отвечает за инициализацию плагина
# Формат возвращаемого сообщения:
{
  "status": "Success"/"Failure"/"Not supported",
  "message": "По какой причине был возвращен именно такой статус",
  // Определяет, использует ли драйвер методы attach/deatach
  "capabilities":{"attach": True/False}
}

Siv Txoj Kev attach и detach yuav txhais qhov xwm txheej uas kubelet yuav ua rau yav tom ntej thaum hu rau tus tsav tsheb. Kuj tseem muaj txoj hauv kev tshwj xeeb expandvolume и expandfs, uas yog lub luag hauj lwm rau dynamically resizing lub ntim.

Raws li ib qho piv txwv ntawm cov kev hloov pauv uas cov txheej txheem ntxiv expandvolume, thiab nrog nws lub peev xwm los hloov qhov ntim hauv lub sijhawm, koj tuaj yeem paub koj tus kheej nrog peb rub thov hauv Rook Ceph Operator.

Thiab ntawm no yog ib qho piv txwv ntawm kev siv ntawm Flexvolume tsav tsheb rau kev ua haujlwm nrog NFS:

usage() {
    err "Invalid usage. Usage: "
    err "t$0 init"
    err "t$0 mount <mount dir> <json params>"
    err "t$0 unmount <mount dir>"
    exit 1
}

err() {
    echo -ne $* 1>&2
}

log() {
    echo -ne $* >&1
}

ismounted() {
    MOUNT=`findmnt -n ${MNTPATH} 2>/dev/null | cut -d' ' -f1`
    if [ "${MOUNT}" == "${MNTPATH}" ]; then
        echo "1"
    else
        echo "0"
    fi
}

domount() {
    MNTPATH=$1

    NFS_SERVER=$(echo $2 | jq -r '.server')
    SHARE=$(echo $2 | jq -r '.share')

    if [ $(ismounted) -eq 1 ] ; then
        log '{"status": "Success"}'
        exit 0
    fi

    mkdir -p ${MNTPATH} &> /dev/null

    mount -t nfs ${NFS_SERVER}:/${SHARE} ${MNTPATH} &> /dev/null
    if [ $? -ne 0 ]; then
        err "{ "status": "Failure", "message": "Failed to mount ${NFS_SERVER}:${SHARE} at ${MNTPATH}"}"
        exit 1
    fi
    log '{"status": "Success"}'
    exit 0
}

unmount() {
    MNTPATH=$1
    if [ $(ismounted) -eq 0 ] ; then
        log '{"status": "Success"}'
        exit 0
    fi

    umount ${MNTPATH} &> /dev/null
    if [ $? -ne 0 ]; then
        err "{ "status": "Failed", "message": "Failed to unmount volume at ${MNTPATH}"}"
        exit 1
    fi

    log '{"status": "Success"}'
    exit 0
}

op=$1

if [ "$op" = "init" ]; then
    log '{"status": "Success", "capabilities": {"attach": false}}'
    exit 0
fi

if [ $# -lt 2 ]; then
    usage
fi

shift

case "$op" in
    mount)
        domount $*
        ;;
    unmount)
        unmount $*
        ;;
    *)
        log '{"status": "Not supported"}'
        exit 0
esac

exit 1

Yog li, tom qab npaj cov ntaub ntawv executable tiag tiag, koj yuav tsum tau upload tus tsav tsheb mus rau Kubernetes pawg. Tus tsav tsheb yuav tsum nyob ntawm txhua pawg ntawm pawg raws li txoj hauv kev tau teem tseg. Los ntawm lub neej ntawd nws tau xaiv:

/usr/libexec/kubernetes/kubelet-plugins/volume/exec/имя_поставщика_хранилища~имя_драйвера/

... tab sis thaum siv Kubernetes sib txawv (OpenShift, Rancher...) txoj kev yuav txawv.

Flexvolume teeb meem: yuav ua li cas pov tus pas nuv ntses kom raug?

Uploading tus tsav tsheb Flexvolume rau pawg nodes tau dhau los ua txoj haujlwm tsis tseem ceeb. Tom qab ua haujlwm manually ib zaug, nws yog ib qho yooj yim mus ntsib ib qho teeb meem uas cov nodes tshiab tshwm nyob rau hauv pawg: vim yog ntxiv ntawm ib tug tshiab node, tsis siv neeg kab rov tav scaling, los yog - dab tsi yog phem - hloov lub node vim ib tug malfunction. Hauv qhov no, ua haujlwm nrog kev khaws cia ntawm cov nodes yuav tsum tau ua yog tsis yooj yim sua, kom txog thaum koj tseem manually ntxiv tus tsav tsheb Flexvolume rau lawv.

Txoj kev daws teeb meem no yog ib qho ntawm Kubernetes primitives - DaemonSet. Thaum ib lub pob tshiab tshwm nyob rau hauv pawg, nws cia li muaj lub plhaub taum pauv los ntawm peb DaemonSet, uas lub ntim hauv zos txuas nrog rau txoj kev mus nrhiav Flexvolume tsav tsheb. Thaum ua tiav kev tsim, lub plhaub taum pauv cov ntaub ntawv tsim nyog rau tus tsav tsheb ua haujlwm rau disk.

Nov yog ib qho piv txwv ntawm xws li DaemonSet rau nteg tawm Flexvolume plugin:

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: flex-set
spec:
  template:
    metadata:
      name: flex-deploy
      labels:
        app: flex-deploy
    spec:
      containers:
        - image: <deployment_image>
          name: flex-deploy
          securityContext:
              privileged: true
          volumeMounts:
            - mountPath: /flexmnt
              name: flexvolume-mount
      volumes:
        - name: flexvolume-mount
          hostPath:
            path: <host_driver_directory>

... thiab ib qho piv txwv ntawm Bash tsab ntawv rau nteg tawm tus tsav tsheb Flexvolume:

#!/bin/sh

set -o errexit
set -o pipefail

VENDOR=k8s.io
DRIVER=nfs

driver_dir=$VENDOR${VENDOR:+"~"}${DRIVER}
if [ ! -d "/flexmnt/$driver_dir" ]; then
  mkdir "/flexmnt/$driver_dir"
fi

cp "/$DRIVER" "/flexmnt/$driver_dir/.$DRIVER"
mv -f "/flexmnt/$driver_dir/.$DRIVER" "/flexmnt/$driver_dir/$DRIVER"

while : ; do
  sleep 3600
done

Nws yog ib qho tseem ceeb kom tsis txhob hnov qab tias daim ntawv ua haujlwm tsis yog atomic. Muaj lub caij nyoog siab uas kubelet yuav pib siv tus neeg tsav tsheb ua ntej nws cov txheej txheem npaj tiav, ua rau lub kaw lus poob. Txoj hauv kev kom raug yog thawj zaug luam cov ntaub ntawv tsav tsheb hauv qab lub npe sib txawv, thiab tom qab ntawd siv atomic rename lag luam.

Daim duab ua haujlwm nrog Ceph hauv Rook tus neeg teb xov tooj: tus tsav tsheb Flexvolume hauv daim duab yog nyob rau hauv tus neeg sawv cev Rook

Qhov teeb meem tom ntej no thaum siv cov tsav tsheb Flexvolume yog rau feem ntau khaws cia ntawm pawg node cov software tsim nyog rau qhov no yuav tsum tau nruab (piv txwv li, ceph-hom pob rau Ceph). Thaum xub thawj, Flexvolume plugin tsis tau tsim los siv cov txheej txheem nyuaj.

Ib qho kev daws teeb meem qub rau qhov teeb meem no tuaj yeem pom hauv Flexvolume tsav tsheb siv ntawm Rook tus neeg teb xov tooj:

Tus tsav tsheb nws tus kheej yog tsim los ua tus neeg siv khoom RPC. IPC qhov (socket) rau kev sib txuas lus yog nyob rau hauv tib lub npe raws li tus neeg tsav tsheb nws tus kheej. Peb nco ntsoov tias kom luam cov ntaub ntawv tsav tsheb nws yuav zoo rau siv DaemonSet, uas txuas cov npe nrog tus tsav tsheb ua lub ntim. Tom qab luam cov ntaub ntawv tsim nyog rook tsav tsheb, lub plhaub taum no tsis tuag, tab sis txuas mus rau IPC lub qhov (socket) los ntawm cov ntim txuas raws li cov neeg ua haujlwm puv RPC. Lub pob ceph-hom yog twb tau nruab rau hauv lub thawv ntim. IPC lub qhov (socket) xyuas kom meej tias cov kubelet yuav sib txuas lus nrog cov pods uas nyob ntawm tib lub node. Txhua yam ingenious yog yooj yim!..

Nyob zoo, peb tus hlub ... hauv-ntoo plugins!

Cov neeg tsim khoom Kubernetes tau tshawb pom tias tus naj npawb ntawm cov plugins rau khaws cia hauv cov tub ntxhais yog nees nkaum. Thiab ib qho kev hloov pauv ntawm txhua tus ntawm lawv, ib txoj hauv kev lossis lwm qhov, mus dhau ntawm Kubernetes tag nrho tso tawm lub voj voog.

Nws hloov tawm hais tias siv tus tshiab version ntawm lub cia plugin, koj yuav tsum tau hloov kho tag nrho pawg. Ntxiv rau qhov no, tej zaum koj yuav xav tsis thoob tias qhov tshiab ntawm Kubernetes yuav dheev dhau los ua tsis sib xws nrog Linux ntsiav koj siv ... Yog li koj so koj lub kua muag thiab, gritting koj cov hniav, koom tes nrog koj cov kev tswj hwm thiab cov neeg siv lub sijhawm. hloov kho Linux kernel thiab Kubernetes pawg. Nrog ua tau downtime hauv kev muab kev pabcuam.

Qhov xwm txheej yog ntau tshaj li comical, koj puas xav? Nws tau pom meej rau tag nrho cov zej zog tias txoj hauv kev tsis ua haujlwm. Los ntawm kev txiav txim siab, Kubernetes cov neeg tsim tawm tshaj tawm tias cov plugins tshiab rau kev ua haujlwm nrog kev cia yuav tsis raug lees txais rau hauv cov ntsiav ntxiv lawm. Tsis tas li ntawd, raws li peb twb paub lawm, ib tug xov tooj ntawm shortcomings tau pom nyob rau hauv qhov kev siv ntawm Flexvolume plugin ...

Qhov tseeb ntxiv plugin rau ntim hauv Kubernetes, CSI, tau raug hu kom kaw qhov teeb meem nrog cov ntaub ntawv tsis tu ncua ib zaug thiab rau tag nrho. Nws cov alpha version, tag nrho hu ua Out-of-Tree CSI Volume Plugins, tau tshaj tawm hauv kev tso tawm Kub Qav Kaws 1.9.

Container Storage Interface, los yog CSI 3000 spinning qws!

Ua ntej ntawm tag nrho cov, kuv xav kom nco ntsoov tias CSI tsis yog ib tug ntim plugin, tab sis ib tug tiag tiag txuj ntawm kev tsim cov khoom siv rau kev ua haujlwm nrog cov ntaub ntawv warehouses. Thawv orchestration systems xws li Kubernetes thiab Mesos yuav tsum "kawm" yuav ua li cas ua haujlwm nrog cov khoom siv raws li tus qauv no. Thiab tam sim no kuv twb kawm Kubernetes lawm.

Tus qauv ntawm CSI plugin hauv Kubernetes yog dab tsi? CSI plugin ua haujlwm nrog cov tsav tsheb tshwj xeeb (CSI drivers) sau los ntawm lwm tus neeg tsim tawm. Tus neeg tsav tsheb CSI hauv Kubernetes yuav tsum muaj tsawg kawg yog ob qho tib si (pods):

xws - tswj kev khaws cia sab nraud. Nws yog siv los ua tus neeg rau zaub mov gRPC, uas yog siv thaum pib StatefulSet.
Ntawm - yog lub luag haujlwm rau mounting tsis tu ncua cia rau pawg nodes. Nws kuj tau siv los ua gRPC server, tab sis nws siv qhov qub DaemonSet.

CSI plugin ua haujlwm li cas hauv Kubernetes

Koj tuaj yeem kawm txog qee cov ntsiab lus ntawm CSI txoj haujlwm, piv txwv li, los ntawm kab lus "Nkag siab txog C.S.I.», kev txhais lus uas peb luam tawm ib xyoos dhau los.

Qhov zoo ntawm xws li kev siv

Rau tej yam yooj yim xws li sau npe tus neeg tsav tsheb rau lub node, Kubernetes developers tau siv ib txheej ntim. Koj tsis tas yuav tsim cov lus teb JSON nrog lub peev xwm ntawm koj tus kheej, raws li tau ua tiav rau Flexvolume plugin.
Es tsis txhob "slipping" cov ntaub ntawv executable mus rau ntawm nodes, peb tam sim no upload pods rau pawg. Qhov no yog qhov peb xav tau los ntawm Kubernetes: tag nrho cov txheej txheem tshwm sim hauv cov thawv xa khoom siv Kubernetes primitives.
Koj tsis tas yuav tsim RPC server thiab RPC tus neeg siv los siv cov tsav tsheb nyuaj. Cov neeg siv khoom tau siv rau peb los ntawm Kubernetes cov neeg tsim khoom.
Kev sib cav dhau los ua haujlwm dhau ntawm gRPC raws tu qauv yog yooj yim dua, yoog raws thiab txhim khu kev qha dua li hla lawv los ntawm cov lus sib cav. Txhawm rau nkag siab yuav ua li cas ntxiv kev txhawb nqa rau qhov ntsuas kev siv ntim rau CSI los ntawm kev ntxiv cov qauv gRPC, koj tuaj yeem nyeem: peb rub thov rau vsphere-csi tsav tsheb.
Kev sib txuas lus tshwm sim ntawm IPC qhov (sockets), kom tsis txhob tsis meej pem seb lub kubelet xa cov lus thov mus rau qhov tseeb pod.

Daim ntawv no puas nco koj txog dab tsi? Qhov zoo ntawm CSI yog daws cov teeb meem tib yam, uas tsis tau coj mus rau hauv tus account thaum tsim Flexvolume plugin.

tshawb pom

CSI raws li tus qauv rau kev siv cov kev cai plugins rau kev cuam tshuam nrog cov ntaub ntawv warehouses tau txais kev sov siab los ntawm cov zej zog. Ntxiv mus, vim lawv qhov zoo thiab kev ua tau zoo, CSI tsav tsheb tau tsim txawm rau cov tshuab khaws cia xws li Ceph lossis AWS EBS, plugins rau kev ua haujlwm nrog uas tau ntxiv rau hauv thawj version ntawm Kubernetes.

Thaum pib ntawm 2019, hauv-ntoo plugins tau tshaj tawm tias obsolete. Peb npaj yuav txuas ntxiv txhawb Flexvolume plugin, tab sis yuav tsis tsim cov haujlwm tshiab rau nws.

Peb tus kheej twb muaj kev paub siv ceph-csi, vsphere-csi thiab npaj tau ntxiv rau daim ntawv no! Txog tam sim no, CSI tab tom daws cov haujlwm uas tau muab rau nws nrog lub suab nrov, tab sis peb yuav tos thiab pom.

Tsis txhob hnov qab tias txhua yam tshiab yog qhov zoo rov xav txog qhov qub!

PS

Nyeem kuj ntawm peb blog:

Tau qhov twg los: www.hab.com

Volume plugins rau Kubernetes cia: los ntawm Flexvolume rau CSI