Kusamuka kwa Cassandra kupita Kubernetes: mawonekedwe ndi mayankho

Timakumana pafupipafupi ndi nkhokwe ya Apache Cassandra ndikufunika kuyigwiritsa ntchito mkati mwa Kubernetes. M'nkhaniyi, tidzagawana masomphenya athu a njira zofunika, njira ndi zothetsera zomwe zilipo (kuphatikizapo mwachidule za ogwira ntchito) posamukira Cassandra ku K8s.

“Amene angathe kulamulira mkazi akhoza kulamuliranso dziko”

Cassandra ndi ndani? Ndilo gawo losungirako logawidwa lopangidwa kuti lizitha kuyang'anira kuchuluka kwa deta ndikuwonetsetsa kupezeka kwakukulu popanda kulephera kumodzi. Pulojekitiyi sifunika kutchula nthawi yayitali, kotero ndingopereka zazikulu za Cassandra zomwe zingakhale zogwirizana ndi nkhani inayake:

• Cassandra amalembedwa ku Java.
• Cassandra topology imaphatikizapo magawo angapo:
  • Node - chitsanzo chimodzi cha Cassandra;
  • Rack ndi gulu la zochitika za Cassandra, zogwirizanitsidwa ndi khalidwe lina, lomwe lili pamalo omwewo;
  • Datacenter - mndandanda wamagulu onse a zochitika za Cassandra zomwe zili pamalo amodzi a data;
  • Cluster ndi mndandanda wama data onse.
• Cassandra amagwiritsa ntchito adilesi ya IP kuzindikira malo.
• Kuti mufulumizitse kulemba ndi kuwerenga ntchito, Cassandra amasunga zina mwazomwe zili mu RAM.

Tsopano - kusamukira ku Kubernetes.

Chongani mndandanda wa kusamutsa

Ponena za kusamuka kwa Cassandra kupita ku Kubernetes, tikukhulupirira kuti ndi kusamuka kudzakhala kosavuta kuyang'anira. Chidzafunika chiyani pa izi, chingathandize ndi chiyani?

1. Kusunga deta

Monga tafotokozera kale, Cassanda amasunga gawo la data mu RAM - mu Memtable. Koma pali gawo lina la data lomwe limasungidwa ku diski - mu mawonekedwe SSTable. Gulu lawonjezedwa ku data iyi Lembani Logi - zolemba zonse zomwe zachitika, zomwe zimasungidwanso ku disk.

Lembani chithunzi cha zochitika mu Cassandra

Ku Kubernetes, titha kugwiritsa ntchito PersistentVolume kusunga deta. Chifukwa cha njira zotsimikiziridwa, kugwira ntchito ndi deta ku Kubernetes kumakhala kosavuta chaka chilichonse.

Tipereka PersistentVolume yathu ku Cassandra pod iliyonse

Ndikofunika kuzindikira kuti Cassandra mwiniyo akutanthauza kubwereza deta, kupereka njira zopangira izi. Choncho, ngati mukumanga gulu la Cassandra kuchokera kumagulu ambiri, ndiye kuti palibe chifukwa chogwiritsira ntchito machitidwe ogawidwa monga Ceph kapena GlusterFS posungira deta. Pachifukwa ichi, zingakhale zomveka kusunga deta pa disk host pogwiritsa ntchito ma disks osalekeza amderalo kapena kukwera hostPath.

Funso lina ndilakuti ngati mukufuna kupanga malo osiyana a omanga panthambi iliyonse. Pankhaniyi, njira yolondola ingakhale kukweza node imodzi ya Cassandra ndikusunga deta muzosungirako zogawidwa, i.e. Ceph ndi GlusterFS zomwe zatchulidwazi zidzakhala zosankha zanu. Ndiye woyambitsayo adzakhala wotsimikiza kuti sadzataya deta yoyesera ngakhale imodzi mwa mfundo zamagulu a Kuberntes itatayika.

2. Kuyang'anira

Chisankho chosatsutsika pakukhazikitsa kuwunika ku Kubernetes ndi Prometheus (tidakambirana mwatsatanetsatane mu lipoti logwirizana). Kodi Cassandra ali bwanji ndi omwe amatumiza metrics ku Prometheus? Ndipo, chofunika kwambiri ndi chiyani, ndi ma dashboard ofananira a Grafana?

Chitsanzo cha maonekedwe a ma graph ku Grafana kwa Cassandra

Pali awiri okha ogulitsa kunja: jmx_exporter и cassandra_exporter.

Tinadzisankhira tokha woyamba chifukwa:

1. JMX Exporter ikukula ndikukula, pamene Cassandra Exporter sanathe kupeza chithandizo chokwanira cha anthu. Cassandra Exporter sichirikizabe mitundu yambiri ya Cassandra.
2. Mutha kuyendetsa ngati javaagent powonjezera mbendera -javaagent:<plugin-dir-name>/cassandra-exporter.jar=--listen=:9180.
3. Pali imodzi ya iye dashboad yokwanira, zomwe sizigwirizana ndi Cassandra Exporter.

3. Kusankha Kubernetes primitives

Malinga ndi kapangidwe kapamwamba ka gulu la Cassandra, tiyeni tiyese kumasulira zonse zomwe zafotokozedwa m'mawu a Kubernetes:

• Cassandra Node → Pod
• Cassandra Rack → StatefulSet
• Cassandra Datacenter → dziwe kuchokera ku StatefulSets
• Cassandra Cluster → ???

Zinapezeka kuti bungwe lina lowonjezera likusowa kuyang'anira gulu lonse la Cassandra nthawi imodzi. Koma ngati chinachake palibe, tikhoza kulenga izo! Kubernetes ali ndi njira yofotokozera zomwe zili zake pazolinga izi - Tanthauzo la Custom Resource.

Kulengeza zowonjezera zolembera ndi zidziwitso

Koma Custom Resource palokha sizitanthauza kalikonse: pambuyo pake, zimafunikira wolamulira. Mungafunike kupempha thandizo Wothandizira Kubernetes...

4. Kuzindikiritsa makoko

M'ndime yomwe ili pamwambapa, tidagwirizana kuti nodi imodzi ya Cassandra ifanana ndi pod ku Kubernetes. Koma ma adilesi a IP a ma pod azikhala osiyana nthawi iliyonse. Ndipo chizindikiritso cha node ku Cassandra chimachokera ku adilesi ya IP ... Zikuoneka kuti pambuyo pochotsa pod, gulu la Cassandra lidzawonjezera node yatsopano.

Pali njira yotulukira, osati imodzi yokha:

1. Titha kusunga zidziwitso pogwiritsa ntchito zizindikiritso (ma UUID omwe amazindikiritsa zochitika za Cassandra) kapena ma adilesi a IP ndikusunga zonse m'matebulo. Njirayi ili ndi zoyipa ziwiri zazikulu:
   • Chiwopsezo cha mtundu wamtundu womwe umachitika ngati node ziwiri zikugwa nthawi imodzi. Pambuyo pakuwuka, ma Cassandra node adzapempha nthawi imodzi adilesi ya IP patebulo ndikupikisana nawo pazomwezi.
   • Ngati node ya Cassandra yataya deta yake, sitidzatha kuizindikira.
2. Yankho lachiwiri likuwoneka ngati kuthyolako pang'ono, komabe: titha kupanga Utumiki ndi ClusterIP pa node iliyonse ya Cassandra. Mavuto ndi kukhazikitsa uku:
   • Ngati pali ma node ambiri mugulu la Cassandra, tidzapanga ma Services ambiri.
   • Mbali ya ClusterIP imayendetsedwa kudzera pa iptables. Izi zitha kukhala vuto ngati gulu la Cassandra lili ndi ma node ambiri (1000... kapena 100?). Ngakhale kusanja kutengera IPVS akhoza kuthetsa vutoli.
3. Yankho lachitatu ndikugwiritsa ntchito netiweki yama node a Cassandra m'malo mwa netiweki yodzipatulira ya ma pod pothandizira kukhazikitsa. hostNetwork: true. Njirayi imakhala ndi malire ena:
   • Kusintha mayunitsi. Ndikofunikira kuti node yatsopanoyo ikhale ndi adilesi ya IP yofanana ndi yapitayo (m'mitambo ngati AWS, GCP izi ndizosatheka kuchita);
   • Pogwiritsa ntchito ma network a cluster node, timayamba kupikisana ndi ma network. Chifukwa chake, kuyika ma pod opitilira imodzi ndi Cassandra pagulu limodzi kumakhala kovuta.

5. Zosunga zobwezeretsera

Tikufuna kusunga deta yonse ya Cassandra node pa ndandanda. Kubernetes imapereka mawonekedwe osavuta kugwiritsa ntchito CronJob, koma pano Cassandra mwiniwake waika spokes m'magudumu athu.

Ndiroleni ndikukumbutseni kuti Cassandra amasunga zina mwazokumbukira. Kuti mupange zosunga zobwezeretsera zonse, muyenera data kuchokera pamtima (Memtables) kupita ku disk (Zithunzi za SSTables). Panthawiyi, mfundo ya Cassandra imasiya kuvomereza kugwirizanitsa, kutseka kwathunthu kuchokera kumagulu.

Pambuyo pake, zosunga zobwezeretsera zimachotsedwa (chithunzi) ndipo dongosolo limasungidwa (keyspace). Ndipo zikuwonekeratu kuti zosunga zobwezeretsera sizimatipatsa kalikonse: tiyenera kusunga zizindikiritso za data zomwe node ya Cassandra inali ndi udindo - izi ndi zizindikiro zapadera.

Kugawidwa kwa ma tokeni kuti mudziwe zomwe ma Cassandra node ali ndi udindo

Chitsanzo cholemba chosungira Cassandra kuchokera ku Google ku Kubernetes chimapezeka pa izi. Mfundo yokhayo yomwe script siyikuganizira ndikukhazikitsanso deta ku node musanatenge chithunzithunzi. Ndiko kuti, zosunga zobwezeretsera sizimachitikira momwe zilili pano, koma za boma posachedwa. Koma izi zimathandiza kuti node isagwire ntchito, zomwe zikuwoneka zomveka kwambiri.

set -eu

if [[ -z "$1" ]]; then
  info "Please provide a keyspace"
  exit 1
fi

KEYSPACE="$1"

result=$(nodetool snapshot "${KEYSPACE}")

if [[ $? -ne 0 ]]; then
  echo "Error while making snapshot"
  exit 1
fi

timestamp=$(echo "$result" | awk '/Snapshot directory: / { print $3 }')

mkdir -p /tmp/backup

for path in $(find "/var/lib/cassandra/data/${KEYSPACE}" -name $timestamp); do
  table=$(echo "${path}" | awk -F "[/-]" '{print $7}')
  mkdir /tmp/backup/$table
  mv $path /tmp/backup/$table
done


tar -zcf /tmp/backup.tar.gz -C /tmp/backup .

nodetool clearsnapshot "${KEYSPACE}"

Chitsanzo cha bash script potengera zosunga zobwezeretsera kuchokera ku node imodzi ya Cassandra

Mayankho okonzeka a Cassandra ku Kubernetes

Ndi chiyani chomwe chimagwiritsidwa ntchito poyika Cassandra ku Kubernetes ndipo ndi iti mwa izi yomwe ikugwirizana bwino ndi zomwe zapatsidwa?

1. Mayankho otengera StatefulSet kapena Helm chart

Kugwiritsa ntchito zofunikira za StatefulSets kuyendetsa gulu la Cassandra ndi njira yabwino. Pogwiritsa ntchito tchati cha Helm ndi ma Go templates, mutha kupatsa wogwiritsa ntchito mawonekedwe osinthika kuti atumize Cassandra.

Izi nthawi zambiri zimagwira ntchito bwino ... mpaka chinachake chosayembekezereka chichitika, monga kulephera kwa node. Zida za Standard Kubernetes sizingaganizire zonse zomwe zafotokozedwa pamwambapa. Kuphatikiza apo, njira iyi ndi yochepa kwambiri momwe ingakulitsire ntchito zovuta kwambiri: kusintha ma node, zosunga zobwezeretsera, kuchira, kuwunika, ndi zina.

Oimira:

• Tchati chochokera kunkhokwe yayikulu ya Helm;
• Tchati kuchokera ku bitnami.

Ma chart onsewa ndi abwino, koma amakumana ndi zovuta zomwe tafotokozazi.

2. Mayankho otengera Kubernetes Operator

Zosankha zoterezi ndizosangalatsa kwambiri chifukwa zimapereka mwayi wokwanira wowongolera masango. Pakupanga woyendetsa Cassandra, monga nkhokwe ina iliyonse, mawonekedwe abwino amawoneka ngati Sidecar <-> Controller <-> CRD:

Chiwembu chowongolera ma Node mu wopangidwa bwino wa Cassandra

Tiyeni tiwone ma opareta omwe alipo.

1. Cassandra-operator kuchokera ku instaclustr

• GitHub
• Kukonzekera: Alpha
• License: Apache 2.0
• Yakhazikitsidwa mu: Java

Iyi ndi pulojekiti yodalirika komanso yokhazikika kuchokera ku kampani yomwe imapereka ntchito zoyendetsedwa ndi Cassandra. Izo, monga tafotokozera pamwambapa, zimagwiritsa ntchito chidebe cham'mbali chomwe chimavomereza malamulo kudzera pa HTTP. Zolembedwa mu Java, nthawi zina zimasowa magwiridwe antchito apamwamba kwambiri a library ya kasitomala. Komanso, wogwiritsa ntchito samathandizira ma Racks osiyanasiyana a Datacenter imodzi.

Koma wogwiritsa ntchitoyo ali ndi zabwino monga kuthandizira kuwunika, kuyang'anira magulu apamwamba pogwiritsa ntchito CRD, komanso zolemba zopangira zosunga zobwezeretsera.

2. Navigator kuchokera ku Jetstack

• GitHub
• Kukonzekera: Alpha
• License: Apache 2.0
• Kukhazikitsidwa mu: Golang

Mawu opangidwa kuti agwiritse ntchito DB-as-a-Service. Pakadali pano imathandizira nkhokwe ziwiri: Elasticsearch ndi Cassandra. Ili ndi mayankho osangalatsa monga kuwongolera kolowera pa database kudzera pa RBAC (pachifukwa ichi ili ndi navigator-apiserver yakeyake). Ntchito yosangalatsa yomwe ingakhale yoyenera kuyang'anitsitsa, koma kudzipereka komaliza kunapangidwa chaka ndi theka chapitacho, zomwe zimachepetsa bwino mphamvu zake.

3. Cassandra-woyendetsa ndi vgkowski

• GitHub
• Kukonzekera: Alpha
• License: Apache 2.0
• Kukhazikitsidwa mu: Golang

Sanaziganizire "mozama", popeza kudzipereka komaliza kumalo osungirako kunali kopitilira chaka chapitacho. Kukula kwa opareshoni kwasiyidwa: mtundu waposachedwa kwambiri wa Kubernetes womwe umathandizidwa ndi 1.9.

4. Cassandra-woyendetsa ndi Rook

• GitHub
• Kukonzekera: Alpha
• License: Apache 2.0
• Kukhazikitsidwa mu: Golang

Wogwiritsa ntchito yemwe chitukuko chake sichikuyenda mwachangu momwe tingafunire. Ili ndi dongosolo la CRD lolingaliridwa bwino la kayendetsedwe ka magulu, limathetsa vuto lozindikiritsa node pogwiritsa ntchito Utumiki ndi ClusterIP ("kuthyolako" komweko) ... koma ndizo zonse tsopano. Pakadali pano palibe kuwunika kapena zosunga zobwezeretsera m'bokosi (mwa njira, tikufuna kuyang'anira tinadzitengera tokha). Chosangalatsa ndichakuti mutha kugwiritsanso ntchito ScyllaDB pogwiritsa ntchito opareshoni.

NB: Tinagwiritsa ntchito opareshoniyi ndi zosintha zazing'ono mu imodzi mwama projekiti athu. Palibe mavuto omwe adawonedwa pakugwira ntchito kwa wogwiritsa ntchito panthawi yonseyi (~ miyezi 4 yogwira ntchito).

5. CassKop kuchokera ku Orange

• GitHub
• Kukonzekera: Alpha
• License: Apache 2.0
• Kukhazikitsidwa mu: Golang

Wogwiritsa ntchito wachichepere kwambiri pamndandanda: kudzipereka koyamba kudapangidwa pa Meyi 23, 2019. Kale tsopano ili ndi zida zake zambiri kuchokera pamndandanda wathu, zambiri zomwe zingapezeke posungira polojekiti. Wothandizira amamangidwa pamaziko a opareta-sdk otchuka. Imathandizira kuyang'anira kunja kwa bokosi. Kusiyanitsa kwakukulu kwa ogwira ntchito ena ndiko kugwiritsa ntchito Pulogalamu ya CassKop, yokhazikitsidwa mu Python ndipo imagwiritsidwa ntchito polumikizana pakati pa ma Cassandra node.

anapezazo

Chiwerengero cha njira ndi zosankha zomwe zingatheke potengera Cassandra kupita ku Kubernetes zimadzilankhula zokha: mutuwo ukufunidwa.

Pakadali pano, mutha kuyesa zilizonse zomwe zili pamwambazi mwangozi komanso pachiwopsezo chanu: palibe omwe akupanga omwe amatsimikizira 100% kuti agwiritse ntchito yankho lawo pamalo opanga. Koma kale, zinthu zambiri zimawoneka zolimbikitsa kuyesa kugwiritsa ntchito mabenchi otukuka.

Ndikuganiza kuti m'tsogolomu mkazi uyu ali m'sitimayo adzabwera bwino!

PS

Werenganinso pa blog yathu:

• «Kusamuka kosasunthika kwa MongoDB kupita ku Kubernetes";
• «Kusamuka kosasunthika kwa RabbitMQ kupita ku Kubernetes";
• «Databases ndi Kubernetes (review and video report)";
• «Malangizo & zidule za K8s: Kufulumizitsa bootstrap pamasamba akulu".

Source: www.habr.com