Update!. Mumashoko, mumwe wevaverengi akakurudzira kuedza (zvichida ari kushanda pazviri pachake) saka ndawedzera chikamu pamusoro pemhinduro iyi. Ndakanyorawo , nokuti maitiro acho akasiyana zvikuru nemamwe ose.
Kutaura chokwadi, ndakakanda mapfumo pasi ndikakanda mapfumo pasi (pamwe pazvino). Ndichashandisa . Sei? Nekuda kwekuchengetedza! Ndiani angadai akafunga kuti ndaizonyanya kufunga nezvekuchengetedza kupfuura neKubernetes pachayo. ndinoshandisa nekuti haina kudhura uye kuita kwayo kwakanaka uye kubva pakutanga ndanga ndichiendesa masumbu ndichishandisa . Ini handina kuyedza yakachengetedzwa Kubernetes masevhisi kubva kuGoogle/Amazon/Microsoft/DigitalOcean, nezvimwewo, nezvimwewo, nekuti ndaida kudzidza zvese ini. Ndinewo mari.
Saka hongu, ndakatora nguva yakawanda ndichiedza kusarudza kuti ndeipi chengetedzo yekusarudza pandakanga ndichiongorora inokwanisika Kubernetes stack. Ini ndinosarudza yakavhurika sosi mhinduro, kwete chete nekuda kwemutengo, asi ini ndakatarisa mune akati wandei akabhadharwa sarudzo nekuda kwekuda kuziva nekuti ivo vane emahara mavhezheni ane zvisingakwanisi. Ndakanyora pasi dzimwe nhamba kubva kumiyedzo ichangoburwa pandakafananidza sarudzo dzakasiyana, uye dzinogona kufarira kune avo vanodzidza nezveKubernetes kuchengetedza. Kunyangwe ini pachangu ndaonekana naKubernetes izvozvi. Ndinodawo kutaura , iyo inogona kupa zvakananga Hetzner Cloud mavhoriyamu, asi ini handisati ndazviedza. Ndakatarisa mukati megore software-yakatsanangurwa chengetedzo nekuti ndaida kudzokorora uye kugona kukurumidza kukwidza mavhoriyamu anoenderera pane chero node, kunyanya kana pane kutadza kwenode uye mamwe mamiriro akafanana. Dzimwe mhinduro dzinopa point-in-time snapshots uye off-saiti backups, zviri nyore.
Ndakaedza 6-7 kuchengetedza mhinduro:
Sezvandamboreva Sezvo ndayedza akawanda esarudzo kubva pane iyo rondedzero, ini pakutanga ndakagara paOpenEBS. OpenEBS iri nyore kwazvo kuisa uye kushandisa, asi kuva akatendeseka, mushure mekuedzwa ne data chaiyo pasi pemutoro, ndakaodzwa mwoyo nekuita kwayo. Iyi ndiyo yakavhurika sosi, uye vanogadzira vari vega nguva dzose inobatsira zvikuru pandaida rubatsiro. Nehurombo, ine kuita kwakashata zvakanyanya kana ichienzaniswa nedzimwe sarudzo, saka bvunzo dzaifanira kuitwazve. OpenEBS parizvino ine matatu ekuchengetedza injini, asi ini ndiri kutumira mabhenji mabhenji e cStor. Handisati ndave nenhamba dzeJiva neLocalPV.
Muchidimbu, Jiva inokurumidza kukurumidza, uye LocalPV inowanzo mhanyisa, haina kushata kupfuura dhisiki bhenji zvakananga. Dambudziko neLocalPV nderekuti vhoriyamu inogona kuwanikwa chete pane node kwayakagadzirirwa, uye hapana kudzokorora zvachose. Ndakaita mamwe matambudziko ekudzoreredza backup kuburikidza pachikwata chitsva nekuti mazita emanodhi aive akasiyana. Kana tikataura nezve backups, cStor ine , iyo iwe yaunogona kugadzira off-saiti backups yezvipfupi pane imwe nguva, iri nyore kupfuura faira-level backups neVelero-Restic. Ndakanyora , kuita kuti zvive nyore kubata ma backups uye kudzoreredza neiyi plugin. Pakazere, ini ndinoda chaizvo OpenEBS, asi kuita kwayo ...
Rook zvakare yakavhurika sosi, uye chii chinoisiyanisa kubva kune dzimwe sarudzo pane iyo rondedzero ndeyekuti inochengetera orchestrator inobata yakaoma ekuchengetedza ekuchengetedza mabasa ane akasiyana backend, semuenzaniso. , uye zvimwe, izvo zvinorerutsa zvikuru basa racho. Ndakanga ndine matambudziko neEfgeFS pandakaiedza mwedzi mishoma yapfuura, saka ndakaedza kunyanya neCeph. Ceph haingopi chete block kuchengetedza, asiwo chinhu chekuchengetedza chinoenderana neS3 / Swift uye yakagoverwa faira system. Chandinofarira nezveCeph kugona kuparadzira vhoriyamu data kune akawanda madhisiki kuitira kuti vhoriyamu ishandise yakawanda dhisiki nzvimbo pane inogona kukwana pane imwechete dhisiki. Zviri nyore. Chimwe chinhu chinotonhorera ndechekuti kana iwe ukawedzera dhisiki kune sumbu, inogovera otomatiki data kune ese madhisiki.
Ceph ine snapshots, asi sekuziva kwangu, haigone kushandiswa zvakananga muRook/Kubernetes. Chokwadi, handina kupinda zvakadzama mune izvi. Asi hapana off-saiti backups, saka uchafanirwa kushandisa chimwe chinhu neVelero / Restic, asi kune chete faira-level backups, kwete point-in-time snapshots. Chandainyanya kufarira nezveRook ndechekuti zviri nyore sei kushanda neCeph - inovanza zvinenge zvese zvakaoma zvinhu uye inopa maturusi ekutaura naCeph zvakanangana nekugadzirisa matambudziko. Nehurombo, panguva yekuyedzwa kwekushushikana kwemavhoriyamu eCeph, ndakaramba ndiine matambudziko nazvo , izvo zvinoita kuti Ceph isagadzikana. Izvo hazvisati zvajeka kuti iyi ibug muCeph pachayo kana dambudziko munzira iyo Rook anotonga nayo Ceph. Ndakafunga nezvezvirongwa zvendangariro, uye zvakava nani, asi dambudziko harina kunyatsogadziriswa. Ceph ine kuita kwakanaka, sezvauri kuona mumabhenji pazasi. Iyo ine zvakare yakanaka dashboard.
Ndinoda chaizvo Longhorn. Mukuona kwangu, iyi ndiyo mhinduro inovimbisa. Ichokwadi, vanogadzira ivo pachavo (Rancher Labs) vanobvuma kuti haisati yave yakakodzera kunharaunda yekushanda, uye izvi zvinoratidza. Iyo yakavhurika sosi uye ine hunhu hwakanaka (kunyangwe vanga vasati vagadzirisa zvakadaro), asi mavhoriyamu anotora nguva yakareba kwazvo kuti abatanidze kune pod, uye mumatambudziko akanyanya zvinotora 15-16 maminetsi, kunyanya mushure mekudzorera yakakura backup kana. kuvandudza mutoro webasa. Iyo ine snapshots uye off-saiti backups yeizvi snapshots, asi inongoshanda kune mavhoriyamu, saka iwe uchazoda chimwe chinhu chakaita seVelero kuchengetedza zvimwe zviwanikwa. Backups uye kudzoreredza kwakavimbika kwazvo, asi kunonoka kunonoka. Serious, kungononoka zvisingaite. Kushandiswa kweCPU uye kurodha system kunowanzo kusimuka kana uchishanda nehuwandu hwepakati data muLonghorn. Pane dashboard iri nyore kubata Longhorn. Ndakambotaura kuti ndinoda Longhorn, asi inoda rimwe basa.
StorageOS ndiyo yekutanga yakabhadharwa chigadzirwa pane iyo rondedzero. Iyo ine vhezheni yekuvandudza ine shoma yakachengetedzwa saizi yekuchengetedza ye500GB, asi ini handifunge kuti pane muganhu pahuwandu hwemanodhi. Dhipatimendi rekutengesa rakandiudza kuti mutengo unotanga pa $ 125 pamwedzi ye1 TB, kana ndikarangarira zvakanaka. Kune yakakosha dashboard uye iri nyore CLI, asi chimwe chinhu chinoshamisa chiri kuitika nekuita: mune mamwe mabhenji zvine huremu, asi muyedzo yekushushikana kwevhoriyamu handina kufarira kumhanya zvachose. Kazhinji, handizivi zvokutaura. Saka handina kunyatsonzwisisa zvakawanda. Iko hakuna off-saiti backups pano uye iwe zvakare uchafanirwa kushandisa Velero neRestic kune backup mavhoriyamu. Zvinoshamisa, nokuti chigadzirwa chinobhadharwa. Uye vanogadzira vanga vasingadi kutaurirana paSlack.
Ndakadzidza nezvaRobin paReddit kubva kune yavo technical director. Ndakanga ndisati ndambonzwa nezvake. Zvichida nokuti ndaitsvaga mhinduro dzekusununguka, asi Robin anobhadharwa. Ivo vane yakanaka rupo yemahara vhezheni ine 10TB yekuchengetedza uye matatu node. Pakazara, chigadzirwa chakanyanya kunaka uye chine maitiro akanaka. Pane CLI yakanaka, asi chinhu chinotonhorera ndechekuti iwe unogona kutora uye backup iyo yese application (mune resource sector iyi inonzi Helm kuburitswa kana "flex apps"), kusanganisira mavhoriyamu uye zvimwe zviwanikwa, saka unogona kuita pasina Velero. Uye zvese zvingave zvakanaka kana zvisiri zvechimwe chidiki diki: kana iwe ukadzoreredza (kana "import", sekudaidzwa kwazvinoitwa muRobin) application pane itsva cluster - semuenzaniso, kana paine kupora kubva munjodzi - kudzoreredza, hongu, inoshanda, asi ramba uchitsigira iyo application inorambidzwa. Izvi hazvigoneke mukuburitswa uku, sezvakasimbiswa nevagadziri. Izvi ndezvekuti, kuzviisa zvinyoro, zvisinganzwisisike, kunyanya kufunga nezvezvimwe zvakanakira (semuenzaniso, zvinoshamisa nekukurumidza mabhakiti uye kudzorera). Vagadziri vanovimbisa kugadzirisa zvese nekuburitswa kunotevera. Kuita kunowanzo kunaka, asi ndakaona zvisinganzwisisike: kana ndikamhanyisa bhenji pavhoriyamu yakasungirirwa kumugadziri, kumhanya kwekuverenga kunokurumidza kupfuura kumhanya vhoriyamu imwechete kubva mukati mepodhi. Mimwe mibairo yese yakafanana, asi mudzidziso hapafanirwe kuve nemusiyano. Kunyangwe ivo vari kushanda pazviri, ndakagumbuka nezve dambudziko rekudzorera nekuchengetedza - ndakafunga kuti ndakanga ndawana mhinduro yakakodzera, uye ini ndaitoda kuibhadhara pandaida imwe nzvimbo kana mamwe maseva.
Handina zvakawanda zvekutaura pano. Ichi chigadzirwa chakabhadharwa, chakaenzana chinotonhorera uye chinodhura. Kuita kunongoshamisa. Ichi ndicho chiratidzo chakanakisa kusvika zvino. Slack akandiudza kuti mitengo inotangira pamadhora mazana maviri neshanu pamwedzi node, sezvakanyorwa muGoogle's GKE Marketplace. Handizivi kuti zvichachipa here kana ukatenga zvakananga. Ini handizvikwanise zvakadaro, saka ndakaodzwa mwoyo zvakanyanya kuti rezinesi rekuvandudza (kusvika 205 TB uye 1 node) harina basa neKubernetes kunze kwekunge uchigutsikana nekupa kwakamira. Ini ndanga ndichitarisira kuti rezinesi revhoriyamu raizodzika pasi kune kuvandudza pakupera kwenguva yekuyedza, asi izvo hazvina kuitika. Rezinesi rekuvandudza rinogona kungoshandiswa zvakananga neDocker, uye kumisikidzwa muKubernetes yakanyanya kuomarara uye ine mashoma. Ehe, ini ndinosarudza yakavhurika sosi, asi dai ndaive nemari, ndaizosarudza Portworx. Parizvino, kuita kwayo hakuenzaniswi nedzimwe sarudzo.
Ndakawedzera chikamu ichi mushure mekuburitswa kwechinyorwa, apo mumwe muverengi akakurudzira kuedza Linstor. Ndakaiedza uye ndakaifarira! Asi tichiri kuda kuchera zvakadzama. Ikozvino ini ndinogona kutaura kuti kuita hakuna kushata (ini ndakawedzera mabhenji mhedzisiro pazasi). Chaizvoizvo, ini ndakawana kuita kwakafanana nediski zvakananga, pasina chero pamusoro. (Usabvunza kuti nei Portworx ine nhamba dziri nani pane dhiraivha bhenji zvakananga. Ini handina zano. Mashiripiti, ndinofungidzira.) Saka Linstor inoratidzika kunge inoshanda zvakanyanya kusvika zvino. Hazvina kuoma kuisa, asi hazvisi nyore sedzimwe sarudzo. Kutanga ndaifanira kuisa Linstor (kernel module uye maturusi / masevhisi) uye kugadzirisa LVM yekuonda kupa uye snapshot rutsigiro kunze kweKubernetes, zvakananga pamugamuchiri, uyezve kugadzira zviwanikwa zvinodiwa kushandisa kuchengetedza kubva Kubernetes. Ini handina kufarira kuti haina kushanda paCentOS uye ndaifanira kushandisa Ubuntu. Hazvina kutyisa, hongu, asi zvinogumbura zvishoma, nekuti zvinyorwa (izvo zvakanakisa, nenzira) inotaura akati wandei mapakeji asingawanikwe mune yakatsanangurwa Epel repositori. Linstor ine snapshots, asi kwete off-saiti backups, saka pano zvakare ndaifanira kushandisa Velero neRestic kune backup mavhoriyamu. Ini ndingada zvidhori pachinzvimbo chefaira-level backups, asi izvi zvinogona kuregererwa kana mhinduro ichiita uye yakavimbika. Linstor ndeye yakavhurika sosi asi yakabhadhara rutsigiro. Kana ndikanzwisisa zvakanaka, inogona kushandiswa pasina zvirambidzo, kunyange kana iwe usina chibvumirano chekutsigira, asi izvi zvinoda kujekeswa. Handizivi kuti Linstor yakaedzwa sei yeKubernetes, asi iyo yekuchengetedza iyo pachayo iri kunze kweKubernetes uye, sezviri pachena, mhinduro haina kuoneka nezuro, saka ingangove yakaedzwa mumamiriro ezvinhu chaiwo. Pane mhinduro here pano ichaita kuti ndichinje pfungwa dzangu ndidzokere kuKubernetes? Handizive. Tichiri kufanira kuchera zvakadzama uye kudzidza kudzokorora. Ngationei. Asi maonero ekutanga akanaka. Ndinonyatsoda kushandisa yangu Kubernetes masumbu panzvimbo yeHeroku kuti ave nerusununguko rwakawanda uye kudzidza zvinhu zvitsva. Sezvo Linstor isiri nyore kuisa sevamwe, ini ndichanyora chinyorwa nezvazvo munguva pfupi.
Benchmarks
Zvinosuruvarisa, handina kuchengeta zvinyorwa zvakawanda pamusoro pekuenzanisa nokuti handina kufunga kuti ndaizonyora nezvazvo. Ini ndinongove nemhedzisiro kubva kune ekutanga fio mabhenji uye chete kune imwechete node masumbu, saka ini handina nhamba dzekudzokororwa zvigadziriso parizvino. Asi kubva pane izvi zvawanikwa unogona kuwana yakaomesesa pfungwa yezvaunotarisira kubva kune yega yega sarudzo, nekuti ini ndakazvienzanisa pamasevha egore mamwechete, 4 cores, 16 GB ye RAM, neimwe 100 GB dhisiki yemavhoriyamu akaedzwa. Ndakamhanyisa mabhenji katatu kune yega yega mhinduro uye ndikaverenga avhareji mhedzisiro, uyezve ini ndinogadzirisa sevhavha kuseta kune yega chigadzirwa. Izvi zvese hazvisi zvesainzi, kuti ndikupe pfungwa yakajairika. Mune mimwe miedzo, ndakakopa 38 GB yemifananidzo nemavhidhiyo kubva pavhoriyamu kuti ndiedze kuverenga nekunyora, asi, nhamo, handina kuchengetedza nhamba. Muchidimbu: Portworkx yakakurumidza zvikuru.
Kune vhoriyamu benchmark ndakashandisa iyi manifest:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: dbench
spec:
storageClassName: ...
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
---
apiVersion: batch/v1
kind: Job
metadata:
name: dbench
spec:
template:
spec:
containers:
- name: dbench
image: sotoaster/dbench:latest
imagePullPolicy: IfNotPresent
env:
- name: DBENCH_MOUNTPOINT
value: /data
- name: FIO_SIZE
value: 1G
volumeMounts:
- name: dbench-pv
mountPath: /data
restartPolicy: Never
volumes:
- name: dbench-pv
persistentVolumeClaim:
claimName: dbench
backoffLimit: 4Ini ndakatanga kugadzira vhoriyamu nekirasi yakakodzera yekuchengetedza uye ndokumhanyisa basa nefio kuseri kwezviitiko. Ndakatora 1 GB yekufungidzira kuita uye kwete kumirira nguva refu. Heino mibairo:
Ndakaratidza kukosha kwakanyanya kune yega metric mune yegirini uye yakaipisisa mune tsvuku.
mhedziso
Sezvauri kuona, kazhinji Portworx yakaita zvirinani pane vamwe. Asi kwandiri zvinodhura. Handizivi kuti Robin inodhura zvakadini, asi vane shanduro huru yekusununguka, saka kana iwe uchida chigadzirwa chakabhadharwa, unogona kuedza (tarisiro vanogadzirisa dambudziko nekudzorera uye kuchengetedza nokukurumidza). Pamatatu emahara, ini ndaive nedambudziko diki neOpenEBS, asi kuita kwayo kwakashata. Zvinosiririsa kuti handina kuchengetedza mamwe maresults, asi ndinovimba manhamba nemacomments angu achakubatsira.
Source: www.habr.com