Mhoroi, ini nguva pfupi yadarika ndakasangana nedambudziko rinonakidza: kumisikidza chengetedzo yekutsigira nhamba huru yemidziyo yebhuroka.
Vhiki rega rega isu tinochengetedza ese makina ari mugore redu, saka isu tinofanirwa kukwanisa kuchengetedza zviuru zvema backups uye nekuzviita nekukurumidza uye nemazvo sezvinobvira.
Sezvineiwo, standard configurations RAID5, RAID6 muchiitiko ichi, isu hatizobvumirwi kuita kudaro, sezvo kudzoreredza muitiro pamadhisiki makuru akadai seyedu kuchave kunorwadza kwenguva refu uye kusingazombofi kwakapera.
Ngatitarisei kuti ndedzipi dzimwe nzira dziripo:
- Yakafanana neRAID5, RAID6, asi ine inogadziriswa parity level. Muchiitiko ichi, kuchengetwa kunoitwa kwete kuvharwa nebhokisi, asi kune chimwe chinhu chakasiyana. Nzira iri nyore yekuyedza erasure coding ndeyekuwedzera .
iri parizvino isina kuburitswa yeZFS chimiro. Kusiyana neRAIDZ, DRAID ine yakagoverwa parity block uye, panguva yekudzoreredza, inoshandisa ese madhisiki eiyo array kamwechete, izvo zvinoita kuti zvive nani kukwanisa kumira kutadza kwe dhisiki uye kupora nekukurumidza mushure mekutadza.
Server iripo Fujitsu Primergy RX300 S7 ne processor Intel Xeon CPU E5-2650L 0 @ 1.80GHz, zvimiti zvipfumbamwe zve RAM Samsung DDR3-1333 8Gb PC3L-10600R ECC Yakanyoreswa (M393B1K70DH0-YH9), disk sherufu Supermicro SuperChassis 847E26-RJBOD1, yakabatana ne Dual LSI SAS2X36 Expander uye 45 discs Seagage ST6000NM0115-1YZ110 pamusoro 6TB chimwe nechimwe.
Tisati tasarudza chero chinhu, isu tinofanira kutanga tanyatso kuyedza zvese.
Kuti ndiite izvi, ndakagadzirira uye ndakaedza zvigadziriso zvakasiyana. Kuti ndiite izvi, ndakashandisa minio, iyo yakaita seS3 backend uye ndakaitanga nenzira dzakasiyana nenhamba dzakasiyana dzezvinangwa.
Chaizvoizvo, iyo minio kesi yakaedzwa mu erasure coding vs software raid ine nhamba yakafanana yemadhisiki uye parity yemadhisiki, uye aya ndeaya: RAID6, RAIDZ2 uye DRAID2.
Zvekureva: paunovhura minio uine chinangwa chimwe chete, minio inoshanda muS3 gedhi modhi, kuendesa yako yemunharaunda faira system muchimiro cheS3 kuchengetedza. Kana iwe ukatangisa minio uchitsanangura akati wandei, iyo Erasure Coding modhi inozobatidza otomatiki, iyo inoparadzira iyo data pakati pezvinangwa zvako uchipa kukanganisa kushivirira.
Nekutadza, minio inokamura zvinangwa mumapoka e16 disks, aine 2 mapari paboka. Avo. Madhisiki maviri anogona kukundikana panguva imwe chete pasina kurasikirwa nedata.
Kuti ndiedze kuita, ndakashandisa madhisiki gumi nematanhatu e16TB imwe neimwe uye ndakanyora zvinhu zvidiki zve 6MB muhukuru pazviri, izvi zvakanyatsotsanangura mutoro wedu weramangwana, sezvo ese maturusi emazuva ano ekuchengetedza akakamura data kuita mabhuroko emamegabyte akati wandei uye anyore nenzira iyi.
Kuitisa bhenji, isu takashandisa iyo s3bench utility, yakatangwa pane iri kure server uye kutumira makumi ezviuru zvezvinhu zvakadaro kune minio mumazana eshinda. Pashure pacho ndakaedza kuvakumbira kuti vadzoke nenzira imwe cheteyo.
Mibairo yekumisikidza inoratidzwa mune inotevera tafura:
Sezvatinoona, minio mune yayo yega erasure coding mode inoita zvakanyanya kuipa pakunyora kupfuura minio inomhanya pamusoro pesoftware RAID6, RAIDZ2 uye DRAID2 mune imwe gadziriro.
Takasiyana ini bvunzo minio pane ext4 vs XFS. Zvinoshamisa kuti, kurudzi rwangu rwemutoro webasa, XFS yakave inononoka kupfuura ext4.
Muchikamu chekutanga chebvunzo, Mdadm yakaratidza hukuru pamusoro peZFS, asi gare gare kuti iwe unogona kuvandudza kuita kweZFS nekuisa zvinotevera sarudzo:
xattr=sa atime=off recordsize=1Muye mushure meizvozvo bvunzo neZFS dzakava nani.
Iwe unogona zvakare kucherechedza kuti DRAID haipe yakawanda kuita budiriro pamusoro peRAIDZ, asi mudzidziso inofanirwa kuve yakachengeteka zvakanyanya.
Mumiyedzo miviri yekupedzisira, ndakaedzawo kuendesa metadata (yakakosha) uye ZIL (log) kugirazi kubva kuSSD. Asi kubvisa metadata hakuna kupa pfuma yakawanda mukurekodha kumhanya, uye pakubvisa ZIL, yangu rova ββsirin'i ne100% kushandiswa, saka ini ndinoona bvunzo iyi kukundikana. Ini handiregi kuti dai ndaive nekukurumidza SSD madhiraivha, saka pamwe izvi zvinogona kuvandudza zvakanyanya mhedzisiro yangu, asi, zvinosuruvarisa, ini ndakanga ndisina.
Pakupedzisira, ndakasarudza kushandisa DRAID uye pasinei nemamiriro ayo e beta, ndiyo inokurumidza uye inonyanya kushanda yekuchengetedza mhinduro munyaya yedu.
Ini ndakagadzira iri nyore DRAID2 mukugadzirisa nemapoka matatu uye maviri akagoverwa spares:
# zpool status data
pool: data
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
data ONLINE 0 0 0
draid2:3g:2s-0 ONLINE 0 0 0
sdy ONLINE 0 0 0
sdam ONLINE 0 0 0
sdf ONLINE 0 0 0
sdau ONLINE 0 0 0
sdab ONLINE 0 0 0
sdo ONLINE 0 0 0
sdw ONLINE 0 0 0
sdak ONLINE 0 0 0
sdd ONLINE 0 0 0
sdas ONLINE 0 0 0
sdm ONLINE 0 0 0
sdu ONLINE 0 0 0
sdai ONLINE 0 0 0
sdaq ONLINE 0 0 0
sdk ONLINE 0 0 0
sds ONLINE 0 0 0
sdag ONLINE 0 0 0
sdi ONLINE 0 0 0
sdq ONLINE 0 0 0
sdae ONLINE 0 0 0
sdz ONLINE 0 0 0
sdan ONLINE 0 0 0
sdg ONLINE 0 0 0
sdac ONLINE 0 0 0
sdx ONLINE 0 0 0
sdal ONLINE 0 0 0
sde ONLINE 0 0 0
sdat ONLINE 0 0 0
sdaa ONLINE 0 0 0
sdn ONLINE 0 0 0
sdv ONLINE 0 0 0
sdaj ONLINE 0 0 0
sdc ONLINE 0 0 0
sdar ONLINE 0 0 0
sdl ONLINE 0 0 0
sdt ONLINE 0 0 0
sdah ONLINE 0 0 0
sdap ONLINE 0 0 0
sdj ONLINE 0 0 0
sdr ONLINE 0 0 0
sdaf ONLINE 0 0 0
sdao ONLINE 0 0 0
sdh ONLINE 0 0 0
sdp ONLINE 0 0 0
sdad ONLINE 0 0 0
spares
s0-draid2:3g:2s-0 AVAIL
s1-draid2:3g:2s-0 AVAIL
errors: No known data errorsZvakanaka, isu tagadzirisa chengetedzo, zvino ngatitaure nezve zvatichatsigira. Pano ndinoda kutaura nekukurumidza nezve mhinduro nhatu dzandakakwanisa kuyedza, uye idzi ndeidzi:
- fork , mhinduro yakasarudzika yevhavha mudziyo backup, ine yakasimba kubatanidzwa neCeph. Inogona kutora misiyano pakati pezvipfukuto uye kugadzira backup yekuwedzera kubva kwavari. Inotsigira nhamba yakakura yekuchengetedza backends, kusanganisira zvese zvemuno uye S3. Inoda dhatabhesi yakaparadzana kuchengetedza deduplication hash tafura. Zvakashata: yakanyorwa mu python, ine zvishoma isingateereri cli.
- fork , chishandiso chave chichizivikanwa kwenguva refu uye chakasimbiswa chekuchengetedza, chinogona kuchengetedza data uye kuidhirowa zvakanaka. Inokwanisa kuchengetedza ma backups emuno uye kune iri kure server kuburikidza ne scp. Inogona kuchengetedza zvishandiso kana yakatangwa nemureza --special, imwe yemaminetsi: pakugadzira backup, iyo repository yakavharwa zvachose, saka zvinokurudzirwa kugadzira yakaparadzana repository yemuchina wega wega wega, musimboti iyi haisi dambudziko, nerufaro ivo vanogadzirwa nyore nyore.
ipurojekiti inoshingairira kusimudzira, yakanyorwa mukuenda, inokurumidza uye inotsigira nhamba huru yekuchengetedza kumashure, kusanganisira kuchengetedza kwenzvimbo, scp, S3 nezvimwe zvakawanda. Takaparadzana, ndinoda kuona kuti kune yakanyatsogadzirwa ye restic, iyo inokutendera iwe kukurumidza kutumira kunze kwekuchengetedza kuti ushandiswe kure. Pane zvese zviri pamusoro, ndakazvifarira zvakanyanya. Inogona kuchengetedza kubva ku stdin. Iyo inenge isina zvipingamupinyi zvinoonekwa, asi pane akati wandei maficha:
-
Chekutanga, ndakaedza kuishandisa mune yakajairika repository modhi yemashini ese chaiwo (seBenji) uye yakatoshanda zvakanaka, asi kudzoreredza mashandiro kwakatora nguva yakareba, nekuti ... Nguva imwe neimwe isati yadzoreredza, restic inoedza kuverenga metadata yeese backups. Dambudziko iri rakagadziriswa zviri nyore, sekunge borg, nekugadzira yakaparadzana repository yemuchina wega wega. Iyi nzira yakaratidza kuve inoshanda zvakanyanya pakugadzirisa ma backups zvakare. Akaparadzana marepositori anogona kuve nepassword yakaparadzana yekuwana data, uye isu hatifanirewo kutya kuti repo repasi rose ringaputsika neimwe nzira. Iwe unogona kuburitsa zvinyorwa zvitsva zviri nyore sekunge muborg backup.
Chero zvazvingaitika, kudonhedza kunoitwa chete maererano neyakapfuura vhezheni yekuchengetedza; iyo yakapfuura backup inotarwa nenzira yeyakatsanangurwa backup, saka kana iwe ukatsigira zvakasiyana zvinhu kubva stdin kuenda kune yakajairwa repository, usakanganwa kutsanangura iyo sarudzo
--stdin-filename, kana kuti tsanangura sarudzo nguva imwe neimwe--parent.
-
Kechipiri, kudzoreredza kune stdout kunotora nguva yakareba kupfuura kudzoreredza kune iyo faira system nekuda kwekuenderana kwayo. Mune ramangwana, isu tinoronga kuwedzera tsigiro yepedyo ye backups ye block zvishandiso.
-
Chechitatu, ikozvino inokurudzirwa kushandisa , nokuti vhezheni 0.9.6 ine tsikidzi ine refu kupora mafaira makuru.
Kuti ndiedze kushanda kweiyo backup uye nekumhanya kwekunyora / kudzoreredza kubva kune backup, ini ndakagadzira yakaparadzana repository uye ndakaedza kuchengetedza mufananidzo mudiki wemuchina chaiwo (21 GB). Mabackups maviri akaitwa pasina kushandura yekutanga, uchishandisa yega yega mhinduro dzakanyorwa kutarisa kuti nekukurumidza/kunonoka sei data rakadhindwa rakakopwa.
Sezvatinoona, Borg Backup ine yakanakisa yekutanga backup performance ratio, asi yakaderera maererano nezvose zviri zviviri kunyora uye kudzoreredza kumhanya.
Restic yakave inokurumidza kupfuura Benji Backup, asi zvinotora nguva yakareba kuti idzorere ku stdout, uye, zvinosuruvarisa, haisati yaziva kunyora zvakananga kune block mudziyo.
Mushure mekuyera zvese zvakanakira nezvayakaipira, ndakasarudza kugadzirisa restic Ρ rest-server seyakanyanya nyore uye inovimbisa backup mhinduro.
Mune ino skrini iwe unogona kuona kuti 10-gigabit chiteshi inoshandiswa sei zvizere panguva dzinoverengeka dzekuchengetedza mashandiro achimhanya panguva imwe chete. Zvakakosha kucherechedza kuti dhisiki recycling haina kukwira pamusoro pe30%.
Ndakafara zvikuru nemhinduro yandakawana!
Source: www.habr.com