Chengetedza Chikamu 3: Ongorora uye Kuyedzwa kweduplicity, duplicati

Ichi chinyorwa chinokurukura maturusi ekuchengetedza anoita ma backups nekugadzira archives pane backup server.

Pakati peayo anosangana nezvinodiwa ndeye duplicity (iyo ine yakanaka interface muchimiro che deja dup) uye duplicati.

Chimwe chinoshamisa chekuchengetedza chishandiso ndeye dar, asi sezvo iine rondedzero yakawandisa yesarudzo - nzira yekuyedza inovhara zvishoma 10% yezvainokwanisa - isu hatisi kuiyedza sechikamu chekutenderera kwazvino.

Mhedzisiro inotarisirwa

Sezvo vese vavhoti vachigadzira matura mune imwe nzira kana imwe, tara yenguva dzose inogona kushandiswa segwara.

Pamusoro pezvo, isu tichaongorora kuti kuchengetwa kwedata sei pasevha yekuchengetera kwakagadziridzwa nekugadzira makopi ekuchengetedza ane chete mutsauko uripo pakati pekopi yakazara uye mamiriro azvino emafaira, kana pakati pezvakapfuura uye zvazvino zvakachengetwa (kuwedzera, kuderera, nezvimwe.) .

Maitiro paunenge uchigadzira backups:

1. Nhamba shoma yemafaira pane sevha yekuchengetedza (inofananidzwa nehuwandu hwemakopi ekuchengetedza kana saizi yedata muGB), asi saizi yavo yakakura kwazvo (makumi kusvika kumazana emamegabytes).
2. Saizi yekuchengetera inongosanganisira shanduko - hapana zvakapetwa zvichachengetwa, saka saizi yekuchengetera ichave idiki pane nersync-based software.
3. Tarisira inorema CPU mutoro paunenge uchishandisa compression uye / kana encryption, uye ingangoita yakanyanya kukwirira network uye dhisiki mutoro kana iyo yekuchengetedza uye / kana encryption maitiro ari kushanda pane backup yekuchengetedza server.

Ngatimhanyei murairo unotevera sereferensi kukosha:

cd /src/dir; tar -cf - * | ssh backup_server "cat > /backup/dir/archive.tar"

Mhedzisiro dzekuita dzaive sezvizvi:

Nguva yekuuraya 3m12s. Zvinogona kuoneka kuti kumhanya kunogumira neiyo disk subsystem yeiyo backup yekuchengetedza server, semuenzaniso ne rsync. Ingokurumidza zvishoma, nekuti... kurekodha kunoenda kune imwe faira.

Zvakare, kuti tiongorore compression, ngatimhanyei sarudzo imwechete, asi gonesa kudzvanya padivi rekuchengetedza server:

cd /src/dir; tar -cf - * | ssh backup_server "gzip > /backup/dir/archive.tgz"

Mibairo yacho ndeiyi:

Nguva yekuuraya 10m11s. Kazhinji ingangove iyo bhodhoro ndiyo imwechete-inoyerera compressor pane yekugamuchira yekupedzisira.

Iwo murairo mumwechete, asi nekumanikidza kunotamirwa kune sevha neiyo yekutanga data kuyedza iyo hypothesis yekuti bhodhoro ndeye imwechete-tambo compressor.

cd /src/dir; tar -czf - * | ssh backup_server "cat > /backup/dir/archive.tgz"

Zvakaitika seizvi:

Nguva yekuuraya yaive 9m37s. Mutoro pane imwe musimboti ne compressor inoonekwa zvakajeka, nekuti Iyo network yekumhanyisa kumhanya uye mutoro pane sosi disk subsystem zvakafanana.

Kuti uongorore encryption, unogona kushandisa openssl kana gpg nekubatanidza mumwe murairo openssl kana gpg mupombi. Pakurevera pachava nemurairo wakaita seuyu:

cd /src/dir; tar -cf - * | ssh backup_server "gzip | openssl enc -e -aes256 -pass pass:somepassword -out /backup/dir/archive.tgz.enc"

Maresults akabuda seizvi:

Nguva yekuuraya yakazoita 10m30s, sezvo 2 maitiro aimhanya padivi rekugamuchira - iyo bhodhoro zvakare iri imwechete-tambo compressor, pamwe nediki encryption pamusoro.

UPS: Pakukumbira kwebliznezz ndiri kuwedzera bvunzo ne pigz. Kana ukashandisa compressor chete, zvinotora 6m30s, kana iwe ukawedzera encryption, inenge inenge 7m. Kunyura mugirafu yepasi is unflushed disk cache:

Duplicate test

Duplicity isoftware yepython yekuchengetedza nekugadzira yakavharidzirwa zvinyorwa mu tar fomati.

Kune ekuwedzera matura, librsync inoshandiswa, saka unogona kutarisira maitiro anotsanangurwa mukati yapfuura post munhevedzano.

Backups inogona kuvharidzirwa uye kusainwa uchishandisa gnupg, iyo yakakosha kana uchishandisa vanopa vakasiyana kuchengetedza mabhapu (s3, backblaze, gdrive, nezvimwewo)

Ngationei kuti maresults ndeapi:

Aya ndiwo mhedzisiro yatakawana tichimhanya pasina encryption

muparadzi

Nguva yekumhanya yebvunzo yega yega:

Kutanga 1
Kutanga 2
Kutanga 3

16m33s
17m20s
16m30s

8m29s
9m3s
8m45s

5m21s
6m04s
5m53s

Uye hezvino mhedzisiro kana gnupg encryption ikagoneswa, ine kiyi saizi ye2048 bits:

Nguva yekushanda pane imwechete data, ine encryption:

Kutanga 1
Kutanga 2
Kutanga 3

17m22s
17m32s
17m28s

8m52s
9m13s
9m3s

5m48s
5m40s
5m30s

Kukura kwebhuroka kwakaratidzwa - 512 megabytes, iyo inonyatsooneka mumagrafu; Iyo processor mutoro wakaramba uri pa50%, zvinoreva kuti chirongwa hachishandise inopfuura imwechete processor core.

Nheyo yekushanda kwechirongwa ichi zvakare inooneka zvakajeka: vakatora chidimbu che data, vakaimanikidza, uye vakatumira kune yekuchengetedza yekuchengetedza server, iyo inogona kunonoka.
Chimwe chimiro inguva inofanotaurwa yenguva yepurogiramu, iyo inoenderana chete nehukuru hwe data yakashandurwa.

Kugonesa encryption hakuna kuwedzera zvakanyanya nguva yekumhanya kwechirongwa, asi yakawedzera processor kurodha ne10%, inogona kunge iri bhonasi yakanaka.

Nehurombo, chirongwa ichi hachina kukwanisa kunyatsoona mamiriro ezvinhu nedhairekitori rekupa zita, uye saizi yekuchengetera yakazove yakaenzana nehukuru hwekuchinja (kureva, ese 18GB), asi kugona kushandisa sevha isina kuvimbika yekusevha zvakajeka. inovhara maitiro aya.

Duplicate test

Iyi software yakanyorwa muC # uye inoshandisa seti yemaraibhurari kubva kuMono. Kune GUI pamwe neCLI vhezheni.

Iyo inofungidzirwa rondedzero yeakanyanya maficha akafanana neduplicity, kusanganisira akasiyana siyana ekuchengetedza ekuchengetedza, zvisinei, kusiyana neduplicity, akawanda maficha anowanikwa pasina wechitatu-bato maturusi. Kunyangwe iyi iri yekuwedzera kana minus zvinoenderana nechaiyo kesi, asi kune vanotanga, zvinonyanya kuve nyore kuve nerunyorwa rwezvese maficha pamberi pavo kamwechete, pane kuti uwedzere kuisa mapakeji epython, sezvazviri. nyaya yacho ine duplicity.

Imwe diki nuance - chirongwa chinoshinga kunyora dhatabhesi yemunharaunda sqlite pachinzvimbo chemushandisi anotanga backup, saka iwe unofanirwa nekuwedzera kuve nechokwadi chekuti dhatabhesi rinodiwa rinotsanangurwa nemazvo pese panotanga maitiro uchishandisa cli. Paunenge uchishanda kuburikidza neGUI kana WEBGUI, ruzivo rwunovanzwa kubva kumushandisi.

Ngationei kuti ndezvipi zviratidzo zvingagadzirwa nemhinduro iyi:

Kana iwe ukadzima encryption (uye WEBGUI isingakurudzire kuita izvi), mibairo yakaita seiyi:

Kushanda maawa:

Kutanga 1
Kutanga 2
Kutanga 3

20m43s
20m13s
20m28s

5m21s
5m40s
5m35s

7m36s
7m54s
7m49s

Ne encryption yakagoneswa, uchishandisa aes, inoita seizvi:

Kushanda maawa:

Kutanga 1
Kutanga 2
Kutanga 3

29m9s
30m1s
29m54s

5m29s
6m2s
5m54s

8m44s
9m12s
9m1s

Uye kana ukashandisa yekunze chirongwa gnupg, zvinotevera zvinobuda:

Kutanga 1
Kutanga 2
Kutanga 3

26m6s
26m35s
26m17s

5m20s
5m48s
5m40s

8m12s
8m42s
8m15s

Sezvauri kuona, chirongwa ichi chinogona kushanda mune akati wandei tambo, asi izvi hazviiti kuti ive mhinduro inobudirira, uye kana iwe ukaenzanisa basa rekunyorera, riri kutangisa chirongwa chekunze.
yakave yakakurumidza kupfuura kushandisa raibhurari kubva kuMono seti. Izvi zvinogona kunge zvakakonzerwa nekuti chirongwa chekunze chakanyanya kugadziridzwa.

Chimwe chinhu chakanaka yaive chokwadi chekuti saizi yenzvimbo yekuchengetera inotora zvakangofanana neiyo chaiyo yakashandurwa data, i.e. duplicati yakaona zita rezita rezita uye yakabata mamiriro aya nemazvo. Izvi zvinogona kuonekwa kana uchimhanya bvunzo yechipiri.

Pakazara, maonero akanaka echirongwa, kusanganisira kuva neushamwari kune vatsva.

Mhinduro

Vese vavhoti vakashanda zvishoma nezvishoma, asi kazhinji, zvichienzaniswa netara yenguva dzose, kune kufambira mberi, pamwe neduplicati. Mutengo wekufambira mberi kwakadaro wakajekawo - mutoro unooneka
processor. Kazhinji, hapana kutsauka kwakakosha mukufanotaura mhedzisiro.

zvakawanikwa

Kana iwe usingade kumhanyisa chero kupi, uye zvakare uine spare processor, chero mhinduro dzakatariswa dzichaita, chero zvakadaro, basa rakawanda rakaitwa risingafanirwe kudzokororwa nekunyora wrapper zvinyorwa pamusoro pe tar. . Kuvapo kwe encryption chinhu chakanyanya kudiwa kana sevha yekuchengetedza makopi ekuchengetedza isingavimbike zvizere.

Kuenzaniswa nemhinduro dzakavakirwa rsync - kuita kunogona kuita kwakapetwa kanoverengeka, zvisinei nekuti muchimiro chayo tar yakashanda 20-30% nekukurumidza kupfuura rsync.
Iko kune kuchengetwa pahukuru hweiyo repository, asi chete neduplicati.

Chiziviso

Backup, chikamu 1: Nei backup ichidiwa, muchidimbu nzira, matekinoroji
Backup Chikamu 2: Kuongorora uye kuyedza rsync-based backup maturusi
Backup Chikamu 3: Ongorora uye kuyedza duplicity, duplicati, deja dup
Backup Chikamu 4: Kuongorora uye kuyedza zbackup, restic, borgbackup
Backup Chikamu 5: Kuedza bacula uye veeam backup yelinux
Negadziriro yeparutivi Chikamu 6: Kuenzanisa negadziriro yeparutivi Tools
Backup Chikamu 7: Mhedziso

Zvakatumirwa na: Pavel Demkovich

Source: www.habr.com