Ichi chinyorwa chinotarisa chekuchengetedza software iyo, nekupwanya dhata data muzvikamu zvakasiyana (chunks), inoumba repository.
Repository zvikamu zvinogona kuwedzeredzwa uye kuvharirwa, uye zvakanyanya kukosha - panguva inodzokororwa backup maitiro - kushandiswazve.
Ikopi yekuchengetedza mune yakadaro repository iketani yakadanwa yezvikamu zvakabatana kune mumwe nemumwe, semuenzaniso, zvichibva pane akasiyana hashi mabasa.
Pane akati wandei akafanana mhinduro, ini ndichatarisa pa3: zbackup, borgbackup uye restic.
Mhedzisiro inotarisirwa
Sezvo vese vanonyorera vachida kuumbwa kwenzvimbo neimwe nzira, chimwe chezvinhu zvakakosha chichava chekufungidzira saizi yenzvimbo. Nenzira yakanaka, saizi yayo haifanirwe kunge inopfuura 13 GB zvichienderana neyakagamuchirwa nzira, kana kunyange zvishoma - zvichienderana nekugadzirisa kwakanaka.
Izvo zvakare zvinodikanwa kwazvo kukwanisa kugadzira makopi ekuchengetedza mafaera zvakananga, pasina kushandisa zvinyorwa senge tar, pamwe nekushanda ne ssh/sftp pasina mamwe maturusi sersync uye sshfs.
Maitiro paunenge uchigadzira backups:
- Saizi ye repository ichave yakaenzana nehukuru hwekuchinja, kana kushoma.
- Inorema CPU mutoro unotarisirwa paunenge uchishandisa compression uye / kana encryption, uye yakanyanya kukwirira network uye dhisiki mutoro ungangoita kana iyo yekuchengetera uye / kana encryption maitiro ari kushanda pane backup yekuchengetedza server.
- Kana iyo repository ikakuvadzwa, chikanganiso chakanonoka chingangove chiri pakugadzira ma backups matsva uye pakuedza kudzoreredza. Izvo zvinodikanwa kuronga mamwe matanho ekuona kutendeseka kweiyo repository kana kushandisa yakavakirwa-mukati maturusi ekutarisa kuvimbika kwayo.
Kushanda netara kunotorwa sereferensi kukosha, sezvakaratidzwa mune chimwe chezvinyorwa zvakapfuura.
Kuedza zbackup
Iyo yakajairwa mashandiro ezbackup ndeyekuti chirongwa chinowana munzvimbo dzekupinza data rwizi dzine data rakafanana, wobva wadzvanya nekuzvinyora, uchichengetedza nzvimbo yega yega kamwe chete.
Deduplication inoshandisa 64-bit ring hashi basa ine hwindo rinotsvedza kutarisa mabyte-by-byte machisi achipesana neapo data blocks (yakafanana nemashandisiro anoita rsync).
Multi-threaded lzma uye lzo anoshandiswa kumanikidza, uye maees ekuvharidzira. Iwo achangoburwa vhezheni ane kugona kudzima data rekare kubva mudura mune ramangwana.
Iyo purogiramu yakanyorwa muC ++ ine zvishoma zvinotsamira. Munyori sezviri pachena akafemerwa neiyo unix-nzira, saka chirongwa chinogashira data pa stdin kana ichigadzira backups, ichigadzira yakafanana data rwizi pa stdout pakudzoreredza. Saka, zbackup inogona kushandiswa seyakanaka kwazvo "chivakwa chekuvaka" paunenge uchinyora yako mhinduro mhinduro. Semuyenzaniso, munyori wechinyorwa ashandisa chirongwa ichi sechishandiso chikuru chekuchengetedza michina yemumba kubvira munenge muna 2014.
Iyo data rwizi ichave yenguva dzose tara kunze kwekunge zvataurwa neimwe nzira.
Ngationei kuti maresults ndeapi:
Basa rakatariswa mune 2 sarudzo:
- repository inogadzirwa uye zbackup inotangwa pane sevha ine sosi data, ipapo zviri mukati meiyo repository zvinoendeswa kune backup yekuchengetedza server.
- repository inogadzirwa pane sevha yekuchengetedza yekuchengetedza, zbackup inotangwa kuburikidza ne ssh pane yekuchengetedza yekuchengetedza server, uye data inotumirwa kwairi kuburikidza nepombi.
Mhedzisiro yesarudzo yekutanga yaive inotevera: 43m11s - kana uchishandisa isina kunyorwa repository uye lzma compressor, 19m13s - pakutsiva compressor nelzo.
Kuremerwa pasevha nedata rekutanga kwaive sekutevera (muenzaniso une lzma unoratidzwa; nelzo paive nemufananidzo wakafanana, asi mugove wersync waive chikamu chechina chenguva):
Zviri pachena kuti maitiro ekuchengetedza akadaro akakodzera chete kune zvisingawanzo uye diki shanduko. Izvo zvakare zvinokurudzirwa zvakanyanya kudzikamisa zbackup kune 1 shinda, zvikasadaro pachave neiyo yakakwira CPU mutoro, nekuti. Iyo purogiramu yakanaka kwazvo pakushanda mumatambo akawanda. Kuremerwa padhisiki kwaive kudiki, iyo kazhinji yaisazoonekwa neazvino ssd-based disk subsystem. Iwe unogona zvakare kuona zvakajeka kutanga kwemaitiro ekuyananisa repository data kune iri kure sevha; kumhanya kwekushanda kunofananidzwa neyakajairwa rsync uye zvinoenderana nekuita kweiyo disk subsystem ye backup yekuchengetedza server. Kuipa kweiyi nzira kuchengetwa kwenzvimbo yekuchengetera nzvimbo uye, semhedzisiro, kudzokorora kwedata.
Zvimwe zvinonakidza uye zvinoshanda mukuita ndiyo yechipiri sarudzo, ichimhanya zbackup yakananga pane yekuchengetedza yekuchengetedza server.
Kutanga, isu tichaedza kushanda pasina kushandisa encryption ne lzma compressor:
Nguva yekumhanya yebvunzo yega yega:
Kutanga 1
Kutanga 2
Kutanga 3
39m45s
40m20s
40m3s
7m36s
8m3s
7m48s
15m35s
15m48s
15m38s
Kana iwe ukagonesa encryption uchishandisa aes, mibairo iri padhuze:
Nguva yekushanda pane imwechete data, ine encryption:
Kutanga 1
Kutanga 2
Kutanga 3
43m40s
44m12s
44m3s
8m3s
8m15s
8m12s
15m0s
15m40s
15m25s
Kana encryption yakasanganiswa nekumanikidza uchishandisa lzo, inoita seizvi:
Kushanda maawa:
Kutanga 1
Kutanga 2
Kutanga 3
18m2s
18m15s
18m12s
5m13s
5m24s
5m20s
8m48s
9m3s
8m51s
Saizi yeyakagadziriswa repository yaive yakafanana pa13GB. Izvi zvinoreva kuti deduplication iri kushanda nemazvo. Zvakare, pane yakatomanikidzwa dhata, kushandisa lzo inopa inooneka mhedzisiro; maererano nenguva yakazara yekushanda, zbackup inosvika pedyo neduplicity/duplicati, asi inosara kumashure kweiyo yakavakirwa pa librsync ne2-5 nguva.
Zvakanakira zviri pachena - kuchengetedza dhisiki nzvimbo pane backup yekuchengetedza server. Kana iri repository yekutarisa maturusi, munyori wezbackup haavapi; zvinokurudzirwa kushandisa kukanganisa-kushivirira disk array kana gore mupi.
Pakazara, fungidziro yakanaka kwazvo, kunyangwe hazvo purojekiti yanga yakamira kweanenge makore matatu (yekupedzisira ficha chikumbiro chaive gore rapfuura, asi pasina mhinduro).
Kuedza borgbackup
Borgbackup iforogo yeattic, imwe system yakafanana nezbackup. Yakanyorwa mupython, ine runyorwa rwezvakawanda zvakafanana nezbackup, asi zvakare inogona:
- Mount backups kuburikidza nefuse
- Tarisa zviri mukati repository
- Shanda muclient-server mode
- Shandisa akasiyana compressor kune data, pamwe neheuristic kutsunga yemhando yefaira paunenge uchiimanikidza.
- 2 encryption sarudzo, aes uye blake
- Yakavakwa-mukati mudziyo we
performance checks
borgbackup benchmark crud ssh://backup_server/repo/path local_dir
Mibairo yaive yakadai:
CZ-BIG 96.51 MB/s (10 100.00 MB ese-zero mafaera: 10.36s)
RZ-BIG 57.22 MB/s (10 100.00 MB ese-zero mafaera: 17.48s)
UZ-BIG 253.63 MB/s (10 100.00 MB ese-zero mafaera: 3.94s)
DZ-BIG 351.06 MB/s (10 100.00 MB ese-zero mafaera: 2.85s)
CR-BIG 34.30 MB/s (10 100.00 MB mafaera asina kurongeka: 29.15s)
RR-BIG 60.69 MB/s (10 100.00 MB mafaera asina kurongeka: 16.48s)
UR-BIG 311.06 MB/s (10 100.00 MB mafaera asina kurongeka: 3.21s)
DR-BIG 72.63 MB/s (10 100.00 MB mafaera asina kurongeka: 13.77s)
CZ-MEDIUM 108.59 MB/s (1000 1.00 MB ese-zero mafaera: 9.21s)
RZ-MEDIUM 76.16 MB/s (1000 1.00 MB ese-zero mafaera: 13.13s)
UZ-MEDIUM 331.27 MB/s (1000 1.00 MB ese-zero mafaera: 3.02s)
DZ-MEDIUM 387.36 MB/s (1000 1.00 MB ese-zero mafaera: 2.58s)
CR-MEDIUM 37.80 MB/s (1000 1.00 MB mafaera asina kurongeka: 26.45s)
RR-MEDIUM 68.90 MB/s (1000 1.00 MB mafaera asina kurongeka: 14.51s)
UR-MEDIUM 347.24 MB/s (1000 1.00 MB mafaera asina kurongeka: 2.88s)
DR-MEDIUM 48.80 MB/s (1000 1.00 MB mafaera asina kurongeka: 20.49s)
CZ-SMALL 11.72 MB/s (10000 10.00 kB ese-zero mafaera: 8.53s)
RZ-SMALL 32.57 MB/s (10000 10.00 kB ese-zero mafaera: 3.07s)
UZ-SMALL 19.37 MB/s (10000 10.00 kB ese-zero mafaera: 5.16s)
DZ-SMALL 33.71 MB/s (10000 10.00 kB ese-zero mafaera: 2.97s)
CR-SMALL 6.85 MB/s (10000 10.00 kB mafaera asina kurongeka: 14.60s)
RR-SMALL 31.27 MB/s (10000 10.00 kB mafaera asina kurongeka: 3.20s)
UR-SMALL 12.28 MB/s (10000 10.00 kB mafaera asina kurongeka: 8.14s)
DR-SMALL 18.78 MB/s (10000 10.00 kB mafaera asina kurongeka: 5.32s)
Paunenge uchiyedzwa, compression heuristics ichashandiswa kuona iyo faira mhando (compression auto), uye mhedzisiro ichave inotevera:
Kutanga, ngatitarisei kuti inoshanda sei pasina encryption:
Kushanda maawa:
Kutanga 1
Kutanga 2
Kutanga 3
4m6s
4m10s
4m5s
56s
58s
54s
1m26s
1m34s
1m30s
Kana iwe ukagonesa repository mvumo (yakavimbika modhi), mibairo ichave pedyo:
Kushanda maawa:
Kutanga 1
Kutanga 2
Kutanga 3
4m11s
4m20s
4m12s
1m0s
1m3s
1m2s
1m30s
1m34s
1m31s
Pakavhurwa aes encryption, mhedzisiro haina kudzikira zvakanyanya:
Kutanga 1
Kutanga 2
Kutanga 3
4m55s
5m2s
4m58s
1m0s
1m2s
1m0s
1m49s
1m50s
1m50s
Uye kana iwe ukachinja aes kuti blake, mamiriro acho achavandudza zvachose:
Kushanda maawa:
Kutanga 1
Kutanga 2
Kutanga 3
4m33s
4m43s
4m40s
59s
1m0s
1m0s
1m38s
1m43s
1m40s
Sezvakaitika kune zbackup, saizi yekuchengetera yaive 13GB uye kunyange zvishoma zvishoma, izvo zvinowanzotarisirwa. Ndakafadzwa zvikuru nenguva yekumhanya; inofananidzwa nemhinduro dzakavakirwa pa librsync, ichipa hunyanzvi hwakakura. Ini zvakare ndakafadzwa nekugona kuseta akasiyana ma paramita kuburikidza nemamiriro ekunze, izvo zvinopa mukana wakakomba kana uchishandisa borgbackup mune otomatiki modhi. Ndakafadzwawo nemutoro panguva yekuchengetedza: kutonga nemutoro we processor, borgbackup inoshanda mu1 thread.
Pakanga pasina zvimwe zvakashata pakuishandisa.
restic test
Kunyangwe chokwadi chekuti restic mhinduro nyowani (vaviri vekutanga vaizivikanwa kumashure muna 2 nepamusoro), ine hunhu hwakanaka. Yakanyorwa muna Go.
Kana ichienzaniswa nezbackup, inopa zvakare:
- Kuongorora kutendeseka kweiyo repository (kusanganisira kutarisa muzvikamu).
- Rondedzero yakakura yemaprotocol anotsigirwa uye vanopa kuchengetedza backups, pamwe nerutsigiro rwerclone - rsync yemafu mhinduro.
- Kuenzanisa 2 backups pamwe chete.
- Kuisa repository kuburikidza nefuse.
Kazhinji, rondedzero yezvimiro iri padyo neborgbackup, mune dzimwe nzvimbo zvakanyanya, mune dzimwe zvishoma. Chimwe chezvimiro ndechekuti hapana nzira yekudzima encryption, uye saka makopi ekuchengetedza anogara akavharirwa. Ngatione mukuita izvo zvinogona kusvinwa kunze kwesoftware iyi:
Mibairo yaive yakadai:
Kushanda maawa:
Kutanga 1
Kutanga 2
Kutanga 3
5m25s
5m50s
5m38s
35s
38s
36s
1m54s
2m2s
1m58s
Mhedzisiro yekuita inofananidzwawo nersync-based solutions uye, kazhinji, padyo neborgbackup, asi CPU mutoro wakakwira (tambo dzakawanda dzinomhanya) uye sawtooth.
Zvingangodaro, purogiramu inogumira nekushanda kwe disk subsystem pane server yekuchengetedza data, sezvakange zvatove zvakadaro ne rsync. Saizi yekuchengetera yaive 13GB, senge zbackup kana borgbackup, pakanga pasina kuipa kuri pachena kana uchishandisa mhinduro iyi.
Mhinduro
Muchokwadi, vese vakakwikwidza vakawana mhedzisiro yakafanana, asi nemitengo yakasiyana. Borgbackup yakaitwa zvakanyanya kupfuura zvese, restic yaive inononoka zvishoma, zbackup haifanirwe kutanga kushandisa,
uye kana yatove kushandiswa, edza kuishandura kuita borgbackup kana restic.
zvakawanikwa
Iyo yakanyanya kuvimbisa mhinduro inoita kunge yakadzikama, nekuti... ndiye ane reshiyo yakanakisa yekugona kumhanyisa kumhanya, asi ngatisamhanye kune zvakajairwa mhedziso ikozvino.
Borgbackup haina kunyanya kuipa, asi zbackup ingangotsiviwa zviri nani. Ichokwadi, zbackup inogona kushandiswa kuona kuti mutemo we3-2-1 unoshanda. Semuenzaniso, kuwedzera kune (lib)rsync-based backup zvivakwa.
Chiziviso
Backup Chikamu 5: Kuedza bacula uye veeam backup yelinux
Negadziriro yeparutivi Chikamu 6: Kuenzanisa negadziriro yeparutivi Tools
Backup Chikamu 7: Mhedziso
Zvakatumirwa na: Pavel Demkovich
Source: www.habr.com