Eli nqaku liza kuthathela ingqalelo isoftware yogcino oluthi, ngokwaphula umjelo wedatha ibe ngamacandelo ahlukeneyo (iingqekembe), yenza indawo yokugcina.
Amacandelo ogcino anokunyanzeliswa ngakumbi kwaye abethelwe, kwaye okona kubaluleke kakhulu - ngexesha leenkqubo zokugcinwa ngokuphindaphindiweyo - zisetyenziswe kwakhona.
Ikopi yokugcina kwindawo yogcino luluhlu olunegama lamacandelo adityaniswe omnye komnye, umzekelo, ngokusekelwe kwimisebenzi eyahlukeneyo ye-hash.
Kukho izisombululo ezininzi ezifanayo, ndiza kugxila kwi-3: i-zbackup, i-borgbackup kunye ne-restic.
Iziphumo ezilindelekileyo
Kuba bonke abafaki zicelo bafuna ukwenziwa kwendawo yokugcina izinto ngendlela enye okanye enye, enye yezona zinto zibalulekileyo ziya kuba kukuqikelela ubungakanani bendawo yokugcina. Ngokufanelekileyo, ubukhulu bayo kufuneka bungabi ngaphezu kwe-13 GB ngokwendlela eyamkelekileyo, okanye nangaphantsi - kuxhomekeke ekuphuculweni kakuhle.
Kukwanqweneleka kakhulu ukukwazi ukwenza iikopi zogcino lwefayile ngokuthe ngqo, ngaphandle kokusebenzisa oovimba njenge tar, kunye nokusebenza nge-ssh/sftp ngaphandle kwezixhobo ezongezelelweyo ezifana ne-rsync kunye ne-sshfs.
Ukuziphatha xa usenza i-backups:
- Ubungakanani bogcino buyalingana nobukhulu botshintsho, okanye ngaphantsi.
- Umthwalo onzima we-CPU ulindeleke xa usebenzisa ucinezelo kunye/okanye uguqulelo oluntsonkothileyo, kwaye inethwekhi ephezulu kakhulu kunye nomthwalo wedisk kunokwenzeka ukuba ugcino kunye/okanye inkqubo yoguqulelo oluntsonkothileyo iyasebenza kwiseva yokugcina yogcino.
- Ukuba indawo yokugcina yonakele, impazamo ebambezelekayo inokwenzeka xa usenza ii-backups ezintsha naxa uzama ukubuyisela. Kuyimfuneko ukucwangcisa amanyathelo ongezelelweyo ukuqinisekisa ukuthembeka kwendawo yokugcina okanye ukusebenzisa izixhobo ezakhelwe ukujonga ukuthembeka kwayo.
Ukusebenza ngetha kuthathwa njengexabiso lereferensi, njengoko kubonisiwe kwelinye lamanqaku angaphambili.
Uvavanyo lwe-zbackup
Indlela eqhelekileyo ye-zbackup kukuba inkqubo ifumana kwiindawo zokungena zedatha yegalelo eziqulethe idatha efanayo, emva koko uzicinezele kwaye uzifihle, ugcine indawo nganye kanye kuphela.
Ukuphinda-phinda kusebenzisa i-64-bit ring hash function with sliding window ukujonga i-byte-by-byte match ngokuchasene neebhloko zedatha esele zikhona (ezifana nendlela i-rsync eyenza ngayo).
I-lzma enemisonto emininzi kunye ne-lzo zisetyenziselwa ucinezelo, kunye nee-ees zoguqulelo oluntsonkothileyo. Iinguqulelo zamva nje ziyakwazi ukucima idatha endala kwindawo yokugcina kwixa elizayo.
Inkqubo ibhalwe kwi-C ++ ngokuxhomekeka okuncinci. Umbhali wayebonakala ephefumlelwe yi-unix-indlela, ngoko ke inkqubo yamkela idatha kwi-stdin xa idala i-backups, ivelisa umzila wedatha ofanayo kwi-stdout xa ubuyisela. Ke, i-zbackup ingasetyenziswa "njengebhloko yokwakha" elungileyo xa ubhala ezakho izisombululo zogcino. Umzekelo, umbhali wenqaku usebenzise le nkqubo njengesona sixhobo siphambili sogcino koomatshini basekhaya ukusukela malunga nowama-2014.
Uluhlu lwedatha luya kuba yitha eqhelekileyo ngaphandle kokuba kuchazwe ngenye indlela.
Makhe sibone ukuba zithini iziphumo:
Umsebenzi uhlolwe kwiinketho ezi-2:
- i-repository yenziwa kwaye i-zbackup iqaliswe kumncedisi kunye nedatha yomthombo, emva koko imixholo yendawo yokugcina idluliselwe kwiseva yokugcina yokugcina.
- i-repository yenziwe kwi-server yokugcina i-backup, i-zbackup iqaliswe nge-ssh kwi-server yokugcina yogcino, kwaye idatha ithunyelwa kuyo ngombhobho.
Iziphumo zokhetho lokuqala zazimi ngolu hlobo lulandelayo: 43m11s - xa usebenzisa i-repository engabhalwanga kunye ne-compressor ye-lzma, i-19m13s - xa ubeka i-compressor nge-lzo.
Umthwalo kumncedisi ngedatha yoqobo wawumi ngolu hlobo lulandelayo (umzekelo nge-lzma ubonisiwe, kunye ne-lzo kwakukho umfanekiso ofanayo, kodwa isabelo se-rsync sasimalunga nekota yexesha):
Kucacile ukuba inkqubo enjalo yokugcina ilungele kuphela utshintsho olunqabileyo kunye noluncinci. Kukwacetyiswa kakhulu ukunciphisa i-zbackup ukuya kumsonto omnye, kungenjalo kuya kubakho umthwalo ophezulu kakhulu we-CPU, kuba Inkqubo ilungile kakhulu ekusebenzeni kwimisonto emininzi. Umthwalo kwidiski wawumncinci, owawungayi kubonakala ngenkqubo ephantsi yediski esekwe kwi-ssd. Ungaphinda ubone ngokucacileyo isiqalo senkqubo yongqamaniso lwedatha yokugcina kwiseva ekude; Ukungalungi kwale ndlela kukugcinwa kwendawo yokugcina indawo kwaye, ngenxa yoko, ukuphindaphinda kwedatha.
Okubangel 'umdla ngakumbi kwaye okusebenzayo ekusebenzeni lukhetho lwesibini, ukuqhuba i-zbackup ngqo kwi-server yokugcina yogcino.
Okokuqala, siya kuvavanya ukusebenza ngaphandle kokusebenzisa i-encryption kunye ne-lzma compressor:
Ixesha lokuqhuba lovavanyo ngalunye:
Yazisa 1
Yazisa 2
Yazisa 3
39m45s
40m20s
40m3s
7m36s
8m3s
7m48s
15m35s
15m48s
15m38s
Ukuba wenza uguqulelo oluntsonkothileyo usebenzisa ii-aes, iziphumo zisondele kakhulu:
Ixesha lokusebenza kwidatha efanayo, ngoguqulelo oluntsonkothileyo:
Yazisa 1
Yazisa 2
Yazisa 3
43m40s
44m12s
44m3s
8m3s
8m15s
8m12s
15m0s
15m40s
15m25s
Ukuba uguqulelo oluntsonkothileyo ludityaniswe noxinzelelo usebenzisa i-lzo, ibonakala ngolu hlobo:
Iiyure zokusebenza:
Yazisa 1
Yazisa 2
Yazisa 3
18m2s
18m15s
18m12s
5m13s
5m24s
5m20s
8m48s
9m3s
8m51s
Ubungakanani bovimba obufunyenweyo babulingana kwaye buba yi-13GB. Oku kuthetha ukuba ukuphindaphinda kusebenza ngokuchanekileyo. Kwakhona, kwidatha esele ixinzelelwe, ukusebenzisa i-lzo inika umphumo obonakalayo ngokwexesha elipheleleyo lokusebenza, i-zbackup isondela kufutshane ne-duplicity / duplicati, kodwa i-lags emva kwezo zisekelwe kwi-librsync ngamaxesha angama-2-5.
Iinzuzo zicacile - ukugcina indawo kwidisk kwi-server yokugcina i-backup. Ngokuphathelele izixhobo zokujonga indawo yokugcina, umbhali we-zbackup akaboneleli kubo;
Ngokubanzi, uluvo oluhle kakhulu, nangona iprojekthi imile malunga ne-3 iminyaka (isicelo sokugqibela senqaku sasimalunga nonyaka odlulileyo, kodwa ngaphandle kwempendulo).
Ukuvavanya iborgbackup
I-Borgbackup yifolokhwe ye-attic, enye inkqubo efana ne-zbackup. Ibhalwe kwipython, inoluhlu lwezakhono ezifanayo ne-zbackup, kodwa ukongeza unako:
- Nyusa i-backups nge-fuse
- Jonga imixholo yokugcina
- Sebenza kwimowudi ye-client-server
- Sebenzisa iicompressors ezahlukeneyo zedatha, kunye nokumiselwa kwe-heuristic yohlobo lwefayile xa uyicinezela.
- Iinketho ezi-2 zofihlo, ii-aes kunye no-blake
- Isixhobo esakhelwe ngaphakathi se
iitshekhi zokusebenza
iborgbackup benchmark crud ssh://backup_server/repo/path local_dir
Iziphumo zibe ngolu hlobo lulandelayo:
CZ-BIG 96.51 MB/s (10 100.00 MB zonke-zero iifayile: 10.36s)
RZ-BIG 57.22 MB/s (10 100.00 MB zonke-zero iifayile: 17.48s)
UZ-BIG 253.63 MB/s (10 100.00 MB zonke-zero iifayile: 3.94s)
DZ-BIG 351.06 MB/s (10 100.00 MB zonke-zero iifayile: 2.85s)
I-CR-BIG 34.30 MB/s (10 100.00 MB iifayile random: 29.15s)
RR-BIG 60.69 MB/s (10 100.00 MB iifayile random: 16.48s)
UR-BIG 311.06 MB/s (10 100.00 MB iifayile random: 3.21s)
DR-BIG 72.63 MB/s (10 100.00 MB iifayile random: 13.77s)
I-CZ-MEDIUM 108.59 MB/s (1000 1.00 MB zonke-zero iifayile: 9.21s)
I-RZ-MEDIUM 76.16 MB/s (1000 1.00 MB zonke-zero iifayile: 13.13s)
UZ-MEDIUM 331.27 MB/s (1000 1.00 MB zonke-zero iifayile: 3.02s)
I-DZ-MEDIUM 387.36 MB/s (1000 1.00 MB zonke-zero iifayile: 2.58s)
I-CR-MEDIUM 37.80 MB/s (1000 1.00 MB iifayile random: 26.45s)
I-RR-MEDIUM 68.90 MB/s (1000 1.00 MB iifayile random: 14.51s)
I-UR-MEDIUM 347.24 MB/s (1000 1.00 MB iifayile random: 2.88s)
I-DR-MEDIUM 48.80 MB/s (1000 1.00 MB iifayile random: 20.49s)
I-CZ-ENCINANE 11.72 MB/s (10000 10.00 kB zonke-zero iifayile: 8.53s)
RZ-NCINCI 32.57 MB/s (10000 10.00 kB zonke-zero iifayile: 3.07s)
UZ-EMNCINANE 19.37 MB/s (10000 10.00 kB zonke-zero iifayile: 5.16s)
I-DZ-SMALL 33.71 MB/s (10000 10.00 kB zonke-zero iifayile: 2.97s)
I-CR-SMALL 6.85 MB/s (10000 10.00 kB iifayile random: 14.60s)
I-RR-SMALL 31.27 MB/s (10000 10.00 kB iifayile random: 3.20s)
UR-NCINCI 12.28 MB/s (10000 10.00 kB iifayile random: 8.14s)
I-DR-SMALL 18.78 MB/s (10000 10.00 kB iifayile random: 5.32s)
Xa kuhlolwa, i-compression heuristics iya kusetyenziswa ukumisela uhlobo lwefayile (compression auto), kwaye iziphumo ziya kuba ngolu hlobo lulandelayo:
Okokuqala, makhe sijonge ukuba isebenza njani ngaphandle koguqulelo oluntsonkothileyo:
Iiyure zokusebenza:
Yazisa 1
Yazisa 2
Yazisa 3
4m6s
4m10s
4m5s
56s
58s
54s
1m26s
1m34s
1m30s
Ukuba wenza ugunyaziso lovimba (indlela engqiniweyo), iziphumo ziya kuvalwa:
Iiyure zokusebenza:
Yazisa 1
Yazisa 2
Yazisa 3
4m11s
4m20s
4m12s
1m0s
1m3s
1m2s
1m30s
1m34s
1m31s
Xa i-aes encryption yenziwe yasebenza, iziphumo azizange zibe mandundu:
Yazisa 1
Yazisa 2
Yazisa 3
4m55s
5m2s
4m58s
1m0s
1m2s
1m0s
1m49s
1m50s
1m50s
Kwaye ukuba utshintshe i-aes ku-blake, imeko iya kuphucuka ngokupheleleyo:
Iiyure zokusebenza:
Yazisa 1
Yazisa 2
Yazisa 3
4m33s
4m43s
4m40s
59s
1m0s
1m0s
1m38s
1m43s
1m40s
Njengakwimeko ye-zbackup, ubungakanani bendawo yokugcina yayiyi-13GB kwaye nangaphantsi kancinci, ekulindeleke ngokubanzi. Ndonwabile kakhulu ngexesha lokuhamba; ithelekiseka nezisombululo ezisekwe kwi-librsync, ibonelela ngezakhono ezibanzi. Ndikwavuyiswa kukukwazi ukuseta iiparitha ezahlukeneyo ngokuguquguquka kokusingqongileyo, okunika inzuzo enkulu xa usebenzisa iborgbackup kwimowudi ezenzekelayo. Ndandivuyiswa kwakhona ngumthwalo ngexesha lokugcina: ukugweba ngomthwalo weprosesa, i-borgbackup isebenza kwi-thread ye-1.
Kwakungekho nto ingalunganga xa uyisebenzisa.
uvavanyo lokuphumla
Ngaphandle kwento yokuba i-restic isisisombululo esitsha ngokufanelekileyo (abagqatswa aba-2 bokuqala babesaziwa ngo-2013 nangaphezulu), ineempawu ezintle kakhulu. Ibhalwe kuGo.
Xa kuthelekiswa ne-zbackup, inika kwakhona:
- Ukujonga imfezeko yendawo yokugcina (kubandakanywa nokukhangela kwiindawo).
- Uluhlu olukhulu lweeprotocol ezixhaswayo kunye nababoneleli bokugcina ii-backups, kunye nenkxaso ye-rclone-rsync yezisombululo zamafu.
- Ukuthelekisa ii-backups ezi-2 kunye nomnye.
- Ukunyuswa kwendawo yokugcina nge-fuse.
Ngokubanzi, uluhlu lweempawu lusondele kakhulu kwi-borgbackup, kwezinye iindawo ngaphezulu, kwezinye ngaphantsi. Enye yeempawu kukuba akukho ndlela yokuvala uguqulelo oluntsonkothileyo, kwaye ke iikopi ezigcinayo ziya kuhlala zifihliwe. Makhe sibone ukuba yintoni enokucudiswa kule software:
Iziphumo zibe ngolu hlobo lulandelayo:
Iiyure zokusebenza:
Yazisa 1
Yazisa 2
Yazisa 3
5m25s
5m50s
5m38s
35s
38s
36s
1m54s
2m2s
1m58s
Iziphumo zokusebenza zikwathelekiswa nezisombululo ezisekelwe kwi-rsync kwaye, ngokubanzi, zisondele kakhulu kwi-borgbackup, kodwa umthwalo we-CPU uphezulu (iintambo ezininzi ezisebenzayo) kunye ne-sawtooth.
Okunokwenzeka, inkqubo inqunyelwe kukusebenza kwenkqubo esezantsi yedisk kwiseva yogcino lwedatha, njengoko kwakunjalo nge-rsync. Ubungakanani bendawo yokugcina yayiyi-13GB, njenge-zbackup okanye i-borgbackup, kwakungekho bubi obucacileyo xa usebenzisa esi sisombululo.
Iziphumo
Enyanisweni, bonke abaviwa bafumana iziphumo ezifanayo, kodwa ngamaxabiso ahlukeneyo. I-Borgbackup yenziwe ngcono kuzo zonke, i-restic yayicotha kancinci, i-zbackup ayifanelanga ukuqalisa ukuyisebenzisa,
kwaye ukuba sele isetyenziswa, zama ukuyitshintsha ibe yiborgbackup okanye irestic.
ezifunyanisiweyo
Esona sisombululo sithembisayo sibonakala sisimo esizinzileyo, kuba... nguye onomlinganiselo ongcono wesakhono kwisantya sokusebenza, kodwa masingangxameli kwizigqibo ngokubanzi okwangoku.
I-Borgbackup ayikho mbi ngakumbi, kodwa i-zbackup mhlawumbi ingcono endaweni. Enyanisweni, i-zbackup isenokusetyenziswa ukuqinisekisa ukuba umthetho we-3-2-1 uyasebenza. Umzekelo, ukongeza kwi (lib) rsync-based backup amaziko.
Isibhengezo
I-Backup Icandelo lesi-5: Ukuvavanya i-bacula kunye ne-veeam backup ye-linux
Backup Icandelo 6: Ukuthelekisa Tools Backup
Ugcino lweCandelo 7: Izigqibo
Ithunyelwe ngu: UPavel Demkovich
umthombo: www.habr.com