Nkhaniyi ifotokoza za pulogalamu yosunga zobwezeretsera yomwe, pophwanya mtsinje wa data kukhala zigawo zosiyana (chunks), imapanga malo osungira.
Zigawo zosungira zimatha kupinikizidwanso ndikubisidwa, ndipo chofunikira kwambiri - panthawi yobwereza mobwerezabwereza - kugwiritsidwanso ntchito.
Kope zosunga zobwezeretsera m'malo oterowo ndi mndandanda wazinthu zomwe zimalumikizidwa wina ndi mzake, mwachitsanzo, kutengera ntchito zosiyanasiyana za hashi.
Pali mayankho angapo ofanana, ndiyang'ana pa 3: zbackup, borgbackup ndi restic.
Zotsatira zoyembekezeredwa
Popeza ofunsira onse amafunikira kuti pakhale malo osungiramo zinthu mwanjira ina, chimodzi mwazinthu zofunika kwambiri ndikuyerekeza kukula kwa malo osungira. Momwemo, kukula kwake kuyenera kukhala kosapitilira 13 GB molingana ndi njira yovomerezeka, kapena kuchepera - kutengera kukhathamiritsa kwabwino.
Ndizofunikiranso kwambiri kuti mutha kupanga zosunga zobwezeretsera za mafayilo mwachindunji, osagwiritsa ntchito zosungira ngati phula, komanso kugwira ntchito ndi ssh/sftp popanda zida zowonjezera monga rsync ndi sshfs.
Khalidwe popanga ma backups:
- Kukula kwa chosungirako kudzakhala kofanana ndi kukula kwa zosintha, kapena kuchepera.
- Kulemera kwa CPU kumayembekezeredwa mukamagwiritsa ntchito kuponderezana ndi / kapena kubisa, ndipo netiweki yayikulu kwambiri ndi disk load ndizotheka ngati kusungitsa ndi/kapena kubisa kukuyenda pa seva yosunga zosunga zobwezeretsera.
- Ngati chosungiracho chawonongeka, cholakwika chochedwetsedwa ndichotheka popanga zosunga zobwezeretsera zatsopano komanso poyesa kubwezeretsa. Ndikofunikira kukonzekera njira zowonjezera kuti zitsimikizire kukhulupirika kwa malo osungiramo zinthu kapena kugwiritsa ntchito zida zomangidwira kuti muwone kukhulupirika kwake.
Kugwira ntchito ndi phula kumatengedwa ngati mtengo wofotokozera, monga momwe tawonetsera m'nkhani zam'mbuyomu.
Kuyesa zbackup
Njira yayikulu ya zbackup ndikuti pulogalamuyo imapeza madera olowera deta omwe ali ndi deta yomweyi, kenako amawakakamiza ndikuwalemba, ndikusunga malo aliwonse kamodzi kokha.
Deduplication imagwiritsa ntchito 64-bit ring hash function yokhala ndi zenera lotsetsereka kuti muwone machesi a byte-by-byte motsutsana ndi midadada yomwe ilipo (mofanana ndi momwe rsync imagwiritsira ntchito).
Multi-threaded lzma ndi lzo amagwiritsidwa ntchito popanikiza, ndi ma aes pobisa. Matembenuzidwe aposachedwa ali ndi kuthekera kochotsa deta yakale m'nkhokwe mtsogolo.
Pulogalamuyi imalembedwa mu C ++ yokhala ndi zodalira zochepa. Wolembayo akuwoneka kuti adauziridwa ndi njira ya unix, kotero pulogalamuyo imavomereza deta pa stdin popanga zosunga zobwezeretsera, ndikupanga njira yofananira ya data pa stdout pobwezeretsa. Chifukwa chake, zbackup itha kugwiritsidwa ntchito ngati "chomangira" chabwino kwambiri polemba mayankho anu osunga zobwezeretsera. Mwachitsanzo, wolemba nkhaniyi wagwiritsa ntchito pulogalamuyi ngati chida chachikulu chosungira makina apanyumba kuyambira pafupifupi 2014.
Kutulutsa kwa data kudzakhala phula wamba pokhapokha zitanenedwa mwanjira ina.
Tiyeni tiwone zotsatira zake:
Ntchitoyi idawunikidwa muzosankha za 2:
- chosungira chimapangidwa ndipo zbackup imayambitsidwa pa seva ndi deta yochokera, ndiye zomwe zili m'malo osungirako zimasamutsidwa ku seva yosungirako zosunga zobwezeretsera.
- chosungira chimapangidwa pa seva yosungira zosunga zobwezeretsera, zbackup imayambitsidwa kudzera pa ssh pa seva yosungira zosunga zobwezeretsera, ndipo deta imatumizidwa kwa iyo kudzera papaipi.
Zotsatira za njira yoyamba zinali motere: 43m11s - mukamagwiritsa ntchito chosungira chosasungidwa ndi lzma kompresa, 19m13s - pochotsa compressor ndi lzo.
Katundu pa seva yokhala ndi chidziwitso choyambirira anali motere (chitsanzo chokhala ndi lzma chikuwonetsedwa; ndi lzo panali pafupifupi chithunzi chomwecho, koma gawo la rsync linali pafupifupi kotala la nthawiyo):
N'zoonekeratu kuti ndondomeko yosunga zobwezeretsera yotereyi ndi yoyenera pazosintha zochepa komanso zazing'ono. Ndikofunikiranso kuchepetsa zbackup ku ulusi wa 1, apo ayi padzakhala katundu wambiri wa CPU, chifukwa Pulogalamuyi imagwira ntchito bwino m'mitundu yambiri. Katundu pa diski anali wocheperako, zomwe sizingawonekere ndi ssd-based disk subsystem. Mutha kuwonanso bwino kuyambika kwa njira yolumikizira deta yosungira ku seva yakutali; liwiro la ntchito likufanana ndi rsync wamba ndipo zimatengera magwiridwe antchito a disk subsystem ya seva yosungira zosunga zobwezeretsera. Kuipa kwa njirayi ndikusungirako malo osungiramo malo ndipo, chifukwa chake, kubwereza deta.
Zosangalatsa komanso zogwiritsidwa ntchito pochita ndi njira yachiwiri, kuyendetsa zbackup molunjika pa seva yosungirako zosunga zobwezeretsera.
Choyamba, tiwona ntchitoyo popanda kugwiritsa ntchito kubisa ndi lzma compressor:
Nthawi yothamanga ya mayeso aliwonse:
Launch 1
Launch 2
Launch 3
39 m45 ku
40 m20 ku
40 m3 ku
7 m36 ku
8 m3 ku
7 m48 ku
15 m35 ku
15 m48 ku
15 m38 ku
Ngati mutsegula kubisa pogwiritsa ntchito aes, zotsatira zake zili pafupi kwambiri:
Nthawi yogwiritsira ntchito pa data yomweyi, ndi kubisa:
Launch 1
Launch 2
Launch 3
43 m40 ku
44 m12 ku
44 m3 ku
8 m3 ku
8 m15 ku
8 m12 ku
15 m0 ku
15 m40 ku
15 m25 ku
Ngati kubisa kumaphatikizidwa ndi kukanikiza pogwiritsa ntchito lzo, zikuwoneka motere:
Maola ogwira ntchito:
Launch 1
Launch 2
Launch 3
18 m2 ku
18 m15 ku
18 m12 ku
5 m13 ku
5 m24 ku
5 m20 ku
8 m48 ku
9 m3 ku
8 m51 ku
Kukula kwa chosungiracho kunali kofanana ndi 13GB. Izi zikutanthauza kuti deduplication ikugwira ntchito bwino. Komanso, pazida zomwe zapanikizidwa kale, kugwiritsa ntchito lzo kumapereka chowoneka bwino; malinga ndi nthawi yonse yogwira ntchito, zbackup imayandikira kubwereza / kubwereza, koma imatsalira kumbuyo kwazomwe zimachokera ku librsync ndi 2-5 nthawi.
Ubwino ndiwodziwikiratu - kupulumutsa malo a disk pa seva yosungira zosunga zobwezeretsera. Ponena za zida zowunikira zosungira, wolemba zbackup samawapatsa; tikulimbikitsidwa kugwiritsa ntchito diski yololera zolakwika kapena wopereka mtambo.
Ponseponse, malingaliro abwino kwambiri, ngakhale kuti ntchitoyi yakhala ikuyimilira kwa zaka pafupifupi 3 (pempho lomaliza linali pafupifupi chaka chapitacho, koma popanda yankho).
Kuyesa borgbackup
Borgbackup ndi foloko ya attic, dongosolo lina lofanana ndi zbackup. Yolembedwa mu python, ili ndi mndandanda wazinthu zofanana ndi zbackup, koma kuwonjezera apo:
- Ikani ma backups kudzera pa fuse
- Onani zomwe zili m'nkhokwe
- Gwirani ntchito mu kasitomala-server mode
- Gwiritsani ntchito ma compressor osiyanasiyana pa data, komanso kutsimikiza kwamtundu wa fayilo mukamayikakamiza.
- 2 encryption options, aes ndi blake
- Chida chomangidwira cha
macheke ntchito
borgbackup benchmark crud ssh://backup_server/repo/path local_dir
Zotsatira zake zinali motere:
CZ-BIG 96.51 MB/s (10 100.00 MB mafayilo onse-ziro: 10.36s)
RZ-BIG 57.22 MB/s (10 100.00 MB mafayilo onse-ziro: 17.48s)
UZ-BIG 253.63 MB/s (10 100.00 MB mafayilo onse-ziro: 3.94s)
DZ-BIG 351.06 MB/s (10 100.00 MB mafayilo onse-ziro: 2.85s)
CR-BIG 34.30 MB/s (10 100.00 MB mafayilo achisawawa: 29.15s)
RR-BIG 60.69 MB/s (10 100.00 MB mafayilo achisawawa: 16.48s)
UR-BIG 311.06 MB/s (10 100.00 MB mafayilo achisawawa: 3.21s)
DR-BIG 72.63 MB/s (10 100.00 MB mafayilo achisawawa: 13.77s)
CZ-MEDIUM 108.59 MB/s (1000 1.00 MB mafayilo onse-ziro: 9.21s)
RZ-MEDIUM 76.16 MB/s (1000 1.00 MB mafayilo onse-ziro: 13.13s)
UZ-MEDIUM 331.27 MB/s (1000 1.00 MB mafayilo onse-ziro: 3.02s)
DZ-MEDIUM 387.36 MB/s (1000 1.00 MB mafayilo onse-ziro: 2.58s)
CR-MEDIUM 37.80 MB/s (1000 1.00 MB mafayilo achisawawa: 26.45s)
RR-MEDIUM 68.90 MB/s (1000 1.00 MB mafayilo achisawawa: 14.51s)
UR-MEDIUM 347.24 MB/s (1000 1.00 MB mafayilo achisawawa: 2.88s)
DR-MEDIUM 48.80 MB/s (1000 1.00 MB mafayilo achisawawa: 20.49s)
CZ-YANG'ONO 11.72 MB/s (10000 10.00 kB mafayilo onse-ziro: 8.53s)
RZ-YANG'ONO 32.57 MB/s (10000 10.00 kB mafayilo onse-ziro: 3.07s)
UZ-YANG'ONO 19.37 MB/s (10000 10.00 kB mafayilo onse-ziro: 5.16s)
DZ-YANG'ONO 33.71 MB/s (10000 10.00 kB mafayilo onse-ziro: 2.97s)
CR-WAMNG'ono 6.85 MB/s (10000 10.00 kB mafayilo achisawawa: 14.60s)
RR-YAMNG'ONO 31.27 MB/s (10000 10.00 kB mafayilo achisawawa: 3.20s)
UR-YANG'ONO 12.28 MB/s (10000 10.00 kB mafayilo achisawawa: 8.14s)
DR-SMALL 18.78 MB/s (10000 10.00 kB mafayilo achisawawa: 5.32s)
Mukayesa, ma compression heuristics adzagwiritsidwa ntchito kudziwa mtundu wa fayilo (compression auto), ndipo zotsatira zake zikhala motere:
Choyamba, tiyeni tiwone momwe zimagwirira ntchito popanda kubisa:
Maola ogwira ntchito:
Launch 1
Launch 2
Launch 3
4 m6 ku
4 m10 ku
4 m5 ku
56
58
54
1 m26 ku
1 m34 ku
1 m30 ku
Ngati mutsegula chilolezo chosungira (njira yovomerezeka), zotsatira zidzakhala pafupi:
Maola ogwira ntchito:
Launch 1
Launch 2
Launch 3
4 m11 ku
4 m20 ku
4 m12 ku
1 m0 ku
1 m3 ku
1 m2 ku
1 m30 ku
1 m34 ku
1 m31 ku
Pamene encryption ya aes idatsegulidwa, zotsatira zake sizinachepe kwambiri:
Launch 1
Launch 2
Launch 3
4 m55 ku
5 m2 ku
4 m58 ku
1 m0 ku
1 m2 ku
1 m0 ku
1 m49 ku
1 m50 ku
1 m50 ku
Ndipo ngati mutasintha ma aes kukhala blake, zinthu zikhala bwino:
Maola ogwira ntchito:
Launch 1
Launch 2
Launch 3
4 m33 ku
4 m43 ku
4 m40 ku
59
1 m0 ku
1 m0 ku
1 m38 ku
1 m43 ku
1 m40 ku
Monga momwe zilili ndi zbackup, kukula kwa chosungirako kunali 13GB komanso ngakhale kucheperako, zomwe zimayembekezeredwa. Ndinasangalala kwambiri ndi nthawi yothamanga; ikufanana ndi mayankho okhudzana ndi librsync, kupereka mphamvu zambiri. Ndidakondweranso ndi kuthekera kokhazikitsa magawo osiyanasiyana kudzera pazosintha zachilengedwe, zomwe zimapereka mwayi waukulu mukamagwiritsa ntchito borgbackup mumachitidwe odziwikiratu. Ndinakondweranso ndi katunduyo panthawi yosunga zobwezeretsera: kuweruza ndi purosesa katundu, borgbackup imagwira ntchito mu ulusi wa 1.
Panalibe kuipa kwapadera poigwiritsa ntchito.
kuyesa kwa restic
Ngakhale kuti restic ndi njira yatsopano (oyamba awiri adadziwika kale mu 2 ndi kupitirira), ili ndi makhalidwe abwino. Zolembedwa mu Go.
Poyerekeza ndi zbackup, imaperekanso:
- Kuwona kukhulupirika kwa chosungira (kuphatikiza kuwunika magawo).
- Mndandanda waukulu wa ma protocol omwe amathandizidwa ndi othandizira kusunga zosunga zobwezeretsera, komanso kuthandizira kwa rclone - rsync kwa mayankho amtambo.
- Kuyerekeza 2 zosunga zobwezeretsera wina ndi mzake.
- Kuyika chosungiramo kudzera mwa fuse.
Nthawi zambiri, mndandanda wazinthu uli pafupi kwambiri ndi borgbackup, m'malo ena mochulukirapo, ena mochepera. Chimodzi mwazinthu ndichakuti palibe njira yoletsera kubisa, chifukwa chake zosunga zobwezeretsera nthawi zonse zimasungidwa. Tiyeni tiwone zomwe zingathe kufinyidwa mu pulogalamuyi:
Zotsatira zake zinali motere:
Maola ogwira ntchito:
Launch 1
Launch 2
Launch 3
5 m25 ku
5 m50 ku
5 m38 ku
35
38
36
1 m54 ku
2 m2 ku
1 m58 ku
Zotsatira zake zimafanananso ndi mayankho a rsync ndipo, kawirikawiri, pafupi kwambiri ndi borgbackup, koma katundu wa CPU ndi wapamwamba (ulusi wambiri ukuyenda) ndi sawtooth.
Mwachiwonekere, pulogalamuyi imakhala yochepa ndi machitidwe a disk subsystem pa seva yosungirako deta, monga momwe zinalili kale ndi rsync. Kukula kosungirako kunali 13GB, monga zbackup kapena borgbackup, panalibe zovuta zodziwikiratu mukamagwiritsa ntchito yankholi.
Zotsatira
M'malo mwake, onse ofuna kusankhidwa adapeza zotsatira zofanana, koma pamitengo yosiyana. Borgbackup idachita bwino koposa zonse, restic inali pang'onopang'ono, zbackup mwina siyiyenera kuyamba kugwiritsa ntchito,
ndipo ngati ikugwiritsidwa ntchito kale, yesani kuyisintha kukhala borgbackup kapena restic.
anapezazo
Njira yodalirika kwambiri ikuwoneka ngati yokhazikika, chifukwa ... ndi iye amene ali ndi chiΕ΅erengero chabwino kwambiri cha kuthekera kwa liwiro la ntchito, koma pakali pano tisathamangire ku lingaliro lonse.
Borgbackup sizoyipa kwambiri, koma zbackup mwina imasinthidwa bwino. Zowona, zbackup itha kugwiritsidwabe ntchito kuwonetsetsa kuti lamulo la 3-2-1 likugwira ntchito. Mwachitsanzo, kuwonjezera pa (lib)rsync-based backup resources.
Kulengeza
Zosunga zobwezeretsera, gawo 5: Kuyesa bacula ndi veeam zosunga zobwezeretsera za linux
Zosunga zobwezeretsera Gawo 6: Kuyerekeza Zida zosunga zobwezeretsera
Zosunga zobwezeretsera Gawo 7: Mapeto
Wolemba: Pavel Demkovich
Source: www.habr.com