Ikhophi yasenqolobaneni Ingxenye 3: Buyekeza Nokuhlolwa kokuphindaphindeka, impinda

Leli nothi lixoxa ngamathuluzi okulondoloza enza izipele ngokudala izingobo zomlando kuseva eyisipele.

Phakathi kwalezo ezihlangabezana nezidingo ukuphindaphindeka (okunokuxhumana okuhle ngendlela ye-deja dup) kanye ne-duplicati.

Elinye ithuluzi lokusekelayo eliphawuleka kakhulu i-dar, kodwa njengoba linohlu olubanzi kakhulu lwezinketho - indlela yokuhlola ihlanganisa kancane u-10% walokho ekwaziyo ukukwenza - asiyihloli njengengxenye yomjikelezo wamanje.

Imiphumela elindelwe

Njengoba womabili amakhandidethi enza izingobo zomlando ngendlela eyodwa noma enye, itiyela elivamile lingasetshenziswa njengomhlahlandlela.

Ukwengeza, sizohlola ukuthi isitoreji sedatha kuseva yesitoreji sithuthukiswa kanjani ngokudala amakhophi ayisipele aqukethe kuphela umehluko phakathi kwekhophi egcwele nesimo samanje samafayela, noma phakathi kwezingobo zomlando zangaphambilini nezamanje (okwenyukayo, okunciphayo, njll.) .

Ukuziphatha lapho udala izipele:

1. Inani elincane lamafayela kuseva yokugcina ikhophi yasenqolobaneni (eliqhathaniswa nenani lamakhophi ayisipele noma usayizi wedatha ku-GB), kodwa usayizi wawo mkhulu kakhulu (amashumi kuya kumakhulu amamegabhayithi).
2. Usayizi wekhosombe uzobandakanya izinguquko kuphela - azikho izimpinda ezizogcinwa, ngakho usayizi wendawo yokugcina uzoba mncane kune-software esekelwe ku-rsync.
3. Lindela umthwalo osindayo we-CPU uma usebenzisa ukuminyanisa kanye/noma ukubethela, kanye nenethiwekhi okungenzeka ukuthi iphezulu impela nomthwalo wediski uma ukugcinwa kwengobo yomlando kanye/noma inqubo yokubethela isebenza kuseva yokugcina isipele.

Masisebenzise umyalo olandelayo njengenani lesithenjwa:

cd /src/dir; tar -cf - * | ssh backup_server "cat > /backup/dir/archive.tar"

Imiphumela yokubulawa yaba kanje:

Isikhathi sokwenza 3m12s. Kungabonakala ukuthi isivinini sinqunyelwe i-subsystem yediski yesiphakeli sokulondoloza isipele, njengesibonelo nge rsync. Ngokushesha nje kancane, ngoba... ukurekhoda kuya kufayela elilodwa.

Futhi, ukuhlola ukucindezelwa, ake sisebenzise inketho efanayo, kodwa vumela ukucindezela ohlangothini lweseva eyisipele:

cd /src/dir; tar -cf - * | ssh backup_server "gzip > /backup/dir/archive.tgz"

Imiphumela yilena:

Isikhathi sokwenza 10m11s. Okungenzeka kakhulu ukuthi i-bottleneck iyi-compressor yokugeleza okukodwa ekugcineni kokwamukela.

Umyalo ofanayo, kodwa ngokucindezelwa kudluliselwe kuseva ngedatha yasekuqaleni ukuze kuhlolwe i-hypothesis yokuthi i-bottleneck iyi-compressor enomucu owodwa.

cd /src/dir; tar -czf - * | ssh backup_server "cat > /backup/dir/archive.tgz"

Kwavela kanje:

Isikhathi sokubulala sasingu-9m37s. Umthwalo kumgogodla owodwa nge-compressor ubonakala ngokucacile, ngoba Isivinini sokudlulisa inethiwekhi nomthwalo ohlelweni olungaphansi lwediski yomthombo kuyefana.

Ukuze uhlole ukubethela, ungasebenzisa i-openssl noma i-gpg ngokuxhuma umyalo owengeziwe openssl noma gpg epayipini. Ukuze uthole ireferensi kuzoba nomyalo ofana nalo:

cd /src/dir; tar -cf - * | ssh backup_server "gzip | openssl enc -e -aes256 -pass pass:somepassword -out /backup/dir/archive.tgz.enc"

Imiphumela iphume kanje:

Isikhathi sokubulala siphenduke ama-10m30s, njengoba izinqubo ezi-2 bezisebenza ohlangothini olutholayo - ibhodlela liphinde libe yi-compressor enomucu owodwa, kanye nekhanda elincane lokubethela.

I-UPS: Ngesicelo se-bliznezz ngengeza izivivinyo nge-pigz. Uma usebenzisa i-compressor kuphela, kuzothatha ama-6m30s, uma futhi wengeza ukubethela, kungaba ngu-7m. Ukucwilisa kugrafu engezansi kuyinqolobane yediski engagudluziwe:

Ukuhlola okuyimpinda

I-Duplicity isofthiwe ye-python yokwenza ikhophi yasenqolobaneni ngokudala izingobo zomlando ezibethelwe ngefomethi ye-tar.

Kuzingobo zomlando ezikhulayo, i-librsync isetshenziswa, ngakho-ke ungalindela ukuziphatha okuchazwe kuyo okuthunyelwe kwangaphambilini ochungechungeni.

Izipele zingabethelwa futhi zisayinwe kusetshenziswa i-gnupg, ebalulekile uma usebenzisa abahlinzeki abahlukene ukuze ulondoloze izipele (s3, backblaze, gdrive, njll.)

Ake sibone ukuthi ithini imiphumela:

Lena imiphumela esiyithole lapho sisebenza ngaphandle kokubethela

umonakalisi

Isikhathi sokusebenza sokuhlolwa ngakunye:

Yethula 1
Yethula 2
Yethula 3

16 m33s
17 m20s
16 m30s

8 m29s
9 m3s
8 m45s

5 m21s
6 m04s
5 m53s

Futhi nansi imiphumela uma ukubethela kwe-gnupg kunikwe amandla, ngosayizi wokhiye wamabhithi angu-2048:

Isikhathi sokusebenza kudatha efanayo, ngokubethela:

Yethula 1
Yethula 2
Yethula 3

17 m22s
17 m32s
17 m28s

8 m52s
9 m13s
9 m3s

5 m48s
5 m40s
5 m30s

Usayizi we-block waboniswa - 512 megabytes, obonakala ngokucacile emagrafu; Umthwalo weprosesa wahlala ku-50%, okusho ukuthi uhlelo alusebenzisi okungaphezu kwe-core processor eyodwa.

Umgomo wokusebenza kohlelo nawo ubonakala ngokucacile: bathatha ucezu lwedatha, balucindezela, baluthumela kuseva yesitoreji esiyisipele, esingahamba kancane.
Esinye isici isikhathi esibikezelwayo sokusebenza sohlelo, esincike kuphela kusayizi wedatha eshintshiwe.

Ukunika amandla ukubethela akuzange kusikhulise kakhulu isikhathi sokusebenza sohlelo, kodwa kunyuse umthwalo weprosesa cishe ngo-10%, okungaba ibhonasi enhle impela.

Ngeshwa, lolu hlelo alukwazanga ukubona kahle isimo ngokuqanjwa kabusha kohla lwemibhalo, futhi usayizi wenqolobane owavela wavela walingana nosayizi wezinguquko (okungukuthi, wonke u-18GB), kodwa ikhono lokusebenzisa iseva engathenjwa ukwenza isipele ngokucacile. ihlanganisa lokhu kuziphatha.

Ukuhlola okuyimpinda

Le softhiwe ibhalwe nge-C# futhi isebenza kusetshenziswa iqoqo lemitapo yolwazi evela ku-Mono. Kukhona i-GUI kanye nenguqulo ye-CLI.

Uhlu olulinganiselwe lwezici eziyinhloko lufana nokuphindaphinda, kufaka phakathi abahlinzeki besitoreji abahlukahlukene, nokho, ngokungafani nokuphindaphinda, izici eziningi zitholakala ngaphandle kwamathuluzi ezinkampani zangaphandle. Ukuthi lokhu ukuhlanganisa noma ukususa kuncike esimweni esithile, kodwa kwabaqalayo, cishe kulula kakhulu ukuba nohlu lwazo zonke izici eziphambi kwabo ngesikhathi esisodwa, kunokuba kufakwe amaphakheji engeziwe e-python, njengoba kunjalo. icala ngokuphindiwe.

Enye i-nuance encane - uhlelo lubhala ngenkuthalo isizindalwazi se-sqlite sendawo egameni lomsebenzisi oqala isipele, ngakho-ke udinga ukuqinisekisa ukuthi i-database edingekayo icaciswe kahle isikhathi ngasinye lapho inqubo iqalwa kusetshenziswa i-cli. Uma usebenza nge-GUI noma i-WEBGUI, imininingwane izofihlwa kumsebenzisi.

Ake sibone ukuthi yiziphi izinkomba ezingakhiqizwa yilesi sixazululo:

Uma uvala ukubethela (futhi i-WEBGUI ingakuncomi ukwenza lokhu), imiphumela imi kanje:

Amahora okusebenza:

Yethula 1
Yethula 2
Yethula 3

20 m43s
20 m13s
20 m28s

5 m21s
5 m40s
5 m35s

7 m36s
7 m54s
7 m49s

Ngokubethela kunikwe amandla, kusetshenziswa ama-aes, kubukeka kanjena:

Amahora okusebenza:

Yethula 1
Yethula 2
Yethula 3

29 m9s
30 m1s
29 m54s

5 m29s
6 m2s
5 m54s

8 m44s
9 m12s
9 m1s

Futhi uma usebenzisa uhlelo lwangaphandle lwe-gnupg, imiphumela elandelayo iyaphuma:

Yethula 1
Yethula 2
Yethula 3

26 m6s
26 m35s
26 m17s

5 m20s
5 m48s
5 m40s

8 m12s
8 m42s
8 m15s

Njengoba ubona, uhlelo lungasebenza emiculweni eminingi, kodwa lokhu akulwenzi lube yisixazululo esikhiqiza kakhudlwana, futhi uma uqhathanisa umsebenzi wokubethela, sethula uhlelo lwangaphandle.
kubonakale kushesha kunokusebenzisa umtapo wolwazi ovela kusethi ye-Mono. Lokhu kungenzeka kungenxa yokuthi uhlelo lwangaphandle luthuthukiswe kakhulu.

Enye into enhle kwaba iqiniso lokuthi ubukhulu benqolobane buthatha ngokufana ncamashi nedatha yangempela eshintshiwe, i.e. i-duplicati ithole ukuqanjwa kabusha kohla lwemibhalo futhi yaphatha lesi simo ngendlela efanele. Lokhu kungabonakala lapho kuqaliswa ukuhlolwa kwesibili.

Sekukonke, okuvelayo okuhle kohlelo, okuhlanganisa nokuba nobungane abasanda kuhlanganyela.

Imiphumela

Womabili amakhandidethi asebenze kancane, kodwa ngokuvamile, uma kuqhathaniswa netiyela elivamile, kukhona inqubekelaphambili, okungenani nge-duplicati. Intengo yentuthuko enjalo nayo icacile - umthwalo obonakalayo
iprosesa. Ngokuvamile, akukho ukuphambuka okukhethekile ekubikezeleni imiphumela.

okutholakele

Uma ungadingi ukujaha noma kuphi, futhi ube neprosesa eyisipele, noma yiziphi izixazululo ezicatshangelwayo zizokwenza, noma kunjalo, mningi umsebenzi osuwenziwe okungafanele uphindwe ngokubhala imibhalo yokugoqa phezu kwetiyela. . Ukuba khona kokubethela kuyimpahla edingeka kakhulu uma iseva yokugcina amakhophi ayisipele ingakwazi ukwethembeka ngokugcwele.

Uma kuqhathaniswa nezixazululo ezisekelwe rsync - ukusebenza kungaba kubi izikhathi eziningana, naphezu kweqiniso lokuthi ngendlela ehlanzekile itiyela lisebenze 20-30% ngokushesha kune-rsync.
Kukhona ukonga ngosayizi wendawo yokugcina, kodwa kuphela nge-duplicati.

Isimemezelo

Ikhophi yasenqolobaneni, ingxenye 1: Kungani kudingeka ukwenza isipele, uhlolojikelele lwezindlela, nobuchwepheshe
Isipele Ingxenye 2: Ukubuyekeza nokuhlola amathuluzi okulondoloza asekelwe ku-rsync
Isipele Ingxenye 3: Buyekeza futhi uhlole ukuphindaphindeka, i-duplicati, i-deja dup
Isipele Ingxenye 4: Ukubuyekeza nokuhlola i-zbackup, i-restic, i-borgbackup
Isipele Ingxenye 5: Ukuhlola i-bacula ne-veeam backup ye-linux
Isipele Ingxenye 6: Ukuqhathanisa Amathuluzi Okwenza Ikhophi Yasenqolobaneni
Isipele Ingxenye 7: Iziphetho

Kuthunyelwe ngu: UPavel Demkovich

Source: www.habr.com