Amakhodi okuphelelwa umsebenzi: ngamagama alula mayelana nendlela yokugcina idatha ngendlela ethembekile futhi eshibhile

Amakhodi okuphelelwa umsebenzi: ngamagama alula mayelana nendlela yokugcina idatha ngendlela ethembekile futhi eshibhile

Yilokhu kubukeka kwe-redundancy

Amakhodi e-redundancy* asetshenziswa kakhulu ezinhlelweni zekhompiyutha ukwandisa ukuthembeka kokugcinwa kwedatha. Ku-Yandex asetshenziswa kumaphrojekthi amaningi. Isibonelo, ukusebenzisa amakhodi okuphelelwa umsebenzi esikhundleni sokuphindaphinda endaweni yethu yokugcina into yangaphakathi konga izigidi ngaphandle kokudela ukwethembeka. Kodwa naphezu kokusetshenziswa kwawo kabanzi, izincazelo ezicacile zendlela amakhodi okuphelelwa umsebenzi asebenza ngayo azivamile kakhulu. Labo abafuna ukuqonda babhekene nalokhu okulandelayo (kusuka I-Wikipedia):

Amakhodi okuphelelwa umsebenzi: ngamagama alula mayelana nendlela yokugcina idatha ngendlela ethembekile futhi eshibhile

Igama lami ngingu-Vadim, kwa-Yandex ngithuthukisa i-MDS yokugcina into yangaphakathi. Kulesi sihloko, ngizochaza ngamagama alula izisekelo zethiyori zamakhodi wokuphindaphinda (amakhodi we-Reed-Solomon kanye ne-LRC). Ngizokutshela ukuthi isebenza kanjani, ngaphandle kwezibalo eziyinkimbinkimbi kanye namagama angavamile. Ekugcineni ngizonikeza izibonelo zokusebenzisa amakhodi wokuphindaphinda ku-Yandex.

Ngeke ngicabangele imininingwane eminingi yezibalo, kodwa ngizohlinzeka ngezixhumanisi kulabo abafuna ukucwila ngokujulile. Ngizophinde ngiqaphele ukuthi ezinye izincazelo zezibalo zingase zingabi lukhuni, ngoba lesi sihloko asihloselwe ochwepheshe bezibalo, kodwa ngonjiniyela abafuna ukuqonda ingqikithi yendaba.

* Ezincwadini zolimi lwesiNgisi, amakhodi okuphelelwa umsebenzi ngokuvamile abizwa ngokuthi amakhodi okusula.

1. Ingqikithi yamakhodi okuphelelwa umsebenzi

Ingqikithi yawo wonke amakhodi okuphelelwa umsebenzi ilula kakhulu: gcina (noma udlulise) idatha ukuze ingalahleki uma kwenzeka amaphutha (ukwehluleka kwediski, amaphutha okudlulisa idatha, njll.).

Emakhodini amaningi* okuphelelwa umsebenzi, idatha ihlukaniswa ngamabhulokhi wedatha n, lapho amabhulokhi angu-m amakhodi okuphinda abalwe, okuholela engqikithini yamabhulokhi angu-n + m. Amakhodi e-redundancy akhiwa ngendlela yokuthi amabhulokhi angu-n edatha angatholwa kusetshenziswa kuphela ingxenye yamabhulokhi angu-n + m. Okulandelayo, sizocubungula kuphela amakhodi we-redundancy block kuphela, okungukuthi, lawo lapho idatha ihlukaniswa ngamabhulokhi.

Amakhodi okuphelelwa umsebenzi: ngamagama alula mayelana nendlela yokugcina idatha ngendlela ethembekile futhi eshibhile

Ukubuyisela wonke amabhulokhi angu-n wedatha, udinga okungenani ube namabhulokhi angu-n + m, njengoba awukwazi ukuthola amabhulokhi angu-n ngokuba ne-n-1 block kuphela (kulokhu, kuzodingeka uthathe ibhulokhi engu-1 umoya"). Ingabe amabhulokhi angu-n angahleliwe we-n + m amabhlogo anele ukuze abuyisele yonke idatha? Lokhu kuncike ohlotsheni lwamakhodi okuphelelwa umsebenzi, ngokwesibonelo, amakhodi e-Reed-Solomon akuvumela ukuthi ubuyisele yonke idatha usebenzisa ama-n blocks angenangqondo, kodwa amakhodi e-LRC okuphelelwa umsebenzi awahlali njalo.

Isitoreji sedatha

Ezinhlelweni zokugcina idatha, njengomthetho, amabhlogo wedatha ngayinye kanye namabhulokhi wekhodi yokuphindaphinda abhalwa kudiski ehlukile. Khona-ke, uma idiski engafanele yehluleka, idatha yoqobo isengabuyiselwa futhi ifundwe. Idatha ingatholwa ngisho noma amadiski amaningana ehluleka ngesikhathi esisodwa.

Ukudluliswa kwedatha

Amakhodi okuphelelwa umsebenzi angasetshenziswa ukudlulisa idatha ngokuthembekile ngenethiwekhi engathembekile. Idatha edlulisiwe ihlukaniswe ngamabhulokhi, futhi amakhodi okukhishwa kabusha abalwa wona. Kokubili amabhulokhi wedatha kanye namabhulokhi wekhodi yokungafuneki asakazwa ngenethiwekhi. Uma kwenzeka amaphutha kumabhulokhi angenasizathu (kufika enanini elithile lamabhulokhi), idatha isengadluliselwa kunethiwekhi ngaphandle kwephutha. Amakhodi we-Reed-Solomon, isibonelo, asetshenziselwa ukudlulisa idatha ngemigqa yokuxhumana ye-optical kanye nokuxhumana ngesathelayithi.

* Kukhona futhi amakhodi okuphelelwa umsebenzi lapho idatha ingahlukaniswa ngamabhulokhi, njengamakhodi e-Hamming namakhodi e-CRC, asetshenziswa kakhulu ekudluliseni idatha kumanethiwekhi e-Ethernet. Lawa amakhodi okulungisa amaphutha, aklanyelwe ukubona amaphutha, hhayi ukuwalungisa (ikhodi ye-Haming iphinde ivumele ukulungiswa okungaphelele kwamaphutha).

2. Amakhodi oMhlanga-Solomon

Amakhodi we-Reed-Solomon angenye yamakhodi asetshenziswa kakhulu e-redundancy, asungulwa emuva ngeminyaka yawo-1960s futhi aqala ukusetshenziswa kabanzi ngeminyaka yawo-1980s ukukhiqizwa ngobuningi kwama-compact disc.

Kukhona imibuzo emibili ebalulekile yokuqonda amakhodi we-Reed-Solomon: 1) indlela yokudala amabhlokhi amakhodi okuphelelwa umsebenzi; 2) indlela yokubuyisela idatha usebenzisa amabhlogo wekhodi yokuphindaphinda. Ake sithole izimpendulo kubo.
Ukwenza kube lula, sizophinde sicabange ukuthi u-n=6 kanye no-m=4. Amanye amasu abhekwa ngokufanisa.

Ungawakha kanjani amabhulokhi wekhodi yokuphinda usebenze

Ibhulokhi ngayinye yamakhodi okuphelelwa umsebenzi ibalwa ngokuzimela kwamanye. Wonke amabhulokhi wedatha we-n asetshenziselwa ukubala ibhulokhi ngayinye. Kumdwebo ongezansi, i-X1-X6 amabhulokhi wedatha, i-P1-P4 amabhulokhi ekhodi yokuphindaphinda.

Amakhodi okuphelelwa umsebenzi: ngamagama alula mayelana nendlela yokugcina idatha ngendlela ethembekile futhi eshibhile

Wonke amabhulokhi wedatha kufanele abe nosayizi ofanayo, futhi amabhithi ayiziro angasetshenziselwa ukuqondanisa. Amabhulokhi wekhodi wokuphinda angasebenzi azoba nosayizi ofanayo namabhulokhi wedatha. Wonke amabhulokhi edatha ahlukaniswe ngamagama (isibonelo, amabhithi ayi-16). Ake sithi sihlukanisa amabhulokhi edatha abe ngamagama angu-k. Khona-ke wonke amabhlogo wamakhodi wokuphinda aphinde ahlukaniswe abe amagama angu-k.

Amakhodi okuphelelwa umsebenzi: ngamagama alula mayelana nendlela yokugcina idatha ngendlela ethembekile futhi eshibhile

Ukuze ubale igama elithi i-th lebhulokhi ngayinye yokuphinda iphindwe, amagama e-i-th awo wonke amabhulokhi wedatha azosetshenziswa. Zizobalwa ngokwefomula elandelayo:

Amakhodi okuphelelwa umsebenzi: ngamagama alula mayelana nendlela yokugcina idatha ngendlela ethembekile futhi eshibhile

Lapha amanani u-x amagama amabhulokhi wedatha, u-p amagama wekhodi yokuphindaphinda amabhlogo, wonke ama-alpha, i-beta, i-gamma ne-delta yizinombolo ezikhethwe ngokukhethekile ezifanayo kuwo wonke ama-i. Kumele kushiwo ngaso leso sikhathi ukuthi wonke lawa magugu akuzona izinombolo ezivamile, kodwa izakhi zenkambu ye-Galois; imisebenzi +, -, *, / akuyona imisebenzi esiyijwayele sonke, kodwa imisebenzi ekhethekile eyethulwe ezintweni zeGalois. inkambu.

Kungani kudingeka izinkambu zeGalois?

Amakhodi okuphelelwa umsebenzi: ngamagama alula mayelana nendlela yokugcina idatha ngendlela ethembekile futhi eshibhile

Kubonakala sengathi yonke into ilula: sihlukanisa idatha ibe amabhulokhi, amabhlogo abe ngamagama, sisebenzisa amagama amabhulokhi wedatha sibala amagama amabhulokhi wekhodi yokuphindaphinda - sithola amabhlogo wekhodi yokuphindaphinda. Ngokuvamile le yindlela esebenza ngayo, kodwa udeveli usemininingwaneni:

  1. Njengoba kushiwo ngenhla, usayizi wegama ulungisiwe, esibonelweni sethu amabhithi ayi-16. Amafomula angenhla amakhodi eReed-Solomon anjengokuthi uma usebenzisa izinombolo ezijwayelekile, umphumela wokubala u-p ungase ungameleleki kusetshenziswa igama losayizi ovumelekile.
  2. Lapho kutholwa idatha, amafomula angenhla azobhekwa njengohlelo lwezibalo okufanele luxazululwe ukuze kutholwe idatha. Phakathi nenqubo yesixazululo, kungase kudingeke ukuba kuhlukaniswe izinombolo eziphelele, okuholela enambeni yangempela engakwazi ukumelwa ngokunembile kumemori yekhompyutha.

Lezi zinkinga zivimbela ukusetshenziswa kwamakhodi e-Reed-Solomon. Isixazululo senkinga singesokuqala, singachazwa kanje: ake siqhamuke nezinombolo ezikhethekile ezingamelwa kusetshenziswa amagama obude obudingekayo (isibonelo, amabhithi ayi-16), kanye nomphumela wokwenza yonke imisebenzi lapho (ukwengeza). , ukususa, ukuphindaphinda, ukuhlukanisa) kuzophinde kwethulwe kumemori yekhompyutha kusetshenziswa amagama anobude obudingekayo.

Izinombolo ezinjalo "ezikhethekile" ziye zacwaningwa yizibalo isikhathi eside; zibizwa ngokuthi amasimu. Inkambu isethi yama-elementi anemisebenzi yokuhlanganisa, ukususa, ukuphindaphinda nokuhlukanisa echazwe yona.

Izinkambu zeGalois* ziyizinkambu okunomphumela wazo oyingqayizivele wokusebenza ngakunye (+, -, *, /) wanoma yiziphi izici ezimbili zenkundla. Izinkambu ze-Galois zingakhelwa izinombolo ezingamandla ka-2: 2, 4, 8, 16, njll. (empeleni amandla anoma iyiphi inombolo eyinhloko p, kodwa empeleni sinentshisekelo emandleni we-2 kuphela). Ngokwesibonelo, kumagama angu-16-bit, lena inkambu equkethe izakhi ezingu-65, kubhangqa ngalinye ongathola umphumela wanoma yikuphi ukusebenza (+, -, *, /). Amanani ka-x, p, alpha, beta, gamma, delta asuka kuzibalo ezingenhla azobhekwa njengezinto zenkambu ye-Galois ukuze kubalwe.

Ngakho, sinohlelo lwezibalo esingakha ngalo amabhulokhi amakhodi okuphelelwa umsebenzi ngokubhala uhlelo olufanele lwekhompyutha. Usebenzisa uhlelo olufanayo lwezibalo, ungenza ukutholwa kwedatha.

* Lena akuyona incazelo eqinile, kodwa kunalokho incazelo.

Indlela yokuthola idatha

Ukubuyisela kuyadingeka lapho amanye amabhulokhi we-n + m engekho. Lokhu kungaba kokubili amabhulokhi wedatha kanye namabhulokhi wekhodi yokungaphindeki umsebenzi. Ukungabi bikho kwamabhulokhi wedatha kanye/noma amabhulokhi ekhodi yokuphindaphinda kuyosho ukuthi okuguquguqukayo okuhambisanayo okungu-x kanye/noma kuka-p akwaziwa ezibalweni ezingenhla.

Izibalo zamakhodi eReed-Solomon zingabukwa njengohlelo lwezibalo lapho wonke amanani e-alpha, e-beta, e-gamma, e-delta engama-constants, wonke u-x kanye no-p ahambisana namabhulokhi atholakalayo yizici eziguquguqukayo ezaziwayo, futhi okusele u-x no-p akwaziwa.

Isibonelo, vumela idatha ivimbe u-1, 2, 3 kanye ne-redundancy code block 2 ingatholakali, khona-ke eqenjini le-i-th lamagama kuzoba nohlelo olulandelayo lwezibalo (okungaziwa kumakwe ngokubomvu):

Amakhodi okuphelelwa umsebenzi: ngamagama alula mayelana nendlela yokugcina idatha ngendlela ethembekile futhi eshibhile

Sinohlelo lwezibalo ezi-4 nezi-4 ezingaziwa, okusho ukuthi singaluxazulula futhi sibuyisele idatha!

Kusukela kulolu hlelo lwezibalo kulandela iziphetho eziningi mayelana nokutholwa kwedatha yamakhodi e-Reed-Solomon (n amabhulokhi wedatha, amabhulokhi wekhodi yokuphindaphinda):

  • Idatha ingatholwa uma kulahleka noma yimaphi amabhulokhi angu-m noma ambalwa. Uma i-m+1 noma amabhulokhi amaningi elahlekile, idatha ayikwazi ukubuyiselwa: akunakwenzeka ukuxazulula uhlelo lwe-m equations nge-m + 1 engaziwa.
  • Ukuze ubuyisele ngisho nebhulokhi yedatha eyodwa, udinga ukusebenzisa noma iyiphi i-n yamabhulokhi asele, futhi ungasebenzisa noma yimaphi amakhodi okungasebenzisi umsebenzi.

Yini enye okudingeka ukwazi

Encazelweni engenhla, ngigwema izinkinga ezimbalwa ezibalulekile ezidinga ukujula kuzibalo ukuze zicatshangelwe. Ikakhulukazi, angisho lutho ngalokhu okulandelayo:

  • Uhlelo lwezibalo lwamakhodi eReed-Solomon kufanele lube nesixazululo (esiyingqayizivele) sanoma iyiphi inhlanganisela yezinto ezingaziwa (ingabi ngaphezu kuka-m okungaziwa). Ngokusekelwe kule mfuneko, amanani e-alpha, i-beta, i-gamma ne-delta akhethiwe.
  • Uhlelo lwezibalo kufanele lukwazi ukwakhiwa ngokuzenzakalelayo (kuye ngokuthi yimaphi amabhlogo angatholakali) futhi axazululwe.
  • Sidinga ukwakha inkambu ye-Galois: kusayizi wegama elinikeziwe, sikwazi ukuthola umphumela wanoma yikuphi ukusebenza (+, -, *, /) kwanoma yiziphi izakhi ezimbili.

Ekupheleni kwe-athikili kunezinkomba zezincwadi ngalezi zindaba ezibalulekile.

Ukukhetha kuka-n kanye no-m

Ungakhetha kanjani u-n no-m ekusebenzeni? Ngokwenza, ezinhlelweni zokugcinwa kwedatha, amakhodi wokuphindaphinda asetshenziselwa ukugcina isikhala, ngakho-ke m ikhethwa njalo ngaphansi kuka-n. Amanani abo athile ancike ezintweni eziningi, okuhlanganisa:

  • Ukuthembeka kokugcinwa kwedatha. I-m enkulu, inani elikhulu lokuhluleka kwediski elingasindiswa, okungukuthi, ukuphakama kokuthembeka.
  • Isitoreji esingadingekile. Uma siphezulu isilinganiso se-m/n, kuzophakama ukuphindaphinda kwesitoreji, futhi uhlelo luzobiza kakhulu.
  • Cela isikhathi sokucubungula. Uma isamba esingu-n + m sikhulu, isikhathi sokuphendula izicelo sizoba side. Njengoba ukufundwa kwedatha (ngesikhathi sokululama) kudinga ukufunda n amabhulokhi agcinwe kumadiski n ahlukene, isikhathi sokufunda sizonqunywa idiski enensa kakhulu.

Ukwengeza, ukugcinwa kwedatha kuma-DC amaningana kubeka imikhawulo eyengeziwe ekukhethweni kwe-n kanye ne-m: uma i-1 DC ivaliwe, idatha kufanele isatholakala ukuze ifundwe. Isibonelo, lapho ugcina idatha kuma-3 DCs, umbandela olandelayo kufanele ufezwe: m >= n/2, ngaphandle kwalokho kungase kube nesimo lapho idatha ingatholakali ukuze ifundwe uma i-1 DC icishiwe.

3. I-LRC - Amakhodi Okwakha Kabusha Endawo

Ukubuyisela idatha usebenzisa amakhodi we-Reed-Solomon, kufanele usebenzise amabhulokhi wedatha angenasizathu. Lokhu kuwukungalungi okuphawulekayo kwezinhlelo zokulondoloza idatha ezisabalalisiwe, ngoba ukubuyisela idatha kudiski eyodwa ephukile, kuzodingeka ufunde idatha kusuka kwezinye eziningi, udale umthwalo omkhulu owengeziwe kumadiski kanye nenethiwekhi.

Amaphutha avame kakhulu ukungafinyeleleki kwebhulokhi eyodwa yedatha ngenxa yokwehluleka noma ukugcwala kwediski eyodwa. Kungenzeka yini ngandlela thize ukunciphisa umthwalo oweqile wokuthola kabusha idatha kuleli cala (elivame kakhulu)? Kuvele ukuthi ungakwazi: kukhona amakhodi e-LRC okuphelelwa umsebenzi aqondene ngqo nale njongo.

I-LRC (Amakhodi Okwakhiwa Kabusha Kwasendaweni) amakhodi okungasebenzi kabusha aqanjwa iMicrosoft ukuze asetshenziswe ku-Windows Azure Storage. Umqondo we-LRC ulula ngangokunokwenzeka: hlukanisa wonke amabhulokhi wedatha abe ngamaqembu amabili (noma ngaphezulu) futhi ufunde ingxenye yamabhulokhi wekhodi yokuphindaphinda yeqembu ngalinye ngokwehlukana. Khona-ke amanye amabhulokhi ekhodi yokuphelelwa umsebenzi azobalwa kusetshenziswa wonke amabhlokhi wedatha (ku-LRC abizwa ngokuthi amakhodi omsebenzi womhlaba wonke), kanti amanye - kusetshenziswa elinye lamaqembu amabili amabhulokhi wedatha (abizwa ngokuthi amakhodi endawo).

I-LRC ichazwa ngezinombolo ezintathu: i-nrl, lapho u-n eyinombolo yamabhulokhi wedatha, r inombolo yamabhulokhi wekhodi yomhlaba wonke, l inombolo yamabhulokhi ekhodi yendawo. Ukuze ufunde idatha lapho ibhulokhi yedatha eyodwa ingatholakali, udinga ukufunda amabhulokhi angu-n/l kuphela - lokhu kungaphansi izikhathi ezingu-l kunakumakhodi we-Reed-Solomon.

Isibonelo, cabanga ngohlelo lwe-LRC 6-2-2. X1–X6 — 6 amabhulokhi edatha, P1, P2 — 2 amabhulokhi womhlaba ongafuneki umsebenzi, P3, P4 — 2 amabhulokhi endawo angasasebenzi.

Amakhodi okuphelelwa umsebenzi: ngamagama alula mayelana nendlela yokugcina idatha ngendlela ethembekile futhi eshibhile

I-redundancy code block blocks P1, P2 ibalwa kusetshenziswa wonke amabhulokhi wedatha. I-redundancy code block P3 - usebenzisa amabhulokhi wedatha X1-X3, i-redundancy code block P4 - usebenzisa amabhulokhi wedatha X4-X6.

Okunye kwenziwa ku-LRC ngokufanisa namakhodi we-Reed-Solomon. Izibalo zokubala amagama amabhulokhi ekhodi yokungaphindi ziyoba:

Amakhodi okuphelelwa umsebenzi: ngamagama alula mayelana nendlela yokugcina idatha ngendlela ethembekile futhi eshibhile

Ukuze ukhethe izinombolo i-alpha, i-beta, i-gamma, i-delta, inani lezimo okufanele kuhlangatshezwane nalo ukuze kuqinisekiswe ukuthi kungenzeka ukutholwa kwedatha (okungukuthi, ukuxazulula uhlelo lwezibalo). Ungafunda kabanzi ngabo ku isihloko.
Futhi ekusebenzeni, ukusebenza kwe-XOR kusetshenziselwa ukubala amakhodi endawo okuphelelwa umsebenzi P3, P4.

Inani leziphetho lilandela kusuka ohlelweni lwezibalo lwe-LRC:

  • Ukubuyisela noma iyiphi ibhulokhi yedatha engu-1, kwanele ukufunda amabhulokhi angu-n/l (n/2 esibonelweni sethu).
  • Uma amabhlogo we-r + l engatholakali, futhi wonke amabhlogo afakwe eqenjini elilodwa, khona-ke idatha ayikwazi ukubuyiselwa. Lokhu kulula ukukuchaza ngesibonelo. Vumela amabhulokhi u-X1–X3 kanye no-P3 angatholakali: lawa amabhulokhi angu-r + l asuka eqenjini elifanayo, angu-4 kithi. Bese siba nohlelo lwezibalo ezi-3 nezingu-4 ezingaziwa ezingaxazululeki.
  • Kuzo zonke ezinye izimo zokungatholakali kwamabhulokhi we-r + l (lapho okungenani ibhulokhi elilodwa litholakala eqenjini ngalinye), idatha ku-LRC ingabuyiselwa.

Ngakho-ke, i-LRC idlula amakhodi we-Reed-Solomon ekubuyiseleni idatha ngemva kwephutha elilodwa. Kumakhodi we-Reed-Solomon, ukuze ubuyisele ngisho nebhulokhi elilodwa ledatha, udinga ukusebenzisa ama-n blocks, futhi ku-LRC, ukuze ubuyisele ibhulokhi elilodwa ledatha, kwanele ukusebenzisa amabhulokhi angu-n/l (n/2 esibonelweni sethu). Ngakolunye uhlangothi, i-LRC ingaphansi kwamakhodi e-Reed-Solomon ngokwenani eliphakeme lamaphutha avumelekile. Ezibonelweni ezingenhla, amakhodi e-Reed-Solomon angabuyisa idatha yanoma yimaphi amaphutha angu-4, futhi ku-LRC kukhona izinhlanganisela ezingu-2 zamaphutha angu-4 lapho idatha ingatholakali.

Okubaluleke kakhulu kuncike esimweni esithile, kodwa ngokuvamile ukonga ngomthwalo oweqile okunikezwa yi-LRC kunedlula isitoreji esingathembeki kancane.

4. Amanye amakhodi okungasebenzi

Ngaphandle kwamakhodi e-Reed-Solomon kanye ne-LRC, kunamanye amakhodi amaningi okuphelelwa umsebenzi. Amakhodi ahlukene okungasebenzi asebenzisa izibalo ezahlukene. Nawa amanye amakhodi okuphelelwa umsebenzi:

  • Ikhodi ye-redundancy kusetshenziswa i-opharetha ye-XOR. Ukusebenza kwe-XOR kwenziwa kumabhulokhi wedatha ye-n, futhi i-1 block of redundancy codes iyatholakala, okungukuthi, uhlelo lwe-n + 1 (n amabhulokhi wedatha, ikhodi ye-redundancy engu-1). Isetshenziswe ku I-RAID 5, lapho amabhulokhi edatha namakhodi okungasebenzi abhalwa ngokujikeleza kuwo wonke amadiski ohlelo.
  • I-algorithm ye-Even-odd esekelwe ekusebenzeni kwe-XOR. Ikuvumela ukuthi wakhe amabhulokhi angu-2 amakhodi okuphelelwa umsebenzi, okungukuthi, uhlelo lwe-n+2.
  • I-algorithm ye-STAR esekelwe ekusebenzeni kwe-XOR. Ikuvumela ukuthi wakhe amabhulokhi angu-3 amakhodi okuphelelwa umsebenzi, okungukuthi, uhlelo lwe-n+3.
  • Amakhodi ephiramidi amanye amakhodi wokuphinda angasebenzi avela kwaMicrosoft.

5. Sebenzisa ku-Yandex

Inqwaba yamaphrojekthi wengqalasizinda ye-Yandex asebenzisa amakhodi okuphelelwa umsebenzi ukuze agcine idatha ethembekile. Nazi izibonelo:

  • Isitoreji sento yangaphakathi ye-MDS, engabhala ngayo ekuqaleni kwesihloko.
  • YT - ImephuYehlisa uhlelo lwe-Yandex.
  • YDB (Yandex DataBase) - i-newSQL egciniwe esabalalisiwe.

I-MDS isebenzisa amakhodi e-LRC okuphelelwa umsebenzi, isikimu esingu-8-2-2. Idatha enamakhodi wokuphindaphinda ibhalwa kumadiski ahlukene we-12 kumaseva ahlukene kuma-DC angu-3 ahlukene: amaseva angu-4 ku-DC ngayinye. Funda kabanzi ngalokhu ku isihloko.

I-YT isebenzisa womabili amakhodi e-Reed-Solomon (I-Scheme 6-3), abengawokuqala ukuwasebenzisa, kanye namakhodi e-LRC okuphelelwa umsebenzi (Uhlelo 12-2-2), kanti i-LRC iyindlela yokugcina ekhethwayo.

I-YDB isebenzisa amakhodi e-redundancy asekelwe ngokulingana (Umfanekiso 4-2). Mayelana namakhodi okuphelelwa umsebenzi ku-YDB kakade utshele kuHighload.

Ukusetshenziswa kwezinhlelo ezihlukene zamakhodi okuphelelwa umsebenzi kungenxa yezidingo ezahlukene zezinhlelo. Isibonelo, ku-MDS, idatha egcinwe kusetshenziswa i-LRC ifakwa kuma-DC ama-3 ngesikhathi esisodwa. Kubalulekile kithi ukuthi idatha ihlale itholakala ukuze ifundwe uma i-1 yanoma yimaphi ama-DC ehluleka, ngakho-ke amabhulokhi kufanele asakazwe phakathi kwe-DCs ukuze uma noma iyiphi i-DC ingatholakali, inani lamabhulokhi angafinyeleleki alikho ngaphezu kokuvunyelwa. Kuhlelo lwe-8-2-2, ungabeka amabhlogo angu-4 ku-DC ngayinye, lapho-ke noma iyiphi i-DC ivaliwe, amabhulokhi angu-4 ngeke atholakale, futhi idatha ingafundwa. Noma yiluphi uhlelo esilukhethayo lapho silubeka ku-3 DCs, kunoma yikuphi kufanele kube (r + l) / n > = 0,5, okungukuthi, ukuphindaphinda kwesitoreji kuyoba okungenani i-50%.

Ku-YT isimo sihlukile: iqoqo ngalinye le-YT litholakala ngokuphelele ku-1 DC (amaqoqo ahlukene kuma-DC ahlukene), ngakho awukho umkhawulo onjalo. Uhlelo lwe-12-2-2 luhlinzeka ngokuphindaphinda okungu-33%, okungukuthi, ukugcinwa kwedatha kushibhile, futhi kungasinda kuze kufike ekunqanyulweni kwediski oku-4 ngesikhathi esisodwa, njengohlelo lwe-MDS.

Kunezici eziningi ezengeziwe zokusebenzisa amakhodi okuphelelwa umsebenzi ekugcinweni kwedatha nezinhlelo zokucubungula: ama-nuances okuthola kabusha idatha, umthelela wokuthola kabusha ngesikhathi sokwenza umbuzo, izici zokuqoshwa kwedatha, njll. Ngizokhuluma ngokwehlukana ngalezi nezinye izici. wokusetshenziswa kwamakhodi okuphelelwa umsebenzi ekusebenzeni, uma isihloko sizothakazelisa.

6. Izixhumanisi

  1. Uchungechunge lwama-athikili mayelana namakhodi we-Reed-Solomon kanye nezinkambu ze-Galois: https://habr.com/ru/company/yadro/blog/336286/
    https://habr.com/ru/company/yadro/blog/341506/
    Babheka kabanzi izibalo ngolimi olufinyeleleka kalula.
  2. I-athikili evela ku-Microsoft mayelana ne-LRC: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/LRC12-cheng20webpage.pdf
    Isigaba 2 sichaza kafushane ithiyori bese sixoxa ngezinto ezenzekayo ne-LRC ngokusebenza.
  3. Uhlelo oluyinqaba: https://people.eecs.berkeley.edu/~kubitron/courses/cs262a-F12/handouts/papers/p245-blaum.pdf
  4. STAR isikimu: https://www.usenix.org/legacy/event/fast05/tech/full_papers/huang/huang.pdf
  5. Amakhodi ephiramidi: https://www.microsoft.com/en-us/research/publication/pyramid-codes-flexible-schemes-to-trade-space-for-access-efficiency-in-reliable-data-storage-systems/
  6. Amakhodi okungasebenzi kahle ku-MDS: https://habr.com/ru/company/yandex/blog/311806
  7. Amakhodi okungasebenzi kahle ku-YT: https://habr.com/ru/company/yandex/blog/311104/
  8. Amakhodi okungasebenzi kahle ku-YDB: https://www.youtube.com/watch?v=dCpfGJ35kK8

Source: www.habr.com

Engeza amazwana