Nā code redundancy: ma nā huaʻōlelo maʻalahi e pili ana i ka mālama ʻana i ka ʻikepili me ka hilinaʻi a me ka maʻalahi

Nā code redundancy: ma nā huaʻōlelo maʻalahi e pili ana i ka mālama ʻana i ka ʻikepili me ka hilinaʻi a me ka maʻalahi

ʻO kēia ke ʻano o ka redundancy

Hoʻohana nui ʻia nā code redundancy* i nā ʻōnaehana kamepiula e hoʻonui i ka hilinaʻi o ka mālama ʻikepili. Ma Yandex hoʻohana ʻia lākou i nā papahana he nui. No ka laʻana, ʻo ka hoʻohana ʻana i nā code redundancy ma mua o ka hoʻopiʻi ʻana i kā mākou waihona mea i loko e mālama ai i nā miliona me ka ʻole o ka ʻimi ʻana i ka hilinaʻi. Akā, ʻoiai kā lākou hoʻohana nui ʻana, ʻaʻole loa ka wehewehe ʻana i ka hana ʻana o nā code redundancy. ʻO ka poʻe makemake e hoʻomaopopo i ke alo o kēia mau mea (mai ʻO Wikipedia):

Nā code redundancy: ma nā huaʻōlelo maʻalahi e pili ana i ka mālama ʻana i ka ʻikepili me ka hilinaʻi a me ka maʻalahi

ʻO Vadim koʻu inoa, ma Yandex ke hoʻomohala nei au i ka mālama ʻana i nā mea kūloko MDS. Ma kēia ʻatikala, e wehewehe au ma nā huaʻōlelo maʻalahi i nā kumu kumu o nā code redundancy (Reed-Solomon a me LRC code). E haʻi wau iā ʻoe pehea e hana ai, me ka makemakika paʻakikī a me nā huaʻōlelo kakaʻikahi. I ka hopena, e hāʻawi wau i nā hiʻohiʻona o ka hoʻohana ʻana i nā code redundancy ma Yandex.

ʻAʻole wau e noʻonoʻo i kekahi mau kikoʻī makemakika, akā hāʻawi wau i nā loulou no ka poʻe makemake e luʻu hohonu. E hoʻomaopopo nō hoʻi au ʻaʻole paʻa loa kekahi mau wehewehe makemakika, no ka mea, ʻaʻole i manaʻo ʻia ka ʻatikala no ka poʻe makemakika, akā no nā ʻenekinia makemake e hoʻomaopopo i ke ʻano o ka pilikia.

* Ma ka palapala ʻōlelo Pelekania, kapa pinepine ʻia nā code redundancy code erasure.

1. ʻO ke kumu o nā code redundancy

He mea maʻalahi loa ke ʻano o nā code redundancy a pau: mālama (a hoʻouna paha) i ka ʻikepili i ʻole e nalo ke loaʻa nā hewa (nā hemahema disk, nā hewa hoʻoili ʻikepili, etc.).

Ma ka hapa nui o nā code redundancy, ua māhele ʻia ka ʻikepili i n poloka ʻikepili, no laila e helu ʻia nā m poloka o nā code redundancy, e hopena i ka huina o n + m poloka. Hoʻokumu ʻia nā code redundancy ma ke ʻano e hiki ke hoʻihoʻi ʻia nā n poloka o ka ʻikepili me ka hoʻohana ʻana i kahi hapa o nā poloka n + m. A laila, e noʻonoʻo mākou i nā code redundancy block wale nō, ʻo ia hoʻi, nā mea i hoʻokaʻawale ʻia ka ʻikepili i mau poloka.

Nā code redundancy: ma nā huaʻōlelo maʻalahi e pili ana i ka mālama ʻana i ka ʻikepili me ka hilinaʻi a me ka maʻalahi

No ka hoʻihoʻi hou ʻana i nā n poloka o ka ʻikepili, pono ʻoe e loaʻa ma ka liʻiliʻi o n o n + m poloka, ʻoiai ʻaʻole hiki iā ʻoe ke loaʻa nā n poloka ma ka loaʻa ʻana o n-1 poloka (i kēia hihia, pono ʻoe e lawe i 1 poloka "mai ka lahilahi. ea”). Ua lawa anei nā poloka random o n + m poloka e hoʻihoʻi i nā ʻikepili a pau? Pili kēia i ke ʻano o nā code redundancy, no ka laʻana, ʻae nā code Reed-Solomon iā ʻoe e hoʻihoʻi i nā ʻikepili āpau me ka hoʻohana ʻana i nā poloka n arbitrary, akā ʻaʻole mau nā code redundancy LRC.

Waihona ʻikepili

Ma nā ʻōnaehana mālama ʻikepili, ma ke ʻano he kānāwai, ua kākau ʻia kēlā me kēia o nā poloka data a me nā poloka code redundancy i kahi disk kaʻawale. A laila, inā ʻaʻole i hāʻule ka disk arbitrary, hiki ke hoʻihoʻi a heluhelu ʻia ka ʻikepili kumu. Hiki ke hoʻihoʻi ʻia ka ʻikepili inā hāʻule kekahi mau disks i ka manawa like.

Hoʻolaha ka ʻike

Hiki ke hoʻohana ʻia nā code redundancy e hoʻouna pono i ka ʻikepili ma luna o kahi pūnaewele hilinaʻi ʻole. Hoʻokaʻawale ʻia nā ʻikepili i hoʻouna ʻia i mau poloka, a helu ʻia nā code redundancy no lākou. Hoʻouna ʻia nā poloka ʻikepili ʻelua a me nā poloka code redundancy ma luna o ka pūnaewele. Inā loaʻa nā hewa i nā poloka ʻokoʻa (a hiki i kahi helu o nā poloka), hiki ke hoʻouna ʻia ka ʻikepili ma luna o ka pūnaewele me ka ʻole o ka hewa. No ka laʻana, hoʻohana ʻia nā code Reed-Solomon e hoʻouna i ka ʻikepili ma luna o nā laina kamaʻilio optical a me nā kamaʻilio satelite.

* Aia kekahi mau code redundancy ʻaʻole i hoʻokaʻawale ʻia ka ʻikepili i nā poloka, e like me nā code Hamming a me nā code CRC, i hoʻohana nui ʻia no ka hoʻouna ʻana i ka ʻikepili ma nā pūnaewele Ethernet. He mau code kēia no ka hoʻoponopono hewa ʻana, ua hoʻolālā ʻia lākou e ʻike i nā hewa, ʻaʻole e hoʻoponopono iā lākou (ʻo ka Hamming code kekahi e ʻae i ka hoʻoponopono hapa o nā hewa).

2. Reed-Solomon code

ʻO nā code Reed-Solomon kekahi o nā code redundancy i hoʻohana nui ʻia, i hana ʻia i ka makahiki 1960 a ua hoʻohana mua ʻia i nā makahiki 1980 no ka hana nui ʻana i nā disc compact.

ʻElua mau nīnau nui no ka hoʻomaopopo ʻana i nā code Reed-Solomon: 1) pehea e hana ai i nā poloka o nā code redundancy; 2) pehea e hoʻihoʻi hou ai i ka ʻikepili me ka hoʻohana ʻana i nā poloka code redundancy. E ʻimi kākou i nā pane iā lākou.
No ka maʻalahi, e manaʻo hou mākou he n = 6 a me m = 4. Manaʻo ʻia nā papahana ʻē aʻe e ka hoʻohālikelike.

Pehea e hana ai i nā poloka code redundancy

Helu kūʻokoʻa kēlā me kēia poloka o nā code redundancy mai nā mea ʻē aʻe. Hoʻohana ʻia nā poloka ʻikepili āpau e helu i kēlā me kēia poloka. Ma ke kiʻikuhi ma lalo nei, ʻo X1-X6 nā poloka ʻikepili, ʻo P1-P4 nā poloka code redundancy.

Nā code redundancy: ma nā huaʻōlelo maʻalahi e pili ana i ka mālama ʻana i ka ʻikepili me ka hilinaʻi a me ka maʻalahi

Pono e like ka nui o nā poloka ʻikepili a pau, a hiki ke hoʻohana ʻia nā ʻāpana ʻole no ka hoʻoponopono ʻana. E like ka nui o nā poloka code redundancy me nā poloka ʻikepili. Hoʻokaʻawale ʻia nā poloka ʻikepili a pau i nā huaʻōlelo (no ka laʻana, 16 bits). E ʻōlelo mākou ua hoʻokaʻawale mākou i nā poloka ʻikepili i k huaʻōlelo. A laila e māhele ʻia nā poloka a pau o nā code redundancy i nā huaʻōlelo k.

Nā code redundancy: ma nā huaʻōlelo maʻalahi e pili ana i ka mālama ʻana i ka ʻikepili me ka hilinaʻi a me ka maʻalahi

No ka helu ʻana i ka huaʻōlelo i-th o kēlā me kēia poloka redundancy, e hoʻohana ʻia nā huaʻōlelo i-th o nā poloka ʻikepili āpau. E helu ʻia lākou e like me ke ʻano ma lalo nei:

Nā code redundancy: ma nā huaʻōlelo maʻalahi e pili ana i ka mālama ʻana i ka ʻikepili me ka hilinaʻi a me ka maʻalahi

Eia nā huaʻōlelo x nā huaʻōlelo o nā poloka data, p nā huaʻōlelo o nā poloka code redundancy, nā alpha āpau, beta, gamma a me delta nā helu i koho kūikawā ʻia no nā i āpau. Pono e ʻōlelo koke ʻia ʻaʻole kēia mau waiwai āpau i nā helu maʻamau, akā nā mea o ke kahua Galois; nā hana +, -, *, / ʻaʻole nā ​​​​hana i maʻa iā mākou a pau, akā nā hana kūikawā i hoʻokomo ʻia ma nā mea o ka Galois. kahua.

No ke aha e pono ai nā māla Galois?

Nā code redundancy: ma nā huaʻōlelo maʻalahi e pili ana i ka mālama ʻana i ka ʻikepili me ka hilinaʻi a me ka maʻalahi

Me he mea lā ua maʻalahi nā mea a pau: hoʻokaʻawale mākou i ka ʻikepili i nā poloka, nā poloka i nā huaʻōlelo, me ka hoʻohana ʻana i nā huaʻōlelo o nā poloka data helu mākou i nā huaʻōlelo o nā poloka code redundancy - loaʻa iā mākou nā poloka code redundancy. Ma ka laulā, penei ka hana, akā aia ka diabolo i nā kikoʻī:

  1. E like me ka mea i ʻōlelo ʻia ma luna, ua paʻa ka nui o ka huaʻōlelo, i kā mākou laʻana 16 bits. ʻO nā kumu hoʻohālike ma luna no nā code Reed-Solomon i ka wā e hoʻohana ai i nā integer maʻamau, ʻaʻole hiki ke hōʻike ʻia ka hopena o ka helu ʻana i ka p me ka hoʻohana ʻana i kahi huaʻōlelo o ka nui kūpono.
  2. I ka hoʻihoʻi ʻana i ka ʻikepili, e noʻonoʻo ʻia nā ʻōkuhi ma luna nei he ʻōnaehana o nā hoʻohālikelike pono e hoʻoponopono ʻia i mea e hoʻihoʻi ai i ka ʻikepili. I loko o ke kaʻina hana hoʻonā, pono paha e puʻunaue i nā integers i kekahi i kekahi, e hopena i kahi helu maoli ʻaʻole hiki ke hōʻike pololei ʻia i ka hoʻomanaʻo kamepiula.

Ke pale nei kēia mau pilikia i ka hoʻohana ʻana i nā helu helu no nā code Reed-Solomon. ʻO ka hopena o ka pilikia he kumu maoli ia, hiki ke wehewehe ʻia penei: e hana i nā helu kūikawā i hiki ke hōʻike ʻia me ka hoʻohana ʻana i nā huaʻōlelo o ka lōʻihi i makemake ʻia (no ka laʻana, 16 bits), a me ka hopena o ka hana ʻana i nā hana āpau (hoʻohui. , unuhi, hoonui, mahele) e hoike pu ia ma ka hoomanao lolouila me ka hoohana ana i na huaolelo o ka loa i makemakeia.

Ua aʻo ʻia kēlā mau helu "kūikawā" e ka makemakika no ka manawa lōʻihi; ua kapa ʻia lākou he kahua. ʻO ke kahua kahi pūʻulu o nā mea me nā hana o ka hoʻohui, unuhi, hoʻonui a me ka mahele i wehewehe ʻia no lākou.

ʻO nā kahua ʻo Galois* he mau kahua kahi e loaʻa ai kahi hopena kūʻokoʻa o kēlā me kēia hana (+, -, *, /) no nā mea ʻelua o ke kahua. Hiki ke kūkulu ʻia nā māla Galois no nā helu he mana o 2: 2, 4, 8, 16, a me nā mea ʻē aʻe. No ka laʻana, no nā huaʻōlelo 2-bit, he kahua kēia i loaʻa nā mea 16, no kēlā me kēia pālua āu e ʻike ai i ka hopena o kekahi hana (+, -, *, /). ʻO nā waiwai o x, p, alpha, beta, gamma, delta mai nā hoʻohālikelike ma luna nei e manaʻo ʻia he mau mea o ka māla Galois no ka helu ʻana.

No laila, loaʻa iā mākou kahi ʻōnaehana hoʻohālikelike e hiki ai iā mākou ke kūkulu i nā poloka o nā code redundancy ma ke kākau ʻana i kahi polokalamu kamepiula kūpono. Me ka hoʻohana ʻana i ka ʻōnaehana like o nā hoohalike, hiki iā ʻoe ke hana i ka hoʻihoʻi ʻikepili.

* ʻAʻole kēia he wehewehe koʻikoʻi, akā he wehewehe.

Pehea e hoʻihoʻi hou ai i ka ʻikepili

Pono ka hoʻihoʻi ʻana i ka wā e nalowale ana kekahi o nā poloka n + m. Hiki i kēia mau poloka ʻikepili a me nā poloka code redundancy. ʻO ka loaʻa ʻole o nā poloka ʻikepili a/a i ʻole nā ​​poloka code redundancy e manaʻo ʻia ʻaʻole ʻike ʻia nā hoʻololi like x a me/p i nā hoʻohālikelike ma luna.

Hiki ke ʻike ʻia nā hoʻohālikelike no nā code Reed-Solomon ma ke ʻano he ʻōnaehana o ka hoʻohālikelike ʻana i nā helu alpha, beta, gamma, delta i mau mau, ʻo nā x a me p e pili ana i nā poloka i loaʻa i ʻike ʻia nā loli, a ʻo ke koena x a me p. ʻaʻole ʻike ʻia.

No ka laʻana, ʻaʻole i loaʻa nā poloka ʻikepili 1, 2, 3 a me redundancy code block 2, a laila no ka pūʻulu i-th e loaʻa kēia ʻōnaehana o nā hoohalike (ʻike ʻole ʻia i ka ʻulaʻula):

Nā code redundancy: ma nā huaʻōlelo maʻalahi e pili ana i ka mālama ʻana i ka ʻikepili me ka hilinaʻi a me ka maʻalahi

Loaʻa iā mākou kahi ʻōnaehana o 4 equations me 4 ʻike ʻole, ʻo ia ka mea hiki iā mākou ke hoʻoponopono a hoʻihoʻi i ka ʻikepili!

Mai kēia ʻōnaehana o nā hoʻohālikelike he nui nā hopena e pili ana i ka hoʻihoʻi ʻana i ka ʻikepili no nā code Reed-Solomon (n nā poloka ʻikepili, m nā poloka code redundancy):

  • Hiki ke hoʻihoʻi ʻia ka ʻikepili inā nalowale kekahi mau poloka m a i ʻole. Inā nalowale nā ​​poloka m+1 a ʻoi aku paha, ʻaʻole hiki ke hoʻihoʻi ʻia ka ʻikepili: ʻaʻole hiki ke hoʻoponopono i kahi ʻōnaehana o nā m like me ka m + 1 ʻike ʻole.
  • No ka hoʻihoʻi ʻana i hoʻokahi poloka data, pono ʻoe e hoʻohana i kekahi n o nā poloka i koe, a hiki iā ʻoe ke hoʻohana i kekahi o nā code redundancy.

He aha hou aʻe kāu e ʻike ai

Ma ka wehewehe ʻana ma luna, ke pale aku nei au i kekahi mau pilikia koʻikoʻi e koi ana i kahi luʻu hohonu i ka makemakika e noʻonoʻo ai. ʻO ka mea nui, ʻaʻole wau e ʻōlelo i kekahi mea e pili ana i kēia:

  • Pono ka ʻōnaehana hoʻohālikelike no nā code Reed-Solomon i kahi hopena (kūikawā) no ka hui pū ʻana o nā mea ʻike ʻole (ʻaʻole i ʻoi aku ma mua o ka m ʻike ʻole). Ma muli o kēia koi, koho ʻia nā waiwai o ka alpha, beta, gamma a me delta.
  • Pono e hana ʻakomi ʻia kahi ʻōnaehana o nā hoohalike (ma muli o ka loaʻa ʻole o nā poloka) a hoʻoholo ʻia.
  • Pono mākou e kūkulu i kahi kahua Galois: no ka nui o ka huaʻōlelo, hiki iā ʻoe ke ʻimi i ka hopena o kekahi hana (+, -, *, /) no nā mea ʻelua.

Aia ma ka hope o ka ʻatikala nā mea e pili ana i kēia mau mea koʻikoʻi.

Koho o n a me m

Pehea e koho ai i n a me m ma ka hana? Ma ka hoʻomaʻamaʻa, ma nā ʻōnaehana mālama ʻikepili, hoʻohana ʻia nā code redundancy e mālama i ka lewa, no laila koho mau ʻia ka m ma lalo o n. ʻO kā lākou mau waiwai kikoʻī e pili ana i nā kumu he nui, me:

  • ʻO ka hilinaʻi o ka mālama ʻikepili. ʻO ka m nui, ʻoi aku ka nui o nā hemahema disk i hiki ke ola, ʻo ia hoʻi, ʻoi aku ka kiʻekiʻe o ka hilinaʻi.
  • Hoʻopaʻa hoʻopaʻa ʻia. ʻOi aku ke kiʻekiʻe o ka lakio m/n, ʻoi aku ka kiʻekiʻe o ka redundancy mālama, a ʻoi aku ka pipiʻi o ka ʻōnaehana.
  • Noi i ka manawa hana. ʻO ka nui o ka huina n + m, ʻoi aku ka lōʻihi o ka manawa pane i nā noi. No ka mea e pono ana ka heluhelu ʻana i ka ʻikepili (i ka wā o ka hoʻihoʻi ʻana) i ka heluhelu ʻana i nā poloka i mālama ʻia ma nā disks like ʻole, e hoʻoholo ʻia ka manawa heluhelu e ka diski lohi.

Eia kekahi, ʻo ka mālama ʻana i nā ʻikepili i loko o kekahi mau DC ke kau nei i nā palena ʻē aʻe i ke koho ʻana o n a me m: inā pio 1 DC, pono e loaʻa ka ʻikepili no ka heluhelu ʻana. No ka laʻana, i ka wā e mālama ai i ka ʻikepili ma 3 DC, pono e hoʻokō ʻia kēia ʻano: m >= n/2, inā ʻaʻole hiki ke loaʻa ka ʻikepili no ka heluhelu ʻana ke pio ʻo 1 DC.

3. LRC - Nā Kānāwai Hoʻoponopono Kūloko

No ka hoʻihoʻi ʻana i ka ʻikepili me ka hoʻohana ʻana i nā code Reed-Solomon, pono ʻoe e hoʻohana i nā poloka data arbitrary. He pōʻino koʻikoʻi kēia no nā ʻōnaehana mālama ʻikepili i hāʻawi ʻia, no ka mea e hoʻihoʻi i ka ʻikepili ma kahi diski haki, pono ʻoe e heluhelu i ka ʻikepili mai ka hapa nui o nā mea ʻē aʻe, e hana ana i kahi ukana nui ma nā disks a me ka pūnaewele.

ʻO nā hewa maʻamau ka hiki ʻole o hoʻokahi poloka o ka ʻikepili ma muli o ka hāʻule ʻole a i ʻole ka hoʻonui ʻana o hoʻokahi disk. Hiki paha ke hoʻemi i ka ukana nui no ka hoʻihoʻi ʻana i ka ʻikepili i kēia hihia (maʻamau)? ʻIke ʻia hiki iā ʻoe: aia nā code redundancy LRC no kēia kumu.

ʻO LRC (Local Reconstruction Codes) he mau code redundancy i haku ʻia e Microsoft no ka hoʻohana ʻana ma Windows Azure Storage. He mea maʻalahi ka manaʻo o ka LRC: e hoʻokaʻawale i nā poloka data āpau i ʻelua (a ʻoi aku paha) mau pūʻulu a heluhelu i kahi ʻāpana o nā poloka code redundancy no kēlā me kēia hui. A laila e helu ʻia kekahi mau poloka code redundancy me ka hoʻohana ʻana i nā poloka data āpau (ma LRC ua kapa ʻia lākou nā code redundancy global), a ʻo kekahi - me ka hoʻohana ʻana i kekahi o nā pūʻulu ʻikepili ʻelua (ua kapa ʻia lākou he code redundancy local).

Hōʻike ʻia ka LRC e nā helu ʻekolu: nrl, kahi n ka helu o nā poloka ikepili, r ka helu o nā poloka code redundancy honua, l ka helu o nā poloka code redundancy local. No ka heluhelu ʻana i ka ʻikepili ke loaʻa ʻole hoʻokahi poloka ʻikepili, pono ʻoe e heluhelu i nā poloka n/l wale nō - ʻoi aku ka liʻiliʻi o kēia ma mua o nā code Reed-Solomon.

No ka laʻana, e noʻonoʻo i ka papahana LRC 6-2-2. X1–X6 — 6 poloka ikepili, P1, P2 — 2 poloka hoemi honua, P3, P4 — 2 poloka hoemi kūloko.

Nā code redundancy: ma nā huaʻōlelo maʻalahi e pili ana i ka mālama ʻana i ka ʻikepili me ka hilinaʻi a me ka maʻalahi

Ua helu ʻia nā poloka code redundancy P1, P2 me ka hoʻohana ʻana i nā poloka ʻikepili āpau. Palena code redundancy P3 - hoʻohana i nā poloka ʻikepili X1-X3, poloka code redundancy P4 - hoʻohana i nā poloka data X4-X6.

Hana ʻia ke koena ma LRC ma ka hoʻohālikelike me nā code Reed-Solomon. ʻO nā hoohalike no ka helu ʻana i nā huaʻōlelo o nā poloka code redundancy:

Nā code redundancy: ma nā huaʻōlelo maʻalahi e pili ana i ka mālama ʻana i ka ʻikepili me ka hilinaʻi a me ka maʻalahi

No ke koho ʻana i nā helu alpha, beta, gamma, delta, pono e hoʻokō ʻia kekahi mau kūlana e hōʻoiaʻiʻo i ka hiki ke hoʻihoʻi i ka ʻikepili (ʻo ia hoʻi, hoʻoholo i ka ʻōnaehana hoʻohālikelike). Hiki iā ʻoe ke heluhelu hou aku e pili ana iā lākou ma 'ōlelo.
Ma ka hoʻomaʻamaʻa ʻana, hoʻohana ʻia ka hana XOR e helu i nā code redundancy kūloko P3, P4.

Aia kekahi mau hopena mai ka ʻōnaehana o nā hoohalike no LRC:

  • No ka hoʻihoʻi ʻana i kekahi poloka data 1, lawa ia e heluhelu i nā poloka n / l (n / 2 i kā mākou hiʻohiʻona).
  • Inā loaʻa ʻole nā ​​poloka r + l, a ua hoʻokomo ʻia nā poloka a pau i hoʻokahi hui, a laila ʻaʻole hiki ke hoʻihoʻi ʻia ka ʻikepili. He mea maʻalahi kēia e wehewehe me kahi laʻana. ʻAʻole loaʻa nā poloka X1–X3 a me P3: ʻo ia nā poloka r + l mai ka hui like, 4 i kā mākou hihia. A laila, loaʻa iā mākou kahi ʻōnaehana o 3 equations me 4 ʻike ʻole hiki ʻole ke hoʻoholo.
  • Ma nā hihia ʻē aʻe o ka loaʻa ʻole o nā poloka r + l (inā loaʻa ka liʻiliʻi hoʻokahi poloka mai kēlā me kēia hui), hiki ke hoʻihoʻi ʻia ka ʻikepili i loko o ka LRC.

No laila, ʻoi aku ka LRC i nā code Reed-Solomon i ka hoʻihoʻi ʻana i ka ʻikepili ma hope o nā hewa hoʻokahi. Ma nā code Reed-Solomon, no ka hoʻihoʻi ʻana i hoʻokahi poloka o ka ʻikepili, pono ʻoe e hoʻohana i nā poloka n, a ma LRC, e hoʻihoʻi i hoʻokahi poloka o ka ʻikepili, ua lawa ia e hoʻohana i nā poloka n / l (n/2 i kā mākou hiʻohiʻona). Ma ka ʻaoʻao ʻē aʻe, ʻoi aku ka liʻiliʻi o LRC i nā code Reed-Solomon ma ke ʻano o ka nui o nā hewa i ʻae ʻia. Ma nā hiʻohiʻona ma luna, hiki i nā code Reed-Solomon ke hoʻihoʻi i ka ʻikepili no nā hewa 4, a no LRC aia nā hui 2 o nā hewa 4 ke hiki ʻole ke hoʻihoʻi ʻia ka ʻikepili.

ʻO ka mea nui e pili ana i ke kūlana kikoʻī, akā ʻoi aku ka nui o ka mālama ʻana i ka ukana i hāʻawi ʻia e LRC ma mua o ka mālama pono ʻole.

4. Nā code redundancy ʻē aʻe

Ma waho aʻe o nā code Reed-Solomon a me LRC, nui nā code redundancy ʻē aʻe. Hoʻohana nā code redundancy ʻokoʻa i ka makemakika like ʻole. Eia kekahi mau code redundancy ʻē aʻe:

  • ʻO ke code redundancy me ka hoʻohana ʻana i ka mea hoʻohana XOR. Hana ʻia ka hana XOR ma nā n poloka ʻikepili, a loaʻa ka 1 poloka o nā code redundancy, ʻo ia hoʻi, kahi hoʻolālā n+1 (n nā poloka data, 1 redundancy code). Hoʻohana ʻia ma RAID 5, kahi i kākau ʻia ai nā poloka o ka ʻikepili a me nā code redundancy i nā diski āpau o ka array.
  • ʻO ka algorithm like ʻole e pili ana i ka hana XOR. Hiki iā ʻoe ke kūkulu i 2 poloka o nā code redundancy, ʻo ia hoʻi, ka papahana n+2.
  • STAR algorithm ma muli o ka hana XOR. Hiki iā ʻoe ke kūkulu i 3 poloka o nā code redundancy, ʻo ia hoʻi, ka papahana n+3.
  • ʻO nā code Pyramide kekahi mau code redundancy mai Microsoft.

5. E hoʻohana ma Yandex

Ke hoʻohana nei kekahi mau papahana hoʻolālā Yandex i nā code redundancy no ka mālama ʻana i ka ʻikepili hilinaʻi. Eia kekahi mau laʻana:

  • MDS kūloko mea waihona, aʻu i kākau e pili ana i ka hoʻomaka 'ana o ka 'atikala.
  • YT - Pūnaehana MapReduce o Yandex.
  • YDB (Yandex DataBase) - newSQL puʻunaue waihona.

Hoʻohana ʻo MDS i nā code redundancy LRC, 8-2-2 scheme. Ua kākau ʻia nā ʻikepili me nā code redundancy i nā diski like ʻole 12 ma nā kikowaena like ʻole ma 3 DC ʻokoʻa: 4 mau kikowaena i kēlā me kēia DC. E heluhelu hou e pili ana i keia ma 'ōlelo.

Hoʻohana ʻo YT i nā code Reed-Solomon ʻelua (Scheme 6-3), ʻo ia ka mea i hoʻokō mua ʻia, a me nā code redundancy LRC (Scheme 12-2-2), me LRC ke ʻano o ka mālama ʻana.

Hoʻohana ʻo YDB i nā code redundancy e pili ana i ke ʻano like ʻole (Figure 4-2). E pili ana i nā code redundancy ma YDB haʻi ʻia ma Highload.

ʻO ka hoʻohana ʻana i nā ʻāpana code redundancy like ʻole ma muli o nā koi like ʻole no nā ʻōnaehana. No ka laʻana, ma MDS, waiho ʻia ka ʻikepili i mālama ʻia me ka LRC i 3 DC i ka manawa hoʻokahi. He mea nui iā mākou ka loaʻa ʻana o ka ʻikepili no ka heluhelu ʻana inā hāʻule ka 1 o nā DCs, no laila pono e puʻunaue ʻia nā poloka ma nā DCs a inā ʻaʻole loaʻa kekahi DC, ʻaʻole i ʻoi aku ka nui o nā poloka hiki ʻole ke ʻae ʻia. Ma ka papahana 8-2-2, hiki iā ʻoe ke kau i nā poloka 4 i kēlā me kēia DC, a laila i ka wā e pio ai kekahi DC, ʻaʻole loaʻa nā poloka 4, a hiki ke heluhelu ʻia ka ʻikepili. ʻO kēlā me kēia kumumanaʻo a mākou e koho ai i ka wā e kau ai i 3 DCs, i kēlā me kēia hihia e loaʻa (r + l) / n> = 0,5, ʻo ia hoʻi, ʻo ka redundancy o ka waiho ʻana ma kahi o 50%.

ʻOkoʻa ke kūlana ma YT: aia kēlā me kēia pūʻulu YT i 1 DC (nā pūʻulu like ʻole i nā DC ʻokoʻa), no laila ʻaʻohe palena. Hāʻawi ka papahana 12-2-2 i ka 33% redundancy, ʻo ia hoʻi, ʻoi aku ka liʻiliʻi o ka mālama ʻana i ka ʻikepili, a hiki ke ola a hiki i ka 4 pau ʻana o ka disk, e like me ka papahana MDS.

Nui aʻe nā hiʻohiʻona o ka hoʻohana ʻana i nā code redundancy i ka mālama ʻana i ka ʻikepili a me nā ʻōnaehana hana: nā nuances o ka hoʻihoʻi ʻana i ka ʻikepili, ka hopena o ka hoʻihoʻi ʻana i ka manawa hoʻokō nīnau, nā hiʻohiʻona o ka hoʻopaʻa ʻana i ka ʻikepili, a me nā mea ʻē aʻe. o ka hoʻohana ʻana i nā code redundancy i ka hoʻomaʻamaʻa, inā e hoihoi ke kumuhana.

6. Nā loulou

  1. He pūʻulu ʻatikala e pili ana i nā code Reed-Solomon a me nā kahua Galois: https://habr.com/ru/company/yadro/blog/336286/
    https://habr.com/ru/company/yadro/blog/341506/
    Nānā hohonu lākou i ka makemakika ma ka ʻōlelo hiki ke loaʻa.
  2. ʻatikala mai Microsoft e pili ana i ka LRC: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/LRC12-cheng20webpage.pdf
    Hōʻike pōkole ʻo Pauku 2 i ke kumumanaʻo a laila kūkākūkā i nā ʻike me LRC ma ka hoʻomaʻamaʻa.
  3. ʻO ka papahana like-odd: https://people.eecs.berkeley.edu/~kubitron/courses/cs262a-F12/handouts/papers/p245-blaum.pdf
  4. ka papahana STAR: https://www.usenix.org/legacy/event/fast05/tech/full_papers/huang/huang.pdf
  5. Nā code Pyramid: https://www.microsoft.com/en-us/research/publication/pyramid-codes-flexible-schemes-to-trade-space-for-access-efficiency-in-reliable-data-storage-systems/
  6. Nā code redundancy ma MDS: https://habr.com/ru/company/yandex/blog/311806
  7. Nā code redundancy ma YT: https://habr.com/ru/company/yandex/blog/311104/
  8. Nā code redundancy ma YDB: https://www.youtube.com/watch?v=dCpfGJ35kK8

Source: www.habr.com

Pākuʻi i ka manaʻo hoʻopuka