Uthelekiso olufutshane loyilo lwe-SDS okanye ukukhangela indawo yokugcina efanelekileyo (GlusterVsCephVsVirtuozzoStorage)

Eli nqaku libhalelwe ukukunceda ukhethe isisombululo esifanelekileyo kwaye uqonde umahluko phakathi kwe-SDS efana neGluster, Ceph kunye neVstorage (Virtuozzo).

Isicatshulwa sisebenzisa amakhonkco kumanqaku anokuchazwa okuneenkcukacha ngakumbi kweengxaki ezithile, ngoko ke iinkcazo ziya kuba mfutshane kangangoko, usebenzisa amanqaku aphambili ngaphandle kolwazi olungeyomfuneko kunye nolwazi oluyintshayelelo onokuthi, ukuba unqwenela, ulufumane ngokuzimeleyo kwi-Intanethi.

Enyanisweni, ngokuqinisekileyo, izihloko eziphakanyisiweyo zifuna iitoni zombhalo, kodwa kwihlabathi lanamhlanje abantu abaninzi abathandi ukufunda kakhulu))), ngoko unokufunda ngokukhawuleza kwaye wenze ukhetho, kwaye ukuba kukho into ethile. akucaci, landela amakhonkco okanye amagama e-google angacacanga))), kwaye eli nqaku lifana ne-wrapper ecacileyo yezi zihloko ezinzulu, ebonisa ukuzaliswa - iingongoma eziphambili eziphambili kwisigqibo ngasinye.

I-Gluster

Masiqale nge-Gluster, esetyenziswa ngokusebenzayo ngabavelisi be-hyperconverged platforms kunye ne-SDS esekelwe kumthombo ovulekileyo weendawo ezibonakalayo kwaye inokufumaneka kwiwebhusayithi ye-RedHat kwicandelo lokugcina, apho unokukhetha khona kwiinketho ezimbini ze-SDS: Gluster okanye i-Ceph.

I-Gluster iqulethe i-stack yabaguquleli - iinkonzo ezenza wonke umsebenzi wokuhambisa iifayile, njl. Isitena yinkonzo ebonelela ngediski enye, uMqulu ngumthamo (iphuli) odibanisa ezi zitena. Okulandelayo kuza inkonzo yokusabalalisa iifayile kumaqela usebenzisa i-DHT (itafile ye-hash esasaziweyo) umsebenzi. Asiyi kubandakanya inkonzo yeShading kwinkcazo kuba amakhonkco angezantsi aya kuchaza iingxaki ezinxulumene nayo.

Xa ubhala, yonke ifayile igcinwe kwisitena kwaye ikopi yayo ibhalwe ngaxeshanye kwisitena kwiseva yesibini. Emva koko, ifayile yesibini iya kubhalwa kwiqela lesibini lezitena ezimbini (okanye ngaphezulu) kumaseva ahlukeneyo.

Ukuba iifayile zimalunga nobukhulu obufanayo kwaye umthamo uquka iqela elinye kuphela, ngoko yonke into ilungile, kodwa phantsi kwezinye iimeko iingxaki ezilandelayo ziya kuvela kwiinkcazo:

• indawo kumaqela isetyenziswe ngokungalinganiyo, kuxhomekeke kubukhulu beefayile kwaye ukuba akukho ndawo yaneleyo kwiqela lokubhala ifayile, uya kufumana impazamo, ifayile ayiyi kubhalwa kwaye ayiyi kuphinda ihanjiswe kwelinye iqela. ;
• xa ubhala ifayile enye, i-IO iya kwiqela elinye kuphela, abanye bangenzi nto;
• awukwazi ukufumana i-IO yomthamo wonke xa ubhala ifayile enye;
• kwaye ingcamango jikelele ibonakala ingaphantsi kwemveliso ngenxa yokunqongophala kwedatha kwiibhloko, apho kulula ukulungelelanisa nokusombulula ingxaki yokusabalalisa okufanayo, kwaye kungekhona ngoku ifayile yonke ingena kwibhloko.

Ukususela kwinkcazo esemthethweni ulwakhiwo siphinde sifikelele ekuqondeni ukuba i-gluster isebenza njengokugcinwa kwefayile phezulu kwe-hardware ye-RAID yakudala. Kubekho iinzame zophuhliso zokusika (Sharding) iifayile zibe ziibhloko, kodwa konke oku kukongeza okubeka ilahleko yokusebenza kwindlela esele ikhona, kunye nokusetyenziswa kwamacandelo asasazwe ngokukhululekileyo kunye nemida yokusebenza njengeFuse. Azikho iinkonzo zemethadatha, ezikhawulela ukusebenza kunye nokunyamezela impazamo yokugcina amandla xa uhambisa iifayile kwiibhloko. Izalathisi zokusebenza ezingcono zinokujongwa ngokucwangciswa kwe "Distributed Replicated" kunye nenani lee-nodes kufuneka libe ubuncinane be-6 ukuququzelela i-replica ethembekileyo ye-3 kunye nokusabalalisa umthwalo ophezulu.

Ezi ziphumo zikwanxulumene nenkcazo yamava omsebenzisi I-Gluster kwaye xa uthelekisa ne UCeph, kwaye kukho kwakhona inkcazo yamava akhokelela ekuqondeni kolu qwalaselo oluvelisayo noluthembekileyo "Kusasazwa ngokuphindaphindiweyo".


Umfanekiso ubonisa ukuhanjiswa komthwalo xa ubhala iifayile ezimbini, apho iikopi zefayile yokuqala zisasazwa kuzo zonke iiseva ezintathu zokuqala, ezidityaniswe kwiqela le-0, kwaye iikopi ezintathu zefayile yesibini zibekwe kwiqela lesibini leqela1 lesithathu. abancedisi. Umncedisi ngamnye unediski enye.

Isiphetho esiqhelekileyo kukuba ungasebenzisa i-Gluster, kodwa ngokuqonda ukuba kuya kuba nemida ekusebenzeni kunye nokunyamezela impazamo okudala ubunzima phantsi kweemeko ezithile zesisombululo se-hyperconverged, apho izibonelelo nazo zifunekayo kwimithwalo yekhompyutheni yeendawo ezibonakalayo.

Kukwakho nezalathisi zokusebenza ze-Gluster ezinokufezekiswa phantsi kweemeko ezithile, ezilinganiselweyo ukunyamezela iimpazamo.

UCeph

Ngoku makhe sijonge kuCeph kwiinkcazo zezakhiwo endikwazileyo fumana. Kukho kwakhona uthelekiso phakathi Glusterfs kunye neCeph, apho unokuqonda ngokukhawuleza ukuba kuyacetyiswa ukuba usebenzise i-Ceph kwiiseva ezihlukeneyo, ekubeni iinkonzo zayo zifuna zonke izixhobo ze-hardware phantsi komthwalo.

izakhiwo Ceph Kunzima ngakumbi kune-Gluster kwaye kukho iinkonzo ezifana neenkonzo zemethadatha, kodwa yonke i-stack yamacandelo inzima kwaye ayiguquki kakhulu ukuyisebenzisa kwisisombululo sokubona. Idatha igcinwe kwiibhloko, ezibonakala zivelisa ngakumbi, kodwa kwi-hierarchy yazo zonke iinkonzo (amacandelo), kukho ilahleko kunye ne-latency phantsi kwemithwalo ethile kunye neemeko zongxamiseko, umzekelo oku kulandelayo. inqaku.

Ukususela kwinkcazo yezakhiwo, intliziyo yi-CRUSH, ngenxa yokuba indawo yokugcina idatha ikhethiwe. Okulandelayo kuza i-PG - le yeyona nto inzima yokuthatha (iqela elinengqiqo) ukuqonda. Ii-PGs ziyafuneka ukwenza i-CRUSH isebenze ngakumbi. Eyona njongo iphambili yePG kukwenza iqela lezinto ukunciphisa ukusetyenziswa kobutyebi, ukwandisa ukusebenza kunye nokulinganisa. Ukujongana nezinto ngokuthe ngqo, ngabanye, ngaphandle kokuzidibanisa zibe yiPG kuyabiza kakhulu. I-OSD yinkonzo yediski nganye.

Iqela linokuba nedatha enye okanye ezininzi kwiinjongo ezahlukeneyo kunye nezicwangciso ezahlukeneyo. Amachibi ahlulwe ngokwamaqela okubeka. Amaqela okubeka agcina izinto ezifikelelwa ngabathengi. Apha kulapho inqanaba lengqiqo iphela khona, kwaye umgangatho womzimba uqala, kuba iqela ngalinye lokubeka labelwe idiski enye engundoqo kunye neediski ezininzi ze-replica (zingaphi kanye ezixhomekeke kwi-pool replication factor). Ngamanye amazwi, kwinqanaba elinengqiqo into igcinwe kwiqela elithile lokubeka, kwaye kwinqanaba lomzimba - kwiidiski ezinikezelwe kuyo. Kule meko, iidiski zinokubekwa ngokwasemzimbeni kwiindawo ezahlukeneyo okanye nakwiindawo ezahlukeneyo zedatha.

Kulo cwangciso, amaqela okubekwa abukeka njengenqanaba eliyimfuneko lokuguquguquka kwesisombululo sonke, kodwa ngexesha elifanayo, njengekhonkco elongezelelweyo kule khonkco, elibonisa ngokungaziboli ukulahlekelwa kwemveliso. Ngokomzekelo, xa ubhala idatha, inkqubo kufuneka ihlukanise kula maqela kwaye emva koko kwinqanaba lomzimba kwi-disk engundoqo kunye neediski zee-replicas. Oko kukuthi, umsebenzi weHash usebenza xa ukhangela kwaye ufaka into, kodwa kukho umphumo - yindleko ephezulu kakhulu kunye nezithintelo zokuvuselela i-hash (xa ukongeza okanye ukususa idiski). Enye ingxaki yehashi yindawo ebethelelwe ngokucacileyo yedatha engenakuguqulwa. Oko kukuthi, ukuba ngandlela ithile idiski iphantsi komthwalo owandisiweyo, ke inkqubo ayinalo ithuba lokungayibhali kuyo (ngokukhetha enye idiski), umsebenzi we-hash ubophelela ukuba idatha ibekwe ngokomgaqo, nokuba imbi kangakanani na. idiski, ke uCeph utya inkumbulo eninzi xa kusakhiwa kwakhona iPG kwimeko yokuziphilisa okanye ukonyuka kogcino. Isiphelo kukuba i-Ceph isebenza kakuhle (nangona icotha), kodwa kuphela xa kungekho kulinganisa, iimeko ezingxamisekileyo, okanye ukuhlaziywa.

Kukho, ewe, ukhetho lokunyusa ukusebenza ngokugcina i-caching kunye nokwabelana nge-cache, kodwa oku kufuna i-hardware efanelekileyo kwaye kuya kubakho ilahleko. Kodwa ngokubanzi, i-Ceph ibonakala ilinga ngakumbi kune-Gluster yemveliso. Kwakhona, xa usebenzisa ezi mveliso, kuyimfuneko ukuthathela ingqalelo into ebalulekileyo - eli linqanaba eliphezulu lobuchule, amava kunye nobugcisa obugxininise kakhulu kwi-Linux, kuba kubaluleke kakhulu ukuhambisa, ukuqwalasela nokugcina yonke into ngokuchanekileyo, nto leyo ibeka uxanduva ngakumbi kunye nomthwalo kumlawuli.

Vstorage

I-architecture ibonakala inomdla ngakumbi Ugcino lweVirtuozzo(Vstorage), enokusetyenziswa ngokubambisana ne-hypervisor kwiindawo ezifanayo, ngokufanayo idlala, kodwa kubaluleke kakhulu ukuqwalasela yonke into ngokuchanekileyo ukufezekisa ukusebenza kakuhle. Oko kukuthi, ukuhambisa imveliso enjalo kwibhokisi kuyo nayiphi na uqwalaselo ngaphandle kokuqwalasela iingcebiso ngokuhambelana nolwakhiwo kuya kuba lula kakhulu, kodwa kungavelisi.

Yintoni enokuhlala kwindawo yokugcina ecaleni kweenkonzo ze-kvm-qemu hypervisor, kwaye ezi ziinkonzo ezimbalwa apho ulawulo olululo lwamacandelo lufunyenwe: inkonzo yomxhasi inyuswe ngeFUSE (ilungisiwe, hayi umthombo ovulekileyo), inkonzo yemetadata yeMDS (Inkonzo yeMetadata), inkonzo yeebhloko zedatha ye-Chunk, leyo kwinqanaba lomzimba lilingana nediski enye kwaye yiyo yonke into. Ngokubhekiselele kwisantya, ewe, kulungile ukusebenzisa iskimu sokunyamezela iimpazamo ezineekopi ezimbini, kodwa ukuba usebenzisa i-caching kunye nelog kwi-SSD drives, ke ikhowudi yokunyamezela iimpazamo (cima ikhowudi okanye i-raid6) inokugqithwa ngokufanelekileyo kwi-SSD. iskimu esixutyiweyo okanye ngcono kuyo yonke iflash. Kukho ukungalungi kunye ne-EC (cima ikhowudi): xa utshintsha ibhloko yedatha enye, kuyimfuneko ukubala kwakhona iimali zokulingana. Ukudlula ilahleko ehambelana nalo msebenzi, uCeph ubhalela i-EC deferredly kwaye iingxaki zokusebenza zingenzeka ngexesha lesicelo esithile, xa, ngokomzekelo, zonke iibhloko kufuneka zifundwe, kwaye kwimeko yeVirtuozzo Storage, ukubhala iibhloko ezitshintshileyo zenziwa. usebenzisa "ilog-structured file system" indlela, enciphisa iindleko zokubala ngokulinganayo. Ukuqikelela malunga neenketho ngokukhawulezisa umsebenzi kunye nangaphandle kwe-EC, kukho isixhobo sokubala. - amanani angaqikelelwa ngokuxhomekeke kwi-coefficient echanekileyo yomenzi wezixhobo, kodwa umphumo wezibalo uncedo oluhle ekucwangciseni uqwalaselo.

Umzobo olula wamacandelo okugcina awuthethi ukuba la macandelo awafuni izixhobo zentsimbi, kodwa ukuba ubala zonke iindleko kwangaphambili, unokuthembela kwintsebenziswano ecaleni kwe-hypervisor.
Kukho iskimu sokuthelekisa ukusetyenziswa kwezixhobo ze-hardware ngeCeph kunye neenkonzo zokugcina zeVirtuozzo.

Ukuba ngaphambili kwakunokwenzeka ukuthelekisa iGluster kunye neCeph usebenzisa amanqaku amadala, usebenzisa imigca ebaluleke kakhulu kubo, ngoko ngeVirtuozzo kunzima kakhulu. Akukho manqaku amaninzi kule mveliso kwaye ulwazi lunokufunyanwa kuphela kuxwebhu ngesiNgesi okanye ngesiRashiya ukuba sijonga iVstorage njengogcino olusetyenziswa kwezinye izisombululo zehyperconverged kwiinkampani ezinjenge Rosplatforma kunye ne-Acronis.

Ndiza kuzama ukunceda ngenkcazo yolu yilo, ngoko kuya kubakho isicatshulwa esincinci, kodwa kuthatha ixesha elide ukuqonda uxwebhu ngokwakho, kwaye amaxwebhu akhoyo anokusetyenziswa kuphela njengesalathisi ngokuhlaziya itafile. yeziqulatho okanye ukukhangela ngegama elingundoqo.

Masithathele ingqalelo inkqubo yokurekhoda kuqwalaselo lwesixhobo esixutyiweyo esinamacandelo achazwe ngasentla: ukurekhodwa kuqala ukuya kwindawo apho umxhasi awuqalisile khona (inkonzo yendawo yokunyusela yeFUSE), kodwa inkonzo yemetadata (MDS) icandelo elilawulayo liya kuba njalo. uqondise umxhasi ngokuthe ngqo kwinkonzo ye-chunk efunwayo (iibhloko zenkonzo yokugcina i-CS), oko kukuthi, i-MDS ayithathi nxaxheba kwinkqubo yokurekhoda, kodwa iqondisa nje inkonzo kwi-chunk efunekayo. Ngokubanzi, sinokunika isifaniso sokurekhoda ngokugalela amanzi kwimibhobho. Umphanda ngamnye yi-256MB yedata block.

То есть один диск, это какое-то количество таких бочек, то есть объем диска разделить на 256МБ. Каждая копия разливается на один узел, вторая почти параллельно уже на другой узел и т.д… Если у нас три реплики и есть SSD диски, для кеша (под чтение и журналы записи), то подтверждение записи будет происходить после записи журнала в SSD, а параллельный сброс с SSD, будет продолжаться на HDD, как бы в фоновом режиме. В случае трех реплик коммит записи будет после подтверждения от SSD третьего узла. Может показаться, что сумму скорости записи трех SSD можно поделить на три и мы получим скорость записи одной реплики, но запись копий идет параллельно и скорость Latency сети обычно выше, чем у SSD, и по сути производительность записи будет зависеть от сети. В связи с этим, чтобы увидеть реальные IOPS необходимо правильно нагрузить весь Vstorage по indlela yokusebenza, oko kukuthi, ukuvavanya umthwalo wangempela, kwaye kungekhona imemori kunye ne-cache, apho kuyimfuneko ukuqwalasela ubungakanani bebhloko yedatha echanekileyo, inani lemicu, njl.

Ilogi yokurekhoda ekhankanywe ngasentla kwi-SSD isebenza ngendlela yokuba ngokukhawuleza ukuba idatha ingene kuyo, ifundeka ngokukhawuleza yinkonzo kwaye ibhalwe kwi-HDD. Kukho iinkonzo ezininzi zemetadata (MDS) ngeqela ngalinye kwaye inani labo limiselwa ikhoram, esebenza ngokwe-algorithm ye-Paxos. Ukusuka kwindawo yokujonga umxhasi, i-FUSE mount point ifolda yokugcina i-cluster ebonakala ngaxeshanye kuzo zonke ii-node kwi-cluster, i-node nganye inomxhasi ophakanyisiweyo ngokwalo mgaqo, ngoko le ndawo yokugcina ifumaneka kwi-node nganye.

Ngokwenziwa kwazo naziphi na iindlela ezichazwe ngasentla, kubaluleke kakhulu, kwisigaba sokucwangcisa kunye nokuthunyelwa, ukuqwalasela ngokuchanekileyo inethiwekhi, apho kuya kubakho ukulinganisa ngenxa yokudibanisa kunye nokukhethwa ngokuchanekileyo kwe-bandwidth yesiteshi senethiwekhi. Ukuhlanganiswa, kubalulekile ukukhetha imowudi ye-hashing efanelekileyo kunye nobukhulu besakhelo. Kukho nomehluko onamandla kakhulu kwi-SDS echazwe ngasentla, le fuse ngeteknoloji yendlela ekhawulezayo kwi-Virtuozzo Storage. Oku, ukongeza kwi-fuse yanamhlanje, ngokungafaniyo nezinye izisombululo zemithombo evulekileyo, yonyusa kakhulu i-IOPS kwaye ikuvumela ukuba ungathintelwa kukukalwa okuthe tye okanye ngokuthe nkqo. Ngokubanzi, xa kuthelekiswa nezakhiwo ezichazwe ngasentla, le nto ibonakala inamandla ngakumbi, kodwa ngenxa yolonwabo olunjalo, ngokuqinisekileyo, kufuneka uthenge iilayisensi, ngokungafaniyo neCeph kunye neGluster.

Ukushwankathela, sinokugqamisa phezulu kwezi zintathu: I-Virtuozzo Storage ithatha indawo yokuqala ngokubhekiselele ekusebenzeni kunye nokuthembeka kwesakhiwo, i-Ceph ithatha indawo yesibini, kwaye i-Gluster ithatha indawo yesithathu.

Iikhrayitheriya apho iVirtuozzo Storage ikhethwe khona: yiseti echanekileyo yezixhobo zokwakha, iphuculwe le ndlela yeFuse kunye nomendo okhawulezayo, iseti eguquguqukayo yoqwalaselo lwehardware, ukusetyenziswa kwezixhobo ezingaphantsi kunye nokukwazi ukwabelana ngecomputing (computing/virtualization), oko kukuthi, ifanelekile ngokupheleleyo kwisisombululo se-hyperconverged, eyinxalenye yayo. Indawo yesibini yiCeph kuba iyisakhiwo esinemveliso ngakumbi xa kuthelekiswa neGluster, ngenxa yokusebenza kwayo kwiibhloko, kunye neemeko eziguquguqukayo kunye nokukwazi ukusebenza kumaqela amakhulu.

Kukho izicwangciso zokubhala uthelekiso phakathi kwe-vSAN, i-Space Direct Storage, i-Vstorage kunye ne-Nutanix Storage, ukuvavanya i-Vstorage kwi-HPE kunye nezixhobo zeHuawei, kunye neemeko zokudibanisa i-Vstorage kunye neenkqubo zokugcinwa kwe-hardware yangaphandle, ngoko ke ukuba uthanda inqaku, kuya kuba kumnandi ukufumana impendulo kuwe, enokunyusa inkuthazo yamanqaku amatsha, kuthathelwa ingqalelo amagqabantshintshi kunye neminqweno yakho.

umthombo: www.habr.com