Bathi yonke into ebomini kufuneka uyizame nokuba kube kanye. Kwaye ukuba uqhele ukusebenza kunye ne-DBMS yobudlelwane, ngoko kuyafaneleka ukuba uqhelane ne-NoSQL ekusebenzeni, okokuqala, ubuncinane kuphuhliso jikelele. Ngoku, ngenxa yophuhliso olukhawulezayo lwale teknoloji, kukho iimbono ezininzi eziphikisanayo kunye neengxoxo ezishushu malunga nesi sihloko, esibangela ngokukodwa umdla.
Ukuba uphonononga undoqo wazo zonke ezi ngxabano, unokubona ukuba zivela ngenxa yendlela engalunganga. Abo basebenzisa i-database ye-NoSQL kanye apho bafuneka khona banelisekile kwaye bafumana zonke iingenelo kwesi sisombululo. Kwaye abavavanyi abaxhomekeke kule teknoloji njenge-panacea apho ingasebenziyo konke konke bayaphoxeka, belahlekelwe amandla ogcino-lwazi olunxulumeneyo ngaphandle kokufumana izibonelelo ezibalulekileyo.
Ndiza kukuxelela ngamava ethu ekuphunyezweni kwesisombululo esisekelwe kwi-Cassandra DBMS: into esimele sibhekane nayo, indlela esiphuma ngayo kwiimeko ezinzima, nokuba sakwazi ukuzuza ngokusebenzisa i-NoSQL kunye nalapho kufuneka sityale imizamo eyongezelelweyo / imali. .
Umsebenzi wokuqala kukwakha inkqubo erekhoda iifowuni kuhlobo oluthile lokugcina.
Umgaqo wokusebenza wenkqubo ngolu hlobo lulandelayo. Igalelo libandakanya iifayile ezinesakhiwo esithile esichaza isakhiwo sefowuni. Isicelo ke siqinisekisa ukuba esi sakhiwo sigcinwe kwiikholamu ezifanelekileyo. Kwixesha elizayo, iifowuni ezigciniweyo zisetyenziselwa ukubonisa ulwazi malunga nokusetyenziswa kwetrafikhi kubabhalisi (iintlawulo, iifowuni, imbali yebhalansi).
Kucace gca ukuba kutheni bekhethe uCassandra-ubhala njengompu womatshini, kulula ukuhla, kwaye uyazinyamezela iimpazamo.
Ngoko, oku koko amava asinike wona
Ewe, i-node engaphumelelanga ayikho intlekele. Lo ngundoqo wokunyamezela iimpazamo zikaCassandra. Kodwa i-node inokuphila kwaye kwangaxeshanye iqale ukuhlupheka ekusebenzeni. Njengoko kwavela, oku kuchaphazela ngokukhawuleza ukusebenza kweqela lonke.
I-Cassandra ayiyi kukukhusela apho i-Oracle ikugcine khona kunye nemiqobo yayo. Kwaye ukuba umbhali wesicelo akazange aqonde oku kwangaphambili, ngoko ke i-double efikile kwiCassandra ayikho mbi kuneyokuqala. Xa sele ifikile, siya kuyifaka.
U-IB akayithandanga kakhulu iCassandra yasimahla ngaphandle kwebhokisi: Akukho kugawulwa kwezenzo zabasebenzisi, akukho mahluko wamalungelo. Ulwazi malunga neefowuni kuthathwa njengedatha yomntu, oku kuthetha ukuba zonke iinzame zokuyicela / ukuyitshintsha nangayiphi na indlela kufuneka ifakwe kunye nokwenzeka kophicotho olulandelayo. Kwakhona, kufuneka uqaphele imfuneko yokwahlula amalungelo kumanqanaba ahlukeneyo kubasebenzisi abohlukeneyo. Injineli yokusebenza elula kunye nomphathi omkhulu onokuthi acime ngokukhululekileyo yonke indawo yesitshixo ziindima ezahlukeneyo, uxanduva olwahlukileyo, kunye nobuchule. Ngaphandle kolwahlulo olunjalo lwamalungelo ofikelelo, ixabiso kunye nemfezeko yedatha iya kungena kumbuzo ngokukhawuleza kunokuba KUNYE inqanaba lokuhambelana.
Khange siyithathele ngqalelo into yokuba iifowuni zifuna zombini uhlalutyo olunzulu kunye nesampulu ngamaxesha athile kwiimeko ezahlukeneyo. Ekubeni iirekhodi ezikhethiweyo zifanele ukuba zicinywe kwaye zibhalwe kwakhona (njengenxalenye yomsebenzi, kufuneka sixhase inkqubo yokuhlaziya idatha xa idatha ingena ekuqaleni i-loop yethu ngokungalunganga), uCassandra akayena umhlobo wethu apha. I-Cassandra ifana nebhanki yehagu - ikulungele ukubeka izinto, kodwa awukwazi ukubala kuyo.
Sifumene ingxaki yokuhambisa idatha kwiindawo zovavanyo (Iindawo ezi-5 kuvavanyo xa zithelekiswa nama-20 kwi-prom). Kule meko, ukulahla akunakusetyenziswa.
Ingxaki ngohlaziyo lwe-schema yedatha yokubhala isicelo kwiCassandra. Ukubuyisela umva kuya kuvelisa amatye amangcwaba amaninzi, anokukhokelela kwilahleko yemveliso ngeendlela ezingalindelekanga.. ICassandra ilungiselelwe ukurekhodwa, kwaye ayicingi kakhulu phambi kokuba ibhale.Nawuphi na umsebenzi onedatha ekhoyo kuyo ukwarekhodwa. Oko kukuthi, ngokususa okungafunekiyo, siya kuvelisa ngakumbi iirekhodi, kwaye ezinye zazo kuphela ziya kumakishwa ngamatye engcwaba.
Ukuphela kwexesha xa ufaka. UCassandra umhle ekurekhodeni, kodwa ngamanye amaxesha ukuhamba okungenayo kunokumxaka kakhulu. Oku kwenzeka xa isicelo siqala ukujikeleza iirekhodi ezininzi ezingenakufakwa ngenxa yesizathu esithile. Kwaye siya kufuna i-DBA yokwenyani eya kubeka iliso kwi-gc.log, inkqubo kunye ne-debug logs kwimibuzo ecothayo, i-metrics kwi-compaction elindileyo.
Amaziko amaninzi edatha kwiqela. Ufunda phi kwaye ubhale phi?
Mhlawumbi yahlulahlulwe ekufundeni nasekubhaleni? Kwaye ukuba kunjalo, kufuneka kubekho iDC ekufutshane nesicelo sokubhala okanye sokufunda? Kwaye asiyi kugqiba ngengqondo yokwahlula okwenyani ukuba sikhetha inqanaba lokungaguquguquki elingalunganga? Kukho imibuzo emininzi, uninzi lwesetingi ezingaziwayo, izinto ezinokwenzeka ofuna ukucinga ngazo.
Indlela esigqibe ngayo
Ukuthintela i-node ekuzikeni, i-SWAP iye yacinywa. Kwaye ngoku, ukuba kukho ukunqongophala kwememori, i-node kufuneka yehle kwaye ingenzi ikhefu elikhulu le-gc.
Ngoko, asisathembeli kwingqiqo kwisiseko sedatha. Abaphuhlisi besicelo bayaziqeqesha kwakhona kwaye baqala ukuthatha amanyathelo okhuseleko kwikhowudi yabo. Ukwahlula okufanelekileyo okucacileyo kokugcinwa kwedatha kunye nokucubungula.
Sithenge inkxaso kwi-DataStax. Ukuphuhliswa kwebhokisi yeCassandra sele kuphelile (isenzo sokugqibela sasingoFebruwari 2018). Ngexesha elifanayo, i-Datastax inikezela ngenkonzo egqwesileyo kunye nenani elikhulu lezisombululo eziguquliweyo nezilungelelanisiweyo kwizisombululo ze-IP ezikhoyo.
Ndifuna kwakhona ukuqaphela ukuba iCassandra ayifanelekanga kakhulu kwimibuzo yokhetho. Ewe, i-CQL linyathelo elikhulu eliya phambili kubasebenzisi (xa kuthelekiswa ne-Trift). Kodwa ukuba unamasebe apheleleyo aqhelene nokudityaniswa okufanelekileyo, ukuhluzwa kwasimahla kuyo nayiphi na intsimi kunye nobuchule bokwenza imibuzo, kwaye la masebe asebenza ukusombulula izikhalazo kunye neengozi, ke isisombululo kwiCassandra sibonakala sinobutshaba kwaye sisidenge kubo. Kwaye saqala ukwenza isigqibo sokuba oogxa bethu kufuneka benze njani iisampulu.
Siqwalasele iinketho ezimbini.Kukhetho lokuqala, asibhali iminxeba kuphela kwi-C*, kodwa nakwindawo egciniweyo yedatha ye-Oracle. Kuphela, ngokungafaniyo ne-C *, le datha yokugcina i-database ibiza kuphela kwinyanga yangoku (ubunzulu obaneleyo bokugcina umnxeba kwiimeko zokutshaja kwakhona). Apha ngokukhawuleza sabona le ngxaki ilandelayo: ukuba sibhala ngokuhambelanayo, emva koko siphulukana nazo zonke iingenelo ze-C* ezinxulumene nokufakwa ngokukhawuleza; ukuba sibhala ngokungafaniyo, akukho siqinisekiso sokuba zonke iifowuni eziyimfuneko zingene kwi-Oracle konke konke. Bekukho enye yokudibanisa, kodwa enkulu: ekusebenzeni kwalowo uqhelekileyo we-PL/SQL woMphuhlisi usahleli, o.k.t. siphumeza ipateni "yeFacade". Sisebenzisa indlela yokothula iminxeba esuka kwi-C*, itsalela idatha ethile yokutyebisa kwiitheyibhile ezihambelanayo kwi-Oracle, idibanisa iisampulu ezisisiphumo kwaye isinike isiphumo, esiya kuthi ngandlela thile sisebenzise (buyela umva, phinda, uhlalutye, uncome). I-Cons: inkqubo inamanyathelo amaninzi, kwaye ukongeza, akukho jongano lwabasebenzi abasebenza.
Ekugqibeleni, sahlala kwinketho yesibini. I-Apache Spark yayisetyenziselwa ukwenza isampuli kwiingqayi ezahlukeneyo. Ingundoqo yendlela iye yancitshiswa kwikhowudi yeJava, leyo, ngokusebenzisa izitshixo ezichaziweyo (umrhumi, ixesha lokufowuna - izitshixo zecandelo), ikhupha idatha kwi-C *, kunye nedatha efunekayo yokutyebisa kuyo nayiphi na enye i-database. Emva koko idibanisa nabo kwimemori yayo kwaye ibonise umphumo kwitheyibhile enesiphumo. Sizoba ubuso bewebhu phezu kwentlantsi kwaye yabonakala isebenziseka.
Xa usombulula ingxaki yokuhlaziya idatha yovavanyo lwamashishini, siphinde saqwalasela izisombululo ezininzi. Zombini ukudluliselwa nge-Sssloader kunye nokhetho lokwahlula iqela kwindawo yovavanyo ibe ziinxalenye ezimbini, nganye kuzo ngokutshintshanayo iyeyeqela elinye kunye nelokwazisa, ngaloo ndlela inikwa amandla yiyo. Xa uhlaziywa uvavanyo, kwakucetywe ukuba batshintshe: inxalenye esebenzayo kuvavanyo iyacinywa kwaye ifakwe kwimveliso, kwaye enye iqala ukusebenza kunye nedatha ngokwahlukileyo. Nangona kunjalo, emva kokucinga kwakhona, sivavanye ngokunengqiqo idatha eyayifanele ukudluliselwa, kwaye saqonda ukuba iifowuni ngokwazo ziyinto engahambelaniyo yovavanyo, eyenziwa ngokukhawuleza xa kuyimfuneko, kwaye yidatha yokuthengisa engenaxabiso lokudluliselwa kwi uvavanyo. Kukho izinto ezininzi zokugcina ezifanele ukuhanjiswa, kodwa ezi ziitafile ezimbalwa, kwaye azinzima kakhulu. Ngoko ke thina njengesisombululo, u-Spark waphinda weza kuhlangula, ngoncedo esibhala ngalo kwaye saqala ukusebenzisa ngokusebenzayo iskripthi sokudlulisa idatha phakathi kweetafile, uvavanyo lwe-prom.
Umgaqo-nkqubo wethu wangoku wokusasaza usivumela ukuba sisebenze ngaphandle kokubuyela umva. Ngaphambi kwepromo, kukho uvavanyo olunyanzelekileyo, apho impazamo ayibizi kangako. Kwimeko yokusilela, ungasoloko ulahla i-casespace kwaye uqengqe iskimu sonke ukusuka ekuqaleni.
Ukuqinisekisa ukufumaneka okuqhubekayo kweCassandra, udinga i-dba hayi yena kuphela. Wonke umntu osebenza kunye nesicelo kufuneka aqonde apho kwaye njani ukujonga imeko yangoku kunye nendlela yokufumanisa iingxaki ngexesha elifanelekileyo. Ukwenza oku, sisebenzisa ngokusebenzayo i-DataStax OpsCenter (Ulawulo kunye nokubeka iliso kwimithwalo yomsebenzi), iimetriki zenkqubo yeCassandra Driver (inani lokuphuma kwexesha lokubhala kwi-C *, inani lexesha lokufunda ukusuka kwi-C *, ubuninzi be-latency, njl.), ukubeka iliso ekusebenzeni yesicelo ngokwaso, sisebenza noCassandra.
Xa sicinga ngombuzo wangaphambili, saqonda apho umngcipheko wethu omkhulu unokulala khona. Ezi ziifom zokubonisa idatha ezibonisa idatha ukusuka kwimibuzo emininzi ezimeleyo ukuya kwindawo yokugcina. Ngale ndlela sinokufumana ulwazi olungangqinelaniyo. Kodwa le ngxaki iya kuba yinto efanelekileyo ukuba sisebenze kunye neziko ledatha enye kuphela. Ke eyona nto isengqiqweni apha, kunjalo, kukwenza umsebenzi webhetshi wokufunda idatha kwisicelo somntu wesithathu, oya kuqinisekisa ukuba idatha ifunyenwe ngexesha elinye. Ngokuphathelele ulwahlulo ekufundeni nasekubhaleni ngokwemigaqo yokusebenza, apha siye samiswa ngumngcipheko wokuba ngokulahleka koqhagamshelwano phakathi kwe-DCs, sinokugqiba ngamaqoqo amabini angahambelani ngokupheleleyo.
Ngenxa yoko, okwangoku ime kwinqanaba lokuvumelana ukuze kubhalwe EACH_QUORUM, ukufunda - LOCAL_QUORUM
Izimvo ezimfutshane kunye nezigqibo
Ukuze sivavanye isisombululo esisisiphumo kwimbono yenkxaso yokusebenza kunye nethemba lophuhliso olongezelelweyo, sagqiba ekubeni sicinge malunga nokuba uphuhliso olunjalo lunokuthi lusetyenziswe phi.
Kanye kwilulwane, emva koko inqaku ledatha kwiinkqubo ezinje ngo “Hlawula xa kufaneleka” (sifaka ulwazi kwi-C*, ukubala kusetyenziswa izikripthi zeSpark), ukubalwa kwamabango ngokudityaniswa kwendawo, ukugcina iindima nokubala amalungelo okufikelela komsebenzisi ngokusekelwe kwindima. imatrix.
Njengoko ubona, irepertoire ibanzi kwaye iyahluka. Kwaye ukuba sikhetha inkampu yabaxhasi / abachasi beNoSQL, ngoko siya kujoyina abaxhasi, ekubeni sifumene izibonelelo zethu, kunye nalapho silindele khona.
Nokuba ukhetho lweCassandra ngaphandle kwebhokisi luvumela ukukala okuthe tye ngexesha langempela, ukusombulula ngokungenabuhlungu umba wokwandisa idatha kwinkqubo. Siye sakwazi ukuhambisa indlela yomthwalo ophezulu kakhulu wokubala ii-aggregates zefowuni kwisekethe eyahlukileyo, kwaye kwakhona ukwahlula i-schema yesicelo kunye nengqiqo, ukulahla isenzo esibi sokubhala imisebenzi yesiko kunye nezinto kwi-database ngokwayo. Sifumene ithuba lokukhetha kunye nokuqwalasela, ukukhawuleza, zeziphi ii-DCs esiya kuzenza izibalo kunye nokuba yeyiphi esiya kurekhoda idatha kuyo, sazikhusela nge-inshurensi malunga nokuphahlazeka kweendawo zombini kunye ne-DC ngokubanzi.
Ukusebenzisa i-architecture yethu kwiiprojekthi ezintsha, kwaye sele ndinamava athile, ndingathanda ukuba ndithathele ingqalelo ngokukhawuleza ama-nuances achazwe ngasentla, kwaye ndithintele iimpazamo ezithile, ndigudise iikona ezibukhali ezingenakuthintelwa ekuqaleni.
Ngokomzekelo, gcina umkhondo wohlaziyo lweCassandra ngexesha elifanelekileyokuba iingxaki ezimbalwa esithe sazifumana sele zaziwa kwaye zilungisiwe.
Musa ukubeka zombini i-database ngokwayo kunye ne-Spark kwiindawo ezifanayo (okanye ulwahlule ngokungqongqo ngenani lokusetyenziswa kobutyebi obuvumelekileyo), kuba i-Spark inokutya i-OP eninzi kunokuba bekulindelwe, kwaye siya kufumana ngokukhawuleza inombolo ye-1 kuluhlu lwethu.
Ukuphucula ukubeka iliso kunye nobuchule bokusebenza kwinqanaba lovavanyo lweprojekthi. Ekuqaleni, thatha ingqalelo kangangoko kunokwenzeka kubo bonke abathengi abanokubakho kwisisombululo sethu, kuba le nto isiseko sesiseko siya kuxhomekeka ekugqibeleni.
Jikelezisa isekethe enesiphumo amatyeli aliqela ukwenzela ukuba kunokwenzeka. Khetha ukuba yeyiphi imimandla enokwenziwa uthotho. Qonda ukuba zeziphi iitheyibhile ezongezelelweyo ekufuneka sizenze ukuze sithathelwe ingqalelo kakhulu ngokuchanekileyo nangokufanelekileyo, kwaye emva koko sinikeze ulwazi olufunekayo xa uceliwe (umzekelo, ngokucinga ukuba sinokugcina idatha efanayo kwiitheyibhile ezahlukeneyo, sithathela ingqalelo ukwahlulahlula ngokwahlukeneyo iindlela ezahlukeneyo, sinokugcina ixesha le-CPU kakhulu kwizicelo zokufunda).
Akukubanga Ubonelele ngoko nangoko ngokuncamathisela i-TTL kunye nokucoca idatha ephelelwe lixesha.
Xa ukhuphela idatha kwiCassandra Ingqiqo yesicelo kufuneka isebenze kumgaqo we-FETCH, ukuze ingabi yonke imiqolo elayishwe kwinkumbulo kanye, kodwa ikhethwe ngokweebhetshi.
Kucetyiswa ngaphambi kokudlulisela iprojekthi kwisisombululo esichazwe khangela unyamezelo lwempazamo yenkqubo ngokuqhuba uthotho lweemvavanyo zokuwa, njengokulahleka kwedatha kwiziko ledatha enye, ukubuyiswa kwedatha eyonakalisiweyo ngexesha elithile, ukuyeka kwenethiwekhi phakathi kwamaziko edatha. Iimvavanyo ezinjalo aziyi kuvumela kuphela ukuba umntu avavanye okulungileyo kunye nokungalunganga koyilo olucetywayo, kodwa luya kubonelela ngokufudumala okulungileyo kwiinjineli eziqhubayo, kwaye isakhono esifunyenweyo siya kuba kude kakhulu ukuba ukusilela kwenkqubo kuveliswa kwimveliso.
Ukuba sisebenza ngolwazi olubalulekileyo (njengedatha yokuhlawula, ukubalwa kwamatyala obhalisile), ngoko kuyafaneleka ukuba sinikele ingqalelo kwizixhobo eziza kunciphisa iingozi ezivela ngenxa yeempawu zeDBMS. Umzekelo, sebenzisa i-nodesync utility (Datastax), uphuhlise isicwangciso esisiso sokusetyenziswa kwayo ngokulandelelana. ngenxa yokungaguquguquki, musa ukwenza umthwalo ogqithisileyo kwiCassandra kwaye uyisebenzise kuphela kwiitafile ezithile kwixesha elithile.
Kwenzeka ntoni kuCassandra emva kweenyanga ezintandathu zobomi? Ngokubanzi, akukho zingxaki zingasonjululwanga. Kananjalo asizange sivumele naziphi na iingozi ezimandundu okanye ilahleko yedatha. Ewe, kuye kwafuneka sicinge malunga nokuhlawulela ezinye iingxaki ezingazange zivele ngaphambili, kodwa ekugqibeleni oku akuzange kusithintele kakhulu isisombululo sethu soyilo. Ukuba ufuna kwaye ungesabi ukuzama into entsha, kwaye ngexesha elifanayo ungafuni ukuphoxeka kakhulu, ke ulungele ukuba akukho nto ikhululekile. Kuya kufuneka uqonde, ujonge kumaxwebhu kwaye uhlanganise eyakho iraki ngaphezulu kwesisombululo selifa elidala, kwaye akukho ithiyori iya kukuxelela kwangaphambili ukuba yeyiphi iraki ekulindeleyo.
umthombo: www.habr.com