Iwo amanena kuti chilichonse m’moyo n’chofunika ngakhale pang’ono. Ndipo ngati mumazolowera kugwira ntchito ndi ma DBMS apaubale, ndiye kuti ndikofunikira kudziwana ndi NoSQL pochita, choyamba, makamaka pachitukuko chonse. Tsopano, chifukwa cha chitukuko chofulumira cha teknolojiyi, pali malingaliro ambiri otsutsana ndi mikangano yotentha pamutuwu, zomwe makamaka zimalimbikitsa chidwi.
Ngati muyang'ana pa chiyambi cha mikangano yonseyi, mutha kuwona kuti imabwera chifukwa cha njira yolakwika. Iwo omwe amagwiritsa ntchito nkhokwe za NoSQL ndendende komwe amafunikira amakhutitsidwa ndikulandila zabwino zonse kuchokera ku yankho ili. Ndipo oyesera omwe amadalira ukadaulo uwu ngati njira yothanirana ndi zomwe sizikugwira ntchito konse amakhumudwitsidwa, atataya mphamvu zamawu achibale osapeza phindu lalikulu.
Ndikuuzani za zomwe takumana nazo pokwaniritsa njira yothetsera vutoli pogwiritsa ntchito Cassandra DBMS: zomwe tinayenera kukumana nazo, momwe tinachokera ku zovuta, kaya titha kupindula pogwiritsa ntchito NoSQL komanso komwe tinayenera kuyika ndalama zowonjezera / ndalama. .
Ntchito yoyamba ndikumanga dongosolo lomwe limalemba mafoni mumtundu wina wosungira.
Mfundo ntchito dongosolo ndi motere. Kulowetsa kumaphatikizapo mafayilo omwe ali ndi dongosolo linalake lomwe limalongosola kamangidwe ka kuyitana. Ntchitoyi imatsimikizira kuti dongosololi likusungidwa m'mizere yoyenera. M'tsogolomu, mafoni osungidwa amagwiritsidwa ntchito kusonyeza zambiri zokhudzana ndi kugwiritsidwa ntchito kwa magalimoto kwa olembetsa (ndalama, mafoni, mbiri yakale).
Ndizodziwikiratu chifukwa chake adasankha Cassandra - amalemba ngati mfuti yamakina, ndiowopsa komanso wololera.
Kotero, izi ndi zomwe zinatipatsa ife
Inde, node yolephera si tsoka. Ichi ndiye chiyambi cha kulekerera kwa Cassandra. Koma node ikhoza kukhala yamoyo ndipo nthawi yomweyo imayamba kuvutika mukuchita. Monga momwe zinakhalira, izi zimakhudza nthawi yomweyo ntchito ya gulu lonse.
Cassandra sangakutetezeni komwe Oracle adakupulumutsirani ndi zovuta zake. Ndipo ngati wolemba ntchitoyo sanamvetse izi pasadakhale, ndiye kuti kuwirikiza komwe kunafika kwa Cassandra sikuli koyipa kuposa choyambirira. Ikafika, tiziyikamo.
IB sanakonde kwambiri Cassandra waulere kunja kwa bokosi: Palibe kudulidwa kwa zochita za ogwiritsa ntchito, palibe kusiyanitsa kwa ufulu. Zambiri zokhudza mafoni zimatengedwa kuti ndi zaumwini, zomwe zikutanthauza kuti zoyesayesa zonse zopempha / kuzisintha mwanjira iliyonse ziyenera kulowetsedwa ndi kuthekera kwa kafukufuku wotsatira. Komanso, muyenera kudziwa kufunika kolekanitsa maufulu pamagulu osiyanasiyana kwa ogwiritsa ntchito osiyanasiyana. Katswiri wosavuta wogwiritsa ntchito komanso woyang'anira wamkulu yemwe amatha kufufuta malo onse ofunikira ndi maudindo osiyanasiyana, maudindo osiyanasiyana, komanso luso. Popanda kusiyanitsa koteroko kwa ufulu wopeza, mtengo ndi kukhulupirika kwa deta nthawi yomweyo zidzakayikiridwa mofulumira kusiyana ndi mlingo wa ALIYENSE wosasinthasintha.
Sitinaganizire kuti kuyimba kumafunikira kusanthula kwakanthawi komanso kusanja pafupipafupi pamikhalidwe yosiyanasiyana. Popeza zolemba zosankhidwa ziyenera kuchotsedwa ndikulembedwanso (monga gawo la ntchitoyo, tiyenera kuthandizira njira yosinthira deta pomwe deta idalowa molakwika molakwika), Cassandra si bwenzi lathu pano. Cassandra ali ngati banki ya nkhumba - ndizosavuta kuyika zinthu, koma simungathe kuziwerengera.
Tinakumana ndi vuto posamutsa deta kupita kumalo oyesera (Node 5 pamayeso motsutsana ndi 20 mu prom). Pankhaniyi, kutaya sikungagwiritsidwe ntchito.
Vuto pakukonzanso schema ya data ya pulogalamu yolembera ku Cassandra. Kubweza kumabweretsa miyala yambiri yam'manda, yomwe ingayambitse kutayika kwa zokolola m'njira zosayembekezereka.. Cassandra ndiwokonzeka kujambula, ndipo saganiza zambiri asanalembe. Ndiko kuti, pochotsa zosafunikira, tidzangopanga zolemba zambiri, ndipo zina mwazo ndizo zomwe zidzalembedwa ndi miyala yamanda.
Kutha kwa nthawi pakulowetsa. Cassandra ndi wokongola mu kujambula, koma nthawi zina kutuluka kwake komwe kukubwera kumatha kumusokoneza kwambiri. Izi zimachitika pomwe pulogalamuyo iyamba kuzungulira zolemba zingapo zomwe sizingayikidwe pazifukwa zina. Ndipo tidzafunika DBA yeniyeni yomwe idzayang'anire gc.log, system ndi debug logs kuti tifufuze pang'onopang'ono, ma metrics pa compaction akudikirira.
Ma data angapo ali mgulu. Kuwerenga kuchokera komanso komwe mungalembe?
Mwina anagawanika kuŵerenga ndi kulemba? Ndipo ngati ndi choncho, payenera kukhala DC pafupi ndi ntchito yolemba kapena kuwerenga? Ndipo kodi sitidzatha ndi ubongo weniweni wogawanika ngati tisankha mulingo wolakwika wosasinthasintha? Pali mafunso ambiri, zosintha zambiri zosadziwika, zotheka zomwe mukufunadi kuziganizira.
Momwe tinasankha
Kuletsa node kuti isamire, SWAP idayimitsidwa. Ndipo tsopano, ngati pali kusowa kwa kukumbukira, mfundoyi iyenera kutsika osati kupanga kuyimitsa kwakukulu kwa gc.
Chifukwa chake, sitidaliranso logic mu database. Opanga mapulogalamu akudziphunzitsanso ndipo akuyamba kusamala ndi ma code awo. Kulekanitsa komveka bwino kosungirako ndi kukonza deta.
Tinagula chithandizo kuchokera ku DataStax. Kukula kwa bokosi la Cassandra kwatha kale (chomaliza chinali mu February 2018). Panthawi imodzimodziyo, Datastax imapereka ntchito yabwino kwambiri komanso mayankho ambiri osinthidwa ndi osinthidwa a IP omwe alipo.
Ndikufunanso kudziwa kuti Cassandra siyothandiza kwambiri pazosankha. Inde, CQL ndi sitepe yaikulu kwa ogwiritsa ntchito (poyerekeza ndi Trift). Koma ngati muli ndi madipatimenti athunthu omwe amazolowera kujowina kosavuta kotere, kusefa kwaulere ndi gawo lililonse ndi kuthekera kokwanira kwamafunso, ndipo madipatimentiwa akugwira ntchito kuti athetse madandaulo ndi ngozi, ndiye kuti yankho la Cassandra likuwoneka lodana ndi lopusa kwa iwo. Ndipo tinayamba kusankha momwe anzathu ayenera kupanga zitsanzo.
Tidawona njira ziwiri: Munjira yoyamba, timalemba mafoni osati mu C *, komanso mu database ya Oracle yosungidwa. Pokhapokha, mosiyana ndi C *, malo osungirako malowa amangoyitanira mwezi wapano (kuzama kokwanira kosungirako kuyimba kwa milandu yoyitanitsa). Apa tidawona nthawi yomweyo vuto ili: ngati tilemba molumikizana, ndiye kuti timataya zabwino zonse za C * zomwe zimalumikizidwa ndikuyika mwachangu; ngati tilemba mosagwirizana, palibe chitsimikizo kuti mafoni onse ofunikira adalowa mu Oracle nkomwe. Panali kuphatikizira kumodzi, koma kwakukulu: kuti mugwiritse ntchito PL/SQL Developer yemweyo amakhalabe, mwachitsanzo, timakhazikitsa dongosolo la "Facade". Timagwiritsa ntchito makina omwe amatsitsa mafoni kuchokera ku C *, amakoka zambiri kuti alemeredwe kuchokera pamatebulo ofananirako ku Oracle, kujowina zitsanzozo ndikutipatsa zotsatira, zomwe timagwiritsa ntchito mwanjira ina (kubwereza, kubwereza, kusanthula, kusilira). kuipa: ndondomekoyi ndithu Mipikisano masitepe, ndipo kuwonjezera, palibe mawonekedwe ogwira ntchito.
Pamapeto pake, tinakhazikika pa njira yachiwiri. Apache Spark idagwiritsidwa ntchito kuyesa mitsuko yosiyanasiyana. Chofunika kwambiri cha makinawo chachepetsedwa kukhala code ya Java, yomwe, pogwiritsa ntchito makiyi otchulidwa (olembetsa, nthawi yoyitana - makiyi a gawo), imakoka deta kuchokera ku C *, komanso deta yofunikira kuti alemere kuchokera ku database ina iliyonse. Pambuyo pake imawaphatikiza mu kukumbukira kwake ndikuwonetsa zotsatira zake patebulo lotsatira. Tinajambula nkhope yapaintaneti pamwamba pa spark ndipo zidawoneka ngati zothandiza.
Pothetsa vuto la kukonzanso deta yoyesa mafakitale, tinaganiziranso njira zingapo. Kusamutsa konseku kudzera pa Ssloader ndi mwayi wogawa gululo mugawo loyesa m'magawo awiri, lomwe limakhala la gulu lomwelo ndi lotsatsa, motero likuyendetsedwa nalo. Pokonzanso mayesowo, adakonzedwa kuti asinthe: gawo lomwe lidagwirapo mayeso limachotsedwa ndikulowa mukupanga, ndipo linalo limayamba kugwira ntchito ndi deta padera. Komabe, titaganiziranso, tidayesanso momveka bwino zomwe zidayenera kusamutsidwa, ndipo tidazindikira kuti kuyimba kwawoko ndi chinthu chosagwirizana ndi mayeso, opangidwa mwachangu ngati kuli kofunikira, ndipo ndizomwe zili zotsatsira zomwe zilibe phindu losamutsira ku mayeso. Pali zinthu zingapo zosungira zomwe zikuyenera kusuntha, koma awa ndi matebulo angapo, osati olemetsa kwambiri. Choncho ife monga yankho, Spark adabweranso kudzapulumutsa, mothandizidwa ndi zomwe tidalemba ndikuyamba kugwiritsa ntchito mwachangu script kusamutsa deta pakati pa matebulo, prom-test.
Ndondomeko yathu yamakono yotumizira imatilola kugwira ntchito popanda kubweza. Pamaso pa promo, pali kuyeserera kovomerezeka, komwe kulakwitsa sikuli kokwera mtengo. Ngati zalephera, mutha kugwetsa nthawi zonse ndikuyendetsa chiwembu chonsecho kuyambira pachiyambi.
Kuti muwonetsetse kupezeka kwa Cassandra mosalekeza, muyenera dba osati iye yekha. Aliyense amene amagwira ntchito ndi pulogalamuyi ayenera kumvetsetsa komwe ndi momwe angayang'anire zomwe zikuchitika komanso momwe angadziwire mavuto munthawi yake. Kuti tichite izi, timagwiritsa ntchito mwachangu DataStax OpsCenter (Kuyang'anira ndi kuyang'anira kuchuluka kwa ntchito), ma metrics a Cassandra Driver system (chiwerengero cha nthawi yolembera ku C *, kuchuluka kwa nthawi yowerengera kuchokera ku C *, kuchedwa kwambiri, ndi zina zotero), kuyang'anira ntchitoyo. ya pulogalamuyo, ikugwira ntchito ndi Cassandra.
Pamene tinaganizira za funso lapitalo, tinazindikira pamene chiwopsezo chathu chachikulu chingakhale. Awa ndi mafomu owonetsera deta omwe amawonetsa deta kuchokera kumafunso angapo odziyimira pawokha kupita kumalo osungira. Mwanjira iyi titha kupeza zidziwitso zosagwirizana. Koma vutoli lingakhale loyenera ngati titagwira ntchito ndi malo amodzi okha a data. Kotero chinthu chomveka kwambiri apa ndi, ndithudi, kupanga batch ntchito yowerengera deta pa ntchito ya chipani chachitatu, yomwe idzawonetsetse kuti deta ikulandiridwa mu nthawi imodzi. Ponena za kugawikana kwa kuwerenga ndi kulemba ponena za ntchito, apa tinayimitsidwa ndi chiopsezo kuti ndi kutayika kwina kwa kugwirizana pakati pa ma DC, tikhoza kukhala ndi magulu awiri omwe sakugwirizana kwathunthu.
Chifukwa chake, pakadali pano yayima pamlingo wosasinthasintha kuti mulembe EACH_QUORUM, kuti muwerenge - LOCAL_QUORUM
Kuwona mwachidule ndi kutsimikizira
Kuti tiwonetsetse yankho lomwe limachokera ku lingaliro la chithandizo cha ntchito ndi chiyembekezo cha chitukuko chowonjezereka, tinaganiza zoganizira za komwe chitukuko choterocho chingagwiritsidwe ntchito.
Kungoyambira pa bat, ndiyeno kugoletsa deta pamapulogalamu monga “Lipirani pakafunika” (timayika zambiri mu C*, kuwerengera pogwiritsa ntchito zilembo za Spark), kuwerengera zonena mophatikizana ndi dera, kusunga maudindo komanso kuwerengera ufulu wa ogwiritsa ntchito kutengera gawo. matrix.
Monga mukuonera, repertoire ndi yotakata komanso yosiyanasiyana. Ndipo ngati tisankha msasa wa othandizira / otsutsa a NoSQL, ndiye kuti tidzalowa nawo othandizira, popeza tinalandira ubwino wathu, komanso komwe tikuyembekezera.
Ngakhale njira ya Cassandra kuchokera mubokosilo imalola kukweza kopingasa mu nthawi yeniyeni, kuthetsa mopanda ululu pankhani yochulukitsa deta mudongosolo. Tinatha kusuntha makina olemetsa kwambiri owerengera ma call aggregates m'gawo losiyana, ndikulekanitsa schema ndi malingaliro ogwiritsira ntchito, kuchotsa mchitidwe woyipa wolemba ntchito ndi zinthu zomwe zili patsamba lokha. Tili ndi mwayi wosankha ndikukonzekera, kuti tifulumizitse, ma DC omwe tidzawerengerepo ndi omwe tidzalembapo deta, tinadziteteza tokha ku kuwonongeka kwa ma node onse ndi DC yonse.
Kugwiritsa ntchito zomanga zathu kumapulojekiti atsopano, komanso kukhala ndi chidziwitso, ndikufuna kuti nthawi yomweyo ndiganizire zamitundu yomwe tafotokozazi, ndikupewa zolakwika zina, kusalaza ngodya zakuthwa zomwe sizingapeweke poyambira.
Mwachitsanzo, sungani zosintha za Cassandra munthawi yakechifukwa zovuta zingapo zomwe tidapeza zidadziwika kale ndikukonzedwa.
Osayika zonse zomwe zili patsamba lokha komanso Spark pama node omwewo (kapena gawani mosamalitsa ndi kuchuluka kwa kugwiritsidwa ntchito kovomerezeka), popeza Spark imatha kudya OP yochulukirapo kuposa momwe amayembekezera, ndipo tidzapeza zovuta nambala 1 pamndandanda wathu.
Kupititsa patsogolo kalondolondo ndi luso la kagwiridwe ka ntchito poyezetsa polojekiti. Poyamba, ganizirani momwe mungathere onse omwe angagwiritse ntchito yankho lathu, chifukwa izi ndi zomwe dongosolo la database lidzadalira.
Tembenukirani kangapo kuti muthe kukhathamiritsa. Sankhani minda yomwe ingathe kutsatiridwa. Mvetsetsani kuti ndi magome owonjezera ati omwe tiyenera kupanga kuti tiziganizira moyenera komanso moyenera, ndiyeno tipereke zidziwitso zofunikira mukafunsidwa (mwachitsanzo, poganiza kuti titha kusunga zidziwitso zomwezo m'matebulo osiyanasiyana, poganizira zosweka mosiyanasiyana malinga ndi njira zosiyanasiyana, titha kupulumutsa nthawi ya CPU pazofunsira zowerenga).
Osati zoipa Nthawi yomweyo perekani zophatikizira TTL ndikuyeretsa deta yakale.
Mukatsitsa deta kuchokera ku Cassandra Lingaliro la kugwiritsa ntchito liyenera kugwira ntchito pa mfundo ya FETCH, kuti mizere yonse isalowe m'makumbukidwe nthawi imodzi, koma imasankhidwa m'magulu.
Ndikoyenera kusamutsa pulojekiti ku yankho lomwe likufotokozedwa yang'anani kulolerana kwa zolakwika za dongosolo poyesa mayeso angapo owonongeka, monga kutayika kwa deta mu malo amodzi a deta, kubwezeretsedwa kwa deta yowonongeka pa nthawi inayake, kutaya kwa intaneti pakati pa malo opangira deta. Mayesero oterowo sangalole kuti munthu aone ubwino ndi kuipa kwa zomangamanga zomwe akufuna, komanso adzapereka machitidwe abwino otenthetsera kwa akatswiri omwe amawatsogolera, ndipo luso lomwe adapeza lidzakhala lopanda phindu ngati zolephera zadongosolo zimabwerezedwanso popanga.
Ngati tigwira ntchito ndi chidziwitso chovuta (monga deta yolipira, kuwerengera ngongole ya olembetsa), ndiye kuti ndi bwino kumvetsera zida zomwe zingachepetse zoopsa zomwe zimabwera chifukwa cha mawonekedwe a DBMS. Mwachitsanzo, gwiritsani ntchito chida cha nodesync (Datastax), mutapanga njira yabwino yogwiritsira ntchito chifukwa cha kusasinthika, musapange katundu wochuluka pa Cassandra ndikugwiritsa ntchito pa matebulo ena okha mu nthawi inayake.
Kodi chimachitika ndi chiyani kwa Cassandra pambuyo pa miyezi isanu ndi umodzi ya moyo? Kawirikawiri, palibe mavuto omwe sanathe. Sitinalolenso ngozi zazikulu kapena kutayika kwa data. Inde, tinkayenera kulingalira za kubweza mavuto ena omwe anali asanabwere, koma pamapeto pake izi sizinasokoneze kwambiri njira yathu yomanga. Ngati mukufuna ndipo musawope kuyesa chinthu chatsopano, ndipo nthawi yomweyo simukufuna kukhumudwa kwambiri, konzekerani kuti palibe chomwe chili chaulere. Muyenera kumvetsetsa, fufuzani zolembazo ndikusonkhanitsa nokha chotengera chanu kuposa momwe munapangira cholowa, ndipo palibe chiphunzitso chomwe chingakuuzeni pasadakhale kuti ndi njira iti yomwe ikukuyembekezerani.
Source: www.habr.com