Zvimiro zvekugadzira data data yeNoSQL

Nhanganyaya

“Unofanira kumhanya nekukurumidza sezvaunokwanisa kuti ungogara panzvimbo,
uye kuti usvike kumwe, unofanira kumhanya kanenge kaviri!”
(c) Alice muWonderland

Imwe nguva yapfuura ndakakumbirwa kupa hurukuro vaongorori kambani yedu panhau yekugadzira data data, nekuti kugara pamapurojekiti kwenguva yakareba (dzimwe nguva kwemakore akati wandei) tinotadza kuona zviri kuitika zvakatipoteredza munyika yeIT technologies. Mukambani yedu (zvinongoitika) mapurojekiti mazhinji haashandise NoSQL dhatabhesi (zvichida izvozvi), saka muhurukuro yangu ndakavapa pfungwa dzakasiyana ndichishandisa muenzaniso weHBase uye ndikaedza kutungamira kuratidzwa kwezvinyorwa kune avo. vasina kumboashandisa vakashanda. Kunyanya, ini ndakaratidza mamwe maficha eiyo data modhi dhizaini ndichishandisa muenzaniso wandakaverenga makore akati wandei apfuura muchinyorwa "Sumo kuHB ase Schema Dhizaini" naAmandeep Khurana. Pakunzvera mienzaniso, ndakaenzanisa nzira dzinoverengeka dzokupedza chinetso chimwe chete kuti ndipe zviri nani mifungo mikuru kuvateereri.

Nguva pfupi yadarika, "pasina chekuita," ndakazvibvunza mubvunzo (kupera kwevhiki kwaMay mukuvharirwa kunonyanya kubatsira kune izvi), maverengero edzidziso achaenderana nekuita? Chaizvoizvo, iyi ndiyo nzira iyo pfungwa yechinyorwa ichi yakazvarwa. Mugadziri anga achishanda neNoSQL kwemazuva akati wandei anogona kusadzidza chero chinhu chitsva kubva pairi (uye nekudaro anogona kungosvetuka hafu yechinyorwa). Asi nokuda vaongororiKune avo vasati vashanda padhuze neNoSQL, ndinofunga zvichave zvinobatsira kuwana nzwisiso yekutanga yezvimiro zvekugadzira data data yeHBase.

Muenzaniso kuongorora

Sekuona kwangu, usati watanga kushandisa NoSQL dhatabhesi, unofanirwa kunyatso funga uye kuyera zvakanakira nezvayakaipira. Kazhinji dambudziko rinogona kugadziriswa uchishandisa zvechinyakare relational DBMSs. Naizvozvo, zviri nani kusashandisa NoSQL pasina zvikonzero zvakakosha. Kana iwe zvakadaro ukafunga kushandisa dhatabhesi reNoSQL, saka iwe unofanirwa kufunga kuti magadzirirwo emaitiro pano akasiyana. Kunyanya mamwe acho anogona kunge asina kujairika kune avo vakambobata chete nehukama DBMSs (maererano nekuona kwangu). Saka, munyika ye "hukama", isu tinowanzotanga nekuenzanisira dambudziko redunhu, uye chete ipapo, kana zvichidikanwa, denormalize modhi. MuNoSQL isu inofanirwa kukurumidza kufunga nezve zvinotarisirwa zviitiko zvekushanda nedata uye kutanga denormalize data. Mukuwedzera, pane zvimwe zvakasiyana zvakasiyana, izvo zvichakurukurwa pasi apa.

Ngationgororei inotevera "synthetic" dambudziko, iro isu ticharamba tichishanda:

Izvo zvinodikanwa kugadzira chigadziriso chekuchengetedza rondedzero yeshamwari dzevashandisi veimwe abstract social network. Kurerutsa, isu tichafungidzira kuti zvese zvatinobatanidza zvinotungamirwa (sepa Instagram, kwete Linkedin). Chimiro chinofanira kukubvumira kuti uite zvinobudirira:

• Pindura mubvunzo kana mushandisi A achiverenga mushandisi B (patani yekuverenga)
• Bvumira kuwedzera / kubvisa zvinongedzo kana kunyorera / kusanyoreswa kwemushandisi A kubva mushandisi B (data shanduko template)

Zvechokwadi, pane zvakawanda zvingasarudzwa zvekugadzirisa dambudziko racho. Mune dhatabhesi rehukama, isu tingangoita tafura yehukama (pamwe inofananidzirwa kana, semuenzaniso, tichida kuchengetedza boka revashandisi: mhuri, basa, nezvimwewo, izvo zvinosanganisira iyi "shamwari"), uye kugadzirisa. kukurumidza kuwana kwaizowedzera indexes/partitioning. Zvichida tafura yekupedzisira yaizotaridzika seizvi:

mushandisi_id
shamwari_id

Vasya
Peter

Vasya
Olya

pano, kuitira kujeka uye kunzwisisa zviri nani, ndicharatidza mazita pane zvitupa

Panyaya yeHBase, tinoziva kuti:

• Kutsvaga kwakanaka kusingakonzerese tafura yakazara inogoneka chete nekiyi
  • kutaura zvazviri, ndosaka kunyora SQL mibvunzo inozivikanwa nevakawanda kune dhatabhesi akadaro ipfungwa yakaipa; tekinoroji, hongu, unogona kutumira mubvunzo weSQL neJoins uye imwe pfungwa kuHBase kubva kuImpala imwechete, asi ichaita sei ...

Naizvozvo, tinomanikidzwa kushandisa ID yemushandisi sekiyi. Uye kufunga kwangu kwekutanga pane musoro wekuti "kupi uye sei kuchengeta maID eshamwari?" pamwe zano rekuzvichengeta mumakoramu. Iyi inonyanya kujeka uye "kusaziva" sarudzo ichaita senge ichi (ngatidaidzei Sarudzo 1 (yagara iripo)kuti uwane zvimwe):

RowKey
Columns

Vasya
1: Petya
2: Olya
3: Dasha

Peter
1: Masha
2: Vasya

Pano, mutsara wega wega unofanana nemushandisi wetiweki. Makoramu ane mazita: 1, 2, ... - maererano nenhamba yeshamwari, uye maID eshamwari akachengetwa mumakoramu. Zvakakosha kuziva kuti mutsara wega wega uchange uine nhamba yakasiyana yemakoramu. Mumuenzaniso mumufananidzo uri pamusoro, mutsara mumwe une makoramu matatu (1, 2 uye 3), uye wechipiri ane maviri chete (1 uye 2) - pano isu pachedu takashandisa zvivakwa zviviri zveHBase izvo zvisina hukama dhatabhesi:

• kugona kushandura zvine simba kuumbwa kwemakoramu (wedzera shamwari -> wedzera koramu, bvisa shamwari -> bvisa koramu)
• mitsara yakasiyana inogona kunge iine zvinyorwa zvakasiyana-siyana

Ngatitarisei chimiro chedu kuti chinoenderana nezvinodiwa zvebasa racho:

• Kuverenga data: kuti tinzwisise kana Vasya akanyoreswa kuOlya, isu tichada kubvisa mutsara wose nekiyi RowKey = "Vasya" uye rongedza kuburikidza nekoramu tsika kusvika isu "tasangana" naOlya mavari. Kana kudzokorora kuburikidza nehunhu hwemakoramu ese, "kwete kusangana" Olya uye dzorera mhinduro Nhema;
• Kugadzirisa data: kuwedzera shamwari: kune basa rakafanana isu tinodawo kubvisa mutsara wose uchishandisa kiyi RowKey = "Vasya" kuverenga nhamba yese yeshamwari dzake. Tinoda iyi nhamba yese yeshamwari kuti tione nhamba yekoramu yatinoda kunyora pasi ID yeshamwari itsva.
• Kuchinja data: kudzima shamwari:
  • Kuda kubvisa mutsara wose nekiyi RowKey = "Vasya" uye ronga kuburikidza nemakoramu kuitira kuti uwane iyo iyo shamwari ichadzimwa yakanyorwa;
  • Tevere, mushure mekudzima shamwari, isu tinofanirwa "kuchinja" data rese muchikamu chimwe chete kuitira kuti tisawane "magapu" mukuverenga kwavo.

Ngationgororei kuti maalgorithms aya anogadzirwa sei, atinozoda kuita padivi re "conditional application", achave, tichishandisa. O-chiratidzo. Ngatiratidze saizi yedu yekufungidzira social network se n. Ipapo iyo nhamba yepamusoro yeshamwari inogona kuwanikwa nemushandisi ndeye (n-1). Tinogona kuramba tichiregeredza izvi (-1) nekuda kwezvinangwa zvedu, sezvo mukati megadziriro yekushandiswa kweO-symbols hazvina kukosha.

• Kuverenga data: zvakakosha kubvisa mutsara wose uye kudzokorora kuburikidza nemakoramu ayo ose mumuganhu. Izvi zvinoreva kuti fungidziro yepamusoro yemitengo inosvika O(n)
• Kugadzirisa data: kuwedzera shamwari: kuti uone huwandu hweshamwari, iwe unofanirwa kudzokorora kuburikidza nemakoramu ese emutsetse, wobva waisa koramu nyowani => O (n)
• Kuchinja data: kudzima shamwari:
  • Zvakafanana nekuwedzera - iwe unofanirwa kupfuura nemakoramu ese mumuganhu => O (n)
  • Mushure mekubvisa mbiru, tinoda "kufambisa" iwo. Kana iwe ukashandisa iyi "musoro-pamusoro", ipapo mumuganhu iwe uchada kusvika (n-1) mashandiro. Asi pano uye mberi muchikamu chinoshanda tichashandisa nzira yakasiyana, iyo ichashandisa "pseudo-shift" yenhamba yakatarwa yekushanda - ndiko kuti, nguva inogara ichashandiswa pairi, pasinei n. Iyi nguva inogara iripo (O(2) kuve chaiyo) inogona kuregeredzwa kana ichienzaniswa neO(n). Iyo nzira inoratidzwa mumufananidzo uri pazasi: isu tinongokopa iyo data kubva "yekupedzisira" column kune iyo yatinoda kudzima data, tobva tadzima koramu yekupedzisira:
    

Pakazara, mune zvese zviitiko takagamuchira asymptotic computational yakaoma yeO (n).
Iwe watove watoona kuti isu tinofanirwa kuverenga mutsara wese kubva mudhatabhesi, uye muzviitiko zviviri kubva pamatatu, kungopfuura nepakati pemakoramu uye kuverenga nhamba yese yeshamwari. Naizvozvo, sekuyedza kugadzirisa, iwe unogona kuwedzera "kuverenga" column, iyo inochengeta huwandu hwese hweshamwari dzeumwe neumwe mushandisi wetiweki. Muchiitiko ichi, hatigoni kuverenga mutsara wose kuti tiverenge nhamba yose yeshamwari, asi verenga imwe chete "kuverenga" column. Chinhu chikuru ndechekuti usakanganwa kugadzirisa "kuverenga" paunenge uchibata data. Izvozvo. tinova nani Sarudzo 2 (kuverenga):

RowKey
Columns

Vasya
1: Petya
2: Olya
3: Dasha
kuverenga: 3

Peter
1: Masha
2: Vasya

kuverenga: 2

Kuenzaniswa neyekutanga sarudzo:

• Kuverenga data: kuwana mhinduro kumubvunzo unoti "Vasya anoverenga Olya here?" hapana chakachinja => O (n)
• Kugadzirisa data: kuwedzera shamwari: Isu takarerutsa kuiswa kweshamwari itsva, sezvo ikozvino hatifanire kuverenga mutsara wese uye nekudzokorora pamusoro pemakoramu ayo, asi tinogona kuwana kukosha kwe "kuverenga" column, nezvimwe. pakarepo sarudza nhamba yekoramu kuti uise shamwari itsva. Izvi zvinotungamira mukudzikiswa kwecomputational kuomarara kuO (1)
• Kuchinja data: kudzima shamwari: Pakudzima shamwari, isu tinogona zvakare kushandisa iyi koramu kudzikisa huwandu hweI / O mashandiro kana "kuchinja" data sero rimwe kuruboshwe. Asi kukosha kwekudzokorora kuburikidza nemakoramu kuti uwane iyo inoda kubviswa ichiripo, saka => O(n)
• Kune rumwe rutivi, ikozvino kana tichivandudza data tinoda kugadzirisa "kuverenga" column nguva dzose, asi izvi zvinotora nguva yakawanda, iyo inogona kuregeredzwa mukati mehurongwa hweO-symbols.

Kazhinji, sarudzo yechipiri inoita seyakanyanya kunaka, asi yakafanana ne "evolution pane revolution." Kuti tiite “revolution” tichada Sarudzo 3 (col).
Ngatitendeutse zvese "zvitsigiro pasi": isu tichagovera column zita remushandisi ID! Izvo zvichange zvichinyorwa mumutsara pachazvo hazvichakoshi kwatiri, regai ive nhamba 1 (kazhinji, zvinhu zvinobatsira zvinogona kuchengetwa ipapo, somuenzaniso, boka "mhuri / shamwari / nezvimwewo."). Iyi nzira inogona kushamisa "munhu asina kugadzirira" asina ruzivo rwekare achishanda neNoSQL dhatabhesi, asi ndiyo nzira iyi inokubvumira kushandisa mukana weHBase mubasa iri zvakanyanya.

RowKey
Columns

Vasya
Petiya: 1
Olya: 1
Dhavhidha: 1

Peter
Masha: 1
Vasya: 1

Pano tinowana mabhenefiti akati wandei kamwechete. Kuti uzvinzwisise, ngationgororei chimiro chitsva uye tifungidzire iyo computational yakaoma:

• Kuverenga data: kuitira kuti upindure mubvunzo kana Vasya akanyoreswa kuna Olya, zvakakwana kuverenga imwe koramu "Olya": kana iripo, ipapo mhinduro ndeyechokwadi, kana isiri - Nhema => O (1)
• Kugadzirisa data: kuwedzera shamwari: Kuwedzera shamwari: ingo wedzera koramu itsva “Shamwari ID” => O(1)
• Kuchinja data: kudzima shamwari: ingobvisa iyo Shamwari ID column => O(1)

Sezvauri kuona, mukana wakakosha weiyi modhi yekuchengetera ndeyekuti mune ese mamiriro atinoda, isu tinoshanda nekoramu imwe chete, tichinzvenga kuverenga mutsara wese kubva kudhatabhesi uye, zvakare, kuverengera makoramu ese emutsara uyu. Tinogona kumira ipapo, asi...

Iwe unogona kushamisika uye kuenda mberi zvishoma munzira yekunatsiridza mashandiro uye kuderedza I/O mashandiro kana uchinge wawana iyo dhatabhesi. Ko kana tikachengeta ruzivo rwakakwana rwehukama zvakananga mumutsara kiyi pachayo? Kureva, ita kiyi inoumbwa senge userID.friendID? Muchiitiko ichi, isu hatitombofanire kuverenga makoramu emutsara zvachose (Sarudzo 4(mutsara)):

RowKey
Columns

Vasya.Petya
Petiya: 1

Vasya.Olya
Olya: 1

Vasya.Dasha
Dhavhidha: 1

Petya.Masha
Masha: 1

Petya.Vasya
Vasya: 1

Zviripachena, kuongororwa kwese kwese data manipulation scenarios muchimiro chakadaro, senge mune yapfuura vhezheni, ichave O (1). Musiyano nesarudzo 3 uchave chete mukushanda kweI/O mashandiro mudura.

Zvakanaka, "uta" hwekupedzisira. Zviri nyore kuona kuti mukusarudza 4, mutsara wemutsara uchava nehurefu hwakasiyana, izvo zvingangokanganisa kushanda (pano tinoyeuka kuti HBase inochengetedza data seti yebhaiti uye mitsara mumatafura inorongwa nekiyi). Uyezve isu tine separator ingangoda kubatwa mune mamwe mamiriro. Kuti ubvise pesvedzero iyi, unogona kushandisa hashes kubva kuuserID uye shamwariID, uye sezvo ese maviri hashe aine hurefu hwenguva dzose, unogona kungoabatanidza, pasina kupatsanura. Ipapo iyo data iri patafura ichaita seizvi (Sarudzo 5(hashi)):

RowKey
Columns

dc084ef00e94aef49be885f9b01f51c01918fa783851db0dc1f72f83d33a5994
Petiya: 1

dc084ef00e94aef49be885f9b01f51c0f06b7714b5ba522c3cf51328b66fe28a
Olya: 1

dc084ef00e94aef49be885f9b01f51c00d2c2e5d69df6b238754f650d56c896a
Dhavhidha: 1

1918fa783851db0dc1f72f83d33a59949ee3309645bd2c0775899fca14f311e1
Masha: 1

1918fa783851db0dc1f72f83d33a5994dc084ef00e94aef49be885f9b01f51c0
Vasya: 1

Zviripachena, iyo algorithmic yakaoma yekushanda nechimiro chakadaro mumamiriro ezvinhu atiri kufunga nezvawo achafanana neaya esarudzo 4 - kureva, O (1).
Pakazara, ngatipei muchidimbu fungidziro dzedu dzese dze computational kuomarara mune imwe tafura:

Kuwedzera shamwari
Kutarisa shamwari
Kubvisa shamwari

Sarudzo 1 (yagara iripo)
O (n)
O (n)
O (n)

Sarudzo yechipiri (kuverenga)
O (1)
O (n)
O (n)

Sarudzo 3 (chikamu)
O (1)
O (1)
O (1)

Sarudzo 4 (mutsara)
O (1)
O (1)
O (1)

Sarudzo 5 (hashi)
O (1)
O (1)
O (1)

Sezvauri kuona, sarudzo 3-5 dzinoita sedzinonyanya kufarirwa uye ne theoretically inovimbisa kuitwa kwese kunodiwa data manipulation scenarios munguva inogara iripo. Mumamiriro ezvinhu ebasa redu, hapana chakajeka chinodiwa kuwana runyoro rweshamwari dzese dzemushandisi, asi muzviitiko zvepurojekiti chaiyo, zvingava zvakanaka kwatiri, sevaongorori vakanaka, "kutarisira" kuti basa rakadaro ringamuka uye “kuwaridza tsanga.” Nokudaro, tsitsi dzangu dziri parutivi rwechisarudzo 3. Asi zvinowanzoitika kuti mune imwe purojekiti chaiyo chikumbiro ichi chingave chave chagadziriswa nedzimwe nzira, saka, pasina chiono chakakwana chechinetso chose, zviri nani kusaita. mhedziso dzekupedzisira.

Kugadzirira kwekuedza

Ini ndinoda kuyedza nharo dzepamusoro dzedzidziso mukuita - ichi chaive chinangwa chepfungwa yakamuka mukupera kwesvondo. Kuti uite izvi, zvinodikanwa kuongorora kumhanya kwekushanda kwe "conditional application" yedu mune zvese zvakatsanangurwa mamiriro ekushandisa dhatabhesi, pamwe nekuwedzera munguva ino nekuwedzera kukura kwesocial network (n). Iyo inotarirwa parameter inotifarira uye yatichayera panguva yekuedza ndiyo nguva inoshandiswa ne "conditional application" kuita "bhizinesi rekushanda" rimwe. Ne "bhizinesi rekutengeserana" tinoreva chimwe chezvinotevera:

• Kuwedzera imwe shamwari itsva
• Kutarisa kana Mushandisi A ishamwari yemushandisi B
• Kubvisa shamwari imwe

Nokudaro, tichifunga nezvezvinodiwa zvakataurwa muchirevo chekutanga, mamiriro ekusimbisa anobuda sezvinotevera:

• Data kurekodha. Nenguva isipi gadzira network yekutanga saizi n. Kuti uswedere pedyo ne "nyika chaiyo", nhamba yeshamwari yega yega mushandisi inewo inongo shanduka. Misa nguva iyo "conditional application" yedu inonyora ese anogadzirwa data kuHBase. Wobva wapatsanura nguva inoguma nenhamba yakazara yeshamwari dzakawedzerwa - iyi ndiyo nzira yatinowana nayo avhareji yenguva yeimwe "bhizinesi mashandiro"
• Kuverenga data. Kune wega wega mushandisi, gadzira runyoro rwe "hunhu" hwaunoda kuwana mhinduro kuti mushandisi akanyoreswa kwavari here kana kuti kwete. Kureba kwechinyorwa = inenge nhamba yeshamwari dzevashandisi, uye kwehafu yeshamwari dzakaongororwa mhinduro inofanira kuva "Hongu", uye kune imwe hafu - "Kwete". Cheki inoitwa mukurongeka zvekuti mhinduro dzekuti "Hongu" uye "Kwete" dzinochinjana (kureva kuti, muchiitiko chechipiri chega chega tichafanirwa kupfuura nemakoramu ese emutsara wesarudzo 1 uye 2). Iyo yakazara nguva yekuongorora inozopatsanurwa nehuwandu hweshamwari dzakaedzwa kuti dziwane avhareji yenguva yekuongorora pachidzidzo.
• Kudzima data. Bvisa shamwari dzese kubva kumushandisi. Zvakare, iyo yekudzima odha haina kujairika (kureva kuti, isu "tinokanganisa" rondedzero yepakutanga inoshandiswa kurekodha data). Yese yenguva yekutarisa inozopatsanurwa nenhamba yeshamwari dzakabviswa kuti uwane avhareji yenguva pacheki.

Mamiriro acho anoda kumhanyiswa kune yega yega ye5 data modhi sarudzo uye kune akasiyana saizi yesocial network kuti uone kuti nguva inoshanduka sei sezvainokura. Mukati meimwe n, zvinongedzo mune network uye rondedzero yevashandisi yekutarisa inofanira, hongu, yakafanana kune ese mashanu sarudzo.
Kuti unzwisise zviri nani, pazasi pane muenzaniso we data yakagadzirwa ye n = 5. Iyo yakanyorwa "jenereta" inoburitsa matatu ID dictionaries seyakabuda:

• yekutanga ndeyekuisa
• yechipiri ndeyekutarisa
• chechitatu - kubviswa

{0: [1], 1: [4, 5, 3, 2, 1], 2: [1, 2], 3: [2, 4, 1, 5, 3], 4: [2, 1]} # всего 15 друзей

{0: [1, 10800], 1: [5, 10800, 2, 10801, 4, 10802], 2: [1, 10800], 3: [3, 10800, 1, 10801, 5, 10802], 4: [2, 10800]} # всего 18 проверяемых субъектов

{0: [1], 1: [1, 3, 2, 5, 4], 2: [1, 2], 3: [4, 1, 2, 3, 5], 4: [1, 2]} # всего 15 друзей

Sezvauri kuona, maID ese anopfuura zviuru gumi muduramazwi rekutarisa ndiwo chaiwo ayo anozopa mhinduro Nhema. Kupinza, kutarisa uye kudzima "shamwari" kunoitwa chaizvo munhevedzano yakataurwa muduramazwi.

Kuedza uku kwakaitwa palaptop inomhanya Windows 10, uko HBase yaimhanya mune imwe Docker mudziyo, uye Python ine Jupyter Notebook yaimhanya mune imwe. Docker yakapihwa 2 CPU cores uye 2 GB ye RAM. Zvese zvine musoro, sekutevedzera kwe "conditional application" uye "pombi" yekugadzira bvunzo data uye yekuyera nguva, yakanyorwa muPython. Raibhurari yakashandiswa kushanda neHBase happybase, kuverenga hashes (MD5) yesarudzo 5 - hahlib

Tichifunga nezvesimba rekombuta reimwe laptop, kutangwa kwen = 10, 30, ... kwakasarudzwa kuyedza. 170 - apo nguva yose yekushanda yenguva yakazara yekuedza (zvese zviitiko zvezvose zvingasarudzwa zvevose n) zvaive zvakatonyanya kana zvishoma uye zvakakodzera panguva imwe chete yetii (paavhareji maminitsi gumi nemashanu).

Pano zvakakosha kutaura kuti mukuedza uku hatisi kunyanya kuongorora nhamba dzekuita zvakakwana. Kunyange kuenzanisa kwakasiyana kwezvisarudzo zviviri zvakasiyana zvinogona kunge zvisiri izvo zvakakwana. Iye zvino isu tiri kufarira mamiriro ekuchinja kwenguva zvichienderana nen, kubva tichifunga nezvekugadzirisa kwepamusoro kwe "test stand", zvakaoma zvikuru kuwana nguva yekufungidzira "yakacheneswa" yekukanganiswa kwemaitiro uye zvimwe zvinhu ( uye basa rakadaro rakanga risina kuiswa).

Edza mhinduro

Muedzo wekutanga ndewekuti nguva inopedzwa kuzadza rondedzero yeshamwari inochinja sei. Mhedzisiro iri mugirafu pazasi.

Sarudzo 3-5, sezvaitarisirwa, inoratidza nguva inenge inogara "bhizinesi rekutengeserana" nguva, iyo isingaenderane nekukura kwehukuru hwemambure uye musiyano usingaoneki mukushanda.
Sarudzo 2 inoratidzawo zvinogara, asi zvishoma zvakanyanya kuita, kanenge kaviri chaiko zvine chekuita nesarudzo 2-3. Uye izvi hazvigone asi kufara, sezvo zvichifambirana nedzidziso - mushanduro iyi nhamba yeI/O mashandiro kuenda/kubva kuHBase yakanyatso kupeta ka5. Izvi zvinogona kushanda sehumbowo husina kunanga hwekuti bhenji redu rekuyedza, mumusimboti, rinopa chokwadi chakanaka.
Sarudzo yekutanga zvakare, sezvaitarisirwa, inoshanduka kuita inononoka uye inoratidza kuwedzera kwemutsara munguva yakashandiswa pakuwedzera mumwe kune saizi yetiweki.
Ngatitarisei zvino mhedzisiro yebvunzo yechipiri.

Sarudzo 3-5 zvakare zvibate sezvaitarisirwa - nguva dzose, yakasununguka nehukuru hwetiweki. Sarudzo 1 uye 2 inoratidza kuwedzera kwemutsara munguva sezvo saizi yetiweki inowedzera uye kuita kwakafanana. Zvakare, sarudzo 2 inoshanduka kuita zvishoma zvishoma - sezviri pachena nekuda kwekuda kwekuverenga uye kugadzirisa iyo yekuwedzera "kuverenga" koramu, inova yakanyanya kuoneka sezvo n inokura. Asi ini ndicharega kutora mhedziso chero ipi zvayo, sezvo kururama kwekuenzanisa uku kwakaderera. Mukuwedzera, izvi zviyero (iyo sarudzo, 1 kana 2, inokurumidza) yakashandurwa kubva pakumhanya kusvika pakumhanya (uchiri kuchengetedza hunhu hwekutsamira uye "kuenda mutsipa nemutsipa").

Zvakanaka, girafu yekupedzisira ndiyo mhedzisiro yekubvisa kuyedzwa.

Zvakare, hapana zvinoshamisa pano. Sarudzo 3-5 inoita kubviswa munguva isingaperi.
Uyezve, zvinofadza, sarudzo 4 uye 5, kusiyana nemamiriro ekutanga, anoratidza zvinooneka zvishoma zvakanyanya kuipa kudarika sarudzo 3. Sezviri pachena, mutsara wekudzima mutsara unodhura kudarika kubviswa kwekoramu, iyo inowanzova inonzwisisika.

Sarudzo 1 uye 2, sezvaitarisirwa, inoratidza kuwedzera kwemutsara munguva. Panguva imwecheteyo, sarudzo 2 inogara ichinonoka pane sarudzo 1 - nekuda kwekuwedzera kweI / O mashandiro e "kuchengetedza" chikamu chekuverenga.

Mhedziso yezviyedzo:

• Sarudzo 3-5 dzinoratidza kugona kukuru sezvavanotora mukana weHBase; Uyezve, kuita kwavo kunosiyana kune mumwe nemumwe nekugara uye hakunei nehukuru hwetiweki.
• Musiyano pakati pezvisarudzo 4 ne5 hauna kunyorwa. Asi izvi hazvirevi kuti sarudzo yechishanu haifaniri kushandiswa. Zvingangodaro kuti mamiriro ekuedza anoshandiswa, achifunga nezvemaitiro ekuita kwebhenji rekuedza, haana kubvumira kuti zvionekwe.
• Mamiriro ekuwedzera kwenguva inodiwa kuita "bhizinesi mashandiro" nedata kazhinji yakasimbisa iyo yakambowanikwa maverengero edzidziso kune ese sarudzo.

Epilogue

Kuedza kwakakasharara kunoitwa hakufanirwe kutorwa sechokwadi chakakwana. Pane zvinhu zvakawanda zvisina kucherechedzwa uye kukanganisa mhedzisiro (idzi shanduko dzinonyanya kuoneka mumagirafu ane diki network saizi). Semuenzaniso, kumhanya kwekusimudzira, iyo inoshandiswa ne happybase, vhoriyamu uye nzira yekushandisa iyo pfungwa yandakanyora muPython (ini handigone kutaura kuti iyo kodhi yakanyorwa zvakanaka uye zvinobudirira kushandisa kugona kwezvinhu zvese), pamwe. iwo maficha eHBase caching, kumashure chiitiko che Windows 10 palaptop yangu, nezvimwe. Kazhinji, tinogona kufunga kuti kuverenga kwese kwedzidziso kwakaratidza kutendeka kwavo. Zvakanaka, kana kuti zvakange zvisingabviri kuvapikisa ne "musoro-kurwisa" kwakadaro.

Mukupedzisa, kurudziro kune wese achangotanga kugadzira data data muHBase: abstract kubva pane yapfuura ruzivo uchishanda nehukama dhatabhesi uye rangarira iyo "mirairo":

• Kana tichigadzira, tinoenderera mberi kubva kune basa uye mapatani ekugadzirisa data, uye kwete kubva kune iyo domain modhi
• Kubudirira kuwana (pasina yakazara tafura scan) - chete nekiyi
• Denormalization
• Mitsara yakasiyana inogona kunge iine makoramu akasiyana
• Dynamic kuumbwa kwevatauri

Source: www.habr.com