Mau oyamba
βUyenera kuthamanga mwachangu momwe ungathere kuti ukhale pamalo,
ndipo kuti ukafike kwinakwake, uyenera kuthamanga kuwirikiza kawiri!β
(c) Alice ku Wonderland
Nyengo yikati yajumphapo nkhapempha kuti nkhambe nkhani openda kampani yathu pamutu wopanga zitsanzo za data, chifukwa chokhala pama projekiti kwa nthawi yayitali (nthawi zina kwa zaka zingapo) timayiwala zomwe zikuchitika kuzungulira ife mdziko laukadaulo wa IT. Pakampani yathu (zimangochitika) mapulojekiti ambiri sagwiritsa ntchito nkhokwe za NoSQL (makamaka pakadali pano), kotero munkhani yanga ndidawasamalira padera pogwiritsa ntchito chitsanzo cha HBase ndikuyesera kuwongolera mafotokozedwe azinthuzo kwa iwo. amene sanazigwiritsepo ntchito agwira ntchito. Makamaka, ndidawonetsa zina mwazinthu zamapangidwe amtundu wa data pogwiritsa ntchito chitsanzo chomwe ndidawerenga zaka zingapo zapitazo . Popenda zitsanzo, ndinayerekezera njira zingapo zothetsera vuto lomwelo kuti ndifotokoze bwino mfundo zazikulu kwa omvera.
Posachedwa, "palibe choti ndichite," ndidadzifunsa funso (kumapeto kwa sabata yatha ya Meyi kukhala kwaokha kumakhala kothandiza kwambiri pa izi), kuwerengera kwamalingaliro kumagwirizana bwanji ndi machitidwe? Kwenikweni, umu ndi momwe lingaliro la nkhaniyi linayambira. Wopanga mapulogalamu yemwe wakhala akugwira ntchito ndi NoSQL kwa masiku angapo sangaphunzire zatsopano kuchokera pamenepo (ndiye chifukwa chake akhoza kulumpha theka la nkhaniyo). Koma za opendaKwa iwo omwe sanagwirepo ntchito limodzi ndi NoSQL, ndikuganiza kuti zingakhale zothandiza kuti mumvetsetse bwino momwe mungapangire ma data a HBase.
Chitsanzo kusanthula
M'malingaliro anga, musanayambe kugwiritsa ntchito ma database a NoSQL, muyenera kulingalira mosamala ndikuyesa zabwino ndi zoyipa. Nthawi zambiri vutoli limatha kuthetsedwa pogwiritsa ntchito ma DBMS achikhalidwe. Chifukwa chake, ndibwino kuti musagwiritse ntchito NoSQL popanda zifukwa zazikulu. Ngati mutasankha kugwiritsa ntchito database ya NoSQL, ndiye kuti muyenera kuganizira kuti njira zopangira pano ndizosiyana. Makamaka ena aiwo angakhale achilendo kwa iwo omwe adachitapo kale ndi ma DBMS ogwirizana (malinga ndi zomwe ndawonera). Chifukwa chake, m'dziko la "zachiyanjano", nthawi zambiri timayamba ndikuwonetsa dera lamavuto, ndipo pokhapokha, ngati kuli kofunikira, kusokoneza chitsanzocho. Mu NoSQL ife ayenera nthawi yomweyo kuganizira zochitika zoyembekezeka zogwirira ntchito ndi deta ndikuyamba kusokoneza deta. Kuonjezera apo, palinso zosiyana zina, zomwe zidzakambidwe pansipa.
Tiyeni tiwone vuto ili "lopanga", lomwe tipitilize kugwira ntchito:
Ndikofunikira kupanga dongosolo losungiramo mndandanda wa abwenzi a ogwiritsa ntchito ena ochezera a pa intaneti. Kuti tifewetse, tidzaganiza kuti maulumikizidwe athu onse amawongoleredwa (monga pa Instagram, osati Linkedin). Dongosololi liyenera kukulolani kuchita bwino:
- Yankhani funso ngati wogwiritsa A amawerenga wogwiritsa B (panjira yowerengera)
- Lolani kuwonjezera/kuchotsa zolumikizira ngati mutalembetsa/kusiya kulembetsa kwa wogwiritsa ntchito A kuchokera kwa wogwiritsa B (chithunzi chosinthira deta)
Inde, pali njira zambiri zothetsera vutoli. Pazosungirako nthawi zonse zaubale, titha kungopanga mndandanda wamaubwenzi (mwina woyimira ngati, mwachitsanzo, tifunika kusunga gulu la ogwiritsa ntchito: banja, ntchito, ndi zina zotere, zomwe zikuphatikiza "bwenzi" ili), ndikukulitsa liwiro lofikira likhoza kuwonjezera ma index / magawo. Mwachidziwikire tebulo lomaliza lingawoneke motere:
Dzina Lolowera
friend_id
Vasya
Peter
Vasya
Olya
kuyambira pano, kuti zimveke bwino komanso kuti mumvetsetse bwino, ndikuwonetsa mayina m'malo mwa ma ID
Pankhani ya HBase, tikudziwa kuti:
- kusaka koyenera komwe sikupangitsa kuti sikeni yonse ya tebulo ndi kotheka kokha ndi kiyi
- m'malo mwake, ndichifukwa chake kulemba mafunso a SQL odziwika kwa ambiri kuzinthu zotere ndizolakwika; mwaukadaulo, inde, mutha kutumiza funso la SQL ndi Joins ndi malingaliro ena ku HBase kuchokera ku Impala yomweyo, koma zikhala zogwira mtima bwanji ...
Chifukwa chake, timakakamizika kugwiritsa ntchito ID ngati kiyi. Ndipo lingaliro langa loyamba pamutu wakuti "kuti ndi momwe mungasungire ma ID a anzanga?" mwina lingaliro kusunga iwo mu mizati. Njira yodziwikiratu komanso "yopanda pake" idzawoneka motere (tiyeni tiyitchule Njira 1 (zofikira)kuti mumve zambiri):
RowKey
Mizati
Vasya
1 :Peti
2: Olya
3: Dasha
Peter
1: Masha
2: Vasya
Apa, mzere uliwonse umafanana ndi wogwiritsa ntchito maukonde. Mizati ili ndi mayina: 1, 2, ... - malinga ndi chiwerengero cha abwenzi, ndipo ma ID a abwenzi amasungidwa m'mizere. Ndikofunika kuzindikira kuti mzere uliwonse udzakhala ndi chiwerengero chosiyana cha mizati. Muchitsanzo chomwe chili pamwambapa, mzere umodzi uli ndi mizati itatu (1, 2 ndi 3), ndipo yachiwiri ili ndi ziwiri zokha (1 ndi 2) - apa ife tokha tidagwiritsa ntchito zinthu ziwiri za HBase zomwe nkhokwe zaubale zilibe:
- Kutha kusintha mawonekedwe amipingo (onjezani bwenzi -> onjezerani gawo, chotsani bwenzi -> chotsani ndime)
- mizere yosiyanasiyana ikhoza kukhala ndi zolemba zosiyanasiyana
Tiyeni tiwone dongosolo lathu kuti likugwirizana ndi zofunikira za ntchitoyi:
- Kuwerenga deta: kuti timvetse ngati Vasya adalembetsa ku Olya, tidzafunika kuchotsa mzere wonse ndi kiyi RowKey = "Vasya" ndikusintha magawo mpaka "titakumana" ndi Olya mwa iwo. Kapena bwerezaninso zikhalidwe zamagawo onse, "osakumana" ndi Olya ndikubweza yankho Labodza;
- Kusintha deta: kuwonjezera bwenzi: pa ntchito yofananayo tiyeneranso kuchotsa mzere wonse pogwiritsa ntchito kiyi RowKey = "Vasya" kuti awerenge chiwerengero cha abwenzi ake. Tikufuna abwenzi onsewa kuti tidziwe kuchuluka kwa gawo lomwe tikuyenera kulemba ID ya bwenzi latsopano.
- Kusintha deta: kufufuta mnzako:
- Kufunika kuchotsa mzere wonse ndi kiyi RowKey = "Vasya" ndikusankha mizati kuti mupeze yomwe bwenzi lanu lichotsedwe limalembedwa;
- Kenako, mutatha kuchotsa mnzathu, tiyenera "kusintha" deta yonse mu gawo limodzi kuti tisapeze "mipata" pamawerengero awo.
Tiyeni tsopano tiwone momwe ma aligorivimuwa adzapindulira, omwe tidzafunika kuwagwiritsa ntchito pa "conditional application", adzakhalire, pogwiritsa ntchito . Tiyeni tiwone kukula kwa malo athu ochezera a pa Intaneti monga n. Ndiye chiwerengero chachikulu cha abwenzi omwe munthu akhoza kukhala nawo ndi (n-1). Titha kunyalanyazanso izi (-1) pazolinga zathu, chifukwa mkati mwa kugwiritsa ntchito zizindikiro za O ndizosafunika.
- Kuwerenga deta: m'pofunika kuchotsa mzere wonse ndi kubwereza mizati yake yonse mu malire. Izi zikutanthauza kuti kuyerekeza kwapamwamba kwa ndalama kudzakhala pafupifupi O(n)
- Kusintha deta: kuwonjezera bwenzi: kuti mudziwe kuchuluka kwa anzanu, muyenera kubwereza mizati yonse ya mzere, ndiyeno ikani ndime yatsopano => O(n)
- Kusintha deta: kufufuta mnzako:
- Zofanana ndi kuwonjezera - muyenera kudutsa mizati yonse mu malire => O(n)
- Pambuyo pochotsa mizati, tiyenera "kusuntha" iwo. Ngati mugwiritsa ntchito "mutu-pa", ndiye kuti mukufunika mpaka (n-1) ntchito. Koma apa ndi kupitirira mu gawo lothandiza tidzagwiritsa ntchito njira yosiyana, yomwe idzagwiritse ntchito "pseudo-shift" pa chiwerengero chokhazikika cha ntchito - ndiko kuti, nthawi yokhazikika idzagwiritsidwa ntchito, mosasamala kanthu za n. Nthawi yokhazikika iyi (O(2) kukhala yeniyeni) ikhoza kunyalanyazidwa poyerekeza ndi O(n). Njirayi ikuwonetsedwa pachithunzi chomwe chili pansipa: timangotengera zomwe zili pagawo "lomaliza" kupita komwe tikufuna kuchotsapo, ndikuchotsa gawo lomaliza:
Pazonse, muzochitika zonse tidalandira zovuta zofananira za O(n).
Mwinamwake mwazindikira kale kuti pafupifupi nthawi zonse timayenera kuwerenga mzere wonse kuchokera ku database, ndipo muzochitika ziwiri mwa zitatu, kungodutsa mzati zonse ndikuwerengera chiwerengero cha abwenzi. Chifukwa chake, poyesa kukhathamiritsa, mutha kuwonjezera gawo la "kuwerengera", lomwe limasunga chiwerengero cha abwenzi a aliyense wogwiritsa ntchito maukonde. Pankhaniyi, sitingathe kuwerenga mzere wonse kuti tiwerenge chiwerengero cha abwenzi, koma werengani gawo limodzi la "kuwerengera". Chinthu chachikulu ndichoti musaiwale kusintha "kuwerengera" pamene mukuyendetsa deta. Kuti. timakhala bwino Njira 2 (kuwerengera):
RowKey
Mizati
Vasya
1 :Peti
2: Olya
3: Dasha
chiwerengero: 3
Peter
1: Masha
2: Vasya
chiwerengero: 2
Poyerekeza ndi njira yoyamba:
- Kuwerenga deta: kupeza yankho la funso lakuti "Kodi Vasya amawerenga Olya?" palibe chomwe chasintha => O (n)
- Kusintha deta: kuwonjezera bwenzi: Tafewetsa kuyika kwa bwenzi latsopano, popeza tsopano sitiyenera kuwerenga mzere wonse ndikubwereza mizati yake, koma titha kupeza mtengo wa "kuwerengera", ndi zina zotero. nthawi yomweyo dziwani nambala yazambiri kuti muyike bwenzi latsopano. Izi zimabweretsa kuchepa kwa zovuta zowerengera kukhala O (1)
- Kusintha deta: kufufuta mnzako: Pochotsa bwenzi, tingagwiritsenso ntchito ndimeyi kuti tichepetse chiwerengero cha ntchito za I / O pamene "tikusuntha" deta selo limodzi kumanzere. Koma kufunika kobwereza mizati kuti mupeze yomwe ikufunika kuchotsedwa ikadalipo, kotero => O(n)
- Kumbali inayi, tsopano pokonzanso deta timafunika kusintha gawo la "kuwerengera" nthawi zonse, koma izi zimatenga nthawi yokhazikika, yomwe ikhoza kunyalanyazidwa mkati mwa zizindikiro za O-symbols.
Nthawi zambiri, kusankha 2 kumawoneka ngati koyenera, koma kuli ngati "chisinthiko m'malo mosintha." Kuti tichite "kusintha" tidzafunika Njira 3 (col).
Tiyeni titembenuzire zonse "zozondoka": tidzasankha Dzina lachitsanzo ID! Zomwe zidzalembedwe pamzatizo sizilinso zofunika kwa ife, zikhale nambala 1 (zambiri, zinthu zothandiza zikhoza kusungidwa pamenepo, mwachitsanzo, gulu "banja / abwenzi / etc."). Njirayi ikhoza kudabwitsa "wamba" wosakonzekera yemwe alibe chidziwitso cham'mbuyo chogwira ntchito ndi ma database a NoSQL, koma ndi njira iyi yomwe imakulolani kugwiritsa ntchito mphamvu za HBase pa ntchitoyi bwino kwambiri:
RowKey
Mizati
Vasya
Petiya: 1
Olya: 1
Dasha: 1
Peter
Masha: 1
Vasya: 1
Apa timapeza zabwino zingapo nthawi imodzi. Kuti tiwamvetse, tiyeni tiwunikenso kamangidwe katsopanoko ndikuyerekeza zovuta zowerengera:
- Kuwerenga deta: kuti tiyankhe funso ngati Vasya adalembetsa ku Olya, ndikwanira kuwerenga ndime imodzi "Olya": ngati ilipo, ndiye yankho ndilowona, ngati silo - zabodza => O (1)
- Kusintha deta: kuwonjezera bwenzi: Kuwonjeza bwenzi: ingowonjezerani gawo latsopano la "ID ya Bwenzi" => O(1)
- Kusintha deta: kufufuta mnzako: ingochotsani gawo la ID ya Bwenzi => O(1)
Monga mukuonera, ubwino waukulu wa chitsanzo chosungirachi ndi chakuti muzochitika zonse zomwe timafunikira, timagwira ntchito ndi gawo limodzi lokha, popewa kuwerenga mzere wonse kuchokera ku database, komanso, kuwerengera mizati yonse ya mzerewu. Tikhoza kuyima pamenepo, koma...
Mutha kudabwitsidwa ndikupita patsogolo pang'ono panjira yokhathamiritsa magwiridwe antchito ndikuchepetsa magwiridwe antchito a I/O mukalowa mu database. Bwanji ngati titasunga zonse zokhudzana ndi ubale mwachindunji mu kiyi ya mzere womwewo? Ndiye kuti, pangani makiyi ophatikizika ngati userID.friendID? Pankhaniyi, sitiyenera ngakhale kuwerenga mizati ya mzere konse (Njira 4(mzere)):
RowKey
Mizati
Vasya.Petya
Petiya: 1
Vasya.Olya
Olya: 1
Vasya.Dasha
Dasha: 1
Petya.Masha
Masha: 1
Petya.Vasya
Vasya: 1
Mwachiwonekere, kuwunika kwa zochitika zonse zosokoneza deta mumtundu wotere, monga momwe zinalili kale, zidzakhala O (1). Kusiyanitsa ndi njira 3 kudzakhala kokha pakuchita bwino kwa ntchito za I/O mu nkhokwe.
Chabwino, "uta" wotsiriza. N'zosavuta kuona kuti posankha 4, fungulo la mzere lidzakhala ndi kutalika kosiyana, zomwe zingakhudze ntchito (pano tikukumbukira kuti HBase imasunga deta monga ma byte ndi mizere mu matebulo amasankhidwa ndi fungulo). Komanso tili ndi cholekanitsa chomwe chingafunike kusamaliridwa muzochitika zina. Kuti muchotse chikokachi, mutha kugwiritsa ntchito ma hashes kuchokera ku userID ndi friendID, ndipo popeza ma hashi onse azikhala ndi kutalika kosalekeza, mutha kungowagwirizanitsa, popanda olekanitsa. Ndiye zomwe zili patebulo ziziwoneka motere (Njira 5 (hashi)):
RowKey
Mizati
dc084ef00e94aef49be885f9b01f51c01918fa783851db0dc1f72f83d33a5994
Petiya: 1
dc084ef00e94aef49be885f9b01f51c0f06b7714b5ba522c3cf51328b66fe28a
Olya: 1
dc084ef00e94aef49be885f9b01f51c00d2c2e5d69df6b238754f650d56c896a
Dasha: 1
1918fa783851db0dc1f72f83d33a59949ee3309645bd2c0775899fca14f311e1
Masha: 1
1918fa783851db0dc1f72f83d33a5994dc084ef00e94aef49be885f9b01f51c0
Vasya: 1
Mwachiwonekere, zovuta za algorithmic zogwirira ntchito ndi dongosolo loterolo muzochitika zomwe tikuziganizira zidzakhala zofanana ndi za chisankho 4 - ndiko kuti, O (1).
Ponseponse, tiyeni tifotokoze mwachidule kuyerekezera kwathu konse kwazovuta zamakompyuta patebulo limodzi:
Kuwonjezera bwenzi
Kufufuza bwenzi
Kuchotsa mnzako
Njira 1 (zofikira)
O (n)
O (n)
O (n)
Njira 2 (kuwerengera)
O (1)
O (n)
O (n)
Njira 3 (gawo)
O (1)
O (1)
O (1)
Njira 4 (mzere)
O (1)
O (1)
O (1)
Njira 5 (hashi)
O (1)
O (1)
O (1)
Monga mukuonera, zosankha 3-5 zimawoneka ngati zokondedwa kwambiri ndipo zimatsimikizira kuchitidwa kwa zochitika zonse zofunikira zowonongeka kwa deta nthawi zonse. M'mikhalidwe ya ntchito yathu, palibe chofunikira chodziwikiratu kuti tipeze mndandanda wa abwenzi onse a wogwiritsa ntchito, koma muzochitika zenizeni za polojekiti, zingakhale bwino kwa ife, monga akatswiri abwino, "kuyembekezera" kuti ntchito yotereyi ichitike ndipo "kuwaza udzu." Choncho, chifundo changa chili kumbali ya chisankho 3. Koma ndizotheka kuti mu polojekiti yeniyeni pempholi likhoza kuthetsedwa kale ndi njira zina, choncho, popanda masomphenya ambiri a vuto lonse, ndibwino kuti musapange. mapeto omaliza.
Kukonzekera kuyesa
Ndikufuna kuyesa mfundo zomwe zili pamwambazi pochita - ichi chinali cholinga cha lingaliro lomwe linabuka kumapeto kwa sabata lalitali. Kuti tichite izi, m'pofunika kuwunika kuthamanga kwa ntchito ya "ntchito zovomerezeka" muzochitika zonse zomwe zafotokozedwa pogwiritsa ntchito nkhokwe, komanso kuwonjezeka kwa nthawiyi ndi kukula kwa malo ochezera a pa Intaneti (n). Zolinga zomwe zimatisangalatsa komanso zomwe tidzayesere panthawi yoyesera ndi nthawi yomwe "ntchito yovomerezeka" ikugwira ntchito imodzi "bizinesi". Ponena za "bizinesi" tikutanthauza chimodzi mwa izi:
- Kuwonjezera bwenzi latsopano
- Kuwona ngati Wogwiritsa A ndi bwenzi la Mtumiki B
- Kuchotsa bwenzi limodzi
Chifukwa chake, poganizira zofunikira zomwe zafotokozedwa m'mawu oyamba, zochitika zotsimikizira zimawonekera motere:
- Kujambula kwa data. Mwachisawawa pangani maukonde oyamba kukula n. Kuti muyandikire ku "dziko lenileni", chiwerengero cha abwenzi omwe aliyense wogwiritsa ntchito ali nacho chimakhala chosiyana. Yesani nthawi yomwe "ntchito yathu yovomerezeka" imalemba zonse zopangidwa ku HBase. Kenako gawani nthawi yotsatila ndi chiwerengero cha abwenzi owonjezera - umu ndi momwe timapezera nthawi ya "bizinesi" imodzi.
- Kuwerenga deta. Kwa wogwiritsa ntchito aliyense, pangani mndandanda wa "umunthu" womwe muyenera kupeza yankho ngati wogwiritsa ntchitoyo adawalembetsa kapena ayi. Kutalika kwa mndandanda = pafupifupi chiwerengero cha abwenzi a wogwiritsa ntchito, ndi theka la abwenzi omwe afufuzidwa yankho liyenera kukhala "Inde", ndi theka lina - "Ayi". Chekecho chimachitika mwadongosolo kotero kuti mayankho a "Inde" ndi "Ayi" asinthana (ndiko kuti, muzochitika zachiwiri zilizonse tidzayenera kudutsa mizati yonse ya mzere pazosankha 1 ndi 2). Nthawi yonse yowunika imagawidwa ndi kuchuluka kwa abwenzi omwe adayesedwa kuti apeze nthawi yowunikira pa phunziro lililonse.
- Kuchotsa deta. Chotsani anzanu onse kwa ogwiritsa ntchito. Kuphatikiza apo, kufufutidwa kumangochitika mwachisawawa (ndiko kuti, "timasokoneza" mndandanda woyambirira womwe umagwiritsidwa ntchito pojambulira deta). Nthawi yowerengera yonse imagawidwa ndi kuchuluka kwa anzawo omwe achotsedwa kuti apeze nthawi yapakati pa cheke.
Zochitikazo ziyenera kuyendetsedwa pamtundu uliwonse wa 5 data model options ndi kukula kosiyana kwa malo ochezera a pa Intaneti kuti muwone momwe nthawi imasinthira pamene ikukula. Mkati mwa n imodzi, zolumikizira pa netiweki ndi mndandanda wa ogwiritsa ntchito kuti awone ziyenera kukhala zofanana pazosankha zonse 5.
Kuti mumvetse bwino, pansipa pali chitsanzo cha deta yopangidwa ya n = 5. "Jenereta" yolembedwa imapanga madikishonale atatu a ID monga zotsatira:
- yoyamba ndi yoyikapo
- yachiwiri ndi yofufuza
- chachitatu - kufufutidwa
{0: [1], 1: [4, 5, 3, 2, 1], 2: [1, 2], 3: [2, 4, 1, 5, 3], 4: [2, 1]} # Π²ΡΠ΅Π³ΠΎ 15 Π΄ΡΡΠ·Π΅ΠΉ
{0: [1, 10800], 1: [5, 10800, 2, 10801, 4, 10802], 2: [1, 10800], 3: [3, 10800, 1, 10801, 5, 10802], 4: [2, 10800]} # Π²ΡΠ΅Π³ΠΎ 18 ΠΏΡΠΎΠ²Π΅ΡΡΠ΅ΠΌΡΡ
ΡΡΠ±ΡΠ΅ΠΊΡΠΎΠ²
{0: [1], 1: [1, 3, 2, 5, 4], 2: [1, 2], 3: [4, 1, 2, 3, 5], 4: [1, 2]} # Π²ΡΠ΅Π³ΠΎ 15 Π΄ΡΡΠ·Π΅ΠΉ
Monga mukuwonera, ma ID onse opitilira 10 mumtanthauzira mawu kuti awonedwe ndendende ndi omwe angapereke yankho labodza. Kuyika, kuyang'ana ndi kuchotsa "abwenzi" kumachitika ndendende mumndandanda womwe wafotokozedwa mudikishonale.
Kuyeseraku kunachitika pa laputopu yomwe ikuyenda Windows 10, pomwe HBase inali ikuyenda mu chidebe chimodzi cha Docker, ndipo Python yokhala ndi Jupyter Notebook inali kuyenda ina. Docker adapatsidwa 2 CPU cores ndi 2 GB ya RAM. Malingaliro onse, kutsanzira kwa "ntchito zovomerezeka" ndi "kupopera" popanga deta yoyesera ndi nthawi yoyezera, zidalembedwa mu Python. Laibulaleyi idagwiritsidwa ntchito kugwira ntchito ndi HBase , kuwerengera ma hashes (MD5) pazosankha 5 - hahlib
Poganizira mphamvu zamakompyuta za laputopu inayake, kukhazikitsidwa kwa n = 10, 30, ... kunasankhidwa moyesera. 170 - pamene nthawi yonse yogwiritsira ntchito nthawi yonse yoyezetsa (zochitika zonse za zosankha zonse kwa onse n) zinali zomveka komanso zoyenerera pa phwando limodzi la tiyi (pafupifupi mphindi 15).
Apa m'pofunika kunena kuti mu kuyesaku sitikuyesa kwenikweni ziwerengero zogwira ntchito. Ngakhale kuyerekeza kwachibale kwa zosankha ziwiri zosiyana sikungakhale kolondola. Tsopano tili ndi chidwi ndi chikhalidwe cha kusintha kwa nthawi malingana ndi n, popeza poganizira za kasinthidwe pamwamba pa "test stand", n'zovuta kupeza nthawi yowerengera "kuchotsedwa" kwa chikoka chachisawawa ndi zinthu zina ( ndipo ntchito yotereyi sinakhazikitsidwe).
Zotsatira zoyesera
Chiyeso choyamba ndi momwe nthawi yogwiritsira ntchito mndandanda wa abwenzi imasinthira. Chotsatira chiri mu graph ili pansipa.
Zosankha 3-5, monga momwe zikuyembekezeredwa, zikuwonetsa pafupifupi nthawi zonse "zochita bizinesi", zomwe sizidalira kukula kwa maukonde ndi kusiyana kosadziwika bwino.
Njira 2 ikuwonetsanso magwiridwe antchito nthawi zonse, koma oyipa pang'ono, pafupifupi ndendende nthawi 2 pokhudzana ndi zosankha 3-5. Ndipo izi sizingatheke koma kusangalala, chifukwa zimagwirizana ndi chiphunzitso - mu bukuli chiwerengero cha ntchito za I/O kupita ku/kuchokera ku HBase ndizokulirapo kawiri. Izi zitha kukhala umboni wosalunjika wosonyeza kuti benchi yathu yoyeserera, kwenikweni, imapereka kulondola kwabwino.
Njira 1 nayonso, monga momwe ikuyembekezeredwa, imakhala yochepetsetsa kwambiri ndipo ikuwonetsa kuwonjezeka kwa nthawi yomwe mumagwiritsa ntchito powonjezerana kukula kwa netiweki.
Tiyeni tsopano tione zotsatira za mayeso achiwiri.
Zosankha 3-5 zikuchitanso momwe zimayembekezeredwa - nthawi yokhazikika, osadalira kukula kwa netiweki. Zosankha 1 ndi 2 zikuwonetsa kuwonjezeka kwanthawi yayitali pamene kukula kwa netiweki kumawonjezeka komanso magwiridwe antchito ofanana. Kuphatikiza apo, kusankha 2 kumakhala kocheperako - mwachiwonekere chifukwa chakufunika kowerengera ndikukonza gawo lowonjezera la "kuwerengera", lomwe limawonekera kwambiri n kukula. Koma ndidzapeΕ΅abe kutsimikizira, popeza kuti kulondola kwa kuyerekezera kumeneku nβkochepa. Kuonjezera apo, ziwerengerozi (zosankha, 1 kapena 2, ndizofulumira) zinasintha kuchoka ku kuthamanga mpaka kuthamanga (posunga chikhalidwe cha kudalira ndi "kupita khosi ndi khosi").
Chabwino, graph yomaliza ndi zotsatira za kuyesa kuchotsa.
Apanso, palibe zodabwitsa apa. Zosankha 3-5 zimachotsa nthawi zonse.
Komanso, chochititsa chidwi, zosankha 4 ndi 5, mosiyana ndi zochitika zam'mbuyomo, zikuwonetsa ntchito yoipitsitsa pang'ono kusiyana ndi njira 3. Zikuoneka kuti ntchito yochotsa mzere ndi yokwera mtengo kuposa ntchito yochotsa ndime, yomwe nthawi zambiri imakhala yomveka.
Zosankha 1 ndi 2, monga zikuyembekezeredwa, zikuwonetsa kuwonjezeka kwa nthawi. Nthawi yomweyo, kusankha 2 kumakhala kocheperako kuposa njira 1 - chifukwa cha ntchito yowonjezera ya I/O "kusunga" gawo lowerengera.
Zotsatira zake zonse pakuyesa:
- Zosankha 3-5 zikuwonetsa kuchita bwino kwambiri akamapezerapo mwayi pa HBase; Komanso, machitidwe awo amasiyana wachibale wina ndi mzake ndi nthawi zonse ndipo sizidalira kukula kwa maukonde.
- Kusiyana pakati pa zosankha 4 ndi 5 sikunalembedwe. Koma izi sizikutanthauza kuti njira 5 siyenera kugwiritsidwa ntchito. Zikuoneka kuti zochitika zoyesera zomwe zimagwiritsidwa ntchito, poganizira za machitidwe a benchi yoyesera, sizinalole kuti zidziwike.
- Mkhalidwe wa kuchuluka kwa nthawi yofunikira kuchita "ntchito zamabizinesi" ndi data nthawi zambiri zimatsimikizira zowerengera zomwe zidapezedwa kale pazosankha zonse.
Epilogue
Zoyeserera movutikira zomwe zachitika siziyenera kutengedwa ngati zoona zenizeni. Pali zinthu zambiri zomwe sizinaganiziridwe ndikusokoneza zotsatira (kusinthasintha uku kumawoneka makamaka pama graph okhala ndi kukula kochepa kwa netiweki). Mwachitsanzo, kuthamanga kwa thrift, komwe kumagwiritsidwa ntchito ndi happybase, voliyumu ndi njira yogwiritsira ntchito mfundo zomwe ndidalemba mu Python (sindinganene kuti code idalembedwa bwino komanso mogwira ntchito zamagulu onse), mwina mawonekedwe a HBase caching, maziko a Windows 10 pa laputopu yanga, etc. Kawirikawiri, tikhoza kuganiza kuti zowerengera zonse zamaganizo zasonyeza kuti ndizovomerezeka. Chabwino, kapena sikunali kotheka kuwatsutsa ndi "mutu pamutu" wotere.
Pomaliza, malingaliro a aliyense amene angoyamba kumene kupanga zitsanzo za data mu HBase: mwachidule pazomwe zidachitika m'mbuyomu akugwira ntchito ndi nkhokwe zaubale ndikukumbukira "malamulo":
- Popanga, timapita ku ntchito ndi machitidwe akusintha kwa data, osati kuchokera pamtundu wa domain
- Kufikira bwino (popanda kusanthula tebulo lonse) - kokha ndi kiyi
- Denormalization
- Mizere yosiyana ikhoza kukhala ndi mizati yosiyana
- Kupanga kwamphamvu kwa olankhula
Source: www.habr.com