Isingeniso
“Kufanele ugijime ngokushesha ngangokunokwenzeka ukuze uhlale endaweni,
futhi ukuze ufike endaweni ethile, kufanele ugijime okungenani ngokuphindwe kabili!”
(c) U-Alice e-Wonderland
Esikhathini esidlule ngacelwa ukuba nginikeze inkulumo abahlaziyi Inkampani yethu igxile ekwakhiweni kwemodeli yedatha, njengoba sichitha isikhathi eside sisebenza kumaphrojekthi (ngezinye izikhathi iminyaka eminingana) futhi silahlekelwa umbono walokho okwenzekayo emhlabeni we-IT. Enkampanini yethu, njengoba kwenzeka, amaphrojekthi amaningi awasebenzisi imininingwane yolwazi ye-NoSQL (okungenani hhayi okwamanje), ngakho-ke enkulumweni yami, ngigxile kuwo ngokuqondile, ngisebenzisa i-HBase njengesibonelo, futhi ngazama ukulungisa isethulo kulabo abangakaze basebenze nabo. Ngokukhethekile, ngibonise ezinye zemininingwane yomklamo wemodeli yedatha ngisebenzisa isibonelo engasifunda eminyakeni embalwa edlule. Lapho ngihlaziya izibonelo, ngaqhathanisa izixazululo eziningana zenkinga efanayo ukuze ngidlulisele kangcono izilaleli imiqondo eyinhloko.
Muva nje, "ngenxa yesithukuthezi," ngizibuze (amaholide amade kaMeyi ngesikhathi sokuvalelwa kwabantu yedwa ayewulungele kakhulu lo mbuzo): ngabe imiqondo yethiyori ingahumusheka kanjani ekusebenzeni? Wazalwa kanjalo umbono walesi sihloko. Umthuthukisi osebenze ne-NoSQL isikhashana angase angazuzi lutho olusha kuyo (ngakho-ke angase eqe ingxenye yendatshana). Kodwa ngoba abahlaziyiKulabo abangakasebenzi kakhulu nge-NoSQL okwamanje, ngicabanga ukuthi kuzoba usizo ekutholeni ukuqonda okuyisisekelo kwemininingwane yokudizayina amamodeli wedatha we-HBase.
Ukuhlaziywa kwesibonelo
Ngokubona kwami, ngaphambi kokusebenzisa imininingwane yolwazi ye-NoSQL, udinga ukucabangisisa futhi ukale ubuhle nobubi. Ngokuvamile, umsebenzi ungase uxazululwe kusetshenziswa ama-DBMS angokwesiko ahlobene. Ngakho-ke, kungcono kakhulu ukugwema ukusebenzisa i-NoSQL ngaphandle kwesizathu esiphoqayo. Uma uthatha isinqumo sokusebenzisa i-database ye-NoSQL, khumbula ukuthi izindlela zokuklama zihluke ngandlela-thile. Ezinye zazo zingase zingajwayelekile ikakhulukazi kulabo abake basebenza kuphela nama-DBMS ahlobene (kokuhlangenwe nakho kwami). Isibonelo, emhlabeni wobudlelwano, sivamise ukuqala ngokumodela indawo yesihloko bese, uma kunesidingo, senze imodeli ibe yisimo. Ku-NoSQL, noma kunjalo, thina kufanele icabangele ngokushesha izimo ezilindelekile zokusebenza ngedatha futhi ekuqaleni wenze idatha ibe yengajwayelekile. Kukhona futhi nenani leminye umehluko, okuzoxoxwa ngakho ngezansi.
Ake sicabangele inkinga elandelayo "yokwenziwa", esizoqhubeka nokusebenza nayo:
Sidinga ukudizayina isakhiwo sesitoreji sohlu lwabangane lwabasebenzisi benethiwekhi ethile yokuxhumana nabantu. Ukwenza kube lula, sizothatha ukuthi konke ukuxhumana kuqondiswe (njengaku-Instagram, hhayi i-LinkedIn). Isakhiwo kufanele sivumele ukuthi okulandelayo kusetshenziswe ngempumelelo:
- Phendula umbuzo wokuthi ngabe umsebenzisi A ufunda umsebenzisi B (iphethini yokufunda)
- Vumela ukwengeza/ukukhipha ukuxhumana lapho umsebenzisi A elandela/engalandeli umsebenzisi B (isifanekiso sokushintsha idatha)
Yebo, ziningi izixazululo ezingenzeka kule nkinga. Kusizindalwazi esivamile sobudlelwano, singavele sidale ithebula lobudlelwano (mhlawumbe elithayiphiwe, uma, ngokwesibonelo, sidinga ukugcina amaqembu abasebenzisi—umndeni, umsebenzi, njll—okuhlanganisa “umngane” othile, futhi sengeze izinkomba/ukuhlukanisa ukuze sithuthukise isivinini sokufinyelela. Ithebula eliwumphumela lingase libukeke kanje:
USER_ID
ubunikazi_bomngane
I-Vasya
UPetya
I-Vasya
Olya
Kusukela lapha kuqhubeke, ukuze kucace futhi ngiqonde kangcono, ngizosebenzisa amagama esikhundleni sama-ID
Endabeni ye-HBase, siyazi ukuthi:
- ukusesha okuphumelelayo ngaphandle kokuskena kwetafula okugcwele kungenzeka kuphela ngokhiye
- Kungakho ukubhala imibuzo ejwayelekile ye-SQL kusizindalwazi esinjalo kuwumbono omubi; ngobuchwepheshe, ungakwazi, vele, ukuthumela umbuzo we-SQL ngokujoyina kanye nokunye okunengqondo kusuka ku-Impala kuya ku-HBase, kodwa izosebenza kangakanani?
Ngakho-ke, siphoqeleka ukuthi sisebenzise i-ID yomsebenzisi njengokhiye. Umcabango wokuqala esihlokweni esithi "kuphi futhi kanjani ukugcina ama-ID abangani?" kungase kube umqondo wokuwagcina kumakholomu. Le nketho esobala kakhulu futhi "engenangqondo" ingabukeka kanje (asiyibize Inketho 1 (okuzenzakalelayo), ukuze kusetshenziswe isikhathi esizayo):
I-RowKey
Amakholomu
I-Vasya
1: uPeti
2: ulala
3 :idla
UPetya
1: Masha
2: Vasya
Lapha, umugqa ngamunye uhambisana nomsebenzisi wenethiwekhi oyedwa. Amakholomu aqanjwe ngokuthi 1, 2, ...—ngokusekelwe enanini labangane—futhi ugcine ama-ID abangani. Kubalulekile ukuqaphela ukuthi umugqa ngamunye uzoba nenani elihlukile lamakholomu. Esibonelweni esingenhla, umugqa owodwa unamakholomu amathathu (1, 2, no-3), kanti owesibili unamakholomu amabili kuphela (1 kanye no-2). Lapha, sisebenzise izici ezimbili ze-HBase isizindalwazi esihlobene esingenazo:
- ikhono lokushintsha ukwakheka kwamakholomu (engeza umngane -> engeza ikholomu, susa umngane -> susa ikholomu)
- imigqa ehlukene ingaba nokuqanjwa kwamakholomu okuhlukene
Ake sihlole isakhiwo sethu ukuthi siyahambisana yini nezidingo zomsebenzi:
- Ukufunda idatha: ukuze siqonde ukuthi u-Vasya ubhalisele u-Olya, kuzodingeka sisuse umugqa wonke ngo-RowKey = "Vasya" futhi uphindaphinda amanani ekholomu size "sihlangane" no-Olya. Noma phinda kuwo wonke amanani ekholomu, "ungahlangani" no-Olya, bese ubuyisela amanga;
- Imininingwane yokuhlela: ukwengeza umngane: ngenkinga efanayo nathi sizodinga ukususa umugqa wonke ngu-RowKey = "Vasya" ukubala inani eliphelele labangane bakhe. Sidinga le nombolo ephelele yabangani ukuze sinqume inombolo yekholomu okufanele sigcine kuyo i-ID yomngane omusha.
- Ukuhlela idatha: ukususa umngane:
- Kuyadingeka ukususa umugqa wonke ngokhiye u-RowKey = "Vasya" futhi udlule kumakholomu ukuze uthole leyo lapho umngane osuswayo aqoshwa khona;
- Okulandelayo, ngemva kokususa umngane, sidinga "ukushintsha" yonke idatha kukholomu eyodwa ukuze singatholi "izikhala" kuzinombolo zabo.
Manje ake sihlole ukuthi akhiqiza kanjani lawa ma-algorithms, esizodinga ukuwasebenzisa ngasohlangothini "lwesicelo esinemibandela", azosetshenziswa. Ake sikhombise usayizi wenethiwekhi yethu yokuxhumana nabantu ecatshangelwayo njengo-n. Bese inani eliphezulu labangane umsebenzisi oyedwa angaba nalo ngu-(n-1). Singakuziba lokhu (-1) ngezinjongo zethu lapha, njengoba kungenamsebenzi phakathi komongo wokusebenzisa izimpawu ze-O.
- Ukufunda idatha: kuyadingeka ukususa wonke umugqa futhi, emkhawulweni, uphindaphinde kuwo wonke amakholomu awo. Lokhu kusho ukuthi isilinganiso sezindleko eziphezulu sizoba cishe u-O(n)
- Imininingwane yokuhlela: ukwengeza umngane: ukuze unqume inani labangane, udinga ukuphindaphinda kuwo wonke amakholomu omugqa, bese ufaka ikholomu entsha => O(n)
- Ukuhlela idatha: ukususa umngane:
- Ngokufanayo nokwengeza, kuyadingeka ukuphindaphinda kuwo wonke amakholomu emkhawulweni => O(n)
- Ngemva kokususa amakholomu, sidinga "ukuwashintsha". Indlela eqondile ingadinga kufika ku-(n-1) imisebenzi eyengeziwe. Kodwa-ke, lapha nakuso sonke isigaba esisebenzayo, sizosebenzisa indlela ehlukile esebenzisa "i-pseudo-shift" enanini elimisiwe lemisebenzi—okusho ukuthi kuthatha isikhathi esingaguquki kungakhathaliseki ukuthi n. Lesi sikhathi esingaguquki (O(2) ukunemba) asinaki uma siqhathaniswa no-O(n). Indlela yokwenza iboniswe esithombeni esingezansi: simane sikopishe idatha kusuka kukholomu "yokugcina" kuya kuleyo lapho idatha idinga ukususwa khona, bese sisusa ikholomu yokugcina:
Sekukonke, kuzo zonke izimo sithole inkimbinkimbi yekhompyutha ye-asymptotic ye-O(n).
Cishe usuqaphele kakade ukuthi cishe ngaso sonke isikhathi kufanele sifunde wonke umugqa ovela kusizindalwazi, futhi ezimweni ezimbili kwamathathu, ukuze nje singene kuwo wonke amakholomu futhi sibale inani eliphelele labangane. Ngakho-ke, njengomzamo wokwenza ngokugcwele, singakwazi ukwengeza ikholomu "yokubala" ukuze sigcine inani labangane kumsebenzisi ngamunye wenethiwekhi. Kulesi simo, akudingekile ukuthi sifunde umugqa wonke ukuze sibale isamba senani labangane, kodwa funda kuphela ikholomu "yokubala". Into esemqoka ukukhumbula ukubuyekeza "ukubala" lapho ukhohlisa idatha. Ngale ndlela, sithola ngcono Inketho 2 (isibalo):
I-RowKey
Amakholomu
I-Vasya
1: uPeti
2: ulala
3 :idla
inani: 3
UPetya
1: Masha
2: Vasya
inani: 2
Uma kuqhathaniswa nenketho yokuqala:
- Ukufunda idatha: ukuthola impendulo yombuzo othi "Ingabe u-Vasya ufunda u-Olya?" akukho okushintshile => O(n)
- Imininingwane yokuhlela: ukwengeza umnganeSenze lula ukufaka umngane omusha ngoba asisadingi ukufunda wonke umugqa futhi siphindaphinde kumakholomu awo. Manje singakwazi ukubuyisa kuphela inani lekholomu elithi "bala" futhi ngaleyo ndlela sinqume ngokushesha inombolo yekholomu yokufaka umngane omusha. Lokhu kunciphisa inkimbinkimbi yokubala ku-O(1).
- Ukuhlela idatha: ukususa umnganeLapho sisusa umngane, singasebenzisa le kholomu ukuze sinciphise inani lemisebenzi ye-I/O lapho sihambisa idatha iseli elilodwa kwesokunxele. Kodwa-ke, sisadinga ukuphindaphinda amakholomu ukuze sithole ezosuswa, ngakho => O(n)
- Ngakolunye uhlangothi, manje lapho sibuyekeza idatha sidinga ukubuyekeza ikholomu "yokubala" njalo, kodwa lokhu kuthatha isikhathi esingaguquki, esinganakwa ngaphakathi kohlaka lwezimpawu ze-O.
Sekukonke, inketho yesi-2 ibonakala ingcono kakhulu, kodwa ifana "nokuguquguquka esikhundleni soguquko." Ukuze sifinyelele "uguquko," sizodinga Inketho 3 (ikholomu).
Masiphendule yonke into phansi: asiqoke Igama lekholomu ID yomsebenzisi! Okulotshwe kukholamu ngokwayo akubalulekile kithi; ake sisebenzise inombolo 1 (ngokujwayelekile, izinto eziwusizo zingagcinwa lapho, isibonelo, njengeqembu elifana "nomndeni/abangane/njll."). Le ndlela ingase imangaze "umuntu ongenzi lutho" ongenalwazi lwangaphambilini ngemininingwane yolwazi ye-NoSQL, kodwa isivumela ukuthi sisebenzise amandla e-HBase ngempumelelo kakhulu kulo msebenzi:
I-RowKey
Amakholomu
I-Vasya
Phethiya: 1
Ulala: 1
Dasha: 1
UPetya
Masha: 1
Vasya: 1
Lokhu kunikeza izinzuzo eziningana. Ukuze siziqonde, ake sihlaziye isakhiwo esisha futhi silinganisele ukuba yinkimbinkimbi kwaso:
- Ukufunda idatha: ukuze uphendule umbuzo wokuthi u-Vasya ubhalisele u-Olya, kwanele ukufunda ikholomu eyodwa "Olya": uma kunjalo, khona-ke impendulo iyiqiniso, uma kungenjalo - Amanga => O(1)
- Imininingwane yokuhlela: ukwengeza umngane: Ukwengeza umngane: mane ungeze ikholomu entsha "I-ID yomngane" => O(1)
- Ukuhlela idatha: ukususa umngane: vele ususe ikholomu ye-"ID yomngane" => O(1)
Njengoba sibona, inzuzo enkulu yale modeli yesitoreji ukuthi kuzo zonke izimo ezidingekayo, sisebenza ngekholomu eyodwa kuphela, sigwema ukufunda wonke umugqa kusizindalwazi, kuncane kakhulu ukuphindaphinda kuwo wonke amakholomu kulowo mugqa. Singama lapho, kodwa...
Singathola ubuciko obuthe xaxa futhi siye phambili kancane ekuthuthukiseni ukusebenza kanye nokunciphisa imisebenzi ye-I/O lapho sifinyelela kusizindalwazi. Kuthiwani uma sigcina imininingwane yobudlelwano obugcwele ngqo kukhiye womugqa ngokwawo? Okusho ukuthi, yenza ukhiye ube yinhlanganisela, njenge-userID.friendID? Uma kunjalo, ngeke kudingeke nokuthi siwafunde nhlobo amakholomu emigqa (Inketho yesi-4 (umugqa)):
I-RowKey
Amakholomu
Vasya. UPetya
Phethiya: 1
Vasya.Olya
Ulala: 1
I-Vasya.Dasha
Dasha: 1
UPetya. UMasha
Masha: 1
UPetya. Vasya
Vasya: 1
Ngokusobala, ukuhlolwa kwazo zonke izimo zokukhohlisa idatha esakhiweni esinjalo kuzoba ngu-O(1), njengakunketho yangaphambilini. Umehluko ngenketho 3 uzoba kuphela ekusebenzeni kahle kwedatha ye-I/O imisebenzi.
Futhi ekugcineni, i-tidbit yokugcina. Kulula ukubona ukuthi kuKhetho 4, ukhiye wethu womugqa uzoba nobude obuguquguqukayo, okungase kube nomthelela ekusebenzeni (khumbula ukuthi i-HBase igcina idatha njengesethi yamabhayithi, futhi imigqa yethebula ihlelwa ngokhiye). Futhi, sinesihlukanisi, esingase sidinge ukucutshungulwa kwezinye izimo. Ukuqeda lo mthelela, singasebenzisa ama-hashes we-userID ne-friendID. Njengoba womabili ama-hashe azoba nobude obungaguquki, singamane sihlanganise, ngaphandle kwesihlukanisi. Bese idatha ekuthebula izobukeka kanje:Inketho 5 (hashi)):
I-RowKey
Amakholomu
dc084ef00e94aef49be885f9b01f51c01918fa783851db0dc1f72f83d33a5994
Phethiya: 1
dc084ef00e94aef49be885f9b01f51c0f06b7714b5ba522c3cf51328b66fe28a
Ulala: 1
dc084ef00e94aef49be885f9b01f51c00d2c2e5d69df6b238754f650d56c896a
Dasha: 1
1918fa783851db0dc1f72f83d33a59949ee3309645bd2c0775899fca14f311e1
Masha: 1
1918fa783851db0dc1f72f83d33a5994dc084ef00e94aef49be885f9b01f51c0
Vasya: 1
Ngokusobala, inkimbinkimbi ye-algorithmic yokusebenza nesakhiwo esinjalo ngokuya ngezimo esizicabangelayo izofana nenketho yesi-4 - okungukuthi, u-O(1).
Ngakho-ke, ake sifingqe zonke izilinganiso zethu zokuba yinkimbinkimbi kwekhompyutha etafuleni elilodwa:
Ingeza umngane
Ihlola umngane
Ukususa umngane
Inketho 1 (okuzenzakalelayo)
O (n)
O (n)
O (n)
Inketho 2 (ukubala)
O (1)
O (n)
O (n)
Inketho 3 (ikholomu)
O (1)
O (1)
O (1)
Inketho yesi-4 (umugqa)
O (1)
O (1)
O (1)
Inketho 5 (hashi)
O (1)
O (1)
O (1)
Njengoba ungabona, izinketho 3-5 zibonakala zikhethwa futhi ngokwethiyori ziqinisekisa ukwenziwa kwazo zonke izimo zokukhohlisa idatha ngesikhathi esifanayo. Isitatimende sethu senkinga asidingi ngokusobala ukubuyisa uhlu lwabo bonke abangani bomsebenzisi, kodwa kumaphrojekthi omhlaba wangempela, njengabahlaziyi abahle, sizobe selulekwa kahle ukuthi siyilindele kusengaphambili inkinga enjalo futhi sithathe izinyathelo zokuphepha. Ngakho-ke, ngikhetha inketho yesi-3. Nokho, kungenzeka ukuthi kuphrojekthi yomhlaba wangempela, lesi sicelo kungenzeka ukuthi sesivele sidingidwe ngezinye izindlela, ngakho-ke ngaphandle kokuqonda okujwayelekile kwayo yonke inkinga, kungcono ukuthi ungafinyeleli isiphetho sokugcina.
Ilungiselela ukuhlolwa
Ngingathanda ukuhlola indlela yokucabanga engenhla ngokoqobo—leyo bekuyinhloso yombono ovele ngempelasonto ende. Ukuze senze lokhu, sidinga ukuhlola ukusebenza "kohlelo lwethu lokusebenza olukhohlisayo" kuzo zonke izimo zokusetshenziswa kwesizindalwazi esichazwe, kanye nokuthi lesi sikhathi sikhula kanjani ngosayizi wenethiwekhi yokuxhumana nomphakathi (n). Ipharamitha eqondiwe esiyithandayo, futhi esizoyilinganisa phakathi nokuhlolwa, isikhathi esithatha "uhlelo lokusebenza olucatshangelwayo" ukwenza "umsebenzi webhizinisi" owodwa. Ngokuthi "ukusebenza kwebhizinisi," sisho okukodwa kokulandelayo:
- Yengeza umngane oyedwa omusha
- Ibheka ukuthi umsebenzisi A ungumngane womsebenzisi B
- Ukususa umngane oyedwa
Ngakho-ke, kucatshangelwa izidingo ezishiwo esitatimendeni sokuqala, isimo sokuqinisekisa simi kanje:
- Ukuqoshwa kwedathaKhiqiza ngokungahleliwe inethiwekhi yokuqala kasayizi n. Ukuze ulinganisele kangcono "umhlaba wangempela," inombolo yabangane umsebenzisi ngamunye anabo nayo ayihleliwe. Linganisa isikhathi esisithatha "uhlelo lwethu lokusebenza lwe-dummy" ukubhala yonke idatha ekhiqiziwe ku-HBase. Bese uhlukanisa lesi sikhathi ngenani eliphelele labangane abengeziwe—lokhu kuzosinika isikhathi esimaphakathi "sokusebenza kwebhizinisi" okukodwa.
- Ukufunda idathaKumsebenzisi ngamunye, dala uhlu "lwezinto zobuntu" okudingeka unqume ukuthi umsebenzisi uyabalandela noma cha. Ubude bohlu bucishe bulingane nenani labangane bomsebenzisi, impendulo ithi "Yebo" engxenyeni yabangani abahloliwe futhi "Cha" kwenye ingxenye. Ukuhlola kwenziwa ngendlela yokuthi izimpendulo zika-"Yebo" kanye "no-Cha" zishintshane (okusho ukuthi, kuzo zonke ezinye izimo, kuzodingeka sijikeleze kuwo wonke amakholomu omugqa ukuze uthole izinketho 1 kanye no-2). Isikhathi sokuhlola esiphelele sibe sesihlukaniswa ngenani labangane abahloliwe ukuze kutholwe isilinganiso sesikhathi ngesheke ngalinye.
- Isusa idathaSusa bonke abangani bomsebenzisi. I-oda lokususa alihleliwe (okungukuthi, sishova uhlu lwangempela olusetshenziswa ukurekhoda idatha). Isikhathi esiphelele sokuqinisekisa sibe sesihlukaniswa ngenani labangane abasuswayo ukuze kutholwe isilinganiso sesikhathi sokuqinisekisa ngakunye.
Izimo kufanele zisetshenziselwe ukwahluka kwemodeli ngayinye yedatha emihlanu kanye nosayizi abahlukene benethiwekhi yokuxhumana nomphakathi ukuze kubonwe ukuthi isikhathi sishintsha kanjani njengoba inethiwekhi ikhula. Ngaphakathi kwe-n-model eyodwa, uxhumo lwenethiwekhi kanye nohlu lwabasebenzisi abazohlolwa kufanele, vele, kufane kuzo zonke izinhlobo ezinhlanu.
Ukuze uthole ukuqonda okungcono, ngezansi isibonelo sedatha ekhiqiziwe ye-n = 5. "Ijeneretha" engiyibhalile ikhiqiza izichazamazwi ezintathu ze-ID:
- eyokuqala ngeyokufaka
- eyesibili ngeyokuhlola
- eyesithathu ngeyokususwa
{0: [1], 1: [4, 5, 3, 2, 1], 2: [1, 2], 3: [2, 4, 1, 5, 3], 4: [2, 1]} # всего 15 друзей
{0: [1, 10800], 1: [5, 10800, 2, 10801, 4, 10802], 2: [1, 10800], 3: [3, 10800, 1, 10801, 5, 10802], 4: [2, 10800]} # всего 18 проверяемых субъектов
{0: [1], 1: [1, 3, 2, 5, 4], 2: [1, 2], 3: [4, 1, 2, 3, 5], 4: [1, 2]} # всего 15 друзей
Njengoba ubona, wonke ama-ID angaphezu kuka-10,000 kusichazamazwi ukuze aqinisekiswe yilawo azobuya nakanjani Amanga. Ukufakwa, ukuqinisekiswa, nokususwa kokuthi "abangane" kwenziwa ngokulandelana okucaciswe kusichazamazwi.
Ukuhlolwa kwenziwe kukhompyutha ephathekayo esebenza Windows 10, lapho i-HBase yayisebenza khona esitsheni esisodwa se-Docker, kanti i-Python ene-Jupyter Notebook yayisebenza kwenye. I-Docker yanikezwa ama-core amabili e-CPU kanye ne-RAM engu-2 GB. Yonke i-logic, okuhlanganisa nokulingisa "uhlelo lokusebenza oluyi-dummy" kanye nohlaka lokukhiqiza idatha yokuhlola kanye nesikhathi sokulinganisa, yabhalwa nge-Python. Umtapo wolwazi , ukubala ama-hashes (MD5) ngenketho 5 - hahlib
Kucatshangelwa amandla ekhompuyutha ekhompuyutha ephathwayo ethile, ukwethulwa kwakhethwa ngokuhlola ku-n = 10, 30, …. 170 – lapho isamba sesikhathi sokusebenza somjikelezo wokuhlola ogcwele (zonke izimo zazo zonke izinketho zabo bonke n) sisafaneleka kakhulu futhi silingana ngesikhathi sephathi yetiye eyodwa (ngokwesilinganiso imizuzu eyi-15).
Kubalulekile ukuqaphela lapha ukuthi kulokhu kuhlolwa, asihloli ngokuyinhloko izibalo zokusebenza eziphelele. Ngisho nokuqhathanisa okuhlobene kwezinhlobonhlobo ezimbili ezihlukene kungase kungabi nembayo ngokuphelele. Okwamanje sinentshisekelo kumvelo yoshintsho lwesikhathi njengomsebenzi ka-n, njengoba, uma kubhekwa ukumiswa kwe-testbed okukhulunywe ngenhla, ukuthola izilinganiso zesikhathi "ezisusiwe" ethonyeni lokungahleliwe nezinye izici kunzima kakhulu (futhi, empeleni, bekungewona umgomo).
Umphumela wokuhlolwa
Ukuhlolwa kokuqala kubheke ukuthi sishintsha kanjani isikhathi sokugcwalisa uhlu lwabangani. Imiphumela ikhonjiswe kugrafu engezansi.
Izinketho 3-5, njengoba kulindelekile, zibonisa isikhathi "sokusebenza kwebhizinisi" esingaguquki, esizimele ekukhuleni kosayizi wenethiwekhi, kanye nomehluko ongenakuhlukaniswa ekusebenzeni.
Okuhlukile 2 nakho kukhombisa ukusebenza okungaguquki, kodwa kubi kakhulu, cishe izikhathi ezi-2 kunalezo ezihlukile 3-5. Lokhu kuyakhuthaza, njengoba kuhambisana nethiyori—kulokhu okuhlukile, inani lemisebenzi ye-I/O ukuya/kusuka ku-HBase inkulu ngokuphindwe izikhathi ezingu-2. Lokhu kungase kube ubufakazi obungaqondile bokuthi ukusethwa kokuhlolwa kwethu ngokuvamile kunikeza ukunemba okuhle.
Inketho 1 iphinda ibonakale ihamba kancane, njengoba bekulindelekile, futhi ikhombisa ukukhuphuka komugqa ngesikhathi esithathwayo ukwengeza umngani oyedwa, kuye ngosayizi wenethiwekhi.
Manje ake sibheke imiphumela yokuhlolwa kwesibili.
Okuhlukile okungu-3-5 futhi kusebenza njengoba kulindelekile—isikhathi esivamile, esizimele kusayizi wenethiwekhi. Okuhlukile koku-1 no-2 kubonisa ukukhula kwesikhathi somugqa ngosayizi wenethiwekhi nokusebenza okufanayo. Okuhlukile 2, nokho, kuhamba kancane—ngokusobala ngenxa yesidingo sokuhlaziya nokucubungula ikholomu "yokubala" eyengeziwe, ebonakala kakhulu njengoba u-n ekhula. Nokho, ngizogodla ukwahlulela, njengoba ukunemba kwalokhu kuqhathanisa kuphansi kakhulu. Ngaphezu kwalokho, ukuhlobana (okuyinto ehlukile, engu-1 noma engu-2, esheshayo) kuye kwahluka ukusuka ekugijimeni kuye kokuthi kusebenze (ngenkathi kugcinwa iphethini engaguquki nokuba yintamo nentamo).
Hhayi-ke, igrafu yokugcina ingumphumela wokuhlolwa kokususa.
Nalapha, azikho izimanga lapha. Izinketho 3-5 zenza ukususa ngesikhathi esingashintshi.
Kuyathakazelisa ukuthi izinketho 4 kanye no-5, ngokungafani nezimo zangaphambilini, zibonisa ukusebenza okubi kakhulu kunenketho yesi-3. Ngokusobala, umsebenzi wokususa umugqa ubiza kakhulu kunomsebenzi wokususa ikholomu, okuvame ukunengqondo.
Izinketho 1 no-2, njengoba kulindelekile, zibonisa ukukhula kwesikhathi somugqa. Nokho, inketho yesi-2 ihlala ihamba kancane kunenketho 1 ngenxa yomsebenzi owengeziwe we-I/O odingekayo ukuze kugcinwe ikholomu yokubala.
Iziphetho ezijwayelekile zokuhlolwa:
- Izinketho 3-5 zibonisa ukusebenza kahle okukhulu, njengoba zisebenzisa i-HBase; Nokho, ukusebenza kwabo kuyahluka komunye nomunye ngokuqhubekayo futhi akuxhomekile kusayizi wenethiwekhi.
- Awukho umehluko obonwe phakathi kwezinketho 4 kanye nezi-5. Nokho, lokhu akusho ukuthi inketho yesi-5 akufanele isetshenziswe. Kungenzeka ukuthi isimo sokuhlola esisetshenzisiwe, uma kubhekwa izici zokusebenza ze-testbed, sivimbele lokhu ukuthi kungatholwa.
- Imvelo yokwenyuka kwesikhathi esidingekayo ukwenza "imisebenzi yebhizinisi" ngedatha ngokuvamile iqinisekisa izibalo zethiyori ezitholwe ngaphambilini zazo zonke izinketho.
Epilogue
Lezi zivivinyo ezingahleliwe akufanele zithathwe njengeqiniso eliphelele. Kunezici eziningi ezingazange zicatshangelwe futhi zahlanekezela imiphumela (lokhu kuguquguquka kubonakala kakhulu kumagrafu ngobukhulu benethiwekhi encane). Isibonelo, ijubane le-thrift, elisetshenziswa yi-happybase, ivolumu kanye nendlela yokusebenzisa ye-logic engiyibhalile ku-Python (angikwazi ukuthi ikhodi yabhalwa kahle noma yasetshenziswa ngempumelelo kuzo zonke izingxenye), mhlawumbe izici ze-HBase caching, kanye nomsebenzi wangemuva. Windows 10 Ku-laptop yami, njll. Sekukonke, kungaphethwa ngokuthi zonke izinkolelo-mbono ziye zafakazelwa ngokuhlola ukuthi ziyiqiniso. Noma okungenani, bekungenakwenzeka ukuziphikisa ngokuhlasela okunjalo ngqo.
Sengiphetha, nazi izincomo zanoma ubani oqala ukuklama amamodeli edatha ku-HBase: beka eceleni ulwazi lwakho lwangaphambilini lwemininingwane yokuxhumana futhi ukhumbule "imiyalo":
- Lapho siklama, siqhubeka nomsebenzi kanye namaphethini okukhohlisa idatha, hhayi kumodeli yesizinda
- Ukufinyelela okusebenzayo (ngaphandle kokuskena kwethebula eligcwele) - ngokhiye kuphela
- Ukunciphisa umzimba
- Imigqa ehlukene ingaba namakholomu ahlukene.
- Ukwakheka okunamandla kwamakholomu
Source: www.habr.com
