Ntchito iliyonse yayikulu ya data imafuna mphamvu zambiri zamakompyuta. Kusuntha kwanthawi zonse kuchokera ku database kupita ku Hadoop kumatha kutenga milungu ingapo kapena kuwononga ndalama zambiri ngati phiko la ndege. Simukufuna kudikirira ndikugwiritsa ntchito ndalama? Yerekezerani katunduyo pamapulatifomu osiyanasiyana. Njira imodzi ndiyo kukhathamiritsa kwa pushdown.
Ndinapempha mphunzitsi wamkulu wa Russia pa chitukuko ndi kayendetsedwe ka mankhwala a Informatica, Alexey Ananyev, kuti alankhule za kukhathamiritsa kukhathamiritsa ntchito mu Informatica Big Data Management (BDM). Kodi mudaphunzirapo kugwira ntchito ndi zinthu za Informatica? Mwinamwake, anali Alexey amene anakuuzani zoyambira za PowerCenter ndikufotokozera momwe mungapangire mapu.
Alexey Ananyev, wamkulu wa maphunziro ku DIS Group
Kodi pushdown ndi chiyani?
Ambiri a inu mumadziwa kale za Informatica Big Data Management (BDM). Chogulitsacho chikhoza kuphatikizira deta yaikulu kuchokera kuzinthu zosiyanasiyana, kusuntha pakati pa machitidwe osiyanasiyana, kumapereka mwayi wosavuta, kukulolani kuti muwonetsere, ndi zina zambiri.
M'manja oyenera, BDM imatha kuchita zodabwitsa: ntchito zidzamalizidwa mwachangu komanso ndi zida zochepa zamakompyuta.
Kodi inunso mukufuna zimenezo? Phunzirani kugwiritsa ntchito kankhira pansi mu BDM kugawa katundu wamakompyuta pamapulatifomu osiyanasiyana. Ukadaulo wa Pushdown umakupatsani mwayi wosintha mapu kukhala script ndikusankha malo omwe script iyi idzayendere. Kusankha uku kumakupatsani mwayi wophatikiza mphamvu zamapulatifomu osiyanasiyana ndikukwaniritsa magwiridwe antchito awo.
Kuti mukonze malo ochitira script, muyenera kusankha mtundu wa pushdown. Zolemba zimatha kuyendetsedwa kwathunthu pa Hadoop kapena kugawidwa pang'ono pakati pa gwero ndi kumira. Pali 4 zotheka pushdown mitundu. Kujambula sikuyenera kusinthidwa kukhala script (yachibadwidwe). Mapu amatha kuchitidwa momwe angathere pa gwero (gwero) kapena kwathunthu pa gwero (lodzaza). Mapu amathanso kusinthidwa kukhala zolemba za Hadoop (palibe).
Kukhathamiritsa kwa Pushdown
Mitundu 4 yomwe yatchulidwa imatha kuphatikizidwa m'njira zosiyanasiyana - kukankhira pansi kumatha kukonzedwa pazosowa zadongosolo. Mwachitsanzo, nthawi zambiri zimakhala zoyenera kuchotsa deta kuchokera ku database pogwiritsa ntchito mphamvu zake. Ndipo deta idzasinthidwa pogwiritsa ntchito Hadoop, kuti musalowetse deta yokha.
Tiyeni tiganizire za nkhaniyi pamene gwero ndi kopita zili m'dawunilodi, ndipo nsanja yosinthira kusintha ingasankhidwe: kutengera makonda, idzakhala Informatica, seva ya database, kapena Hadoop. Chitsanzo choterocho chidzakulolani kuti mumvetsetse bwino kwambiri mbali yaumisiri yogwiritsira ntchito makinawa. Mwachilengedwe, m'moyo weniweni, izi sizichitika, koma ndizoyenera kuwonetsa magwiridwe antchito.
Tiyeni titenge mapu kuti tiwerenge matebulo awiri munkhokwe imodzi ya Oracle. Ndipo mulole zotsatira zowerengera zilembedwe patebulo mu nkhokwe yomweyi. Ndondomeko ya mapu idzakhala motere:
Mu mawonekedwe a mapu pa Informatica BDM 10.2.1 zikuwoneka motere:
Mtundu wa Pushdown - mbadwa
Ngati tisankha mtundu wamtundu wakukankhira, ndiye kuti mapu achitika pa seva ya Informatica. Deta idzawerengedwa kuchokera ku seva ya Oracle, yotumizidwa ku seva ya Informatica, kusinthidwa kumeneko ndikusamutsidwa ku Hadoop. Mwanjira ina, tipeza njira ya ETL.
Mtundu wa Pushdown - gwero
Posankha mtundu wa gwero, timapeza mwayi wogawa ndondomeko yathu pakati pa seva ya database (DB) ndi Hadoop. Ndondomeko ikachitidwa ndi zochunirazi, zopempha zochotsa deta kuchokera pamatebulo zidzatumizidwa kunkhokwe. Ndipo zina zonse zidzachitidwa ngati masitepe pa Hadoop.
Chiwonetsero chokonzekera chidzawoneka motere:
Pansipa pali chitsanzo chokhazikitsa malo othamanga.
Pankhaniyi, kupanga mapu kuchitidwa munjira ziwiri. M'makonzedwe ake tidzawona kuti yasanduka script yomwe idzatumizidwa ku gwero. Kuphatikiza apo, kuphatikiza matebulo ndikusintha deta kudzachitika ngati funso lopanda tanthauzo pagwero.
Pachithunzi chomwe chili pansipa, tikuwona mapu okometsedwa pa BDM, ndi funso lofotokozedwanso pagwero.
Udindo wa Hadoop mu kasinthidwe uku udzachepetsedwa kuti usamayendetse kayendedwe ka deta - kuyiyika. Zotsatira zafunso zidzatumizidwa ku Hadoop. Kuwerenga kukamalizidwa, fayilo yochokera ku Hadoop idzalembedwa pamadzi.
Mtundu wa Pushdown - wodzaza
Mukasankha mtundu wonse, kupanga mapu kusandulika kukhala funso la database. Ndipo zotsatira za pempholo zidzatumizidwa ku Hadoop. Chithunzi cha ndondomeko yotereyi chikuwonetsedwa pansipa.
Chitsanzo chokhazikitsa chikuwonetsedwa pansipa.
Zotsatira zake, tipeza mapu okongoletsedwa ofanana ndi am'mbuyomu. Kusiyana kokha ndiko kuti malingaliro onse amasamutsidwa kwa wolandira mwa mawonekedwe opitilira kuyika kwake. Chitsanzo cha kukhathamiritsa kwa mapu chili pansipa.
Apa, monga momwe zinalili m'mbuyomu, Hadoop amasewera ngati wokonda. Koma apa gwero likuwerengedwa lonse, ndiyeno ndondomeko yokonza deta ikuchitika pa mlingo wolandila.
Mtundu wa Pushdown ndi null
Chabwino, njira yomaliza ndi mtundu wokankhira pansi, momwe mapu athu amasinthira kukhala zolemba za Hadoop.
Mapu okongoletsedwa tsopano aziwoneka motere:
Apa deta kuchokera ku mafayilo oyambira idzawerengedwa koyamba pa Hadoop. Kenako, pogwiritsa ntchito njira zake, mafayilo awiriwa adzaphatikizidwa. Pambuyo pake, deta idzasinthidwa ndikuyika ku database.
Pomvetsetsa mfundo za kukhathamiritsa kutsitsa, mutha kukonza bwino njira zambiri zogwirira ntchito ndi data yayikulu. Chifukwa chake, posachedwa, kampani ina yayikulu, m'masabata ochepa chabe, idatsitsa deta yayikulu kuchokera kusungidwe kupita ku Hadoop, yomwe idasonkhanitsa kale kwa zaka zingapo.
Source: www.habr.com