Apache Storm 2.0 yogawa makina apakompyuta omwe alipo

adawona kuwala kutulutsidwa kwakukulu kwa kachitidwe kogawa zochitika Apache Storm 2.0, yodziwika chifukwa chakusintha kwa zomangamanga zatsopano zomwe zakhazikitsidwa ku Java, m'malo mwa chilankhulo cha Clojure chomwe chidagwiritsidwa ntchito kale.

Pulojekitiyi imakupatsani mwayi wokonza zotsimikizika za zochitika zosiyanasiyana munthawi yeniyeni. Mwachitsanzo, Mkuntho ukhoza kugwiritsidwa ntchito kusanthula mitsinje ya data mu nthawi yeniyeni, kuyendetsa ntchito zophunzirira pamakina, kukonza makompyuta opitilira, kukhazikitsa RPC, ETL, ndi zina zambiri. Dongosololi limathandizira kusanjika, kupanga masinthidwe osagwirizana ndi zolakwika, njira yotsimikizika yosinthira deta ndipo imakhala ndi magwiridwe antchito apamwamba, okwanira kukonza zopempha zopitilira miliyoni pamphindi pagawo limodzi lamagulu.

Kuphatikizana ndi machitidwe osiyanasiyana opangira mizere ndi matekinoloje a database kumathandizidwa. Zomangamanga za Storm zimaphatikizapo kulandira ndi kukonza mitsinje yosasinthika, yosinthidwa kosalekeza pogwiritsa ntchito ma processor ovuta omwe amatha kugawa pakati pa magawo osiyanasiyana owerengera. Ntchitoyi idaperekedwa kwa gulu la Apache pambuyo poti Twitter idapeza BackType, kampani yomwe idapanga dongosololi. M'malo mwake, Storm idagwiritsidwa ntchito mu BackType kusanthula zomwe zikuchitika mu ma microblogs, poyerekeza ma tweets atsopano akuwuluka ndi maulalo omwe amagwiritsidwa ntchito mwa iwo (mwachitsanzo, adawunikidwa momwe maulalo akunja kapena zolengeza zomwe zidasindikizidwa pa Twitter zidawulutsidwanso ndi ena omwe adatenga nawo gawo. ).

Ntchito ya Storm ikufananizidwa ndi nsanja ya Hadoop, kusiyana kwakukulu ndikuti deta siisungidwa m'nyumba yosungiramo katundu, koma imalowetsedwa kunja ndikukonzedwa mu nthawi yeniyeni. Storm ilibe chosungiramo chosungiramo ndipo funso lowunikira limayamba kugwiritsidwa ntchito ku data yomwe ikubwera mpaka itathetsedwa (pamene Hadoop amagwiritsa ntchito nthawi yocheperako MapReduce ntchito, Storm amagwiritsa ntchito lingaliro lopitilira "topology"). Kuphatikizika kwa othandizira kumatha kugawidwa pamaseva angapo - Storm imangofananiza ntchito ndi ulusi pamagulu osiyanasiyana.

Dongosololi lidalembedwa koyambirira ku Clojure ndipo likuyenda mkati mwa makina a JVM. Apache Foundation yakhazikitsa njira yosamutsira Storm kupita ku kernel yatsopano yolembedwa ku Java, zomwe zotsatira zake zimaperekedwa pakutulutsidwa kwa Apache Storm 2.0. Zida zonse zoyambira papulatifomu zimalembedwanso mu Java. Thandizo la olemba zolemba ku Clojure lasungidwa, koma tsopano likuperekedwa ngati zomangira. Storm 2.0.0 imafuna Java 8. Makina opangira ulusi wambiri adakonzedwanso, kulola kukwaniritsa kuwonjezeka kowoneka bwino kwa magwiridwe antchito (kwa ma topology ena, kuchedwa kunachepetsedwa ndi 50-80%).

Mtundu watsopanowu ulinso ndi mtundu watsopano wa Mitsinje API womwe umakupatsani mwayi wofotokozera othandizira pogwiritsa ntchito machitidwe opangira mapulogalamu. API yatsopano imayikidwa pamwamba pa API yokhazikika ndipo imathandizira kuphatikizana kwa magwiridwe antchito kuti akwaniritse kukonza kwawo. Windowing API ya ntchito zazenera yawonjezera chithandizo chopulumutsa ndi kubwezeretsa dziko kumbuyo.

Thandizo pakuganizira zowonjezera zowonjezera popanga zisankho zosachepera
CPU ndi kukumbukira, monga maukonde ndi GPU zoikamo. Zosintha zambiri zapangidwa kuti zitsimikizire kuphatikizidwa ndi nsanja Kafka. Dongosolo lowongolera zofikira lakulitsidwa kuti liphatikizepo kuthekera kopanga magulu owongolera ndikugawa ma tokeni. Zowonjezera zowonjezera zokhudzana ndi SQL ndi thandizo la ma metrics. Malamulo atsopano awonekera mu mawonekedwe a administrator kuti athetse vuto la cluster state.

Malo ogwiritsira ntchito Storm:

• Kukonza mitsinje ya data yatsopano kapena zosintha za database mu nthawi yeniyeni;
• Computing mosalekeza: Mkuntho ukhoza kuyankha mafunso mosalekeza ndi kukonza mitsinje yosalekeza, ndikupereka zotsatira zoyeserera kwa kasitomala munthawi yeniyeni.
• Call Distributed Remote Procedure Call (RPC): Mkuntho ungagwiritsidwe ntchito popereka mafunso okhudzana ndi zofunikira. Ntchito ("topology") mu Storm ndi ntchito yogawidwa m'malo onse omwe amadikirira kuti mauthenga afike omwe akufunika kukonzedwa. Pambuyo polandira uthenga, ntchitoyi imagwira ntchito m'deralo ndikubwezeretsa zotsatira zake. Chitsanzo chogwiritsa ntchito RPC yogawidwa chingakhale kukonza mafunso osaka molumikizana kapena kuchita ma seti ambiri.

Mawonekedwe a Storm:

• Mtundu wosavuta wamapulogalamu womwe umathandizira kwambiri kukonza kwanthawi yeniyeni;
• Thandizo la zilankhulo zilizonse zamapulogalamu. Ma modules alipo a Java, Ruby ndi Python, kusintha kwa zilankhulo zina ndikosavuta chifukwa cha njira yosavuta yolumikizirana yomwe imafunikira mizere 100 ya code kuti ithandizire;
• Kulekerera zolakwika: kuti mugwiritse ntchito ntchito yokonza deta, muyenera kupanga fayilo ya mtsuko ndi code. Storm igawira payokha fayilo ya mtsukowu m'magulumagulu, kulumikiza zowongolera zomwe zikugwirizana nazo, ndikukonzekera kuyang'anira. Ntchitoyo ikatha, codeyo idzayimitsidwa pamfundo zonse;
• Horizontal scalability. Mawerengedwe onse amachitidwa mofanana; pamene katundu akuwonjezeka, ndikwanira kungogwirizanitsa mfundo zatsopano kumagulu;
• Kudalirika. Storm imawonetsetsa kuti uthenga uliwonse womwe ukubwera wakonzedwa kamodzi kokha. Uthengawu udzasinthidwa kamodzi kokha ngati palibe zolakwika podutsa onse ogwira ntchito; ngati mavuto abuka, ndiye kuti kuyesa kosatheka kudzabwerezedwa.
• Liwiro. Khodi ya Storm imalembedwa ndikuchita bwino m'maganizo ndipo imagwiritsa ntchito makina otumizira mauthenga mwachangu ZeroMQ.

Source: opennet.ru