ukukhululwa okubalulekileyo kwenkqubo yokuhanjiswa kwesiganeko , iphawuleka ngotshintsho lwayo kulwakhiwo olutsha oluphunyezwe kwiJava, endaweni yolwimi lweClojure olwalusetyenziswa ngaphambili.
Iprojekthi ikuvumela ukuba uququzelele ukusetyenzwa okuqinisekisiweyo kweziganeko ezahlukeneyo ngexesha langempela. Ngokomzekelo, i-Storm ingasetyenziselwa ukuhlalutya imijelo yedatha ngexesha langempela, ukuqhuba imisebenzi yokufunda ngomatshini, ukulungelelanisa i-computing eqhubekayo, ukuphumeza i-RPC, i-ETL, njl. Inkqubo isekela ukudibanisa, ukudalwa kokulungelelaniswa kokunyamezela, imowudi yokuqinisekisa idatha kunye nokusebenza okuphezulu, okwaneleyo ukucubungula izicelo ezingaphezu kwesigidi ngesibini kwi-node enye yeqela.
Ukudityaniswa neenkqubo ezahlukeneyo zokusetyenzwa kwemigca kunye nobuchwepheshe besiseko sedatha kuyaxhaswa. Uyilo lweStorm lubandakanya ukufumana kunye nokusetyenzwa okungacwangciswanga, okuhlala kuhlaziywa rhoqo imijelo yedatha kusetyenziswa iiprosesa ezintsonkothileyo ezingenamqathango ezikwaziyo ukwahlula phakathi kwezigaba ezahlukeneyo zokubala. Le projekthi yanikezelwa kuluntu lwase-Apache emva kokuba i-Twitter ifumene i-BackType, inkampani eyaphuhlisa isakhelo. Ukusebenza, i-Storm yayisetyenziswe kwi-BackType ukuhlalutya ukubonakaliswa kweziganeko kwi-microblogs, ngokuthelekisa kwi-fly ii-tweets ezintsha kunye neekhonkco ezisetyenziswe kuzo (umzekelo, kwavavanywa ukuba izixhumanisi zangaphandle okanye izibhengezo ezipapashwe kwi-Twitter zaphinda zasasazwa ngabanye abathathi-nxaxheba. ).
Ukusebenza kweStorm kuthelekiswa neqonga leHadoop, umahluko ophambili kukuba idatha ayigcinwanga kwindawo yokugcina impahla, kodwa ifakwe ngaphandle kwaye iqhutywe ngexesha langempela. I-Storm ayinayo indawo yokugcina eyakhelwe-ngaphakathi kwaye umbuzo wohlalutyo uqala ukusetyenziswa kwidatha engenayo ide icinywe (ngelixa i-Hadoop isebenzisa i-Map yexesha elilinganiselweyo Nciphisa imisebenzi, i-Storm isebenzisa ingcamango yokuqhuba "i-topology" ngokuqhubekayo). Ukuphunyezwa kwabaphathi kungasasazwa kuzo zonke iiseva ezininzi - I-Storm ingqamanisa ngokuzenzekelayo umsebenzi kunye nemisonto kwiindawo ezahlukeneyo zeqela.
Inkqubo yabhalwa kuqala kwiClojure kwaye isebenza ngaphakathi kumatshini we-JVM obonakalayo. I-Apache Foundation iqalise inyathelo lokufudukela kwiStorm kwi-kernel entsha ebhalwe kwiJava, iziphumo ezicetywayo ekukhutshweni kwe-Apache Storm 2.0. Zonke izinto ezisisiseko zeqonga zibhalwa kwakhona kwiJava. Inkxaso yabaphathi bokubhala kwiClojure igcinwe, kodwa ngoku inikezelwa ngendlela yokubopha. I-Storm 2.0.0 ifuna iJava 8. Imodeli yokucubungula imisonto emininzi iye yahlengahlengiswa ngokupheleleyo, ivumela ukwanda okubonakalayo kokusebenza (kwezinye ii-topology, ukulibaziseka kwancitshiswa ngama-50-80%).
Inguqulelo entsha ikwabonelela nge-API entsha echwetheziweyo ye-Streams ekuvumela ukuba uchaze abaphathi usebenzisa imisebenzi yendlela yokusebenza. I-API entsha iphunyezwa phezulu kwesiseko esisemgangathweni se-API kwaye ixhasa ukudibanisa ngokuzenzekelayo kwemisebenzi ukunyusa ukusebenza kwayo. IWindowing API yemisebenzi yefestile yongeze inkxaso yogcino nokubuyisela kwimeko yangasemva.
Inkxaso yokuthathela ingqalelo izibonelelo ezongezelelweyo xa kusenziwa izigqibo ezingaphelelanga apho
I-CPU kunye nememori, njengenethiwekhi kunye noseto lwe-GPU. Inani elikhulu lophuculo lwenziwe ukuqinisekisa ukudibanisa kunye neqonga . Inkqubo yolawulo lokufikelela yandisiwe ukubandakanya ukukwazi ukwenza amaqela omlawuli kunye nokuthumela amathokheni. Uphuculo olongeziweyo olunxulumene ne-SQL kunye nenkxaso yeemetriki. Imiyalelo emitsha ivele kujongano lomlawuli ukwenzela ukulungisa ingxaki yelizwe leqela.
Iindawo ekufakwe kuzo isicelo seStorm:
- Ukucubungula imijelo yedatha entsha okanye uhlaziyo lwedatha ngexesha langempela;
- I-Computing eqhubekayo: Isiqhwithi sinokuqhuba imibuzo eqhubekayo kunye nokuqhuba imisinga eqhubekayo, ukuhambisa iziphumo zokuqhuba umxhasi ngexesha langempela.
- Umnxeba osasazwayo weNkqubo yokuLawulwa (i-RPC): Isiqhwithi singasetyenziselwa ukubonelela ngokuqhutywa ngokuhambelanayo kwemibuzo yobutyebi. Umsebenzi (“i-topology”) kwiStorm ngumsebenzi osasazwe kuzo zonke iindawo ezilinda ukuba imiyalezo ifike ekufuneka iqwalaselwe. Emva kokufumana umyalezo, umsebenzi usebenza kwimeko yendawo kwaye ubuyisela umphumo. Umzekelo wokusetyenziswa kwe-RPC esasaziweyo iya kuba kukuqhuba imibuzo yokukhangela ngokuhambelanayo okanye ukwenza imisebenzi kwiseti enkulu yeeseti.
Iimpawu zesiphango:
- Imodeli yenkqubo elula eyenza lula kakhulu ukusetyenzwa kwedatha ngexesha lokwenyani;
- Inkxaso yalo naluphi na ulwimi lwenkqubo. Iimodyuli ziyafumaneka kwiJava, iRuby kunye nePython, ukulungelelaniswa kwezinye iilwimi kulula enkosi kwiprotocol yonxibelelwano elula kakhulu efuna malunga nemigca yekhowudi eyi-100 ukuxhasa;
- Ukunyamezela impazamo: ukuqhuba umsebenzi wokucubungula idatha, kufuneka uvelise ifayile yejar ngekhowudi. I-Storm iya kusasaza ngokuzimeleyo le fayile yejagi ngapha kweenodi zeqela, qhagamshela abaphathi abanxulumene nayo, kwaye uququzelele ukubeka iliso. Xa umsebenzi ugqiba, ikhowudi iya kucinywa ngokuzenzekelayo kuzo zonke iindawo;
- I-Horizontal scalability. Zonke izibalo zenziwa kwimodi ehambelanayo; njengoko umthwalo usanda, kwanele ukudibanisa iindawo ezintsha kwiqela;
- Ukuthembeka. I-Storm iqinisekisa ukuba wonke umyalezo ongenayo uqhutywe ngokupheleleyo kanye. Umyalezo uyakusetyenzwa kube kanye kuphela ukuba akukho ziphoso xa udlula kubo bonke abaphathi; ukuba iingxaki ziyavela, iinzame zokuqhubekeka ezingaphumelelanga ziya kuphindwa.
- Isantya. Ikhowudi yeStorm ibhalwe ngokusebenza okuphezulu engqondweni kwaye isebenzisa inkqubo yomyalezo okhawulezayo we-asynchronous .
umthombo: opennet.ru