ukukhishwa okubalulekile kwesistimu yokucubungula umcimbi esabalalisiwe , ephawuleka ngokushintshela esakhiweni esisha esisetshenziswe ku-Java, esikhundleni solimi lwe-Clojure olwalusetshenziswa ngaphambilini.
Iphrojekthi ikuvumela ukuthi uhlele ukucutshungulwa okuqinisekisiwe kwemicimbi ehlukahlukene ngesikhathi sangempela. Isibonelo, i-Storm ingasetshenziswa ukuhlaziya ukusakazwa kwedatha ngesikhathi sangempela, ukwenza imisebenzi yokufunda ngomshini, ukuhlela ikhompuyutha eqhubekayo, ukusebenzisa i-RPC, i-ETL, njll. Uhlelo lusekela ukuhlanganisa, ukudalwa kokucushwa okubekezelela amaphutha, imodi yokucubungula idatha eqinisekisiwe futhi inokusebenza okuphezulu, okwanele ukucubungula izicelo ezingaphezu kwesigidi ngomzuzwana endaweni eyodwa yeqoqo.
Ukuhlanganiswa nezinhlelo ezihlukahlukene zokucubungula ulayini kanye nobuchwepheshe besizindalwazi kuyasekelwa. Isakhiwo seStorm sihlanganisa ukwamukela nokucubungula imifudlana yedatha engahlelekile, evuselelwe njalo kusetshenziswa amaphrosesa ayinkimbinkimbi anamandla okuhlukanisa phakathi kwezigaba ezahlukene zokubala. Le phrojekthi yanikezwa umphakathi wama-Apache ngemuva kokuthi i-Twitter ithole i-BackType, inkampani eyasungula uhlaka ekuqaleni. Empeleni, i-Storm yasetshenziswa ku-BackType ukuze kuhlaziywe ukuvela kwezehlakalo kuma-microblogs, ngokuqhathanisa nama-tweets amasha andizayo kanye nezixhumanisi ezisetshenziswa kuwo (ngokwesibonelo, kwahlolwa ukuthi izixhumanisi zangaphandle noma izimemezelo ezishicilelwe ku-Twitter zaphinde zasakazwa kanjani abanye ababambiqhaza. ).
Ukusebenza kwe-Storm kuqhathaniswa nesiteji se-Hadoop, umehluko oyinhloko ukuthi idatha ayigcinwa endaweni yokugcina impahla, kodwa idliwe ngaphandle futhi icutshungulwe ngesikhathi sangempela. I-Storm ayinaso isendlalelo sesitoreji esakhelwe ngaphakathi futhi umbuzo wokuhlaziya uqala ukusetshenziswa kudatha engenayo uze ukhanselwe (ngenkathi i-Hadoop isebenzisa imisebenzi ye-MapReduce yesikhathi esilinganiselwe, i-Storm isebenzisa umqondo wokuqhubeka nokusebenzisa "i-topology"). Ukwenziwa kwezibambi kungasatshalaliswa kuzo zonke iziphakeli ezimbalwa - Isiphepho sifanisa ngokuzenzakalelayo umsebenzi nochungechunge kumanodi eqoqo ahlukene.
Uhlelo ekuqaleni lwalubhalwe nge-Clojure futhi lusebenza ngaphakathi komshini we-JVM obonakalayo. I-Apache Foundation yethule isinyathelo sokuthuthela i-Storm ku-kernel entsha ebhalwe ku-Java, imiphumela yayo ehlongozwayo ekukhululweni kwe-Apache Storm 2.0. Zonke izingxenye eziyisisekelo zeplathifomu zibhalwa kabusha nge-Java. Ukusekelwa kwezibambi zokubhala ku-Clojure kugciniwe, kodwa manje kunikezwa ngendlela yokubopha. I-Storm 2.0.0 idinga i-Java 8. Imodeli yokucubungula enezintambo eziningi isiklanywe kabusha ngokuphelele, ivumela ukwanda okubonakalayo kokusebenza (kwezinye i-topology, ukubambezeleka kwancishiswa ngo-50-80%).
Inguqulo entsha futhi inikezela nge-Streams API entsha ethayiphiwe ekuvumela ukuthi uchaze izibambi usebenzisa imisebenzi yesitayela sokuhlela esisebenzayo. I-API entsha isetshenziswa ngaphezulu kwesisekelo se-API esijwayelekile futhi isekela ukuhlanganiswa okuzenzakalelayo kwemisebenzi ukuze kuthuthukiswe ukucubungula kwayo. I-Windowing API yemisebenzi yewindi yengeze usekelo lokulondoloza nokubuyisela esimweni ngemuva.
Ukusekela ukucabangela izinsiza ezengeziwe lapho wenza izinqumo ezingagcini lapho
I-CPU nememori, njengenethiwekhi nezilungiselelo ze-GPU. Inani elikhulu lentuthuko lenziwe ukuze kuqinisekiswe ukuhlanganiswa neplatifomu . Isistimu yokulawula ukufinyelela inwetshiwe ukuze ifake ikhono lokudala amaqembu omlawuli kanye namathokheni okuthumela. Kungezwe ukuthuthukiswa okuhlobene nosekelo lwe-SQL namamethrikhi. Imiyalo emisha ivele kusixhumi esibonakalayo somlawuli sokulungisa iphutha lesimo seqoqo.
Izindawo zokufaka isicelo seStorm:
- Ukucubungula ukusakazwa kwedatha entsha noma izibuyekezo zesizindalwazi ngesikhathi sangempela;
- I-Computing Eqhubekayo: Isiphepho singasebenzisa imibuzo eqhubekayo futhi sicubungule imifudlana eqhubekayo, ilethe imiphumela yokucubungula kuklayenti ngesikhathi sangempela.
- Ikholi Yenqubo Ekude Esabalalisiwe (i-RPC): Isiphepho singasetshenziswa ukuhlinzeka ngokusebenza okuhambisanayo kwemibuzo edinga izinsiza. Umsebenzi (“i-topology”) ku-Storm umsebenzi osatshalaliswa kuzo zonke izindawo olinda ukuthi imilayezo ifike edinga ukucutshungulwa. Ngemva kokuthola umlayezo, umsebenzi uwucubungula kumongo wendawo bese ubuyisela umphumela. Isibonelo sokusebenzisa i-RPC esabalalisiwe kungaba ukucubungula imibuzo yosesho ngokuhambisana noma ukwenza imisebenzi kusethi enkulu yamasethi.
Izici Zesiphepho:
- Imodeli yokuhlela elula eyenza kube lula kakhulu ukucubungula idatha ngesikhathi sangempela;
- Ukusekela kwanoma yiziphi izilimi zokuhlela. Amamojula ayatholakala kuJava, Ruby kanye nePython, ukuzivumelanisa nezinye izilimi kulula ngenxa yephrothokholi yokuxhumana elula kakhulu edinga imigqa yekhodi eyi-100 ukuyisekela;
- Ukubekezelela iphutha: ukwenza umsebenzi wokucubungula idatha, udinga ukukhiqiza ifayela lembiza ngekhodi. I-Storm izosabalalisa ngokuzimela leli fayela lembiza kuwo wonke amanodi eqoqo, ixhume izibambi ezihlotshaniswa nayo, futhi ihlele ukuqapha. Lapho umsebenzi usuqedile, ikhodi izokhutshazwa ngokuzenzakalelayo kuwo wonke ama-node;
- I-scalability evundlile. Zonke izibalo zenziwa ngemodi efanayo; njengoba umthwalo ukhula, kwanele ukumane uxhume ama-node amasha kuqoqo;
- Ukwethembeka. I-Storm iqinisekisa ukuthi wonke umlayezo ongenayo ucutshungulwa ngokugcwele okungenani kanye. Umlayezo uzocutshungulwa kanye kuphela uma kungekho amaphutha uma udlula kuzo zonke izibambi; uma izinkinga zivela, imizamo yokucubungula engaphumelelanga izophindwa.
- Isivinini. Ikhodi ye-Storm ibhalwe ngokusebenza okuphezulu engqondweni futhi isebenzisa isistimu ukuthumela imiyalezo esheshayo .
Source: opennet.ru