Apache Storm 2.0 yakagovera computing system iripo

Ndakaona chiedza kuburitswa kwakakosha kweiyo yakagoverwa chiitiko chekugadzirisa system Apache Dutu 2.0, inozivikanwa neshanduko yayo kuenda kuchivako chitsva chakaitwa muJava, pachinzvimbo chewakamboshandiswa mutauro weClojure.

Iyo purojekiti inokutendera iwe kuronga yakavimbiswa kugadziriswa kwezviitiko zvakasiyana munguva chaiyo. Semuenzaniso, Dutu rinogona kushandiswa kuongorora data hova munguva chaiyo, kumhanya muchina kudzidza mabasa, kuronga inoenderera komputa, kuita RPC, ETL, nezvimwe. Iyo sisitimu inotsigira kubatanidza, kusikwa kwekukanganisa-kushivirira zvigadziriso, yakavimbiswa data process mode uye ine yakakwirira kuita, inokwana kugadzirisa zvinopfuura miriyoni zvikumbiro pasekondi pane imwe cluster node.

Kubatanidzwa neakasiyana siyana ekugadzirisa mitsara masisitimu uye dhatabhesi tekinoroji inotsigirwa. Iyo Storm architecture inosanganisira kugamuchira uye kugadzirisa isina kurongeka, inogara ichigadziridzwa data hova uchishandisa anopokana akaoma processors ane kugona kugovera pakati pematanho akasiyana ekuverenga. Iyo purojekiti yakapihwa kunharaunda yeApache mushure mekunge Twitter yawana BackType, iyo kambani yakatanga kugadzira chimiro. Mukuita, Storm yakashandiswa muBackType kuongorora kuratidzwa kwezviitiko mumamicroblogs, nekuenzanisa pane inobhururuka matweets matsva uye matani anoshandiswa mazviri (semuenzaniso, yakaongororwa kuti zvekunze zvinongedzo kana zviziviso zvakaburitswa paTwitter zvakaburitswazve nevamwe vatori vechikamu. )

Kushanda kweStorm kunofananidzwa neHadoop chikuva, nemusiyano wakakosha uri wekuti data harina kuchengetwa mudura, asi rinopinzwa kunze uye rinogadziriswa munguva chaiyo. Dutu harina yakavakirwa-mukati yekuchengetera layer uye mubvunzo wekuongorora unotanga kushandiswa kune inouya data kudzamara yadzimwa (apo Hadoop ichishandisa magumo-nguva MapReduce mabasa, Storm inoshandisa pfungwa yekuenderera mberi ichimhanya "topology"). Kuitwa kwevanobata kunogona kugoverwa kune akati wandei maseva - Dutu rinofananidza basa netambo pane akasiyana masumbu.

Iyo sisitimu yakatanga kunyorwa muClojure uye inomhanya mukati meJVM chaiyo muchina. Iyo Apache Foundation yakatanga danho rekutamisa Dutu kune nyowani kernel yakanyorwa muJava, mhedzisiro yacho inotsanangurwa mukuburitswa kweApache Storm 2.0. Zvese zvakakosha zvikamu zvepuratifomu zvinonyorwa patsva muJava. Tsigiro yekunyora vanobata muClojure yakachengetwa, asi ikozvino yakapihwa nenzira yekusungira. Dutu 2.0.0 inoda Java 8. Iyo yakawanda-threaded yekugadzirisa modhi yakagadziridzwa zvachose, ichibvumira. achieve kuwedzera kunoonekwa mukuita (kune mamwe matopology, kunonoka kwakaderedzwa ne50-80%).

Iyo vhezheni nyowani inopawo nyowani yakanyorwa Streams API iyo inokutendera kuti utsanangure vanobata uchishandisa inoshanda hurongwa-maitiro maitiro. Iyo API nyowani inoiswa pamusoro peiyo yakajairwa base API uye inotsigira otomatiki kusanganisa kwemashandiro kukwenenzvera magadzirirwo awo. Iyo Windowing API yekushanda kwehwindo yakawedzera rutsigiro rwekuchengetedza uye kudzoreredza mamiriro kumashure.

Tsigiro yekufunga nezve zvimwe zviwanikwa pakuita sarudzo dzisina kuganhurirwa
CPU uye ndangariro, senge network uye GPU marongero. Nhamba huru yekuvandudzwa kwakaitwa kuti kuve nechokwadi chekubatanidzwa nepuratifomu Kafka. Iyo yekupinda yekudzora sisitimu yakawedzerwa kuti ibatanidze kugona kugadzira vatariri mapoka uye kugovera tokeni. Yakawedzera kuvandudzwa kwakabatana neSQL uye metrics rutsigiro. Mitemo mitsva yaonekwa mune yemutungamiriri interface yekugadzirisa iyo cluster state.

Nzvimbo dzekunyorera Storm:

• Kugadzirisa nzizi dze data nyowani kana dhatabhesi zvigadziriso munguva chaiyo;
• Inoenderera Komputa: Dutu rinogona kumhanyisa mibvunzo uye kugadzirisa hova dzinoramba dzichienderera, ichiendesa mhinduro kumutengi munguva chaiyo.
• Distributed Remote Procedure Call (RPC): Dutu rinogona kushandiswa kupa kuenzanirana kwekuita kwemibvunzo-yakanyanya-kubvunza. Basa ("topology") muStorm ibasa rakagoverwa munzvimbo dzese rinomirira kuti meseji isvike inoda kugadziriswa. Mushure mekugamuchira meseji, basa rinorigadzirisa mumamiriro enzvimbo uye rinodzosera mhedzisiro. Muenzaniso wekushandisa RPC yakagoverwa kungave kugadzirisa mibvunzo yekutsvaga nenzira yakafanana kana kuita mashandiro pane yakakura seti.

Storm Features:

• A simple programming model iyo inorerutsa zvakanyanya-chaiyo-nguva data processing;
• Kutsigira chero mitauro yepurogiramu. Mamodule anowanikwa kuJava, Ruby uye Python, kuchinjika kune mimwe mitauro kuri nyore nekuda kweiyo yakapusa kutaurirana protocol inoda anenge zana mitsara yekodhi kutsigira;
• Kukanganisa kushivirira: kuti uite basa rekugadzirisa data, unofanirwa kugadzira faira rejagi nekodhi. Storm inogovera yakazvimirira iyi faira rejagi mhiri kwemasumbu masumbu, batanidza vanobata vanobatana nayo, uye kuronga kutarisa. Kana basa racho rapera, iyo kodhi ichadzimwa otomatiki pane ese node;
• Horizontal scalability. Ese maverengero anoitwa muparallel modhi; sezvo mutoro unowedzera, zvakakwana kungobatanidza node nyowani kune cluster;
• Kuvimbika. Storm inova nechokwadi chekuti meseji yese inouya inogadziriswa zvizere kamwechete. Iyo meseji ichagadziriswa kamwe chete kana pasina zvikanganiso kana uchipfuura nevose vanobata; kana matambudziko amuka, zvino kusabudirira kugadzirisa kuedza kunodzokororwa.
• Speed. Dutu kodhi yakanyorwa nekuita kwepamusoro mupfungwa uye inoshandisa sisitimu yekukurumidza asynchronous messaging ZeroMQ.

Source: opennet.ru