Kwemwedzi inenge mitanhatu yapfuura, ndanga ndichigadzira hurongwa hwekurwisa hunyengeri (chitsotsi, kubiridzira, nezvimwewo) pasina zvivakwa zvekutanga zveizvi. Mazano anhasi atakawana nekuita muhurongwa hwedu anotibatsira kuona nekuongorora zvinhu zvakawanda zvehutsotsi. Muchinyorwa chino, ndinoda kutaura nezve misimboti yatakatevera uye zvatakaita kuti tiwane mamiriro azvino ehurongwa hwedu, tisingatarise muchikamu chehunyanzvi.
Nheyo dzehurongwa hwedu
Paunonzwa mazwi akadai se "otomatiki" uye "hutsotsi" iwe unogona kutanga kufunga nezve muchina kudzidza, Apache Spark, Hadoop, Python, Airflow, uye humwe hunyanzvi muApache Foundation ecosystem uye Data Science field. Ini ndinofunga pane chimwe chikamu chekushandisa maturusi aya asingawanzo kutaurwa: anoda zvimwe zvinodikanwa kuti zvive munzvimbo pane yako bhizinesi system usati wazvishandisa. Muchidimbu, iwe unoda bhizinesi data chikuva chinosanganisira dziva re data uye kuchengetedza. Asi ko kana iwe usina chikuva chakadaro uye uchiri kuda kuvandudza tsika iyi? Nheyo dzinotevera, dzandinotsanangura pasi apa, dzakatibatsira kuti tisvike patinokwanisa kuisa pfungwa pakuvandudza mazano edu, pane kutsvaga kushanda. Zvisinei, iyi haisi "plateau" yeprojekti. Kune zvakawanda zvimwe zvinhu muchirongwa kubva kune tekinoroji uye zvigadzirwa zvemaonero.
Nheyo yekutanga: Kukosha kweBhizinesi Kutanga
Isu tinoisa "bhizinesi kukosha" pamberi pese kuedza kwedu. Kazhinji, chero otomatiki yekuongorora sisitimu ndeyeboka reakaomesesa masisitimu ane yakakwirira mwero we otomatiki uye hunyanzvi kuoma. Kugadzira mhinduro yakakwana inotora nguva yakawanda kana iwe ukaigadzira kubva pakutanga. Takasarudza kuisa kukosha kwebhizinesi kutanga uye kukura kwetekinoroji kechipiri. Muhupenyu chaihwo, izvi zvinoreva kuti hatibvume tekinoroji yepamusoro sedzidziso. Isu tinosarudza tekinoroji inoshanda zvakatinakira panguva ino. Nekufamba kwenguva, zvingaite sekuti tichafanirwa kuitazve mamwe mamodule. Uku ndiko kubvumirana kwatakagamuchira.
Nheyo 2: Kuwedzera njere
Ini ndinobheja vanhu vazhinji vasina kubatanidzwa zvakanyanya mukugadzira michina yekudzidza mhinduro vanogona kufunga kuti kutsiva kwevanhu ndicho chinangwa. Muchokwadi, mhinduro yemuchina iri kure nekukwana uye mune dzimwe nzvimbo chete ndidzo dzinogoneka. Isu takasiya pfungwa iyi kubva pakutanga nekuda kwezvikonzero zvakati: data isina kuenzana pachiitiko chehutsotsi uye kusakwanisa kupa runyoro rwakakwana rwezvimiro zvemhando dzekudzidza dzemuchina. Kusiyana neizvi, isu takasarudza iyo yakawedzerwa njere sarudzo. Iyi ndiyo imwe pfungwa yehungwaru hwekugadzira inotarisana nebasa rekutsigira reAI, ichisimbisa chokwadi chekuti matekinoroji ekuziva akagadzirirwa kuwedzera hungwaru hwevanhu, kwete kuitsiva. [1]
Tichifunga izvi, kugadzira mhinduro yemuchina wakazara kubva pakutanga kunoda huwandu hukuru hwekuedza kunonotsa kuumba kukosha kwebhizinesi redu. Isu takasarudza kuvaka sisitimu ine inoramba ichikura chikamu chekudzidza muchina tichitungamirwa nenyanzvi dzedu dzedunhu. Chikamu chinonyengera chekugadzira hurongwa hwakadaro ndechekuti inofanirwa kupa vaongorori vedu zvidzidzo zvenyaya kwete chete maererano nekuti iyi ibasa rekubiridzira kana kwete. Kazhinji, chero kusanzwisisika mune maitiro evatengi inyaya inofungidzira iyo nyanzvi dzinofanirwa kuongorora uye neimwe nzira kupindura. Chete shoma dzenyaya idzi dzakarekodhwa dzinogona kunyatso kurongeka sehutsotsi.
Nheyo 3: Yakapfuma Insights Platform
Chikamu chakaomesesa chehurongwa hwedu ndechekupera-kusvika-kumagumo kweiyo masisitimu mafambiro ekushanda. Vaongorori nevagadziri vanofanirwa kuwana zviri nyore dhatabheti dzenhoroondo nemametric ese akashandiswa pakuongorora. Mukuwedzera, iyo data platform inofanira kupa nzira iri nyore yekuwedzera iyo iripo seti yezviratidzo neitsva. Maitiro atinogadzira, uye aya haangori maitiro esoftware, anofanirwa kuita kuti zvive nyore kuverenga zvakare nguva dzakapfuura, kuwedzera metrics matsva uye kushandura fungidziro yedata. Isu tinogona kuita izvi nekuunganidza data rese rinogadzirwa nehurongwa hwedu hwekugadzira. Mumamiriro ezvinhu akadaro, data yaizove zvishoma nezvishoma inova chipingamupinyi. Tinoda kuchengetedza huwandu huri kukura hwe data isu hatishandise uye nekuichengetedza. Mumamiriro ezvinhu akadai, iyo data inozowedzera kusakosha nekufamba kwenguva, asi zvichiri kuda kuedza kwedu kuibata. Kwatiri, kuchengetwa kwedata hakuna kuita zvine musoro, uye takasarudza kushandisa imwe nzira. Isu takasarudza kuronga chaiyo-nguva matura data kutenderedza masangano atinoda kuisa, uye kuchengeta chete iyo data inotibvumira kutarisa nguva dzazvino uye dzemazuva ano. Dambudziko nekuedza uku nderekuti sisitimu yedu ine heterogeneous ine akawanda data zvitoro uye software module inoda kunyatsoronga kushanda nenzira inowirirana.
Dhizaini pfungwa dzehurongwa hwedu
Isu tine zvinhu zvina zvakakosha muhurongwa hwedu: hurongwa hwekupinza, computational system, ongororo yeBI, uye yekutevera system. Vanoshandira zvinangwa zvakasarudzika, uye isu tinovachengeta vakaparadzaniswa nekutevera dzimwe nzira dzekusimudzira.
Contract based design
Chokutanga pane zvose, takabvumirana kuti zvikamu zvinongofanira kuvimba nezvimwe zvinyorwa zve data (zvibvumirano) zvinopfuudzwa pakati pavo. Izvi zvinoita kuti zvive nyore kubatanidza pakati pavo uye kwete kumanikidza chaiyo kuumbwa (uye kurongeka) kwezvikamu. Semuenzaniso, mune zvimwe zviitiko izvi zvinotibvumira kuti tibatanidze zvakananga hurongwa hwekugamuchira nealert tracking system. Mumamiriro ezvinhu akadaro, izvi zvichaitwa maererano nechibvumirano chekuzivisa chakabvumiranwa. Izvi zvinoreva kuti zvikamu zviviri izvi zvichabatanidzwa pachishandiswa kondirakiti iyo chero chimwe chikamu chinogona kushandisa. Isu hatisi kuwedzera imwe kondirakiti yekuwedzera yambiro kune yekutevera system kubva kune yekuisa system. Iyi nzira inoda kushandiswa kweiyo yakatemerwa nhamba shoma yezvibvumirano uye inorerutsa sisitimu nekutaurirana. Chaizvoizvo, isu tiri kutora nzira inonzi "Contract Yekutanga Dhizaini" uye tichiishandisa kune yekutepfenyura zvibvumirano. [2]
Kutenderera Kwese
Kuchengetedza uye kutonga nyika muhurongwa zvinozotungamira kune matambudziko mukuitwa kwayo. Kazhinji, iyo nyika inofanirwa kuwanikwa kubva kune chero chikamu, inofanirwa kuenderana uye ichipa iyo yakanyanya-kusvika-zuva kukosha kune ese mazvikamu, uye inofanirwa kuve yakavimbika neiyo chaiyo kukosha. Pamusoro pezvo, kuve nemafoni ekuenderera mberi kuchengetedza kuti uwane iyo yazvino mamiriro kuchawedzera huwandu hweI / O uye kuoma kweiyo algorithms inoshandiswa mumapaipi edu chaiwo-nguva. Nekuda kweizvi, takasarudza kubvisa kuchengetedza kwenyika, kana zvichibvira, zvachose kubva kuhurongwa hwedu. Iyi nzira inoda kuti data yese inodiwa ibatanidzwe mune inotumirwa data unit (message). Semuenzaniso, kana tichida kuverenga nhamba yese yezvimwe zvakacherechedzwa (nhamba yekushanda kana makesi ane humwe hunhu), tinoiverengera mundangariro uye tinoburitsa rwizi rwemhando dzakadaro. Dependent modules inoshandisa partitioning uye batching kupatsanura rukova nemasangano uye kushanda pane zvazvino hunhu. Iyi nzira yakabvisa kudiwa kwekuramba kuchengetwa kwedhisiki kune data rakadaro. Sisitimu yedu inoshandisa Kafka semeseji broker uye inogona kushandiswa sedhatabhesi neKSQL. [3] Asi kuishandisa kwaizosunga zvakasimba mhinduro yedu kuKafka, uye takasarudza kusaishandisa. Nzira yatakasarudza inotibvumira kutsiva Kafka neimwe meseji broker pasina shanduko huru yemukati kune system.
Iyi pfungwa hairevi kuti hatishandisi dhisiki yekuchengetedza uye dhatabhesi. Kuti titarise uye tiongorore mashandiro ehurongwa, tinoda kuchengetedza huwandu hwakakosha hwe data pa diski, iyo inomiririra zviratidzo zvakasiyana uye nyika. Chinhu chakakosha apa ndechekuti real-time algorithms haibvi pane data rakadaro. Kazhinji, isu tinoshandisa iyo data yakachengetwa yekuongorora pasina Indaneti, kugadzirisa, uye kuronda nyaya dzakanangana nemhedzisiro inoburitswa nehurongwa.
Matambudziko muhurongwa hwedu
Pane mamwe matambudziko atakagadzirisa kusvika padanho rakati, asi anoda mhinduro dzine mufungo. Parizvino, ndoda kuzvitaura pano, nekuti chinhu chimwe nechimwe chakakodzera chinyorwa chayo.
- Tichiri kuda kutsanangura maitirwo nematongerwo anobatsira kuburitsa data ine musoro uye yakakosha yekuongorora kwedu otomatiki, kuwanikwa uye kuongorora data.
- Kuunzwa kwemhedzisiro yekuongororwa nemunhu ari mukati mekugadzirisa otomatiki sisitimu kuti ivandudze nedata razvino. Uku hakusi kungovandudza kune modhi yedu, asiwo gadziriso kune maitiro edu uye nekunzwisisa kuri nani kwedata redu.
- Kuwana chiyero pakati peiyo deterministic maitiro eIF-ELSE neML. Mumwe munhu akati: "ML chishandiso chevanopererwa." Izvi zvinoreva kuti iwe uchada kushandisa ML kana iwe usingachanzwisise maitiro ekugadzirisa uye kugadzirisa ako algorithms. Nekune rumwe rutivi, iyo deterministic nzira haibvumiri kuonekwa kweanomalies asina kufanoona.
- Tinoda nzira iri nyore yekuyedza fungidziro dzedu kana kuwirirana pakati pemetrics mune data.
- Iyo sisitimu inofanirwa kuve neyakawanda nhanho yechokwadi yakanaka mhedzisiro. Mhosva dzehutsotsi ingori chidimbu chezviitiko zvese zvinogona kunzi zvakanaka kune system. Semuenzaniso, vaongorori vanoda kugashira nyaya dzese dzinofungidzirwa kuti dziongororwe, uye chikamu chidiki chete chadzo chine hunyengeri. Iyo sisitimu inofanirwa kupa vanoongorora zviitiko zvese, ingave hutsotsi chaihwo kana kungofungira maitiro.
- Iyo data platform inofanirwa kukwanisa kudzoreredza nhoroondo dataset ine maverengero akagadzirwa uye akaverengerwa panhunzi.
- Kuendesa kuri nyore uye otomatiki kwechero chikamu chesystem munzvimbo dzinosvika nhatu dzakasiyana: kugadzira, kuyedza (beta), uye kune vanogadzira.
- Uye pakupedzisira asi kwete zvishoma. Isu tinofanirwa kugadzira yakakura bhenji yekumisikidza chikuva patinogona kuongorora mamodheru edu. [4]
nezvakanyorwa
Source: www.habr.com