Cishe ezinyangeni eziyisithupha ezedlule, bengidala uhlelo lokulwa nokukhwabanisa (ukukhwabanisa, ukukhwabanisa, njll.) ngaphandle kwengqalasizinda yokuqala yalokhu. Imibono yanamuhla esiyitholile futhi sayisebenzisa ohlelweni lwethu isisiza ukuthi sithole futhi sihlaziye imisebenzi eminingi yokukhwabanisa. Kulesi sihloko, ngithanda ukukhuluma ngezimiso esizilandele kanye nalokho esikwenzile ukuze sifinyelele isimo samanje sesistimu yethu, ngaphandle kokungena engxenyeni yezobuchwepheshe.
Izimiso zesistimu yethu
Uma uzwa amagama anjengokuthi "okuzenzakalelayo" kanye "nokukhwabanisa" cishe uqala ukucabanga ngokufunda komshini, i-Apache Spark, i-Hadoop, i-Python, i-Airflow, nobunye ubuchwepheshe ku-Apache Foundation ecosystem kanye nenkambu Yesayensi Yedatha. Ngicabanga ukuthi kunesici esisodwa sokusebenzisa lawa mathuluzi esingavamisile ukushiwo: adinga izimfuneko ezithile ukuze zibe khona ohlelweni lwakho lwebhizinisi ngaphambi kokuthi uwasebenzise. Ngamafuphi, udinga inkundla yedatha yebhizinisi ehlanganisa ichibi ledatha nesitoreji. Kodwa kuthiwani uma ungenayo inkundla enjalo futhi usadinga ukuthuthukisa lo mkhuba? Izimiso ezilandelayo, engizichaza ngezansi, zisisize ukuba sifinyelele lapho singagxila khona ekuthuthukiseni imibono yethu, kunokuba sithole esebenzayo. Nokho, leli akulona "ithafa" lephrojekthi. Kukhona ezinye izinto eziningi ohlelweni ngokubuka kwezobuchwepheshe kanye nomkhiqizo.
Isimiso 1: Inani Lebhizinisi Okokuqala
Sibeka βinani lebhizinisiβ phambili kuyo yonke imizamo yethu. Ngokuvamile, noma iyiphi isistimu yokuhlaziya okuzenzakalelayo ingeyeqembu lezinhlelo eziyinkimbinkimbi ezinezinga eliphezulu lokuzenzakalela kanye nobunzima bezobuchwepheshe. Ukudala isisombululo esiphelele kuzothatha isikhathi esiningi uma usidala kusukela ekuqaleni. Sinqume ukubeka inani lebhizinisi kuqala kanye nokuvuthwa kwezobuchwepheshe okwesibili. Empilweni yangempela, lokhu kusho ukuthi asibamukeli ubuchwepheshe obuthuthukisiwe njengemfundiso-nkolo. Sikhetha ubuchwepheshe obusisebenzela kangcono njengamanje. Ngokuhamba kwesikhathi, kungase kubonakale sengathi kuzodingeka siphinde sisebenzise amanye amamojula. Lokhu ukuvumelana esamukele.
Isimiso 2: Ukuhlakanipha okungeziwe
Ngibheja abantu abaningi abangabandakanyi ngokujulile ekuthuthukiseni izixazululo zokufunda ngomshini bangase bacabange ukuthi ukushintshwa komuntu kuwumgomo. Eqinisweni, izixazululo zokufunda ngomshini aziphelele futhi ezindaweni ezithile kuphela lapho kungashintshwa khona. Sishiye lo mbono kusukela ekuqaleni ngenxa yezizathu ezimbalwa: idatha engalingani ngomsebenzi wokukhwabanisa kanye nokungakwazi ukunikeza uhlu oluphelele lwezici zamamodeli okufunda omshini. Ngokuphambene, sikhethe inketho ye-augmented intelligence. Lona omunye umqondo wobuhlakani bokwenziwa ogxile endimeni esekelayo ye-AI, egcizelela iqiniso lokuthi ubuchwepheshe bengqondo buklanyelwe ukuthuthukisa ubuhlakani bomuntu, hhayi obumiselele. [1]
Unalokhu engqondweni, ukwakha isixazululo esiphelele sokufunda umshini kusukela ekuqaleni kungadinga inani elikhulu lomzamo ongabambezela ukudalwa kwenani lebhizinisi lethu. Sinqume ukwakha isistimu enesici esikhula ngokuphindaphindiwe sokufunda komshini ngaphansi kokuqondisa kochwepheshe bethu besizinda. Ingxenye ekhohlisayo yokuthuthukisa uhlelo olunjalo ukuthi kufanele lunikeze abahlaziyi bethu ngezibonelo zezifundo hhayi kuphela mayelana nokuthi lokhu kuwumsebenzi wokukhwabanisa noma cha. Ngokuvamile, noma yikuphi ukuphazamiseka ekuziphatheni kwamakhasimende kuyindaba esolisayo ochwepheshe okudingeka bayiphenye futhi ngandlela thize baphendule. Ambalwa kuphela kulawa macala arekhodiwe angahlukaniswa njengokukhwabanisa.
Isimiso sesi-3: I-Rich Insights Platform
Ingxenye enzima kakhulu yesistimu yethu ukuqinisekiswa kokuphela kuye ekupheleni kokusebenza kwesistimu. Abahlaziyi nonjiniyela kufanele bathole kalula amasethi edatha omlando anawo wonke ama-metrics asetshenziswe ekuhlaziyeni. Ukwengeza, inkundla yedatha kufanele inikeze indlela elula yokwengeza isethi ekhona yezinkomba entsha. Izinqubo esizenzayo, futhi lezi akuzona nje izinqubo zesofthiwe, kufanele zenze kube lula ukubala kabusha izikhathi zangaphambilini, sengeze amamethrikhi amasha futhi siguqule isibikezelo sedatha. Lokhu singakufeza ngokuqongelela yonke idatha ekhiqizwa isistimu yethu yokukhiqiza. Esimeni esinjalo, idatha kancane kancane izoba yisithiyo. Kuzodingeka sigcine inani elikhulayo ledatha esingayisebenzisi futhi siyivikele. Esimeni esinjalo, idatha izoba ingasasebenzi ngokuhamba kwesikhathi, kodwa isadinga imizamo yethu yokuyilawula. Kithina, ukuqoqwa kwedatha akuzange kube nengqondo, futhi sanquma ukusebenzisa indlela ehlukile. Sinqume ukuhlela izinqolobane zedatha yesikhathi sangempela ezinkampanini eziqondiwe esifuna ukuzihlukanisa, futhi sigcine kuphela idatha esivumela ukuthi sihlole izikhathi zakamuva nezesikhathi samanje. Inselele ngalo mzamo ukuthi isistimu yethu ihlukile ngezitolo eziningi zedatha namamojula esofthiwe adinga ukuhlela ngokucophelela ukuze asebenze ngendlela engaguquki.
Dizayina imiqondo yesistimu yethu
Sinezingxenye ezine eziyinhloko kusistimu yethu: isistimu yokungenisa, isistimu yokubala, ukuhlaziywa kwe-BI, kanye nesistimu yokulandelela. Afeza izinjongo ezithile ezihlukene, futhi siwagcina ehlukanisiwe ngokulandela izindlela ezithile zokuthuthukisa.
Umklamo osuselwe kwinkontileka
Okokuqala, sivumelene ngokuthi izingxenye kufanele zithembele kuphela ezakhiweni ezithile zedatha (izinkontileka) eziphasiswa phakathi kwazo. Lokhu kwenza kube lula ukuhlanganisa phakathi kwazo futhi ungaphoqeleli ukwakheka okuthile (nokuhleleka) kwezingxenye. Isibonelo, kwezinye izimo lokhu kusivumela ukuthi sihlanganise ngokuqondile isistimu yokwamukela nesistimu yokulandelela izexwayiso. Esimeni esinjalo, lokhu kuzokwenziwa ngokuvumelana nenkontileka yesaziso okuvunyelwene ngayo. Lokhu kusho ukuthi zombili izingxenye zizohlanganiswa kusetshenziswa inkontileka noma iyiphi enye ingxenye engayisebenzisa. Ngeke singeze inkontileka eyengeziwe ukuze singeze izexwayiso ohlelweni lokulandela ngomkhondo kusuka kusistimu yokufaka. Le ndlela idinga ukusetshenziswa kwenani elincane elinqunywe kusengaphambili lezinkontileka futhi yenza uhlelo nokuxhumana lube lula. Empeleni, sithatha indlela ebizwa ngokuthi "Umklamo Wokuqala Wenkontileka" futhi siyisebenzise kuzinkontileka zokusakaza bukhoma. [2]
Ukusakaza yonke indawo
Ukonga nokuphatha umbuso ohlelweni nakanjani kuzoholela ezinkingeni ekusetshenzisweni kwawo. Ngokuvamile, umbuso kufanele ufinyeleleke kunoma iyiphi ingxenye, kufanele uhambisane futhi unikeze inani lakamuva kakhulu kuzo zonke izingxenye, futhi kufanele uthembeke ngamavelu alungile. Ngaphezu kwalokho, ukuba nezingcingo eziya kusitoreji esiqhubekayo ukuze uthole isimo sakamuva kuzokhuphula inani le-I/O nobunkimbinkimbi be-algorithms esetshenziswa emigqeni yethu yesikhathi sangempela. Ngenxa yalokhu, sinqume ukususa indawo yokugcina indawo, uma kungenzeka, ngokuphelele ohlelweni lwethu. Le ndlela idinga ukuthi yonke imininingwane edingekayo ifakwe kuyunithi yedatha edluliselwayo (umlayezo). Isibonelo, uma sidinga ukubala isamba senani lokunye ukuqaphela (inani lemisebenzi noma izimo ezinezici ezithile), sibala ngenkumbulo futhi sikhiqize ukusakazwa kwamanani anjalo. Amamojula ancikile azosebenzisa ukuhlukanisa nokuhlanganisa ukuze ahlukanise ukusakaza ngamabhizinisi futhi asebenze ngamavelu akamuva. Le ndlela yaqeda isidingo sokuba nesitoreji esiqhubekayo sediski kudatha enjalo. Isistimu yethu isebenzisa i-Kafka njengomthengisi wemilayezo futhi ingasetshenziswa njengendawo egciniwe ene-KSQL. [3] Kodwa ukuyisebenzisa kuzobopha kakhulu isisombululo sethu ku-Kafka, futhi sanquma ukungasisebenzisi. Indlela esiyikhethile isivumela ukuthi simisele i-Kafka omunye umthengisi womlayezo ngaphandle kwezinguquko ezinkulu zangaphakathi ohlelweni.
Lo mqondo awusho ukuthi asisebenzisi isitoreji sediski kanye nemininingwane yolwazi. Ukuze sihlole futhi sihlaziye ukusebenza kwesistimu, sidinga ukugcina inani elibalulekile ledatha kudiski, elimele izinkomba nezifunda ezihlukahlukene. Iphuzu elibalulekile lapha ukuthi ama-algorithms wesikhathi sangempela awancikile kudatha enjalo. Ezimweni eziningi, sisebenzisa idatha elondoloziwe ukuze sihlaziye ungaxhunyiwe ku-inthanethi, silungise iphutha, futhi silandelele izimo ezithile kanye nemiphumela ekhiqizwa isistimu.
Izinkinga ohlelweni lwethu
Kunezinkinga ezithile esizixazulule zafika ezingeni elithile, kodwa zidinga izixazululo ezicatshangelwayo. Okwamanje, ngithanda ukubalula lapha, ngoba into ngayinye ifanele isihloko sayo.
- Kusadingeka sichaze izinqubo nezinqubomgomo ezisiza ukukhiqiza idatha enengqondo nefanelekile yokuhlaziya kwethu okuzenzakalelayo, ukutholwa nokuhlola idatha.
- Ukwethulwa kwemiphumela yokuhlaziywa komuntu ohlelweni lokushuna ngokuzenzakalelayo uhlelo ukuze lubuyekeze ngedatha yakamuva. Lokhu akusona nje isibuyekezo kumodeli yethu, kodwa futhi kuyisibuyekezo sezinqubo zethu nokuqonda kangcono idatha yethu.
- Ukuthola ibhalansi phakathi kwendlela yokunquma ye-IF-ELSE ne-ML. Othile uthe: "ML iyithuluzi labaphelelwe ithemba." Lokhu kusho ukuthi uzofuna ukusebenzisa i-ML lapho ungasayiqondi indlela yokuthuthukisa nokuthuthukisa ama-algorithms akho. Ngakolunye uhlangothi, indlela yokunquma ayikuvumeli ukutholwa kokudidayo obekungabonwanga kusengaphambili.
- Sidinga indlela elula yokuhlola imibono yethu noma ukuhlobana phakathi kwamamethrikhi kudatha.
- Uhlelo kumele lube namazinga amaningi emiphumela emihle yeqiniso. Amacala okukhwabanisa ayingxenyana kuphela yazo zonke izimo ezingathathwa njengezihle ohlelweni. Isibonelo, abahlaziyi bafuna ukuthola zonke izigameko ezisolisayo ukuze zibuyekezwe, futhi ingxenye encane kuphela yazo enokukhwabanisa. Uhlelo kumele luhlinzeke ngempumelelo abahlaziyi ngazo zonke izimo, noma ngabe ukukhwabanisa kwangempela noma ukuziphatha okusolisayo.
- Inkundla yedatha kufanele ikwazi ukubuyisa amasethi edatha omlando anezibalo ezidaliwe futhi ezibalwe ngokuphazima kweso.
- Ukuthunyelwa okulula nokuzenzakalelayo kwanoma yiziphi izingxenye zesistimu okungenani ezindaweni ezintathu ezihlukene: ukukhiqiza, ukuhlola (i-beta), kanye nokonjiniyela.
- Futhi okokugcina. Kudingeka sakhe inkundla yokulinganisa ebanzi lapho singahlaziya khona amamodeli ethu. [4]
izithenjwa
Source: www.habr.com