Kwiinyanga ezidlulileyo malunga neenyanga ezintandathu ndidala inkqubo yokulwa nobuqhetseba (ubuqhetseba, ubuqhetseba, njl.) Iimbono zanamhlanje esizifumeneyo kwaye sazizalisekisa kwinkqubo yethu zisinceda sibhaqe kwaye sihlalutye imisebenzi emininzi yobuqhophololo. Kule nqaku, ndingathanda ukuthetha ngemigaqo esiyilandelayo kunye nento esiyenzayo ukufezekisa imeko yangoku yenkqubo yethu, ngaphandle kokungena kwicandelo lobugcisa.
Imigaqo yenkqubo yethu
Xa usiva amagama afana ne "automatic" kunye "nobuqhophololo," mhlawumbi uqala ukucinga ngokufunda koomatshini, i-Apache Spark, i-Hadoop, i-Python, i-Airflow, kunye nobunye ubuchwepheshe obuvela kwi-Apache Foundation ecosystem kunye ne-Data Science field. Ndicinga ukuba kukho inkalo enye yokusebenzisa ezi zixhobo edla ngokungakhankanywa: zifuna iimfuneko ezithile kwinkqubo yeshishini lakho ngaphambi kokuba uqalise ukuzisebenzisa. Ngamafutshane, udinga iqonga ledatha yeshishini elibandakanya ichibi ledatha kunye nendawo yokugcina impahla. Kodwa kuthekani ukuba awunalo iqonga elinjalo kwaye kusafuneka uphuhlise lo mkhuba? Le migaqo ilandelayo endibelana ngayo ngezantsi isincede ukuba sifikelele kwinqanaba apho sinokugxila ekuphuculeni iingcamango zethu kunokuba sifumane enye esebenzayo. Nangona kunjalo, oku akusiyo iprojekthi yeplateau. Kusekho izinto ezininzi kwisicwangciso ukusuka kumbono wezobuchwepheshe kunye nemveliso.
Umgaqo 1: Ixabiso leShishini kuqala
Sibeka "ixabiso leshishini" phambili kuyo yonke imizamo yethu. Ngokubanzi, nayiphi na inkqubo yokuhlalutya ngokuzenzekelayo yeqela leenkqubo eziyinkimbinkimbi ezinomgangatho ophezulu wokuzenzekelayo kunye nobunzima bobugcisa. Ukudala isisombululo esipheleleyo kuya kuthatha ixesha elininzi ukuba udala ukusuka ekuqaleni. Sigqibe kwelokuba sibeke ixabiso leshishini kuqala kunye nokugqibelela kobuchwepheshe okwesibini. Kubomi bokwenyani, oku kuthetha ukuba asibamkeli ubugcisa obuphambili njengemfundiso. Sikhetha itekhnoloji esebenza kakuhle kuthi okwangoku. Ngokuhamba kwexesha, kunokubonakala ngathi kuya kufuneka siphinde siphumeze ezinye iimodyuli. Olu lulungelelwano esamkeleyo.
Umgaqo 2: Ubukrelekrele obandisiweyo
Ndibheja uninzi lwabantu abangabandakanyekanga nzulu ekuphuhliseni izisombululo zokufunda ngomatshini banokucinga ukuba ukutshintsha abantu yinjongo. Ngapha koko, izisombululo zokufunda ngoomatshini azigqibelelanga kwaye ziindawo ezithile kuphela ezinokutshintshwa. Sayikhaba le ngcamango ukususela ekuqaleni ngenxa yezizathu ezininzi: idatha engalinganiyo kumsebenzi wobuqhophololo kunye nokungakwazi ukubonelela uluhlu olubanzi lweempawu kwiimodeli zokufunda ngomatshini. Ngokwahlukileyo koko, sikhethe ukhetho olongeziweyo lobukrelekrele. Le yenye ingqikelelo yobukrelekrele bokwenziwa ejolise kwindima exhasayo ye-AI, igxininisa into yokuba ubugcisa bokuqonda bunenjongo yokuphucula ubukrelekrele bomntu endaweni yokububuyisela endaweni yabo. [1]
Ngenxa yoku, ukuphuhlisa isisombululo sokufunda koomatshini esipheleleyo kwasekuqaleni kuya kufuna umgudu omkhulu, oya kulibazisa ukuyilwa kwexabiso leshishini lethu. Sigqibe ekubeni sakhe inkqubo enenkalo yokufunda koomatshini ekhula ngokuphindaphindiweyo phantsi kwesikhokelo seengcali zethu zesizinda. Inxalenye engumceli mngeni yokuphuhlisa inkqubo enjalo kukuba kufuneka ibonelele abahlalutyi bethu ngamatyala kungekhona nje kuphela malunga nokuba ngumsebenzi wobuqhophololo okanye cha. Ngokubanzi, nayiphi na i-anomaly ekuziphatheni kwabathengi yimeko ekrokrisayo ukuba iingcali kufuneka ziphonononge kwaye ziphendule ngandlela thile. Liqhezu kuphela la matyala axeliweyo anokubekwa ngokwenyani njengorhwaphilizo.
Umgaqo 3: I-Platform ye-Analytics ecebileyo
Elona candelo licela umngeni kwinkqubo yethu kukuqinisekisa ukuphela ukuya ekupheleni kokuhamba komsebenzi. Abahlalutyi kunye nabaphuhlisi kufuneka bafumane ngokulula iiseti zedatha yembali nazo zonke iimethrikhi ezisetyenziselwa uhlalutyo. Ukongezelela, iqonga ledatha kufuneka libonelele ngendlela elula yokuncedisa iseti esele ikhona yeemethrikhi ezintsha. Iinkqubo esizenzayo, kwaye ezi ayizonkqubo zesoftware kuphela, kufuneka zisivumele ukuba sibale kwakhona amaxesha angaphambili ngokulula, songeze iimetriki ezintsha kwaye sitshintshe uqikelelo lwedatha. Sinokukufezekisa oku ngokuqokelela zonke iinkcukacha eziveliswa yinkqubo yethu yokuvelisa. Kule meko, idatha iya kuba yinkathazo ngokuthe ngcembe. Kuya kufuneka sigcine inani elikhulayo ledatha esingayisebenzisi kwaye siyikhusele. Kwimeko enjalo, idatha iya kuba ngakumbi kwaye ingabinamsebenzi ekuhambeni kwexesha, kodwa isafuna iinzame zethu zokuyilawula. Kithina, ukugcinwa kwedatha kwakungekho ngqiqweni, ngoko sagqiba ekubeni sithathe indlela eyahlukileyo. Sigqibe kwelokuba siququzelele iivenkile zedatha yexesha lokwenyani malunga nezinto ekujoliswe kuzo esifuna ukuzihlela, kwaye sigcine kuphela idatha esivumela ukuba sijonge awona maxesha amva nje nafanelekileyo. Umceli mngeni kule nzame kukuba inkqubo yethu iyahluka, ineendawo ezininzi zokugcina idatha kunye neemodyuli zesoftware ezifuna ukucwangciswa ngononophelo ukuze kusebenze ngendlela engaguqukiyo.
Iikhonsepthi zoyilo lwenkqubo yethu
Sinamacandelo amane aphambili kwinkqubo yethu: inkqubo yokungenisa, ukubala, uhlalutyo lwe-BI kunye nenkqubo yokulandelela. Basebenzela iinjongo ezithile, ezizimeleyo, kwaye sibagcina bebodwa ngokulandela iindlela ezithile zoyilo.
Uyilo olusekwe kwikhontrakthi
Okokuqala, sivumelene ukuba amacandelo kufuneka athembele kuphela kulwakhiwo lwedatha ethile (iikhontrakthi) ezigqithiswe phakathi kwazo. Oku kwenza kube lula ukudibanisa phakathi kwabo kwaye unganyanzelisi ukubunjwa okukhethekileyo (kunye nomyalelo) wamacandelo. Ngokomzekelo, kwezinye iimeko oku kusivumela ukuba sidibanise ngokuthe ngqo inkqubo yokuthatha kunye nenkqubo yokulandelela isilumkiso. Kwimeko enjalo, oku kuya kwenziwa ngokuhambelana nesivumelwano esivunyiweyo esivunyelweneyo. Oku kuthetha ukuba omabini amacandelo aya kudityaniswa kusetyenziswa ikhontrakthi enokuthi isetyenziswe naliphi na elinye icandelo. Asiyi kuba songeza ikhontrakthi eyongezelelweyo ukongeza izilumkiso kwisistim yokulandelela ukusuka kwinkqubo yokufaka. Le ndlela ifuna ukusetyenziswa kwenani eliqingqiweyo elimiselweyo leekhontrakthi kwaye yenza lula inkqubo kunye nonxibelelwano. Ngokusisiseko, sithatha indlela ebizwa ngokuba yi "Contract First Design" kwaye siyisebenzise kwiikontraka zokusasaza. [2]
Ukusasaza kuyo yonke indawo
Ukugcina nokulawula urhulumente kwinkqubo ngokuqinisekileyo kuya kukhokelela kwiingxaki ekuphunyezweni kwayo. Ngokubanzi, urhulumente kufuneka afikeleleke kulo naliphi na icandelo, kufuneka angqinelane kwaye anike elona xabiso langoku kuwo onke amacandelo, kwaye kufuneka athembeke ngamaxabiso achanekileyo. Ukongezelela, ukuba neefowuni zokugcina ukugcinwa ngokuqhubekayo ukubuyisela imeko yamva nje kuya kwandisa inani lemisebenzi ye-I / O kunye nobunzima be-algorithms esetyenziswa kwimibhobho yethu yexesha langempela. Ngenxa yoku, sigqibe kwelokuba sisuse ugcino lukarhulumente, ukuba kunokwenzeka, ngokupheleleyo kwinkqubo yethu. Le ndlela idinga ukuba yonke idatha eyimfuneko ifakwe kwibhloko yedatha edlulisiweyo (umyalezo). Umzekelo, ukuba sifuna ukubala inani elipheleleyo leminye imigqaliselo (inani lemisebenzi okanye iimeko ezineempawu ezithile), sibala kwimemori kwaye sivelise umsinga waloo maxabiso. Iimodyuli ezixhomekeke kuzo ziya kusebenzisa ulwahlulo kunye nokudibanisa ukwahlula umlambo ube ngamaqumrhu kwaye usebenze kumaxabiso amva nje. Le ndlela yaphelisa imfuneko yokugcinwa kwedisk eqhubekayo kwidatha enjalo. Inkqubo yethu isebenzisa i-Kafka njengomthengisi womyalezo kwaye ingasetyenziswa njengendawo yogcino-lwazi nge-KSQL. [3] Kodwa ukuyisebenzisa bekuya kubopha isisombululo sethu kakhulu kwiKafka, kwaye sagqiba ekubeni singayisebenzisi. Indlela esiyikhethileyo ivumela ukuba sitshintshe iKafka ngomnye umthengisi womyalezo ngaphandle kotshintsho olukhulu lwangaphakathi kwinkqubo.
Le ngcamango ayithethi ukuba asisebenzisi ukugcinwa kwediski kunye nedatha. Ukuvavanya kunye nokuhlalutya ukusebenza kwenkqubo, kufuneka sigcine isixa esibalulekileyo sedatha kwidiski emele iimethrikhi ezahlukeneyo kunye namazwe. Inqaku elibalulekileyo apha kukuba i-algorithms yexesha langempela ayixhomekeke kwidatha enjalo. Kwiimeko ezininzi, sisebenzisa idatha egciniweyo ukuhlalutya ngaphandle kwe-intanethi, ukulungisa iimpazamo kunye nokulandelwa kweemeko ezithile kunye neziphumo eziveliswa yinkqubo.
Iingxaki zesixokelelwano sethu
Kukho iingxaki ezithile esizisombulule ukuya kwinqanaba elithile, kodwa zifuna izisombululo ezicingisisiweyo. Ngoku ndingathanda ukuzikhankanya apha kuba inqaku ngalinye lixabisa inqaku lalo.
- Kusafuneka sichaze iinkqubo kunye nemigaqo-nkqubo exhasa ukuqokelelwa kwedatha enentsingiselo nefanelekileyo ukulungiselela uhlalutyo lwedatha oluzenzekelayo, ukufunyanwa kunye nokuhlola.
- Ukubandakanywa kokuhlalutya komntu kuphumela kwinkqubo yokumisela ngokuzenzekelayo inkqubo ukuyihlaziya ngedatha yakutshanje. Oku akwanelanga ukuhlaziya imodeli yethu kuphela, kodwa nokuhlaziya iinkqubo zethu kunye nokuphucula ukuqonda kwethu idatha yethu.
- Ukufumana ibhalansi phakathi kwendlela yokumisela ye-IF-ELSE kunye ne-ML. Omnye umntu uthe, "ML sisixhobo sabasweleyo." Oku kuthetha ukuba uya kufuna ukusebenzisa i-ML xa ungasayiqondi indlela yokuphucula nokuphucula i-algorithms yakho. Ngakolunye uhlangothi, indlela yokumisela ayivumeli ukufunyanwa kwezinto ezingaqhelekanga ezazingacingelwanga.
- Sidinga indlela elula yokuvavanya iingcamango zethu okanye ulungelelwaniso phakathi kweemetrics kwidatha.
- Inkqubo kufuneka ibe namanqanaba amaninzi eziphumo ezilungileyo zokwenyani. Amatyala orhwaphilizo aliqhezwana lawo onke amatyala anokuthi athathwe njengelungileyo kwinkqubo. Ngokomzekelo, abahlalutyi bafuna ukufumana zonke iimeko ezikrokrelayo zokuqinisekisa, kwaye inxalenye encinci yazo yinkohliso. Le nkqubo kufuneka iveze zonke iimeko kubahlalutyi ngokufanelekileyo, nokuba bubuqhophololo bokwenene okanye ukuziphatha okukrokrisayo.
- Iqonga ledatha kufuneka likwazi ukubuyisela iiseti zedatha yembali kunye nezibalo ezenziweyo kwaye zibalwe kwi-fly.
- Ngokulula nangokuzenzekela naliphi na ilungu lenkqubo ubuncinci kwiindawo ezintathu ezahlukeneyo: imveliso, umfuniselo (beta) kunye nabaphuhlisi.
- Kwaye okokugqibela kodwa kungancinci. Kufuneka sakhe iqonga lokuvavanya ukusebenza okutyebileyo apho sinokuhlalutya iimodeli zethu. [4]
iimbekiselo
umthombo: www.habr.com