Ii-MLOps: I-DevOps kwihlabathi lokuFunda ngoomatshini

Ngo-2018, kwizangqa zobuchwephesha kunye neenkomfa zethematic ezinikezelwe kwi-AI, umbono we-MLOps wavela, othe wafumana indawo ngokukhawuleza kwishishini kwaye ngoku uphuhlisa njengesikhokelo esizimeleyo. Kwixesha elizayo, i-MLOps inokuba yenye yezona ndawo zifunwa kakhulu kwi-IT. Yintoni na kwaye yintoni edliwa ngayo, siyaqonda phantsi kokusikwa.

Yintoni iMLOps

I-MLOps (ukudibanisa iteknoloji kunye neenkqubo zokufunda koomatshini kunye neendlela zokuphunyezwa kweemodeli eziphuhlisiwe kwiinkqubo zoshishino) yindlela entsha yentsebenziswano phakathi kwabameli bezoshishino, izazinzulu, iimathematika, iingcali zokufunda ngomatshini kunye neenjineli ze-IT ekudalweni kweenkqubo zobukrelekrele bokwenziwa.

Ngamanye amazwi, yindlela yokuguqula iindlela zokufunda koomatshini kunye nobuchwepheshe zibe sisixhobo esiluncedo sokusombulula iingxaki zeshishini. 

Kufuneka kuqondwe ukuba ikhonkco lemveliso liqala ixesha elide ngaphambi kokuphuhliswa kwemodeli. Isinyathelo salo sokuqala kukuchaza injongo yeshishini, i-hypothesis malunga nexabiso elinokuthi likhutshwe kwidatha, kunye nembono yeshishini lokuyisebenzisa. 

Ingcamango ye-MLOps yavela njengomzekeliso kwingcamango ye-DevOps ngokumalunga neemodeli kunye nobuchwepheshe bokufunda koomatshini. I-DevOps yindlela yokuphuhlisa isoftware ekuvumela ukuba ukwandise isantya sotshintsho lomntu ngelixa ugcina ukuguquguquka kunye nokuthembeka usebenzisa iindlela ezininzi, kubandakanya uphuhliso oluqhubekayo, ukwahlulwa kwemisebenzi kwinani leenkonzo ezincinci ezizimeleyo, uvavanyo oluzenzekelayo kunye nokuhanjiswa kotshintsho lomntu ngamnye, kwihlabathi liphela. ukubeka iliso kwezempilo, inkqubo yokuphendula ngokukhawuleza kwiintsilelo ezifunyenweyo, njl. 

I-DevOps ichaze umjikelo wobomi besoftware, kwaye umbono weza kuluntu lwesoftware ukusebenzisa ubuchule obufanayo kwidatha enkulu. I-DataOps yinzame yokulungelelanisa kunye nokwandisa indlela yokusebenza, kuthathelwa ingqalelo izinto ezikhethekileyo zokugcina, ukuhambisa kunye nokucubungula inani elikhulu ledatha kwiiplatifti ezahlukeneyo kunye nokusebenzisana.
  
Ngokufika kobunzima obuthile obubalulekileyo beemodeli zokufunda koomatshini ezifakwe kwiinkqubo zoshishino lwamashishini, ukufana okuqinileyo phakathi komjikelo wobomi beemodeli zemathematika zokufunda koomatshini kunye nomjikelo wobomi besoftware kwaqatshelwa. Ukwahlukana kuphela kukuba i-algorithms yemodeli yenziwa ngokusebenzisa izixhobo zokufunda ngomatshini kunye neendlela. Ke ngoko, umbono uvele ngokwendalo wokusebenzisa kunye nokulungelelanisa iindlela esele zaziwa kuphuhliso lwesoftware yeemodeli zokufunda koomatshini. Ke, ezi zigaba zilandelayo ziphambili zinokwahlulwa kumjikelo wobomi beemodeli zokufunda koomatshini:

• ukuchaza umbono weshishini;
• uqeqesho lwemodeli;
• uvavanyo kunye nokuphunyezwa kwemodeli kwinkqubo yeshishini;
• ukusebenza kwemodeli.

Xa ngexesha lokusebenza kuba yimfuneko ukutshintsha okanye ukuqeqesha kwakhona imodeli kwidatha entsha, umjikelezo uqala ngokutsha - imodeli igqitywe, ihlolwe, kwaye inguqu entsha isetyenziswe.

Ukuhlehla. Kutheni uphinda ufundise kwaye ungaphindi ufundise? Igama elithi "ukuqeqeshwa kwakhona kwemodeli" linentsingiselo ephindwe kabini: phakathi kweengcali lithetha isiphene kwimodeli, xa imodeli iqikelele kakuhle, ngokwenene iphinda iparameter eqikelelweyo kwisethi yoqeqesho, kodwa isebenza kakubi kakhulu kwisethi yedatha yangaphandle. Ngokwemvelo, imodeli enjalo iyisiphene, kuba esi siphene asivumeli ukusetyenziswa kwayo.

Kulo mjikelezo wobomi, kubonakala kunengqiqo ukusebenzisa izixhobo ze-DevOps: uvavanyo oluzenzekelayo, ukuthunyelwa kunye nokubeka iliso, ukubhaliswa kwemodeli yokubala njenge-microservices eyahlukileyo. Kodwa kukho inani leempawu ezithintela ukusetyenziswa ngokuthe ngqo kwezi zixhobo ngaphandle kokubopha i-ML eyongezelelweyo.

Indlela yokwenza iimodeli zisebenze kwaye zenze inzuzo

Njengomzekelo, apho siza kubonisa ukusetyenziswa kwendlela ye-MLOps, siya kuthatha umsebenzi we-classic ngoku we-roboizing ingxoxo yenkxaso yebhanki (okanye nayiphi na enye) imveliso. Inkqubo yeshishini yenkxaso yengxoxo yile ilandelayo: umthengi ufaka umbuzo kwingxoxo kwaye ufumana impendulo evela kwingcali ngaphakathi komthi wencoko echazwe ngaphambili. Umsebenzi wokuzenzekelayo ingxoxo enjalo idla ngokusonjululwa kusetyenziswa iiseti ezichazwe ngobuchule zemithetho enzima kakhulu ukuphuhlisa nokugcina. Ukusebenza kwe-automation enjalo, kuxhomekeke kwinqanaba lobunzima bomsebenzi, unokuba ngu-20-30%. Ngokwendalo, kuvela umbono wokuba kuluncedo ngakumbi ukuphumeza imodyuli yobukrelekrele eyenziweyo - imodeli ephuhliswe kusetyenziswa umatshini wokufunda othi:

• ukukwazi ukucubungula izicelo ezininzi ngaphandle kokuthatha inxaxheba komqhubi (kuxhomekeke kwisihloko, kwezinye iimeko, ukusebenza kakuhle kunokufikelela kwi-70-80%);
• ilungelelanisa ngcono amagama angaqhelekanga kwincoko-iyakwazi ukumisela injongo, umnqweno wokwenene womsebenzisi wesicelo esiqulunqwe ngokungacacanga;
• iyakwazi ukugqiba xa impendulo yomzekelo yanele, kwaye xa kukho ukuthandabuza malunga "nokuqonda" kwale mpendulo kwaye kuyimfuneko ukubuza umbuzo owongezelelweyo wokucacisa okanye ukutshintshela kumqhubi;
• inokuhlaziywa ngokuzenzekelayo (endaweni yokuba iqela labaphuhlisi lihlala lilungelelanisa kwaye lilungisa iimeko zokuphendula, imodeli iphinda ihlaziywe yiNzululwazi yeDatha isebenzisa iilayibrari zokufunda zoomatshini ezifanelekileyo). 

Uyenza njani loo modeli iphambili isebenze? 

Njengasekusombululeni nawuphi na omnye umsebenzi, ngaphambi kokuphuhlisa imodyuli enjalo, kuyafuneka ukuba uchaze inkqubo yeshishini kwaye uchaze ngokusemthethweni umsebenzi othile esiya kuwusombulula sisebenzisa indlela yokufunda umatshini. Kweli nqanaba, inkqubo yokusebenza, echazwe yi-Ops yesifinyezo, iqala. 

Следующим шагом специалист Data science в сотрудничестве с инженером по данным проверяет доступность и достаточность данных и гипотезу бизнеса о работоспособности бизнес-идеи, разрабатывая прототип модели и проверяя ее фактическую эффективность. Только после подтверждения бизнесом может начинаться переход от разработки модели к встраиванию ее в системы, выполняющие конкретный бизнес-процесс. Сквозное планирование внедрения, глубокое понимание на каждом этапе, как модель будет использоваться и какой экономический эффект она принесет, является основополагающим моментом в процессах внедрения подходов MLOps  в технологический ландшафт компании.

Ngokuphuhliswa kobuchwepheshe be-AI, inani kunye neentlobo zemisebenzi ezinokusombululwa ngoncedo lokufunda koomatshini ziyanda njenge-avalanche. Inkqubo nganye yeshishini igcina inkampani ngokuzenzekela umsebenzi wabasebenzi bezikhundla ezininzi (iziko lokufowuna, ukujonga kunye nokuhlelwa kwamaxwebhu, njl.njl.), ikhulisa isiseko somthengi ngokongeza imisebenzi emitsha enomtsalane kunye nefanelekileyo, ukonga imali ngenxa yobuninzi babo. ukusetyenziswa nokwabiwa ngokutsha kwezibonelelo nokunye okuninzi. Ekugqibeleni, nayiphi na inkqubo igxile ekudaleni ixabiso kwaye, ngenxa yoko, kufuneka izise umphumo othile wezoqoqosho. Apha kubaluleke kakhulu ukucacisa ngokucacileyo ingcamango yoshishino kwaye ubale inzuzo elindelekileyo ekuphunyezweni komzekelo kwisakhiwo esipheleleyo sokudala ixabiso lenkampani. Kukho iimeko xa ukuphunyezwa komzekelo kungazithethelela, kwaye ixesha elichithwe ziingcali zokufunda ngomatshini libiza kakhulu kunomsebenzi womqhubi owenza lo msebenzi. Yingakho kuyimfuneko ukuzama ukuchonga iimeko ezinjalo kwizigaba zokuqala zokudala iinkqubo ze-AI.

Ngenxa yoko, imifuziselo iqalisa ukuzisa inzuzo kuphela xa umsebenzi weshishini uqulunqwe ngokuchanekileyo kwinkqubo ye-MLOps, izinto eziphambili zamiselwa, kwaye inkqubo yokwazisa imodeli kwinkqubo yaqulunqwa kumanqanaba okuqala ophuhliso.

Inkqubo entsha - imingeni emitsha

Impendulo egcweleyo kumbuzo osisiseko woshishino malunga nendlela imodeli yeML esebenzayo ekusombululeni iingxaki, umbuzo jikelele wokuthembela kwi-AI yenye yemingeni ephambili ekuphuhliseni nasekuphunyezweni kweendlela ze-MLOps. Ekuqaleni, amashishini ayathandabuza malunga nokuqaliswa kokufunda koomatshini kwiinkqubo - kunzima ukuthembela kwiimodeli kwiindawo apho abantu badla ngokusebenza ngaphambili. Kwishishini, iinkqubo zibonakala ngathi "ibhokisi elimnyama", ukufaneleka kweempendulo ezisafuna ukuqinisekiswa. Ukongezelela, kwiibhanki, kwishishini labaqhubi be-telecom kunye nabanye, kukho iimfuno ezingqongqo zabalawuli baseburhulumenteni. Zonke iinkqubo kunye ne-algorithms eziphunyezwa kwiinkqubo zebhanki zixhomekeke kuphicotho. Ukusombulula le ngxaki, ukubonisa ubungqina kwishishini kunye nabalawuli ukunyaniseka kunye nokuchaneka kweempendulo zobukrelekrele bokwenziwa, izixhobo zokubeka iliso ziyaziswa kunye nomzekelo. Ukongezelela, kukho inkqubo yokuqinisekisa ngokuzimeleyo, okunyanzelekileyo kwiimodeli ezilawulayo, ezihlangabezana neemfuno zeBhanki Ephakathi. Iqela leengcali ezizimeleyo liphicotha iziphumo ezifunyenwe yimodeli, ngokuqwalasela idatha yegalelo.

Umngeni wesibini kuvavanyo kunye nokuqwalaselwa kweengozi zemodeli xa kuphunyezwa imodeli yokufunda ngomatshini. Nangona umntu akakwazi ukuphendula umbuzo ngokuqiniseka ngokupheleleyo ukuba loo ngubo yayimhlophe okanye iluhlaza okwesibhakabhaka, ngoko ke ubukrelekrele bokwenziwa bunelungelo lokwenza impazamo. Kukwafanelekile ukuqwalasela ukuba idatha ingatshintsha ngokuhamba kwexesha, kwaye iimodeli kufuneka ziphinde ziqeqeshwe ukuze zivelise umphumo ochanekileyo ngokwaneleyo. Ukuze inkqubo yoshishino ingahlupheki, kuyimfuneko ukulawula imingcipheko yemodeli kunye nokubeka iliso ekusebenzeni komzekelo, ukuyibuyisela rhoqo kwidatha entsha.

Kodwa emva kwenqanaba lokuqala lokungathembani, umphumo ochaseneyo uqala ukubonakala. Iimodeli ezininzi zingeniswa ngempumelelo kwiinkqubo, ngakumbi ishishini linomdla okhulayo wokusetyenziswa kobukrelekrele bokwenziwa - kukho imisebenzi emitsha kunye nemitsha enokusombululwa kusetyenziswa iindlela zokufunda ngomatshini. Umsebenzi ngamnye usungula inkqubo epheleleyo efuna ubuchule obuthile:

• iinjineli zedatha zilungiselela kwaye ziqhube idatha;
• izazinzulu zedatha zisebenzisa izixhobo zokufunda ngomatshini kunye nokuphuhlisa imodeli;
• IT iphumeze imodeli kwinkqubo;
• Injineli ye-ML inquma indlela yokudibanisa ngokuchanekileyo le modeli kwinkqubo, apho izixhobo ze-IT zisetyenziselwa ngokuxhomekeke kwiimfuno zendlela yokusetyenziswa kwemodeli, ngokuqwalasela ukuhamba kwezicelo, ixesha lokuphendula, njl. 
• Umakhi we-ML uyila indlela imveliso yesoftware enokuphunyezwa ngayo ngokwasemzimbeni kwinkqubo yoshishino.

Umjikelo wonke ufuna inani elikhulu leengcali eziqeqeshwe kakhulu. Kwinqanaba elithile lophuhliso kunye neqondo lokungena kwiimodeli ze-ML kwiinkqubo zoshishino, kuvela ukuba ukulinganisa ngokulinganayo inani leengcali ngokulingana nokukhula kwenani lemisebenzi kuba yindleko kwaye ingasebenzi. Ngoko ke, umbuzo uvela wokuzenzekelayo inkqubo ye-MLOps - ichaza iiklasi ezininzi eziqhelekileyo zeengxaki zokufunda koomatshini, ukuphuhlisa imibhobho yokucubungula idatha kunye neemodeli zokuqeqesha kwakhona. Kumfanekiso ofanelekileyo, ukuxazulula iingxaki ezinjalo, iingcali zifunwa ngokulinganayo ngokulinganayo kwizakhono kwi-junction ye-BigData, iSayensi yeDatha, i-DevOps kunye ne-IT. Ke ngoko, eyona ngxaki inkulu kwishishini leSayensi yeDatha kunye nomngeni omkhulu ekuququzeleleni iinkqubo ze-MLOps kukungabikho kobuchule obunjalo kwimarike yoqeqesho ekhoyo. Iingcali ezihlangabezana neemfuno ezinjalo okwangoku zinqabile kwimarike yabasebenzi kwaye zifanelekile ubunzima babo kwigolide.

Kumbuzo wobuchule

Ngokwethiyori, yonke imisebenzi ye-MLOps inokusombulula ngezixhobo zakudala zeDevOps kwaye ngaphandle kokubhenela kulwandiso lwendima ekhethekileyo. Ke, njengoko siphawulile ngasentla, isazi sedatha akufuneki sibe sisazi semathematika kunye nomhlalutyi wedatha kuphela, kodwa kunye ne-guru yepayipi yonke - uphuhliso lwezakhiwo, iimodeli zeprogram kwiilwimi ezininzi kuxhomekeke kuyilo, ukulungiselela i-mart data kunye nokuthunyelwa. iwela emagxeni akhe. Nangona kunjalo, ukudalwa kokubophelela kwezobuchwepheshe kuphunyezwe kwinkqubo yokuphela ukuya ekupheleni kwe-MLOps kuthatha ukuya kutsho kuma-80% eendleko zabasebenzi, nto leyo ethetha ukuba ingcali yemathematika eqeqeshiweyo, eyiNzululwazi yeDatha ekumgangatho ophezulu, iya kunikela kuphela i-20% ixesha kwispesheli sakhe. Ke ngoko, ukwahlulwa kweendima zeengcali eziphumeza inkqubo yokuphumeza iimodeli zokufunda koomatshini kubaluleka. 

Ukuba kufuneka kucaciswe nzulu kangakanani iindima kuxhomekeke kubungakanani beshishini. Yinto enye xa isiqalo sinengcali enye, umsebenzi kwindawo yokugcina iinjineli zamandla, injineli, umyili wezakhiwo, kunye ne-DevOps ngokwakhe. Ingumbandela owahluke ngokupheleleyo xa, kwishishini elikhulu, zonke iinkqubo zophuhliso imodeli zigxininiswe ezimbalwa kwizinga eliphezulu Data Scientists, lo gama umdwelisi nkqubo okanye ingcali yedatha - ubuchule obuqhelekileyo kwaye bungabizi kakhulu kwimarike yabasebenzi - inokuthatha. uninzi lwemisebenzi yesiqhelo.

Ngaloo ndlela, apho umda ulele ekukhethweni kweengcali zokuqinisekisa inkqubo ye-MLOps kunye nendlela inkqubo yokusebenza kweemodeli eziphuhlisiwe ezicwangcisiweyo zichaphazela ngokuthe ngqo isantya kunye nomgangatho weemodeli eziphuhlisiwe, ukuveliswa kweqela kunye ne-microclimate kuyo.

Yintoni esele yenziwe liqela lethu

Kutshanje siqale ukwakha isakhelo sobuchule kunye neenkqubo zeMLOps. Kodwa ngoku, iiprojekthi zethu zokulawula umjikelo wobomi beemodeli kunye nokusebenzisa iimodeli njengenkonzo zikwinqanaba lovavanyo lwe-MVP.

Siphinde samisela ubume obufanelekileyo bobuchule beshishini elikhulu kunye nesakhiwo sombutho sokusebenzisana phakathi kwabo bonke abathathi-nxaxheba kwinkqubo. Amaqela e-Agile ahlelwe ukuba asombulule iingxaki kwi-spectrum yonke yabathengi bezoshishino, kunye nenkqubo yokusebenzisana namaqela eprojekthi ukudala amaqonga kunye neziseko zophuhliso, ezisisiseko se-MLOps esakhiwayo.

Imibuzo yekamva

Ii-MLOps ngummandla okhulayo ofumana ukunqongophala kobuchule kwaye uya kufumana amandla kwixesha elizayo. Okwangoku, kungcono ukwakha kuphuhliso kunye nezenzo zeDevOps. Eyona njongo iphambili ye-MLOps kukusebenzisa iimodeli zeML ngokufanelekileyo ukusombulula iingxaki zoshishino. Kodwa oku kuphakamisa imibuzo emininzi:

• Indlela yokunciphisa ixesha lokuqalisa iimodeli kwimveliso?
• Indlela yokunciphisa ingxabano phakathi kwamaqela anobuchule obahlukeneyo kunye nokwandisa ukugxila kwintsebenziswano?
• Indlela yokulandelela iimodeli, ukulawula iinguqulelo kunye nokuququzelela ukubeka iliso okusebenzayo?
• Uyenza njani umjikelo wobomi ojikeleza ngokwenene kwimodeli yeML yanamhlanje?
• Indlela yokulinganisa inkqubo yokufunda koomatshini?

Iimpendulo zale mibuzo ziya kuqinisekisa ubukhulu becala ukuba iiMLOps ziya kutyhila kangakanani na amandla azo apheleleyo.

umthombo: www.habr.com