Ikamva lifikile, kwaye ubukrelekrele bokwenziwa kunye nobuchwepheshe bokufunda koomatshini sele busetyenziswa ngempumelelo ziivenkile zakho ezizithandayo, iinkampani zothutho kunye neefama zeTurkey.
Kwaye ukuba kukho into ekhoyo, sele kukho into malunga nayo kwi-Intanethi ... iprojekthi evulekileyo! Jonga indlela iHub yeDatha eVulekileyo ekunceda ngayo ukukhulisa itekhnoloji entsha kwaye uthintele imiceli mngeni yophumezo.
Ngazo zonke iingenelo zobukrelekrele bokwenziwa (AI) kunye nokufunda koomatshini (ML), imibutho ihlala inobunzima bokulinganisa obu buchwepheshe. Ezona ngxaki ziphambili kule meko zidla ngokulandelayo:
- Utshintshiselwano ngolwazi kunye nentsebenziswano - phantse akunakwenzeka ukutshintshiselana ngolwazi ngaphandle komzamo kunye nokusebenzisana ngokuphindaphinda ngokukhawuleza.
- Ukufikelela kwidatha - kumsebenzi ngamnye kufuneka wakhiwe ngokutsha kwaye ngesandla, okuthatha ixesha elininzi.
- Ukufikelela kwimfuno -akukho ndlela yokufumana ukufikelela kwimfuno kwizixhobo zokufunda zoomatshini kunye neqonga, kunye neziseko zekhompyutha.
- Imveliso - iimodeli zihlala kwinqanaba leprototype kwaye aziziswa kusetyenziso lweshishini.
- Khangela kwaye uchaze iziphumo ze-AI - ukuveliswa kwakhona, ukulandelela kunye nokuchazwa kweziphumo ze-AI / ML kunzima.
Ishiywe ingalungiswanga, ezi ngxaki zinefuthe elibi kwisantya, ukusebenza kakuhle, kunye nemveliso yezazinzulu zedatha ezixabisekileyo. Oku kukhokelela ekukhungathekeni kwabo, ukuphoxeka emsebenzini wabo, kwaye ngenxa yoko, ulindelo lweshishini malunga ne-AI/ML luya kumosha.
Uxanduva lokusombulula ezi ngxaki luwela kwiingcali ze-IT, ekufuneka zinike abahlalutyi bedatha - kunjalo, into efana nelifu. Ngeenkcukacha ezithe kratya, sifuna iqonga elinika inkululeko yokuzikhethela kwaye linokufikelela ngokulula, kulula. Kwangaxeshanye, iyakhawuleza, ikwazi ukuphinda iqwalaselwe kwakhona, iyakaleka kwimfuno kwaye iyamelana nokungaphumeleli. Ukwakha iqonga elinjalo kubuchwephesha bomthombo ovulekileyo kunceda ukuphepha ukutshixa umthengisi kunye nokugcina inzuzo yesicwangciso sexesha elide ngokwemigaqo yolawulo lweendleko.
Kwiminyaka embalwa edlulileyo, into efanayo yayisenzeka kuphuhliso lwesicelo kwaye yakhokelela ekuveleni kwee-microservices, amafu axubileyo, i-IT automation, kunye neenkqubo ze-agile. Ukujongana nayo yonke le nto, iingcali ze-IT zijike kwii-container, i-Kubernetes kunye namafu avulekileyo adibeneyo.
La mava ngoku asetyenziswa ukuphendula imingeni ka-Al. Yiyo loo nto iingcali ze-IT zisakha amaqonga asekwe kwikhonteyina, enze ukuba kuyilwe iinkonzo ze-AI/ML ngaphakathi kweenkqubo ezikhawulezayo, ukukhawulezisa ukusungula izinto ezintsha, kwaye zakhiwe ngeliso elijonge kwilifu elixubileyo.
Siza kuqalisa ukwakha iqonga ngeRed Hat OpenShift, iqonga lethu leKubernetes elinezikhongozeli zelifu elixutyiweyo, elinenkqubo ekhula ngokukhawuleza yendalo yesoftware kunye nezisombululo zehardware zeML (NVIDIA, H2O.ai, Starburst, PerceptiLabs, njl.). Abanye abathengi be-Red Hat, njenge-BMW Group, i-ExxonMobil kunye nabanye, sele besasaze izixhobo ze-ML ze-container kunye neenkqubo ze-DevOps phezulu kweqonga kunye ne-ecosystem yayo ukuzisa i-architecture yabo ye-ML kwimveliso kunye nokukhawulezisa umsebenzi wabahlalutyi bedatha.
Esinye isizathu sokuba siqalise iprojekthi ye-Open Data Hub kukubonisa umzekelo wolwakhiwo olusekwe kwiiprojekthi ezininzi zesoftware evulelekileyo kwaye sibonise indlela yokuphumeza umjikelo wobomi bonke besisombululo seML esekwe kwiqonga le-OpenShift.
Vula iProjekthi yeHub yeDatha
Le yiprojekthi yomthombo ovulekileyo ophuhliswa ngaphakathi koluntu oluhambelanayo lophuhliso kwaye izalisekisa umjikelo opheleleyo wemisebenzi - ukusuka ekulayisheni nasekuguquleni idatha yokuqala ukuya ekuveliseni, ekuqeqesheni nasekugcineni imodeli - xa usombulula iingxaki ze-AI / ML usebenzisa izitya kunye ne-Kubernetes kwi-OpenShift. iqonga. Le projekthi inokuthathwa njengokuphunyezwa kwereferensi, umzekelo wendlela yokwakha isisombululo se-AI / ML-as-a-service evulekileyo esekelwe kwi-OpenShift kunye nezixhobo ezivulekileyo ezinxulumene nezixhobo ezifana neTensorflow, JupyterHub, Spark kunye nabanye. Kubalulekile ukuqaphela ukuba i-Red Hat ngokwayo isebenzisa le projekthi ukubonelela ngeenkonzo zayo ze-AI/ML. Ukongeza, i-OpenShift idibanisa kunye ne-software engundoqo kunye nezisombululo ze-ML ze-hardware ezivela kwi-NVIDIA, i-Seldon, i-Starbust kunye nabanye abathengisi, okwenza kube lula ukwakha nokusebenzisa iinkqubo zakho zokufunda umatshini.
Iprojekthi ye-Open Data Hub igxile kwezi ndidi zilandelayo zabasebenzisi kunye neemeko zokusetyenziswa:
- Umhlalutyi wedatha ofuna isisombululo sokuphumeza iiprojekthi zeML, eziququzelelwe njengelifu elinomsebenzi wokuzisebenzela.
- Umhlalutyi wedatha ofuna ukhetho oluphezulu ukusuka kumthombo ovulekileyo we-AI/ML izixhobo kunye namaqonga.
- Umhlalutyi wedatha ofuna ukufikelela kwimithombo yedatha xa iimodeli zoqeqesho.
- Umhlalutyi wedatha ofuna ukufikelela kwizixhobo zekhompyutha (CPU, GPU, imemori).
- Umhlalutyi weDatha ofuna ukukwazi ukusebenzisana kunye nokwabelana ngomsebenzi kunye noogxa bakhe, ukufumana impendulo, kunye nokwenza uphuculo lokuphindaphinda ngokukhawuleza.
- Umhlalutyi wedatha ofuna ukusebenzisana nabaphuhlisi (kunye namaqela e-devops) ukuze iimodeli zakhe ze-ML kunye neziphumo zomsebenzi zingene kwimveliso.
- Injineli yedatha efuna ukubonelela umhlalutyi wedatha ngokufikelela kwimithombo eyahlukeneyo yedatha ngelixa ithobela iimfuno zolawulo kunye nokhuseleko.
- Umlawuli wenkqubo ye-IT/umqhubi ofuna ukukwazi ukulawula ngokungenamzamo umjikelo wobomi (ufakelo, uqwalaselo, uphuculo) lwamacandelo omthombo ovulekileyo kunye nobuchwepheshe. Sikwafuna ulawulo olufanelekileyo kunye nezixhobo zekota.
Iprojekthi ye-Open Data Hub idibanisa uluhlu lwezixhobo zomthombo ovulekileyo ukuze kuphunyezwe umjikelo opheleleyo wemisebenzi ye-AI/ML. IJupyter Notebook isetyenziswa apha njengesona sixhobo sisebenzayo sokuhlalutya idatha. I-toolkit ithandwa kakhulu phakathi kwezazinzulu zedatha namhlanje, kwaye i-Open Data Hub ivumela ukuba benze lula kwaye balawule iindawo zokusebenza zeJupyter Notebook usebenzisa i-JupyterHub eyakhelwe-ngaphakathi. Ukongeza ekudaleni nasekungeniseni iincwadi zamanqaku zeJupyter, iprojekthi ye-Open Data Hub ikwaqulathe inani leencwadana zamanqaku esele zenziwe ngendlela ye-AI Library.
Eli thala leencwadi liyingqokelela yamacandelo okufunda koomatshini avulelekileyo kunye nezisombululo zeemeko eziqhelekileyo ezenza lula iprototyping ekhawulezayo. I-JupyterHub idityaniswe nemodeli yokufikelela ye-OpenShift ye-RBAC, ekuvumela ukuba usebenzise ii-akhawunti ezikhoyo ze-OpenShift kwaye uphumeze ukungena kokunye. Ukongeza, i-JupyterHub inikezela nge-interface yomsebenzisi-friendly interface ebizwa ngokuba yi-spawner, apho umsebenzisi angakwazi ukuqwalasela ngokulula inani lezixhobo ze-computing (i-CPU cores, imemori, i-GPU) ye-Jupyter Notebook ekhethiweyo.
Emva kokuba umhlalutyi wedatha enze kwaye alungise i-laptop, zonke ezinye iinkxalabo malunga nazo zinyamekelwa ngumcwangcisi we-Kubernetes, oyinxalenye ye-OpenShift. Abasebenzisi banokwenza kuphela imifuniselo yabo, bagcine kwaye babelane ngeziphumo zomsebenzi wabo. Ukongeza, abasebenzisi abaphambili banokufikelela ngokuthe ngqo kwiqokobhe le-OpenShift CLI ngokuthe ngqo ukusuka kwiincwadana zamanqaku zeJupyter ukuze basebenzise izinto zokuqala zeKubernetes ezinje ngoJob okanye ukusebenza kwe-OpenShift njengeTekton okanye iKnative. Okanye ngale nto ungasebenzisa i-GUI ye-OpenShift efanelekileyo, ebizwa ngokuba yi βOpenShift web consoleβ.
Ukuqhubela phambili kwinqanaba elilandelayo, Vula i-Data Hub yenza kube lula ukulawula imibhobho yedatha. Kule nto, into yeCeph isetyenzisiweyo, enikezelwa njengento yokugcina idatha ye-S3 ehambelanayo. I-Apache Spark ikuvumela ukuba usasaze idatha kwimithombo yangaphandle okanye i-Ceph S3 yokugcina eyakhelwe ngaphakathi, kwaye ikuvumela ukuba wenze utshintsho lokuqala lwedatha. I-Apache Kafka inikezela ngolawulo oluphezulu lwemibhobho yedatha (apho idatha ingalayishwa ngamaxesha amaninzi, kunye nokuguqulwa kwedatha, ukuhlalutya, kunye nokusebenza ngokuzingisa).
Ngoko ke, umhlalutyi wedatha ufikelele kwidatha kwaye wakha imodeli. Ngoku unomnqweno wokwabelana ngeziphumo ezifunyenweyo kunye noogxa bakhe okanye abaphuhlisi bezicelo, kwaye babonelele ngemodeli yakhe kwimigaqo yenkonzo. Oku kufuna iseva ye-inference, kwaye i-Open Data Hub inomncedisi onjalo, ubizwa ngokuba yi-Seldon kwaye ikuvumela ukuba upapashe imodeli njengenkonzo ye-RESTful.
Ngaxa lithile, zininzi iimodeli ezinjalo kwiseva yeSeldon, kwaye kukho imfuneko yokubeka iliso kwindlela ezisetyenziswa ngayo. Ukufezekisa oku, i-Open Data Hub inikezela ngengqokelela yeemetriki ezifanelekileyo kunye ne-injini yokunika ingxelo esekelwe kumthombo ovulekileyo wezixhobo zokubeka iliso ezisetyenziswa ngokubanzi i-Prometheus kunye ne-Grafana. Ngenxa yoko, sifumana ingxelo yokubeka iliso kusetyenziso lweemodeli ze-AI, ngakumbi kwindawo yokuvelisa.
Ngale ndlela, i-Open Data Hub inikezela ngendlela efana nefu kuyo yonke i-AI / ML yokuphila, ukusuka ekufikeleleni kwedatha kunye nokulungiselela ukuya kwimodeli yoqeqesho kunye nemveliso.
Ukubeka konke kunye
Ngoku umbuzo uvela indlela yokucwangcisa konke oku kumlawuli we-OpenShift. Kwaye kulapho umqhubi okhethekileyo we-Kubernetes kwiiprojekthi ze-Open Data Hub engena khona.
Lo msebenzisi ulawula ufakelo, uhlengahlengiso kunye nobomi beprojekthi ye-Open Data Hub, kuquka nokuthunyelwa kwezixhobo ezikhankanywe ngasentla ezifana neJupyterHub, iCeph, iSpark, iKafka, iSeldon, iPrometheus neGrafana. Iprojekthi ye-Open Data Hub inokufumaneka kwi-OpenShift web console, kwicandelo labasebenzi boluntu. Ngaloo ndlela, umlawuli we-OpenShift unokucacisa ukuba iiprojekthi ze-OpenShift ezihambelanayo zihlelwe njenge "Iprojekthi ye-Data evulekileyo". Oku kwenziwa kanye. Emva koko, umhlalutyi wedatha ungena kwindawo yakhe yeprojekthi nge-OpenShift web console kwaye ubona ukuba umqhubi we-Kubernetes ohambelanayo ufakwe kwaye uyafumaneka kwiiprojekthi zakhe. Emva koko udala umzekelo weprojekthi ye-Open Data Hub ngonqakrazo olunye kwaye ngokukhawuleza unokufikelela kwizixhobo ezichazwe ngasentla. Kwaye konke oku kunokuqwalaselwa ngokufumaneka okuphezulu kunye nemodi yokunyamezela impazamo.
Ukuba ungathanda ukuzizamela iprojekthi ye-Open Data Hub, qala ngayo . Iinkcukacha zobuchwephesha boyilo lwe-Open Data Hub zingafunyanwa , izicwangciso zophuhliso lweprojekthi - . Kwixesha elizayo, siceba ukuphumeza ukudibanisa okongeziweyo kunye ne-Kubeflow, ukuxazulula imiba emininzi kunye nokulawulwa kwedatha kunye nokhuseleko, kwaye uququzelele ukuhlanganiswa kunye neenkqubo ezisekelwe kwimithetho ye-Drools kunye ne-Optaplanner. Veza uluvo lwakho kwaye ube ngumthathi-nxaxheba kwiprojekthi kunokwenzeka kwiphepha .
Ukuphinda uhlaziye: Imingeni enzima yokulinganisa ithintela imibutho ekuqondeni amandla apheleleyo obukrelekrele bokwenziwa kunye nokufunda koomatshini. I-Red Hat OpenShift sele isetyenziswe ngempumelelo ukusombulula iingxaki ezifanayo kwishishini lesoftware. Iprojekthi ye-Open Data Hub, ephunyezwe ngaphakathi koluntu lophuhliso lomthombo ovulekileyo, inikezela ngoyilo lwereferensi yokulungiselela umjikelo opheleleyo wemisebenzi ye-AI/ML esekelwe kwifu elixubileyo le-OpenShift. Sinesicwangciso esicacileyo nesicingayo sophuhliso lwale projekthi, kwaye sizimisele ngokudala uluntu olusebenzayo noluneziqhamo olujikeleze kuyo ukuphuhlisa izisombululo ze-AI ezivulekileyo kwiqonga le-OpenShift.
umthombo: www.habr.com