Ikusasa selifikile, futhi ubuhlakani bokwenziwa kanye nobuchwepheshe bokufunda komshini sebuvele busetshenziswa ngempumelelo izitolo zakho eziyintandokazi, izinkampani zokuthutha kanye namapulazi ama-turkey.
Futhi uma kukhona okuthile, khona-ke kukhona kakade okuthile ngakho ku-intanethi ... iphrojekthi evulekile! Bona ukuthi i-Open Data Hub ikusiza kanjani ukuthi ulinganisele ubuchwepheshe obusha futhi ugweme izinselele zokusetshenziswa.
Ngazo zonke izinzuzo ze-artificial intelligence (AI) nokufunda komshini (ML), izinhlangano zivame ukuba nobunzima bokukala lobu buchwepheshe. Izinkinga eziyinhloko kuleli cala ngokuvamile yilezi ezilandelayo:
- Ukushintshisana ngolwazi nokubambisana - cishe akunakwenzeka ukushintshanisa ulwazi ngaphandle komzamo futhi uhlanganyele ngokuphindaphinda okusheshayo.
- Ukufinyelela idatha - ngomsebenzi ngamunye udinga ukwakhiwa kabusha futhi ngesandla, okuthatha isikhathi esiningi.
- Ukufinyelela lapho kudingeka - ayikho indlela yokuthola ukufinyelela okudingekayo kumathuluzi okufunda omshini kanye nesiteji, kanye nengqalasizinda yekhompyutha.
- Ukukhiqizwa - amamodeli ahlala esiteji se-prototype futhi awalethwa ekusetshenzisweni kwezimboni.
- Landelela futhi uchaze imiphumela ye-AI - ukukhiqiza kabusha, ukulandelela kanye nokuchazwa kwemiphumela ye-AI/ML kunzima.
Uma zingaxazululiwe, lezi zinkinga ziba nomthelela omubi kwijubane, ukusebenza kahle, kanye nokukhiqiza kososayensi bedatha ebalulekile. Lokhu kuholela ekukhungathekeni kwabo, ukudumazeka emsebenzini wabo, futhi ngenxa yalokho, okulindelwe yibhizinisi mayelana ne-AI/ML kuyachitheka.
Isibopho sokuxazulula lezi zinkinga siwela kochwepheshe be-IT, okufanele banikeze abahlaziyi bedatha - kunjalo, into efana nefu. Ngemininingwane eyengeziwe, sidinga inkundla enikeza inkululeko yokuzikhethela futhi enokufinyelela okulula, okulula. Ngesikhathi esifanayo, iyashesha, ilungiseka kabusha kalula, iyakala ngokufunwa futhi imelana nokwehluleka. Ukwakha inkundla enjalo ngobuchwepheshe bemithombo evulekile kusiza ukugwema ukukhiya komthengisi nokugcina inzuzo yesikhathi eside yamasu mayelana nokulawulwa kwezindleko.
Eminyakeni embalwa edlule, into efanayo yayisenzeka ekuthuthukisweni kwesicelo futhi kwaholela ekuveleni kwama-microservices, amafu ayi-hybrid, i-IT automation, nezinqubo ezishesha kakhulu. Ukuze ubhekane nakho konke lokhu, ochwepheshe be-IT baphendukele ezitsheni, i-Kubernetes kanye namafu avulekile ayingxubevange.
Lokhu okuhlangenwe nakho manje kusetshenziselwa ukuphendula izinselele zika-Al. Kungakho ochwepheshe be-IT bakha amapulatifomu asekelwe kuma-container, anika amandla ukudalwa kwezinsizakalo ze-AI/ML ngaphakathi kwezinqubo eziguquguqukayo, asheshisa ukuqanjwa okusha, futhi akhiwe ngeso elibheke efwini eliyingxubevange.
Sizoqala ukwakha inkundla enjalo nge-Red Hat OpenShift, inkundla yethu ye-Kubernetes ene-container ye-hybrid cloud, ene-ecosystem ekhula ngokushesha yesofthiwe ne-hardware ML solutions (NVIDIA, H2O.ai, Starburst, PerceptiLabs, njll.). Amanye amakhasimende e-Red Hat, njenge-BMW Group, i-ExxonMobil namanye, asevele athumele ama-toolchain e-ML nezinqubo ze-DevOps phezu kweplatifomu kanye ne-ecosystem yayo ukuze ilethe izakhiwo zabo ze-ML ekukhiqizweni nokusheshisa umsebenzi wabahlaziyi bedatha.
Esinye isizathu esenza ukuthi sethule iphrojekthi ye-Open Data Hub ukukhombisa isibonelo sesakhiwo esisuselwe kumaphrojekthi wesofthiwe yomthombo ovulekile amaningana futhi sibonise indlela yokuqalisa umjikelezo wokuphila wesixazululo se-ML ngokusekelwe kuplathifomu ye-OpenShift.
Vula Iphrojekthi Yehabhu Yedatha
Lena iphrojekthi yomthombo ovulekile eyakhiwa ngaphakathi komphakathi ohambisanayo othuthukayo futhi isebenzisa umjikelezo ogcwele wokusebenza - ukusuka ekulayisheni nasekuguquleni idatha yokuqala iye ekukhiqizeni, ekuqeqesheni nasekugcineni imodeli - lapho kuxazulula izinkinga ze-AI / ML kusetshenziswa iziqukathi kanye ne-Kubernetes ku-OpenShift. inkundla. Le phrojekthi ingabhekwa njengokuqaliswa kwereferensi, isibonelo sendlela yokwakha isixazululo se-AI/ML-as-a-service evulekile ngokusekelwe ku-OpenShift namathuluzi omthombo ovulekile ahlobene njenge-Tensorflow, JupyterHub, Spark namanye. Kubalulekile ukuqaphela ukuthi i-Red Hat ngokwayo isebenzisa le phrojekthi ukuze inikeze amasevisi ayo e-AI/ML. Ngaphezu kwalokho, i-OpenShift ihlanganisa ne-software eyinhloko nezixazululo ze-ML ezivela ku-NVIDIA, Seldon, Starbust nabanye abathengisi, okwenza kube lula ukwakha nokusebenzisa izinhlelo zakho zokufunda umshini.
Iphrojekthi ye-Open Data Hub igxile ezigabeni ezilandelayo zabasebenzisi nezimo zokusebenzisa:
- Umhlaziyi wedatha odinga isixazululo sokuqalisa amaphrojekthi e-ML, ahlelwe njengefu elinezinto zokuzisiza.
- Umhlaziyi wedatha odinga ukukhetha okuphezulu kumathuluzi akamuva omthombo ovulekile we-AI/ML nezinkundla.
- Umhlaziyi wedatha odinga ukufinyelela emithonjeni yedatha lapho eqeqeshwa amamodeli.
- Umhlaziyi wedatha odinga ukufinyelela kuzinsiza zekhompuyutha (i-CPU, i-GPU, inkumbulo).
- Umhlaziyi Wedatha odinga ikhono lokuhlanganyela nokwabelana ngomsebenzi nozakwethu, athole impendulo, futhi enze intuthuko ekuphindaphindeni ngokushesha.
- Umhlaziyi wedatha ofuna ukusebenzelana nonjiniyela (namaqembu e-devops) ukuze amamodeli akhe e-ML nemiphumela yomsebenzi iye ekukhiqizweni.
- Unjiniyela wedatha odinga ukunikeza umhlaziyi wedatha ukufinyelela emithonjeni ehlukahlukene yedatha kuyilapho ethobela izimfuneko zokulawula nokuvikeleka.
- Umlawuli/umsebenzisi wesistimu ye-IT odinga amandla okulawula ngokuzikhandla umjikelezo wempilo (ukufakwa, ukumisa, ukuthuthukiswa) kwezingxenye zomthombo ovulekile nobuchwepheshe. Sidinga futhi amathuluzi okuphatha afanelekile kanye nesabelo.
Iphrojekthi ye-Open Data Hub ihlanganisa ndawonye ububanzi bamathuluzi omthombo ovulekile ukuze kusetshenziswe umjikelezo ogcwele wemisebenzi ye-AI/ML. I-Jupyter Notebook isetshenziswa lapha njengethuluzi eliyinhloko lokusebenza lokuhlaziya idatha. Ikhithi yamathuluzi idume kakhulu phakathi kososayensi bedatha namuhla, futhi i-Open Data Hub ibavumela ukuthi bakhe kalula futhi baphathe izindawo zokusebenza ze-Jupyter Notebook besebenzisa i-JupyterHub eyakhelwe ngaphakathi. Ngokungeziwe ekudaleni nasekungeniseni izincwadi zokubhalela ze-Jupyter, iphrojekthi ye-Open Data Hub nayo iqukethe inqwaba yezincwadi zokubhalela ezenziwe ngomumo ze-AI Library.
Lo mtapo wezincwadi uyiqoqo lezingxenye zokufunda zomshini ovulekile nezisombululo zezimo ezivamile ezenza kube lula ukwenza i-prototyping ngokushesha. I-JupyterHub ihlanganiswe nemodeli yokufinyelela ye-OpenShift ye-RBAC, ekuvumela ukuthi usebenzise ama-akhawunti akhona e-OpenShift futhi usebenzise ukungena ngemvume okukodwa. Ngaphezu kwalokho, i-JupyterHub inikeza isikhombimsebenzisi esisebenziseka kalula esibizwa ngokuthi i-spawner, lapho umsebenzisi angamisa kalula inani lezinsiza zekhompyutha (ama-CPU cores, inkumbulo, i-GPU) ye-Jupyter Notebook ekhethiwe.
Ngemuva kokuthi umhlaziyi wedatha edale futhi alungise ikhompuyutha ephathekayo, zonke ezinye izinto ezikhathazayo ngayo zinakekelwa umhleli we-Kubernetes, oyingxenye ye-OpenShift. Abasebenzisi bangenza kuphela ukuhlola kwabo, balondoloze futhi babelane ngemiphumela yomsebenzi wabo. Ukwengeza, abasebenzisi abathuthukile bangafinyelela ngokuqondile igobolondo le-OpenShift CLI ngokuqondile ezincwadini zokubhalela ze-Jupyter ukuze basebenzise izinto zokuqala ze-Kubernetes ezifana nokusebenza kwe-Job noma kwe-OpenShift okufana ne-Tekton noma i-Knative. Noma kulokhu ungasebenzisa i-GUI elula ye-OpenShift, ebizwa ngokuthi “ikhonsoli yewebhu ye-OpenShift”.
Ukuqhubekela esigabeni esilandelayo, i-Open Data Hub yenza kube nokwenzeka ukuphatha amapayipi edatha. Kulokhu, kusetshenziswa into ye-Ceph, enikezwa njengento yokugcina idatha ehambisana ne-S3. I-Apache Spark ikuvumela ukuthi usakaze idatha kusuka emithonjeni yangaphandle noma isitoreji esakhelwe ngaphakathi se-Ceph S3, futhi ikuvumela ukuthi wenze izinguquko zedatha yokuqala. I-Apache Kafka inikeza ukuphathwa okuthuthukisiwe kwamapayipi edatha (lapho idatha ingalayishwa izikhathi eziningi, kanye nokuguqulwa kwedatha, ukuhlaziya, nokusebenza okuqhubekayo).
Ngakho-ke, umhlaziyi wedatha ufinyelele idatha futhi wakha imodeli. Manje unesifiso sokwabelana ngemiphumela etholwe nozakwabo noma abathuthukisi bezicelo, futhi abanikeze imodeli yakhe ngezimiso zesevisi. Lokhu kudinga iseva ye-inference, futhi i-Open Data Hub ineseva enjalo, ebizwa ngokuthi i-Seldon futhi ikuvumela ukuthi ushicilele imodeli njengesevisi ye-RESTful.
Kwesinye isikhathi, kunamamodeli anjalo amaningana kuseva ye-Seldon, futhi kunesidingo sokuqapha ukuthi asetshenziswa kanjani. Ukuze kuzuzwe lokhu, i-Open Data Hub inikeza iqoqo lamamethrikhi afanelekile kanye nenjini yokubika esekelwe kumathuluzi okuqapha omthombo ovulekile asetshenziswa kabanzi i-Prometheus ne-Grafana. Ngenxa yalokho, sithola impendulo yokuqapha ukusetshenziswa kwamamodeli e-AI, ikakhulukazi endaweni yokukhiqiza.
Ngale ndlela, i-Open Data Hub inikeza indlela efana nefu kuwo wonke umjikelezo wokuphila we-AI/ML, kusukela ekufinyeleleni idatha nokulungiselela ukuya ekuqeqeshweni nasekukhiqizeni okuyimodeli.
Ukuhlanganisa konke
Manje umbuzo uphakama ukuthi ungakuhlela kanjani konke lokhu kumlawuli we-OpenShift. Futhi kulapho osebenza khona okhethekile we-Kubernetes wamaphrojekthi we-Open Data Hub engena khona.
Lesi sisebenzisi siphatha ukufakwa, ukumiswa kanye nomjikelezo wempilo wephrojekthi ye-Open Data Hub, okuhlanganisa nokusetshenziswa kwamathuluzi ashiwo ngenhla njenge-JupyterHub, i-Ceph, i-Spark, i-Kafka, i-Seldon, i-Prometheus ne-Grafana. Iphrojekthi ye-Open Data Hub ingatholakala kukhonsoli yewebhu ye-OpenShift, esigabeni sama-opharetha omphakathi. Ngakho, umlawuli we-OpenShift angacacisa ukuthi amaphrojekthi e-OpenShift ahambisanayo ahlukaniswa ngokuthi "Iphrojekthi Yehabhu Yedatha Evuliwe". Lokhu kwenziwa kanye. Ngemva kwalokhu, umhlaziyi wedatha ungena esikhaleni sakhe sephrojekthi ngokusebenzisa ikhonsoli yewebhu ye-OpenShift futhi abone ukuthi u-opharetha we-Kubernetes ohambisanayo ufakiwe futhi uyatholakala kumaphrojekthi akhe. Ube esedala isibonelo sephrojekthi ye-Open Data Hub ngokuchofoza okukodwa futhi ngokushesha akwazi ukufinyelela amathuluzi achazwe ngenhla. Futhi konke lokhu kungalungiswa ngokutholakala okuphezulu kanye nemodi yokubekezelela iphutha.
Uma ungathanda ukuzizamela iphrojekthi ye-Open Data Hub, qala ngayo . Imininingwane yobuchwepheshe yokwakheka kwe-Open Data Hub ingatholwa , izinhlelo zokuthuthukisa iphrojekthi - . Ngokuzayo, sihlela ukusebenzisa ukuhlanganiswa okwengeziwe ne-Kubeflow, sixazulule izinkinga eziningi ngokulawulwa kwedatha nokuphepha, futhi sihlele ukuhlanganiswa namasistimu asekelwe emithethweni i-Drools ne-Optaplanner. Veza umbono wakho futhi uhlanganyele kuphrojekthi kungenzeka ekhasini .
Ukuphinda: Izinselele ezinkulu zokukala zivimbela izinhlangano ekuboneni amandla aphelele obuhlakani bokwenziwa nokufunda komshini. I-Red Hat OpenShift sekuyisikhathi eside isetshenziswa ngempumelelo ukuxazulula izinkinga ezifanayo embonini yesofthiwe. Iphrojekthi ye-Open Data Hub, esetshenziswa phakathi komphakathi wokuthuthukiswa komthombo ovulekile, inikezela ngendawo eyireferensi yokuhlela umjikelezo ogcwele wemisebenzi ye-AI/ML esekelwe efwini eliyingxube ye-OpenShift. Sinohlelo olucacile nolucatshangelwayo lokuthuthukiswa kwale phrojekthi, futhi sizimisele ngokudala umphakathi osebenzayo nothelayo owuzungezile ukuze sithuthukise izixazululo ze-AI ezivulekile endaweni yesikhulumi se-OpenShift.
Source: www.habr.com