Ramangwana rasvika, uye hungwaru hwekugadzira uye matekinoroji ekudzidza muchina ari kutoshandiswa zvinobudirira nezvitoro zvaunofarira, makambani ekufambisa uye kunyange mapurazi eTurkey.
Uye kana chimwe chinhu chiripo, ipapo pane chatova chimwe chinhu pamusoro payo paInternet ... chirongwa chakazaruka! Ona kuti Vhura Data Hub inokubatsira sei kuyera matekinoroji matsva uye kudzivirira matambudziko ekuita.
Nezvese zvakanakira zvehungwaru hwekugadzira (AI) uye kudzidza muchina (ML), masangano anowanzo kunetseka kuyera matekinoroji aya. Matambudziko makuru munyaya iyi kazhinji ndeaya anotevera:
- Kutsinhana kwemashoko nekushandira pamwe - zvinenge zvisingabviri kuchinjana ruzivo pasina simba uye kushandira pamwe mukukurumidza kudzokorora.
- Kuwana data - kune rimwe nerimwe basa rinoda kuvakwa patsva uye nemaoko, izvo zvinotora nguva yakawanda.
- Kuwana pane zvinodiwa - hapana nzira yekuwana pane-inoda mukana wekushandisa muchina wekudzidza maturusi uye chikuva, pamwe nekombuta zvivakwa.
- Kugadzira - mamodheru anoramba ari padanho reiyo prototype uye haana kuunzwa mukushandiswa kwemaindasitiri.
- Tevera uye tsanangura mhinduro dzeAI - kuberekazve, kuronda uye tsananguro yeAI/ML mhedzisiro yakaoma.
Kusiiwa zvisina kugadziriswa, matambudziko aya anokanganisa kukurumidza, kugona, uye kugadzirwa kweakakosha data masayendisiti. Izvi zvinotungamira mukuvhiringidzika kwavo, kuodzwa mwoyo pabasa ravo, uye semhedzisiro, tarisiro yebhizinesi maererano neAI/ML inoparara.
Basa rekugadzirisa matambudziko aya rinowira kune nyanzvi dzeIT, dzinofanirwa kupa vanoongorora data - ndizvozvo, chimwe chinhu chakafanana negore. Mune rumwe ruzivo, isu tinoda chikuva chinopa rusununguko rwekusarudza uye ine nyore, nyore kuwana. Panguva imwecheteyo, inokurumidza, inogadzirika zviri nyore, inoshomeka painoda uye inoshingirira kukundikana. Kuvaka chikuva chakadai pane yakavhurika sosi matekinoroji kunobatsira kudzivirira kukiya-mukati uye kuchengetedza yenguva refu yehurongwa mukana maererano nekudzora mutengo.
Makore mashoma apfuura, chimwe chinhu chakafanana chaiitika mukuvandudza kwekushandisa uye zvakakonzera kubuda kwemicroservices, makore akasanganiswa, IT otomatiki, uye agile maitiro. Kuti utsungirire zvese izvi, inyanzvi dzeIT dzakatendeukira kumidziyo, Kubernetes uye yakavhurika hybrid makore.
Chiitiko ichi chave kushandiswa kupindura matambudziko aAl. Ndosaka vadzidzisi veIT vari kuvaka mapuratifomu ari mumudziyo-akavakirwa, anogonesa kugadzirwa kweAI/ML masevhisi mukati memaitiro agile, kusimudzira hunyanzvi, uye anovakwa neziso rakananga kune iro hybrid gore.
Tichatanga kuvaka chikuva chakadaro neRed Hat OpenShift, chikuva chedu cheKubernetes chegore rakasanganiswa, iro rine kukurumidza kukura ecosystem yesoftware uye hardware ML mhinduro (NVIDIA, H2O.ai, Starburst, PerceptiLabs, nezvimwewo). Vamwe vevatengi veRed Hat, vakaita seBMW Group, ExxonMobil nevamwe, vakatoisa ML toolchains uye DevOps maitiro pamusoro pepuratifomu uye ecosystem yavo kuunza zvivakwa zvavo zveML mukugadzira uye nekumhanyisa basa revanoongorora data.
Chimwe chikonzero chatakatanga chirongwa cheOpen Data Hub kuratidza muenzaniso wezvivakwa zvichibva pane akati wandei akavhurika sosi software mapurojekiti uye kuratidza maitiro ekuita kutenderera kwehupenyu hwese kweML mhinduro zvichibva papuratifomu yeOpenShift.
Vhura Data Hub Project
Iyi ipurojekiti yakavhurika sosi inogadzirwa mukati menharaunda inoenderana yekusimudzira uye inoshandisa kutenderera kwakazara kwekushanda - kubva pakurodha nekushandura data rekutanga kusvika kugadzira, kudzidzisa uye kuchengetedza modhi - pakugadzirisa matambudziko eAI / ML uchishandisa midziyo uye Kubernetes paOpenShift. platform. Iyi purojekiti inogona kutorwa sereferensi yekushandisa, muenzaniso wekuvaka yakavhurika AI/ML-se-a-sevhisi mhinduro yakavakirwa paOpenShift uye inoenderana yakavhurika sosi maturusi akadai seTensorflow, JupyterHub, Spark nevamwe. Izvo zvakakosha kuti uzive kuti Red Hat pachayo inoshandisa chirongwa ichi kupa yayo AI/ML masevhisi. Pamusoro pezvo, OpenShift inobatanidza neakakosha software uye hardware ML mhinduro kubva kuNVIDIA, Seldon, Starbust nevamwe vatengesi, zvichiita kuti zvive nyore kuvaka uye kumhanya yako wega muchina kudzidza masisitimu.
Iyo Open Data Hub purojekiti yakatarisana neanotevera mapoka evashandisi uye makesi ekushandisa:
- Muongorori wedata anoda mhinduro yekushandisa mapurojekiti eML, akarongwa segore rine mabasa ekuzvishandira.
- Data Analyst anoda yakanyanya sarudzo kubva ichangoburwa yakavhurika sosi AI/ML maturusi uye mapuratifomu.
- Muongorori wedata anoda kuwana kune data masosi kana mamodheru ekudzidzira.
- Muongorori wedata anoda kuwana zviwanikwa zvekombuta (CPU, GPU, ndangariro).
- Data Analyst anoda kugona kubatana uye kugovera basa nevaunoshanda navo, kugamuchira mhinduro, uye kugadzirisa nekukurumidza iteration.
- Muongorori wedata anoda kudyidzana nevagadziri (uye zvikwata zve devops) kuitira kuti mamodeli ake eML uye mhedzisiro yebasa zviende mukugadzirwa.
- Injiniya yedata inoda kupa muongorori wedatha mukana wekuwana kwakasiyana siyana data masosi uchiteedzera zvinodzora uye kuchengetedza zvinodiwa.
- IT system maneja / mushandisi anoda kugona kudzora zvisingaite hupenyu (kumisikidza, kumisikidza, kusimudzira) yeakavhurika sosi zvikamu uye matekinoroji. Isu tinodawo manejimendi akakodzera uye maturusi ekushandisa.
Iyo Open Data Hub purojekiti inounza pamwechete huwandu hweakavhurika sosi maturusi ekushandisa kutenderera kuzere kweAI/ML mashandiro. Jupyter Notebook inoshandiswa pano sechinhu chikuru chekushanda che data analytics. Iyo kiti yekushandisa inofarirwa zvakanyanya pakati pemasainzi edatha nhasi, uye Vhura Data Hub inovabvumira kugadzira uye kubata Jupyter Notebook nzvimbo dzekushandira vachishandisa yakavakirwa-mukati JupyterHub. Pamusoro pekugadzira uye kuendesa kunze zvinyorwa zveJupyter, iyo Open Data Hub purojekiti zvakare ine akati wandei akagadzirira-akagadzirwa zvinyorwa muchimiro cheAI Library.
Raibhurari iyi muunganidzwa wezvakavhurika-sosi muchina kudzidza zvikamu uye mhinduro dzezvakajairwa zviitiko zvinorerutsa kukurumidza prototyping. JupyterHub inosanganisirwa neOpenShift's RBAC yekuwana modhi, iyo inokutendera kuti ushandise aripo OpenShift maakaundi uye shandisa kusaina kamwechete. Uye zvakare, JupyterHub inopa mushandisi-inoshamwaridzika mushandisi interface inonzi spawner, kuburikidza iyo mushandisi anogona nyore kugadzirisa huwandu hwemakomputa zviwanikwa (CPU cores, memory, GPU) yeyakasarudzwa Jupyter Notebook.
Mushure mekunge muongorori wedata agadzira uye nekugadzirisa iyo laptop, zvimwe zvese zvinonetsa nezvazvo zvinotarisirwa neKubernetes scheduler, inova chikamu cheOpenShift. Vashandisi vanogona chete kuita zviedzo zvavo, kuchengetedza uye kugovera mhedzisiro yebasa ravo. Pamusoro pezvo, vashandisi vepamberi vanogona kuwana zvakananga OpenShift CLI ganda zvakananga kubva kuJupyter zvinyorwa zvekusimudzira Kubernetes primitives yakadai saJobho kana OpenShift mashandiro akadai seTekton kana Knative. Kana pane izvi unogona kushandisa OpenShift's iri nyore GUI, inonzi "OpenShift web console".
Kuenderera mberi kune chinhanho chinotevera, Vhura Data Hub inoita kuti zvikwanise kubata mapaipi edatha. Nokuda kweizvi, chinhu cheCeph chinoshandiswa, icho chinopihwa seS3-inoenderana chinhu chekuchengetedza data. Apache Spark inokutendera kuti utumire data kubva kunze kwekunze kana yakavakirwa-mukati Ceph S3 chengetedzo, uye zvakare inobvumidza iwe kuita yekutanga data shanduko. Apache Kafka inopa manejimendi epamberi emapaipi edata (apo data inogona kutakurwa kakawanda, pamwe nekushandurwa kwedata, kuongorora, uye kushingirira kuita).
Saka, muongorori we data akawana data uye akavaka modhi. Iye zvino ane chishuwo chekugovera mhedzisiro yakawanikwa nevaanoshanda navo kana vanogadzira maapplication, uye kuvapa nemuenzaniso wake pamisimboti yebasa. Izvi zvinoda inference server, uye Vhura Data Hub ine sevha yakadaro, inonzi Seldon uye inokutendera kuti uburitse modhi sebasa reRESTful.
Pane imwe nguva, kune akati wandei mamodheru paSeldon server, uye pane kudikanwa kwekutarisa kuti anoshandiswa sei. Kuti uite izvi, Vhura Data Hub inopa muunganidzwa wemametric akakodzera uye injini yekubika yakavakirwa payakavhurika sosi yekutarisa maturusi Prometheus naGrafana. Nekuda kweizvozvo, tinogashira mhinduro yekutarisa kushandiswa kweAI modhi, kunyanya munzvimbo yekugadzira.
Nenzira iyi, Vhura Data Hub inopa nzira-yakafanana nemakore mukati mese AI/ML lifecycle, kubva pakuwana data uye kugadzirira kusvika kumuenzaniso wekudzidzisa nekugadzira.
Kuisa zvose pamwe chete
Zvino mubvunzo unomuka maitiro ekuronga zvese izvi kune OpenShift maneja. Uye apa ndipo panouya akakosha Kubernetes anoshanda kune Open Data Hub mapurojekiti.
Mushandi uyu anokwenenzvera kuisirwa, kumisikidzwa uye kutenderera kwehupenyu hweOpen Data Hub purojekiti, kusanganisira kutumirwa kwezvishandiso zvambotaurwa zvakaita seJupyterHub, Ceph, Spark, Kafka, Seldon, Prometheus uye Grafana. Iyo Open Data Hub purojekiti inogona kuwanikwa muOpenShift web console, muchikamu chevashandi venharaunda. Nekudaro, iyo OpenShift maneja anogona kutsanangura kuti inoenderana OpenShift mapurojekiti akaiswa muchikamu se "Open Data Hub chirongwa". Izvi zvinoitwa kamwe chete. Mushure meizvi, muongorori wedata anopinda munzvimbo yake yeprojekiti kuburikidza neOpenShift web console uye anoona kuti anowirirana Kubernetes opareta akaiswa uye anowanikwa kumapurojekiti ake. Anobva agadzira Open Data Hub purojekiti muenzaniso nekudzvanya kamwe uye anobva awana maturusi anotsanangurwa pamusoro. Uye izvi zvese zvinogona kugadzirwa mukuwanikwa kwakanyanya uye kukanganisa kushivirira maitiro.
Kana iwe uchida kuyedza iyo Open Data Hub chirongwa chako, tanga nazvo . Ruzivo rwehunyanzvi rweOpen Data Hub architecture inogona kuwanikwa , zvirongwa zvekuvandudza mapurojekiti - . Mune ramangwana, tinoronga kuita kuwedzera kubatanidzwa neKubeflow, kugadzirisa nyaya dzinoverengeka nekudzora data uye chengetedzo, uye zvakare kuronga kubatanidzwa nemirairo-based system Drools uye Optaplanner. Taura maonero ako uye uve mutori wechikamu muchirongwa zvinogoneka papeji .
Kudzokorora: Matambudziko akakura ekuyera ari kudzivirira masangano kuti aone kugona kwakazara kwehungwaru hwekugadzira uye kudzidza muchina. Red Hat OpenShift yagara yakashandiswa zvinobudirira kugadzirisa matambudziko akafanana muindasitiri yesoftware. Iyo Open Data Hub purojekiti, yakaitwa mukati meyakavhurika sosi yekusimudzira nharaunda, inopa dhizaini yekuvaka yekuronga kutenderera kuzere kweAI/ML mashandiro zvichienderana neiyo OpenShift hybrid gore. Tine hurongwa hwakajeka uye hunofunga hwekusimudzira chirongwa ichi, uye isu takazvipira kugadzira nharaunda inoshanda uye ine zvibereko yakaitenderedza yekugadzira yakavhurika mhinduro dzeAI papuratifomu yeOpenShift.
Source: www.habr.com