Tsogolo lafika, ndipo matekinoloje opangira nzeru ndi makina ophunzirira makina akugwiritsidwa kale ntchito bwino ndi masitolo omwe mumakonda, makampani oyendetsa komanso minda ya Turkey.
Ndipo ngati chinachake chiripo, ndiye kuti pali kale chinachake pa intaneti ... ntchito yotseguka! Onani momwe Open Data Hub imakuthandizireni kukulitsa matekinoloje atsopano ndikupewa zovuta zomwe mungagwiritse ntchito.
Ndi zabwino zonse zanzeru zopangira (AI) ndi kuphunzira pamakina (ML), mabungwe nthawi zambiri amavutika kukulitsa matekinolojewa. Mavuto akulu pankhaniyi nthawi zambiri amakhala awa:
- Kusinthana chidziwitso ndi mgwirizano - ndizosatheka kusinthanitsa zidziwitso mosavutikira ndikuchita mogwirizana mobwerezabwereza mwachangu.
- Kupeza deta - pa ntchito iliyonse iyenera kumangidwanso mwatsopano komanso pamanja, zomwe zimatenga nthawi yambiri.
- Kufikira pakufunidwa - palibe njira yopezera mwayi wofuna kugwiritsa ntchito zida zophunzirira makina ndi nsanja, komanso zida zamakompyuta.
- Kupanga - zitsanzo zimakhalabe pachiwonetsero ndipo sizimagwiritsidwa ntchito m'mafakitale.
- Tsatani ndi kufotokoza zotsatira za AI - kuchulukitsa, kutsatira ndi kufotokozera zotsatira za AI/ML ndizovuta.
Kusiyidwa, mavutowa amasokoneza liwiro, mphamvu, ndi zokolola za asayansi ofunikira. Izi zimawapangitsa kukhumudwa, kukhumudwa ndi ntchito yawo, ndipo zotsatira zake, ziyembekezo zabizinesi zokhudzana ndi AI/ML zimangowonongeka.
Udindo wothetsera mavutowa ukugwera akatswiri a IT, omwe ayenera kupereka akatswiri a data - ndiko kulondola, chinachake chonga mtambo. Mwatsatanetsatane, timafunikira nsanja yomwe imapereka ufulu wosankha komanso kukhala ndi mwayi, wosavuta. Panthawi imodzimodziyo, imakhala yachangu, yosinthika mosavuta, yowonjezereka pakufunika komanso yosagwirizana ndi zolephera. Kumanga nsanja yotere pa matekinoloje otseguka kumathandiza kupewa kutseka kwa ogulitsa ndikukhalabe ndi mwayi wanthawi yayitali pankhani yowongolera mtengo.
Zaka zingapo zapitazo, china chofananacho chinali kuchitika pakukula kwa ntchito ndikupangitsa kuti ma microservices, mitambo yosakanizidwa, IT automation, ndi njira zofulumira. Kuti athane ndi zonsezi, akatswiri a IT atembenukira ku zotengera, Kubernetes ndi mitambo yotseguka yosakanizidwa.
Izi tsopano zikugwiritsidwa ntchito poyankha zovuta za Al. Ichi ndichifukwa chake akatswiri a IT akumanga nsanja zomwe zimakhala zokhazikika, zomwe zimathandizira kupanga ntchito za AI/ML mkati mwa njira zofulumira, kufulumizitsa zatsopano, ndipo zimamangidwa ndi maso kumtambo wosakanizidwa.
Tidzayamba kupanga nsanja yotere ndi Red Hat OpenShift, nsanja yathu ya Kubernetes yamtambo wosakanizidwa, yomwe ili ndi njira zomwe zikukula mwachangu zamapulogalamu ndi hardware ML (NVIDIA, H2O.ai, Starburst, PerceptiLabs, etc.). Ena mwamakasitomala a Red Hat, monga BMW Gulu, ExxonMobil ndi ena, atumiza kale zida za ML ndi njira za DevOps pamwamba pa nsanja ndi chilengedwe chake kuti abweretse zomanga zawo za ML kuti apange ndikufulumizitsa ntchito ya osanthula deta.
Chifukwa china chomwe tidakhazikitsira pulojekiti ya Open Data Hub ndikuwonetsa chitsanzo cha kamangidwe kotengera mapulojekiti angapo otseguka ndikuwonetsa momwe angagwiritsire ntchito njira yonse ya moyo wa ML potengera nsanja ya OpenShift.
Open Data Hub Project
Iyi ndi pulojekiti yotseguka yomwe imapangidwa mkati mwa gulu lachitukuko lofananira ndikugwiritsa ntchito njira zonse - kuyambira pakutsitsa ndikusintha zomwe zidayambira mpaka kupanga, kuphunzitsa ndi kusunga chitsanzo - pothetsa mavuto a AI / ML pogwiritsa ntchito zida ndi Kubernetes pa OpenShift. nsanja. Pulojekitiyi ikhoza kuonedwa ngati kukhazikitsidwa kwa maumboni, chitsanzo cha momwe mungapangire njira yotseguka ya AI/ML-as-a-service yochokera ku OpenShift ndi zida zotsegulira zotseguka monga Tensorflow, JupyterHub, Spark ndi ena. Ndikofunikira kudziwa kuti Red Hat imagwiritsa ntchito ntchitoyi kuti ipereke ntchito zake za AI/ML. Kuonjezera apo, OpenShift imagwirizanitsa ndi mapulogalamu akuluakulu ndi hardware ML zothetsera kuchokera ku NVIDIA, Seldon, Starbust ndi ogulitsa ena, kuti zikhale zosavuta kupanga ndi kuyendetsa makina anu ophunzirira makina.
Pulojekiti ya Open Data Hub imayang'ana kwambiri magulu otsatirawa a ogwiritsa ntchito ndi milandu yogwiritsira ntchito:
- Katswiri wa data yemwe amafunikira yankho pakukhazikitsa mapulojekiti a ML, okonzedwa ngati mtambo wokhala ndi ntchito zodzipangira okha.
- Katswiri wa Data yemwe amafunikira kusankha kwakukulu kuchokera pazida zaposachedwa za AI/ML ndi nsanja.
- Katswiri wa data yemwe amafunikira mwayi wopeza magwero a data pophunzitsa zitsanzo.
- Katswiri wa data yemwe amafunikira mwayi wogwiritsa ntchito makompyuta (CPU, GPU, memory).
- Katswiri wa Data yemwe amafunikira luso lothandizana nawo ndikugawana ntchito ndi anzawo, kulandira mayankho, ndikusintha kubwereza mwachangu.
- Katswiri wa data yemwe akufuna kuyanjana ndi opanga (ndi magulu a devops) kuti zitsanzo zake za ML ndi zotsatira za ntchito zitheke.
- Katswiri wama data yemwe amayenera kupatsa wowunikira deta kuti azitha kupeza magwero osiyanasiyana a data pomwe akutsatira malamulo ndi chitetezo.
- Woyang'anira kachitidwe ka IT / wogwiritsa ntchito yemwe amafunikira luso lotha kuyendetsa bwino moyo (kukhazikitsa, kukonza, kukweza) kwa magawo otseguka ndi matekinoloje. Timafunikiranso kasamalidwe koyenera ndi zida za quota.
Pulojekiti ya Open Data Hub imabweretsa pamodzi zida zingapo zotseguka kuti zigwiritse ntchito machitidwe a AI/ML. Jupyter Notebook imagwiritsidwa ntchito pano ngati chida chachikulu chothandizira kusanthula deta. Zothandizira ndizodziwika kwambiri pakati pa asayansi a data masiku ano, ndipo Open Data Hub imawalola kupanga ndikuwongolera malo ogwirira ntchito a Jupyter Notebook pogwiritsa ntchito JupyterHub yomangidwa. Kuphatikiza pa kupanga ndi kuitanitsa zolemba za Jupyter, pulojekiti ya Open Data Hub ilinso ndi zolemba zingapo zomwe zidapangidwa kale ngati AI Library.
Laibulale iyi ndi gulu la magawo ophunzirira makina osatsegula komanso mayankho azinthu zomwe zimathandizira kumasulira mwachangu. JupyterHub imaphatikizidwa ndi mtundu wofikira wa OpenShift wa RBAC, womwe umakupatsani mwayi wogwiritsa ntchito maakaunti a OpenShift omwe alipo ndikugwiritsa ntchito kusaina kamodzi. Kuphatikiza apo, JupyterHub imapereka mawonekedwe osavuta ogwiritsa ntchito otchedwa spawner, kudzera momwe wogwiritsa ntchito amatha kukonza mosavuta kuchuluka kwazinthu zamakompyuta (CPU cores, memory, GPU) ya Jupyter Notebook yosankhidwa.
Pambuyo pofufuza deta ndikupanga ndikusintha laputopu, nkhawa zina zonse za izo zimasamalidwa ndi Kubernetes scheduler, yomwe ili gawo la OpenShift. Ogwiritsa ntchito amatha kuchita zoyeserera zawo, kusunga ndikugawana zotsatira za ntchito yawo. Kuphatikiza apo, ogwiritsa ntchito apamwamba amatha kupeza mwachindunji chipolopolo cha OpenShift CLI mwachindunji kuchokera m'mabuku a Jupyter kuti agwiritse ntchito zoyambira za Kubernetes monga Job kapena OpenShift magwiridwe antchito monga Tekton kapena Knative. Kapena pa izi mutha kugwiritsa ntchito GUI yabwino ya OpenShift, yomwe imatchedwa "OpenShift web console".
Kupitilira gawo lotsatira, Open Data Hub imapangitsa kuti zizitha kuyang'anira mapaipi a data. Pachifukwa ichi, chinthu cha Ceph chimagwiritsidwa ntchito, chomwe chimaperekedwa ngati chinthu chosungiramo deta cha S3. Apache Spark imakulolani kuti musunthire deta kuchokera kuzinthu zakunja kapena kusungirako kwa Ceph S3, komanso kukulolani kuti musinthe zoyambira. Apache Kafka imapereka kayendetsedwe kapamwamba ka mapaipi a data (komwe deta imatha kuikidwa kangapo, komanso kusintha kwa deta, kusanthula, ndi kulimbikira ntchito).
Choncho, wofufuza deta adapeza deta ndikumanga chitsanzo. Tsopano ali ndi chikhumbo chogawana zotsatira zomwe adapeza ndi anzake kapena opanga mapulogalamu, ndikuwapatsa chitsanzo chake pa mfundo za ntchito. Izi zimafuna seva yowonetsera, ndipo Open Data Hub ili ndi seva yotereyi, imatchedwa Seldon ndipo imakulolani kufalitsa chitsanzo ngati ntchito ya RESTful.
Panthawi ina, pali mitundu ingapo pa seva ya Seldon, ndipo pamafunika kuwunika momwe amagwiritsidwira ntchito. Kuti mukwaniritse izi, Open Data Hub imapereka mndandanda wazitsulo zoyenera komanso injini yoperekera malipoti kutengera zida zowunikira zomwe zimagwiritsidwa ntchito ndi Prometheus ndi Grafana. Zotsatira zake, timalandira ndemanga kuti tiwunikire kugwiritsa ntchito mitundu ya AI, makamaka pamalo opangira.
Mwanjira iyi, Open Data Hub imapereka njira yofanana ndi mtambo mu moyo wonse wa AI/ML, kuchokera pakupeza deta ndi kukonzekera kupita ku maphunziro achitsanzo ndi kupanga.
Kuziyika zonse pamodzi
Tsopano funso likubuka momwe mungakonzekere zonsezi kwa woyang'anira OpenShift. Ndipo apa ndipamene wogwiritsa ntchito wapadera wa Kubernetes wama projekiti a Open Data Hub amalowa.
Wogwiritsa ntchitoyu amayang'anira kuyika, kukonzanso ndikusintha moyo wa pulojekiti ya Open Data Hub, kuphatikiza kuyika zida zomwe tatchulazi monga JupyterHub, Ceph, Spark, Kafka, Seldon, Prometheus ndi Grafana. Pulojekiti ya Open Data Hub ingapezeke mu OpenShift web console, mu gawo la ogwira ntchito m'deralo. Chifukwa chake, woyang'anira OpenShift atha kufotokoza kuti ma projekiti ofananira a OpenShift amagawidwa ngati "Open Data Hub project". Izi zimachitika kamodzi. Pambuyo pake, wowunikira deta amalowa mu malo ake a polojekiti kudzera pa OpenShift web console ndikuwona kuti wogwiritsa ntchito Kubernetes wogwirizana waikidwa ndipo akupezeka pa ntchito zake. Kenako amapanga projekiti ya Open Data Hub ndikudina kamodzi ndipo nthawi yomweyo amatha kugwiritsa ntchito zida zomwe zafotokozedwa pamwambapa. Ndipo zonsezi zitha kukhazikitsidwa pakupezeka kwakukulu komanso kulekerera zolakwika.
Ngati mukufuna kuyesa polojekiti ya Open Data Hub nokha, yambani . Zambiri zamakina a Open Data Hub zitha kupezeka , mapulani a chitukuko cha polojekiti - . M'tsogolomu, tikukonzekera kukhazikitsa kugwirizanitsa kwina ndi Kubeflow, kuthetsa nkhani zingapo zokhudzana ndi kayendetsedwe ka deta ndi chitetezo, komanso kukonzekera kugwirizanitsa ndi machitidwe oyendetsera malamulo a Drools ndi Optaplanner. Fotokozani maganizo anu ndikukhala nawo mbali mu polojekitiyi zotheka pa tsamba .
Kubwerezanso: Mavuto akulu akulepheretsa mabungwe kuzindikira kuthekera konse kwanzeru zopangira komanso kuphunzira pamakina. Red Hat OpenShift yakhala ikugwiritsidwa ntchito bwino kuthetsa mavuto omwewo mumakampani opanga mapulogalamu. Pulojekiti ya Open Data Hub, yomwe yakhazikitsidwa mkati mwa gulu lotseguka lachitukuko, imapereka kasamalidwe kazinthu zokonzekera zochitika zonse za AI/ML kutengera mtambo wosakanizidwa wa OpenShift. Tili ndi dongosolo lomveka bwino komanso loganiza bwino lachitukuko cha polojekitiyi, ndipo tili ndi chidwi chofuna kupanga gulu lachangu komanso lobala zipatso mozungulira popanga mayankho otseguka a AI papulatifomu ya OpenShift.
Source: www.habr.com