Maitiro ekukunda kutya uye kutanga kushandisa Azure Machine Kudzidza

Ndinoziva mazhinji maData Scientists - uye ini pamwe ndiri mumwe wavo pachangu - anoshanda paGPU michina, yemuno kana chaiyo, iri mugore, kungave kuburikidza neJupyter Notebook kana kuburikidza neimwe mhando yePython budiriro nharaunda. Kushanda kwemakore maviri senyanzvi yeAI / ML yekuvandudza, ndakaita izvi chaizvo, ndichigadzira data pane sevha yenguva dzose kana nzvimbo yekushanda, uye kudzidzira kudzidzira pamuchina chaiwo une GPU muAzure.

Chokwadi, tose takanzwa nezvazvo Azure Machine Kudzidza -yakakosha gore chikuva chekudzidza muchina. Zvisinei, mushure mekutanga kutarisa zvinyorwa zvenhanganyaya, zvinoita sekunge Azure ML ichakuitira mamwe matambudziko pane yainogadzirisa. Semuenzaniso, mudzidziso yataurwa pamusoro, kudzidziswa paAzure ML kunotangwa kubva kuJupyter Notebook, nepo script yekudzidzisa pachayo ichinzi igadzirwe uye igadziriswe senge faira remeseji mune rimwe masero - asi usingashandisi auto-kupedzisa, syntax. kuratidza, uye zvimwe zvakanaka zveyakajairwa budiriro nharaunda. Nechikonzero ichi, hatina kushandisa zvakanyanya Azure ML mubasa redu kwenguva yakareba.

Nekudaro, ini nguva pfupi yadarika ndakawana nzira yekutanga kushandisa Azure ML zvinobudirira mubasa rangu! Kufarira ruzivo?

Chakavanzika chikuru ndechekuti Visual Studio Code yekuwedzera yeAzure ML. Zvinokutendera kuti ugadzire zvinyorwa zvekudzidzisa muVS Code, uchitora mukana wakazara wezvakatipoteredza - uye unogona kumhanyisa script munharaunda wobva wangotumira kune kudzidziswa muAzure ML cluster nekudzvanya kushoma. Zviri nyore, handizvo here?

Mukuita kudaro, iwe unowana zvinotevera mabhenefiti kubva kushandisa Azure ML:

• Iwe unogona kushanda yakawanda yenguva munharaunda yako pamushini wako mune iri nyore IDE, uye shandisa GPU chete kudzidzisa modhi. Panguva imwecheteyo, dziva rekudzidzira zviwanikwa rinogona kuchinjika kumutoro unodiwa, uye nekuisa huwandu hushoma hwemanodhi kusvika ku0, unogona kutanga otomatiki muchina "pakuda" pamberi pekudzidzira mabasa.
• zvingangokukunda chengeta zvese zvinobuda pakudzidza munzvimbo imwechete, kusanganisira mametrics akawanikwa uye mamodheru anoguma - hapana chikonzero chekuuya neimwe mhando yehurongwa kana kurongeka kuchengetedza zvese zvabuda.
• saka Vanhu vakati wandei vanogona kushanda pachirongwa chimwe chete - vanogona kushandisa iyo yakafanana computing cluster, zvese zviedzo zvichaiswa mumutsara, uye vanogona zvakare kuona mhedzisiro yezviyedzo zveumwe neumwe. Imwe scenario yakadaro kushandisa Azure ML mukudzidzisa Kudzidza Kwakadzikaapo pachinzvimbo chekupa mudzidzi wega wega muchina chaiwo une GPU, unogona kugadzira sumbu rimwe rinozoshandiswa nevese vepakati. Mukuwedzera, tafura yakawanda yemigumisiro ine kuenzanisa kwemuenzaniso inogona kushanda sechinhu chakanaka chemakwikwi.
• NeAzure ML, unogona kuitisa nyore nhevedzano yekuyedza, semuenzaniso, ye hyperparameter optimization - izvi zvinogona kuitwa nemitsara mishoma yekodhi, hapana chikonzero chekuitisa nhevedzano yekuedza nemaoko.

Ndinovimba ndakakukurudzira kuti uedze Azure ML! Heino maitiro ekutanga:

• Ita shuwa kuti waisa Visual Studio Code, pamwe chete nekuwedzera Azure Sign In и AzureML
• Clone iyo repository https://github.com/CloudAdvocacy/AzureMLStarter - ine imwe demo kodhi yekudzidzisa yakanyorwa nemaoko dhijiti yekuzivikanwa modhi pane iyo MNIST dataset.
• Vhura iyo cloned repository muVisual Studio Code.
• Verenga!

Azure ML Workpace uye Azure ML Portal

Azure ML yakarongeka yakatenderedza pfungwa nzvimbo yekushanda - nzvimbo yekushanda. Dhata inogona kuchengetwa munzvimbo yebasa, zviedzo zvinotumirwa kwairi kudzidziswa, mhedzisiro yekudzidziswa inochengeterwawo ipapo - iwo anoguma metrics uye modhi. Iwe unogona kuona zviri mukati menzvimbo yekushanda kuburikidza Azure ML portal - uye kubva ipapo iwe unogona kuita mabasa mazhinji, kubva pakurodha data kusvika kuongororo yekuongorora uye kuendesa mhando.

Iwe unogona kugadzira nzvimbo yekushanda kuburikidza newebhu interface Azure Portal (ona nhanho nhanho mirayiridzo), kana kushandisa iyo Azure CLI yekuraira mutsara (mirayiridzo):

az extension add -n azure-cli-ml
az group create -n myazml -l northeurope
az ml workspace create -w myworkspace -g myazml

Zvakare zvakabatana nenzvimbo yebasa ndezvimwe computing zviwanikwa (Compute) Kana uchinge wagadzira chinyorwa chekudzidzisa modhi, unokwanisa tumira kuedza kuti uite kunzvimbo yekushanda, uye tsanangura compute target - mune iyi kesi, iyo script ichaiswa mukati, kumhanya munzvimbo inodiwa yekombuta, uye ipapo mibairo yese yekuyedza ichachengetwa munzvimbo yebasa kuti iwedzere kuongororwa uye kushandiswa.

Chinyorwa chekudzidza cheMNIST

Funga nezvedambudziko rekirasi Kuzivikanwa kwedhijiti yakanyorwa nemaoko uchishandisa iyo MNIST dataset. Saizvozvo, mune ramangwana, unogona kuita chero yako yekudzidziswa zvinyorwa.

Pane script mune yedu repository train_local.py, iyo yatinodzidzisa iyo yakapfava mutsara regression modhi tichishandisa raibhurari yeSkLearn. Zvechokwadi, ndinonzwisisa kuti iyi haisi iyo nzira yakanakisisa yekugadzirisa dambudziko - tinoishandisa semuenzaniso, seyo nyore.

Iyo script inotanga kudhawunirodha data reMNIST kubva kuOpenML yobva yashandisa kirasi LogisticRegression kudzidzisa modhi, uye wozodhinda iko kunobva kwaitika:

mnist = fetch_openml('mnist_784')
mnist['target'] = np.array([int(x) for x in mnist['target']])

shuffle_index = np.random.permutation(len(mist['data']))
X, y = mnist['data'][shuffle_index], mnist['target'][shuffle_index]

X_train, X_test, y_train, y_test = 
  train_test_split(X, y, test_size = 0.3, random_state = 42)

lr = LogisticRegression()
lr.fit(X_train, y_train)
y_hat = lr.predict(X_test)
acc = np.average(np.int32(y_hat == y_test))

print('Overall accuracy:', acc)

Unogona kumhanyisa script pakombuta yako uye wowana mhedzisiro mumasekonzi mashoma.

Mhanya script muAzure ML

Kana tikamhanyisa script yekudzidzira kuburikidza neAzure ML, isu tichave nemabhenefiti maviri makuru:

• Kumhanya kudzidziswa pane yakasarudzika komputa sosi, iyo, sekutonga, inobudirira kupfuura komputa yemuno. Panguva imwecheteyo, Azure ML pachayo ichatarisira kurongedza script yedu nemafaira ese kubva kune yazvino dhairekitori kupinda mumudziyo wedocker, kuisa zvinodiwa, uye kuitumira kuti iitwe.
• Nyora mhinduro kune imwechete registry mukati meAzure ML workspace. Kuti titore mukana wechinhu ichi, isu tinofanirwa kuwedzera mitsetse miviri yekodhi kune yedu script kurekodha iko kunenge kwaitika:

from azureml.core.run import Run
...
try:    
    run = Run.get_submitted_run()
    run.log('accuracy', acc)
except:
    pass

Iyo inoenderana vhezheni yechinyorwa inonzi train_universal.py (inoti namanomano zvishoma pane zvakanyorwa pamusoro, asi kwete zvikuru). Iyi script inogona kumhanyirwa munzvimbo uye pane iri kure komputa sosi.

Kuti uimhanye muAzure ML kubva kuVS Code, unofanirwa kuita zvinotevera:

1. Ita shuwa kuti Azure Extension yakabatana nekunyorera kwako. Sarudza iyo Azure icon kubva pane menyu kuruboshwe. Kana usina kubatana, chiziviso chichaonekwa mukona yezasi yekurudyi (sezvizvi), nekudzvanya paunogona kupinda kuburikidza nebrowser. Unogonawo kudzvanya Ctrl-Shift-P kufonera VS Code command line, uye nyora Azure Sign In.

2. Mushure meizvozvo, muchikamu cheAzure (icon kuruboshwe), tsvaga chikamu MACHINE KUDZIDZA:

Pano iwe unofanirwa kuona mapoka akasiyana ezvinhu mukati menzvimbo yekushanda: komputa zviwanikwa, zviedzo, nezvimwe.

1. Enda kune rondedzero yemafaira, tinya kurudyi pane script train_universal.py uye sarudza Azure ML: Mhanya sekuyedza muAzure.

1. Izvi zvichateverwa nenhevedzano yenhaurirano munzvimbo yekuraira yeVS Code: simbisa kunyoreswa uye Azure ML nzvimbo yebasa yauri kushandisa, uye sarudza. Gadzira chiedzo chitsva:

1. Sarudza kugadzira chinhu chitsva chekombuta Gadzira New Compute:

   • Compute inosarudza komputa sosi iyo kudzidziswa kuchaitika. Iwe unogona kusarudza komputa yemuno, kana AmlCompute cloud cluster. Ini ndinokurudzira kugadzira scalable cluster yemachina STANDARD_DS3_v2, ine huwandu hushoma hwemakina e0 (uye huwandu hwe1 kana kupfuura, zvichienderana nezvido zvako). Izvi zvinogona kuitwa kuburikidza neVS Code interface, kana kare kuburikidza ML Portal.

   

2. Zvadaro, unofanira kusarudza configuration Compute Configuration, iyo inotsanangura maparamendi emudziyo wakagadzirirwa kudzidziswa, kunyanya, ese maraibhurari anodiwa. Muchiitiko chedu, sezvo tiri kushandisa Scikit Dzidza, tinosarudza SkLearn, uye wobva wangosimbisa rondedzero yakarongwa yemaraibhurari nekudzvanya Enter. Kana iwe ukashandisa mamwe maraibhurari ekuwedzera, anofanirwa kutsanangurwa pano.

   
   

3. Izvi zvinovhura hwindo rine JSON faira rinotsanangura kuyedza. Mariri, unogona kugadzirisa mamwe ma parameters - semuenzaniso, zita rekuedza. Mushure meizvozvo tinya pane link Tumira Kuedza mukati meiyi faira:

1. Mushure mekubudirira kuendesa kuyedza kuburikidza neVS Code, kurudyi rwenzvimbo yekuzivisa, iwe uchaona chinongedzo kune Azure ML Portal, kwaunogona kutarisa mamiriro uye mhedzisiro yekuedza.

Zvadaro, iwe unogona kugara uchiwana muchikamu Ongororo Azure ML Portal, kana kuti muchikamu Azure Machine Kudzidza murondedzero yezviedzo:

1. Kana mushure meizvozvo iwe waita zvigadziriso kune kodhi kana kushandura ma parameter, kutangazve kuedza kuchave nekukurumidza uye nyore. Nekudzvanya-kurudyi pane faira, iwe uchaona chinhu chitsva chemenyu Dzokorora kumhanya kwekupedzisira - ingosarudza, uye kuyedza kunobva kwatanga:

Iwe unogona kugara uchiwana mhedzisiro yemametrics kubva kune ese ekutanga paAzure ML Portal, hapana chikonzero chekuanyora pasi.

Zvino iwe unoziva kuti kumhanya kuyedza neAzure ML kuri nyore uye hakurwadze, uye iwe unowana akati wandei mabhenefiti mukuita kudaro.

Asi iwe unogonawo kuona kusabatsirika. Semuenzaniso, zvakatora nguva yakareba kuti iite script. Ehe, kurongedza script mumudziyo uye nekuitumira pane server kunotora nguva. Kana panguva imwechete iyo cluster yakachekwa kusvika pahukuru hwe0 node, zvinotora nguva yakawanda kutanga iyo chaiyo muchina, uye izvi zvese zvinoonekwa zvakanyanya kana isu tichiyedza mabasa akareruka seMNIST, ayo anogadziriswa mumasekondi mashoma. . Nekudaro, muhupenyu chaihwo, kana kudzidziswa kuchitora maawa akati wandei, kana mazuva kana mavhiki, iyi yakawedzera nguva inova isingakoshi, kunyanya pakatarisana nekumashure kwekuita kwepamusoro kwakanyanya uko komputa komputa inogona kupa.

Chii chinotevera?

Ndinovimba kuti mushure mekuverenga chinyorwa ichi, unogona uye kushandisa Azure ML mubasa rako kumhanyisa zvinyorwa, kubata zviwanikwa zvekombuta, uye chengetedza mhedzisiro pakati. Nekudaro, Azure ML inogona kukupa mamwe mabhenefiti!

Mukati menzvimbo yekushanda, unogona kuchengeta data, nekudaro uchigadzira nzvimbo yepakati yemabasa ako ese, ari nyore kuwana. Mukuwedzera, iwe unogona kumhanyisa zviedzo kwete kubva kuVisual Studio Code, asi uchishandisa API - izvi zvinogona kunyanya kubatsira kana iwe uchida kuita hyperparameter optimization uye uchida kumhanya script kakawanda nemaparamita akasiyana. Zvakare, yakakosha tekinoroji yakavakirwa muAzure ML hyper drive, izvo zvinokutendera kuti uite zvakanyanya kunyengera kutsvaga uye optimization ye hyperparameters. Ini ndichataura pamusoro peizvi zvingaitika mune yangu inotevera post.

Inobatsira Zviwanikwa

Kuti udzidze zvakawanda nezve Azure ML, unogona kuwana anotevera Microsoft Dzidza makosi anobatsira:

• Nhanganyaya kuAzure ML Service
• Kuvaka AI mhinduro neAzure ML Service
• Dzidzisa modhi yemuno neAzure ML Service

Source: www.habr.com