Iyo kodhi yemuchina yekudzidza software zvigadzirwa zvinowanzoomesesa uye zvakanyanya kuvhiringa. Kuona uye kubvisa tsikidzi mariri ibasa rakasimba rekushandisa. Kunyange zviri nyore inoda yakakomba nzira yekuvaka network, kutanga kwehuremu, uye network optimization. Kukanganisa kudiki kunogona kutungamirira kumatambudziko asingafadzi.
Ichi chinyorwa chiri nezve algorithm yekubvisa yako neural network.
Skillbox inokurudzira: Nzira inoshanda .
Tinoyeuchidza: kune vese vaverengi veHabr - a 10 ruble mutengo paunenge uchinyoresa mune chero Skillbox kosi uchishandisa iyo Habr promo kodhi.
Iyo algorithm ine zvikamu zvishanu:
- nyore kutanga;
- kusimbiswa kwekurasikirwa;
- kutarisa mhinduro dzepakati nekubatanidza;
- parameter diagnostics;
- kudzora basa.
Kana chimwe chinhu chichiita sechinonakidza kwauri kupfuura chakasara, unogona kusvetuka kuenda kune izvi zvikamu.
Kutanga nyore
Neural network ine dhizaini yakaoma, kugadzika, uye chiyero chekudzidza chinonetsa kugadzirisa pane chenguva dzose. Isu tiri kuita hunyengeri hudiki pano, sezvo iyo poindi pachayo isina kunangana nekugadzirisa, asi iyi ichiri kurudziro yakakosha.
Kutanga kuri nyore kugadzira modhi yakareruka uye kuidzidzisa pane imwe seti (point) yedata.
Kutanga tinogadzira muenzaniso wakareruka
Kuti titange nekukurumidza, tinogadzira diki network ine imwechete yakavanzika layer uye tarisa kuti zvese zvinoshanda nemazvo. Ipapo isu zvishoma nezvishoma tinoomesa modhi, tichitarisa chimwe nechimwe chitsva chechimiro chayo (yekuwedzera layer, parameter, nezvimwewo), uye enderera mberi.
Isu tinodzidzisa modhi pane imwechete seti (point) yedata
Secheki yekukurumidza huchenjeri hwepurojekiti yako, unogona kushandisa imwe kana maviri mapoinzi edhata yekudzidziswa kuratidza kana sisitimu iri kushanda nemazvo. Iyo neural network inofanirwa kuratidza 100% kunyatsoita mukudzidziswa uye kuyedzwa. Kana zvisiri izvo, saka ingave iyo modhi idiki zvakanyanya kana iwe watove nebug.
Kunyangwe zvese zvakanaka, gadzirira modhi yenguva imwe chete kana kupfuura usati waenderera mberi.
Kurasikirwa kwekuongorora
Kurasikirwa kwekufungidzira ndiyo nzira huru yekukwenenzvera kuita kwemuenzaniso. Iwe unofanirwa kuve nechokwadi chekuti kurasikirwa kwakakodzera kune dambudziko uye kuti mabasa ekurasikirwa akayerwa pachiyero chakakodzera. Kana iwe ukashandisa anopfuura rudzi rumwe rwekurasikirwa, ita shuwa kuti ese akaenzana uye akayerwa nemazvo.
Zvakakosha kuve nehanya nekurasikirwa kwekutanga. Tarisa kuti mhedzisiro chaiyo iri padyo sei kune inotarisirwa mhedzisiro kana modhi yatanga nekungofungidzira. IN : "Iva nechokwadi chekuti unowana mhedzisiro yaunotarisira paunotanga nenhamba shoma yemaparamita. Zviri nani kuti utarise nekukurumidza kurasikirwa kwedata (pamwe nedhigirii rekugara rakaiswa kune zero). Semuyenzaniso, yeCIFAR-10 ine Softmax classifier, isu tinotarisira kurasikirwa kwekutanga kuve 2.302 nekuti inotarisirwa kupararira mukana ndeye 0,1 pakirasi yega yega (sezvo paine makirasi gumi) uye kurasikirwa kweSoftmax ndiko kukanganisa kwakashata kwekirasi yekirasi chaiyo. se -ln (10) = 0.1."
Kumuenzaniso webhinari, kuverenga kwakafanana kunongoitwa kune imwe neimwe yemakirasi. Heino, semuenzaniso, iyo data: 20% 0's uye 80% 1's. Kurasikirwa kwekutanga kunotarisirwa kunosvika -0,2ln (0,5) -0,8ln (0,5) = 0,693147. Kana mhedzisiro yakakura kupfuura 1, inogona kuratidza kuti iyo neural network huremu haina kuenzana zvakanaka kana data haina kujaira.
Kutarisa mhinduro dzepakati uye zvinongedzo
Kuti ugadzirise neural network, zvinodikanwa kuti unzwisise masimba emaitiro mukati metiweki uye basa remunhu epakati akaturikidzana sezvaakabatana. Hezvino zvinowanzoitika kukanganisa kwaungasangana nazvo:
- matauriro asiriwo ekuvandudza zvishoma nezvishoma;
- uremu zvigadziriso hazvishandiswe;
- kuputika gradients.
Kana iyo gradient kukosha iri zero, izvi zvinoreva kuti mwero wekufunda mune optimizer wakanyanya kunonoka, kana kuti uri kusangana nekutaura kusina kunaka kwekugadzirisa gradient.
Uye zvakare, zvinodikanwa kuti utarise kukosha kweiyo activation mabasa, uremu uye zvigadziriso zveimwe neimwe yezvikamu. Semuenzaniso, ukuru hweparameter inogadziridza (zviyero uye kusarura) .
Pane chiitiko chinonzi "Kufa ReLU" kana , apo ReLU neurons ichaburitsa zero mushure mekudzidza yakakura yakaipa bias kukosha kune huremu hwayo. Aya ma neurons haambodzingwa zvakare chero nguva mune data.
Unogona kushandisa gradient kutarisa kuti uone zvikanganiso izvi nekuenzanisa gradient uchishandisa nzira yenhamba. Kana iri padhuze neakaverengerwa magradients, ipapo backpropagation yakaitwa nemazvo. Kuti ugadzire cheki yegradient, tarisa izvi zviwanikwa zvikuru kubva kuCS231 ΠΈ uye zvakare ne Andrew Nga pane iyi nyaya.
inoratidza nzira nhatu huru dzekuona neural network:
- Preliminaries inzira dzakareruka dzinotiratidza chimiro chakazara chemuenzaniso wakadzidziswa. Izvi zvinosanganisira kubuda kwemaumbirwo kana mafirita ega ega zvidimbu zveneural network uye maparamita ari mukati mechikamu chimwe nechimwe.
- Activation based. Mavari, isu tinodudzira ma activation eimwe neuroni kana mapoka eeuroni kuti anzwisise mabasa avo.
- Kubva pane gradients. Idzi nzira dzinowanzo kushandura magradients anoumbwa kubva kumberi nekumashure kupasa kwemuenzaniso kudzidziswa (kusanganisira saliency mepu uye kirasi activation mepu).
Kune akati wandei anobatsira maturusi ekuona ma activation nekubatanidza kwezvikamu zvega ega, semuenzaniso. ΠΈ .
Parameter diagnostics
Neural network ine akawanda ma paramita anodyidzana, izvo zvinokanganisa optimization. Chaizvoizvo, chikamu ichi inyaya yekutsvagisa kwakasimba nenyanzvi, saka mazano ari pasi apa anofanirwa kutariswa sezano chete, kutanga mapoinzi ekuvakira.
Saizi yepakeji (batch saizi) - Kana iwe uchida kuti saizi yebatch ive yakakura zvakakwana kuti uwane chaiyo yekukanganisa gradient fungidziro, asi idiki yakakwana kuti stochastic gradient descent (SGD) igadzirise network yako. Madiki batch saizi anozotungamira kukurumidza kusangana nekuda kweruzha panguva yekudzidziswa uye nekuzotevera kune optimization matambudziko. Izvi zvinotsanangurwa zvakadzama .
Kudzidza mwero - kuderera zvakanyanya kunotungamira kunonoka kusangana kana njodzi yekunamira mune minima yemuno. Panguva imwecheteyo, mwero wekufunda wepamusoro unokonzeresa optimization divergence nekuti iwe uri panjodzi yekusvetuka nepakati yakadzika asi yakamanikana chikamu chekurasikirwa basa. Edza kushandisa kukurumidza kuronga kuti udzikise uchidzidzira neural network. Kusvika parizvino neCS231n .
Gradient clippingβ -kuchekerera parameter gradients panguva yekudzokera kumashure pakukosha kwepamusoro kana kumucheto kwakajairwa. Inobatsira kugadzirisa matambudziko chero ari kuputika gradients aungasangana nawo munhanho yetatu.
Batch normalization - inoshandiswa kugadzirisa iyo data yekuisa yega yega, iyo inotibvumira kugadzirisa dambudziko remukati covariate shift. Kana uri kushandisa Dropout uye Batch Norma pamwechete, .
Stochastic gradient descent (SGD) - kune akati wandei emhando dzeSGD dzinoshandisa kukurumidza, inochinja mareti ekudzidza uye nzira yeNesterov. Nekudaro, hapana kana chimwe chazvo chine mukana wakajeka maererano nezvose zviri zviviri kudzidza kunyatsoita uye generalization ().
Regularization -yakakosha pakuvaka modhi inogoneka, sezvo ichiwedzera chirango chemhando yakaoma kana yakanyanyisa parameter kukosha. Iyi inzira yekudzikisa kusiyana kwemuenzaniso pasina kuwedzera zvakanyanya kurerekera kwayo. Zvimwe .
Kuti uongorore zvese iwe pachako, iwe unofanirwa kudzima nguva dzose uye tarisa iyo data kurasikirwa gradient iwe pachako.
Kudonha kunze ndiyo imwe nzira yekugadzirisa network yako kudzivirira kusangana. Munguva yekudzidziswa, kudonha kunoitwa chete nekuchengetedza chiitiko cheneuron neimwe mukana p (hyperparameter) kana kuimisa kune zero mune yakatarisana. Nekuda kweizvozvo, iyo network inofanirwa kushandisa akasiyana subset yemaparamita kune yega yega batch yekudzidziswa, iyo inoderedza shanduko mune mamwe ma paramita anove makuru.
Zvakakosha: Kana iwe ukashandisa zvese zvekudonhedza uye batch normalization, chenjera nezve marongero eaya maoperation kana kutomboashandisa pamwe chete. Zvose izvi zvichiri kukurukurwa zvakasimba uye zvichiwedzerwa. Hedzino nhaurirano mbiri dzakakosha pamusoro penyaya iyi ΠΈ .
Kudzora kwebasa
Zviri pamusoro kunyora workflows uye kuedza. Kana iwe ukasanyora chero chinhu, unogona kukanganwa, semuenzaniso, kuti ndeapi chiyero chekudzidza kana zviyero zvekirasi zvinoshandiswa. Nekuda kwekudzora, unogona kuona uye kuburitsa zvekare zviedzo zvekare. Izvi zvinokutendera kuti uderedze huwandu hwekuedzerwa kwakafanana.
Nekudaro, zvinyorwa zvemanyorero zvinogona kuve basa rakaoma kana iri vhoriyamu yebasa. Apa ndipo panouya maturusi akaita seComet.ml kuti akubatsire iwe kugadzika dataset, shanduko yekodhi, nhoroondo yekuyedza, uye mhando dzekugadzira, kusanganisira ruzivo rwakakosha nezve modhi yako (hyperparameters, modhi maitiro metrics, uye ruzivo rwezvakatipoteredza).
A neural network inogona kutarisisa kune shanduko diki, uye izvi zvinotungamira mukudonha kwekuita modhi. Kutsvaga uye kunyora basa rako ndiro danho rekutanga raunogona kutora kuti uenzanise nharaunda yako uye modhi.
Ndinovimba iyi positi inogona kuve yekutanga kuti iwe utange kugadzirisa yako neural network.
Skillbox inokurudzira:
- Makore maviri anoshanda kosi .
- Yepamhepo kosi .
- Kosi yegore rimwechete .
Source: www.habr.com