Ikhodi yemikhiqizo yesoftware yokufunda ngomshini imvamisa iyinkimbinkimbi futhi iyadida. Ukuthola nokuqeda iziphazamisi kuwo kuwumsebenzi odinga izinsiza. Ngisho elula kakhulu zidinga indlela ejulile yokwakhiwa kwenethiwekhi, ukuqaliswa kwezisindo, nokuthuthukisa inethiwekhi. Iphutha elincane lingaholela ezinkingeni ezingemnandi.
Lesi sihloko simayelana ne-algorithm yokulungisa amanethiwekhi akho e-neural.
I-Skillbox iyancoma: Isifundo esiwusizo .
Siyakukhumbuza: kubo bonke abafundi be-"Habr" - isaphulelo sama-ruble angu-10 lapho ubhalisa kunoma yisiphi isifundo se-Skillbox usebenzisa ikhodi yephromoshini ethi "Habr".
I-algorithm inezigaba ezinhlanu:
- isiqalo esilula;
- ukuqinisekiswa kokulahlekelwa;
- ukuhlola imiphumela emaphakathi kanye nokuxhumana;
- ukuxilongwa kwepharamitha;
- ukulawula umsebenzi.
Uma okuthile kubonakala kuthakazelisa kakhulu kuwe kunabanye, ungakwazi ukweqela ngokushesha kulezi zigaba.
Isiqalo esilula
Inethiwekhi ye-neural enezakhiwo eziyinkimbinkimbi, ukujwayela, kanye nesihleli sezinga lokufunda kunzima kakhulu ukuyisusa kunesivamile. Siyakhohlisa lapha, njengoba iphuzu ngokwalo lihlobene ngokungaqondile nokulungisa iphutha, kodwa lokhu kuseyisincomo esibalulekile.
Isiqalo esilula siwukwenza imodeli eyenziwe lula futhi uyiqeqeshe kusethi eyodwa (iphuzu) ledatha.
Okokuqala sakha imodeli eyenziwe lula
Ukuze uqalise ngokushesha, sakha inethiwekhi encane enesendlalelo esisodwa esifihliwe futhi sihlole ukuthi konke kusebenza kahle. Bese sihlanganisa kancane kancane imodeli, sibheka isici esisha ngasinye sesakhiwo sayo (ungqimba olwengeziwe, ipharamitha, njll.), bese uqhubekela phambili.
Siqeqesha imodeli kusethi eyodwa (iphoyinti) ledatha
Njengesheke elisheshayo lephrojekthi yakho, ungasebenzisa iphuzu ledatha elilodwa noma amabili ukuze uqeqeshwe ukuze uqinisekise ukuthi isistimu isebenza kahle yini. Inethiwekhi ye-neural kufanele ibonise ukunemba okungu-100% ekuqeqeshweni nasekuhlolweni. Uma kungenjalo, khona-ke imodeli incane kakhulu noma usuvele unesiphazamisi.
Noma ngabe konke kuhamba kahle, lungiselela imodeli yenkathi eyodwa noma ngaphezulu ngaphambi kokuthi uqhubeke.
Ukuhlolwa kokulahlekelwa
Isilinganiso sokulahlekelwa kuyindlela eyinhloko yokwenza ngcono ukusebenza kwemodeli. Udinga ukuqinisekisa ukuthi ukulahlekelwa kuyilungele inkinga nokuthi imisebenzi yokulahlekelwa ilinganiselwe esikalini esifanele. Uma usebenzisa ngaphezu kohlobo olulodwa lokulahlekelwa, qiniseka ukuthi zonke zilandelana futhi zikalwa ngendlela efanele.
Kubalulekile ukunaka ukulahlekelwa kokuqala. Hlola ukuthi umphumela wangempela usondele kangakanani kumphumela olindelekile uma imodeli iqale ngokuqagela okungahleliwe. IN : βQiniseka ukuthi uthola umphumela owulindele uma uqala ngenani elincane lamapharamitha. Kungcono ukuthi uhlole ngokushesha ukulahleka kwedatha (ngezinga lokujwayelekile elibekwe ku-zero). Isibonelo, ku-CIFAR-10 ngesigaba se-Softmax, silindele ukulahlekelwa kokuqala kube ngu-2.302 ngoba amathuba okusabalalisa alindelekile ngu-0,1 ekilasini ngalinye (njengoba kunamakilasi angu-10) futhi ukulahlekelwa kwe-Softmax amathuba elogi angalungile ekilasi elilungile. njengoba β ln (0.1) = 2.302.β
Ngesibonelo sakanambambili, ukubala okufanayo kwenzelwa ikilasi ngalinye. Nansi, isibonelo, idatha: 20% 0 kanye 80% 1's. Ukulahlekelwa kokuqala okulindelekile kuzoba ku--0,2ln (0,5) -0,8ln (0,5) = 0,693147. Uma umphumela umkhulu kuno-1, ungase ubonise ukuthi izisindo zenethiwekhi ye-neural azibhalansi kahle noma idatha ayijwayelekanga.
Ihlola imiphumela emaphakathi noxhumo
Ukuze ulungise iphutha lenethiwekhi ye-neural, kuyadingeka ukuqonda ukuguquguquka kwezinqubo ngaphakathi kwenethiwekhi kanye nendima yezendlalelo ezimaphakathi ngazinye njengoba zixhumekile. Nawa amaphutha ajwayelekile ongahlangana nawo:
- izisho ezingalungile zokuvuselela kancane kancane;
- izibuyekezo zesisindo azisetshenziswa;
- ama-gradient aqhumayo.
Uma amanani egrediyenti enguziro, lokhu kusho ukuthi izinga lokufunda kusilungiseleli lihamba kancane kakhulu, noma ukuthi uhlangabezana nenkulumo engalungile yokubuyekeza ukugreda.
Ngaphezu kwalokho, kuyadingeka ukuqapha amanani wemisebenzi yokuvula, izisindo kanye nezibuyekezo zesendlalelo ngasinye. Isibonelo, ubukhulu bezibuyekezo zepharamitha (izisindo nokuchema) .
Kukhona into ebizwa ngokuthi "Dying ReLU" noma , lapho ama-neuron e-ReLU azokhipha uziro ngemva kokufunda inani elikhulu elibi elibi lokukhetha izisindo zalo. Lawa ma-neurons awaphinde axoshwe nganoma isiphi isikhathi kudatha.
Ungasebenzisa ukuhlola i-gradient ukuze uhlonze lawa maphutha ngokusondeza i-gradient usebenzisa indlela yezinombolo. Uma iseduze nama-gradients abaliwe, khona-ke ukusakazwa kwe-backpropagation kwasetshenziswa ngendlela efanele. Ukuze udale isheke le-gradient, hlola lezi zinsiza ezinhle ezivela ku-CS231 ΠΈ futhi futhi nge Andrew Nga kulesi sihloko.
ikhombisa izindlela ezintathu eziyinhloko zokubona ngeso lengqondo inethiwekhi ye-neural:
- Ukuqala kuyizindlela ezilula ezisibonisa isakhiwo esijwayelekile semodeli eqeqeshiwe. Lokhu kufaka phakathi okukhiphayo komumo noma izihlungi zezendlalelo ngazinye zenethiwekhi ye-neural namapharamitha angaphakathi kwesendlalelo ngasinye.
- Ukuqalisa kusekelwe. Kuzo, sicacisa ukusebenza kwama-neurons ngamanye noma amaqembu ama-neuron ukuqonda imisebenzi yawo.
- Ngokusekelwe kuma-gradients. Lezi zindlela zivame ukukhohlisa ama-gradients akheke kusuka kumaphasi aya phambili nangemuva okuqeqeshwa okuyimodeli (okuhlanganisa amamephu ahlakaniphile kanye namamephu okuqalisa ekilasi).
Kunamathuluzi amaningana awusizo okubona ngeso lengqondo ukwenza kusebenze nokuxhumana kwezendlalelo ngazinye, isb. ΠΈ .
Ukuxilongwa kwepharamitha
Amanethiwekhi e-Neural anamapharamitha amaningi asebenzisanayo, okwenza kube nzima ukwenza kahle. Empeleni, lesi sigaba siyisihloko socwaningo olusebenzayo lochwepheshe, ngakho-ke iziphakamiso ezingezansi kufanele zibhekwe njengeseluleko kuphela, amaphuzu okuqala okufanele kwakhiwe kuwo.
Usayizi wephakheji (usayizi weqoqo) - Uma ufuna usayizi weqoqo ukuthi ube mkhulu ngokwanele ukuze uthole izilinganiso ezinembile ze-gradient, kodwa ube mncane ngokwanele ukwehla kwe-stochastic gradient (SGD) ukuze wenze inethiwekhi yakho ibe ngokujwayelekile. Amasayizi amaqoqo amancane azoholela ekuhlanganeni ngokushesha ngenxa yomsindo phakathi nenqubo yokuqeqeshwa kanye nobunzima bokwenza kahle. Lokhu kuchazwa ngokuningiliziwe .
Izinga lokufunda - okuphansi kakhulu kuzoholela ekuhlanganeni kancane noma engcupheni yokubhajwa ku-minima yendawo. Ngesikhathi esifanayo, izinga lokufunda eliphezulu lizodala ukwehluka kokufaneleka ngoba usengozini yokweqa engxenyeni ejulile kodwa ewumngcingo yomsebenzi wokulahlekelwa. Zama ukusebenzisa ukuhlela isivinini ukuze unciphise ngenkathi uqeqesha inethiwekhi ye-neural. Kusesikhathini samanje nge-CS231n .
Ukusika i-Gradientβ - ukusika ama-gradients wepharamitha ngesikhathi sokusakazwa emuva ngenani eliphakeme kakhulu noma inkambiso eseceleni. Iwusizo ekuxazululeni inkinga noma imaphi ama-gradient aqhumayo ongahlangana nawo ephuzwini lesithathu.
Ukujwayela kweqoqo - isetshenziselwa ukwenza kube ngokwejwayelekile idatha yokufaka yesendlalelo ngasinye, esivumela ukuthi sixazulule inkinga ye-covariate shift yangaphakathi. Uma usebenzisa i-Dropout ne-Batch Norma ndawonye, .
Ukwehla kwe-Stochastic gradient (SGD) - kunezinhlobo eziningana ze-SGD ezisebenzisa umfutho, amazinga okufunda aguquguqukayo kanye nendlela ye-Nesterov. Kodwa-ke, akukho neyodwa yazo enenzuzo ecacile mayelana nokusebenza kahle kokufunda kanye nokujwayelekile ().
Ukwenziwa njalo - kubalulekile ekwakheni imodeli ejwayelekile, njengoba yengeza inhlawulo yemodeli eyinkimbinkimbi noma amanani epharamitha eqisayo. Lena indlela yokunciphisa ukuhlukahluka kwamamodeli ngaphandle kokwandisa kakhulu ukuchema kwayo. Okuningi .
Ukuze uhlole yonke into ngokwakho, udinga ukukhubaza ukujwayela futhi uhlole i-gradient yokulahleka kwedatha ngokwakho.
Ukuphuma enye indlela yokwenza inethiwekhi yakho ibe lula ukuvikela ukuminyana. Ngesikhathi sokuqeqeshwa, ukuyeka esikoleni kwenziwa kuphela ngokugcina umsebenzi we-neuron ngamathuba athile okuthi p (hyperparameter) noma ukuyibeka ku-zero kwelinye icala. Ngenxa yalokho, inethiwekhi kufanele isebenzise isethi engaphansi ehlukile yemingcele kuqeqebana ngalinye lokuqeqeshwa, okunciphisa izinguquko kumapharamitha athile abusayo.
Okubalulekile: Uma usebenzisa kokubili ukuyeka ukuphuma kanye ne-batch normalization, qaphela mayelana nokuhleleka kwale misebenzi noma ukukusebenzisa ndawonye. Konke lokhu kusaxoxwa ngakho futhi kwengezwe. Nazi izingxoxo ezimbili ezibalulekile ngalesi sihloko ΠΈ .
Ukulawula umsebenzi
Imayelana nokubhala ukugeleza komsebenzi nezivivinyo. Uma ungabhali lutho, ungase ukhohlwe, isibonelo, ukuthi yiliphi izinga lokufunda noma izisindo zekilasi ezisetshenziswayo. Ngenxa yokulawula, ungakwazi ukubuka kalula futhi ukhiqize kabusha imilingo yangaphambilini. Lokhu kukuvumela ukuthi wehlise inani lokuhlolwa okuyimpinda.
Kodwa-ke, imibhalo eyenziwa ngesandla ingaba umsebenzi onzima uma kunomthamo omkhulu womsebenzi. Yilapho amathuluzi afana ne-Comet.ml efika khona ukuze akusize ungene ngokuzenzakalelayo amasethi edatha, izinguquko zekhodi, umlando wokuhlola, namamodeli okukhiqiza, okuhlanganisa ulwazi olubalulekile mayelana nemodeli yakho (ama-hyperparameter, ama-metrics wokusebenza kwemodeli, nolwazi lwendawo).
Inethiwekhi ye-neural ingaba bucayi kakhulu ezinguqukweni ezincane, futhi lokhu kuzoholela ekwehleni kokusebenza kwemodeli. Ukulandelela nokubhala umsebenzi wakho kuyisinyathelo sokuqala ongasithatha ukuze ulinganise indawo yakho kanye nokumodela.
Ngethemba ukuthi lokhu okuthunyelwe kungaba yisiqalo sokuthi uqale ukulungisa iphutha lenethiwekhi yakho ye-neural.
I-Skillbox iyancoma:
- Isifundo sokwenza iminyaka emibili .
- Inkambo eku-inthanethi .
- Isifundo sonyaka esisebenzayo .
Source: www.habr.com