Miyezi ingapo yapitayo, anzathu ochokera ku Google pa Kaggle mpikisano kuti apange gulu la zithunzi zomwe zapezedwa modabwitsa "Mwamsanga, Jambulani!" Gulu, lomwe linaphatikizapo mapulogalamu a Yandex Roman Vlasov, adatenga malo achinayi pampikisano. Pa maphunziro ophunzirira makina a Januwale, Roman adagawana malingaliro a gulu lake, kukhazikitsidwa komaliza kwa gulu, ndi machitidwe osangalatsa a adani ake.
- Moni nonse! Dzina langa ndine Roma Vlasov, lero ndikuwuzani za Quick, Draw! Doodle Recognition Challenge.
Pagulu lathu panali anthu asanu. Ndinalowa nawo nthawi yophatikiza isanakwane. Tinali ndi mwayi, tinagwedezeka pang'ono, koma tinagwedezeka kuchoka pa malo a ndalama, ndipo iwo anagwedezeka kuchoka pa malo a golide. Ndipo ife tinatenga malo olemekezeka achinayi.
(Panthawi ya mpikisano, matimuwo adadziwonera okha mulingo, womwe udapangidwa potengera zotsatira zomwe zawonetsedwa pagawo limodzi lazosankha zomwe zaperekedwa. Mayeso omaliza adapangidwanso mbali ina ya dataset. Izi zimachitika motere. Chifukwa chake, pomaliza, posinthana pakati pa mavoti, maudindo amagwedezeka pang'ono (kuchokera ku Chingerezi kugwedezeka - kusakaniza): pazidziwitso zina, zotsatira zake zitha kupezeka. kuti tisiyane.timu ya Roman inali yoyamba pa top XNUMX.Panthawiyi atatu otsogola ndi ndalama,monetary rating zone poti malo atatu oyamba okha ndi omwe adapatsidwa mphoto yandalama.Kugwedezana kutatha,timu idalowa kale. malo achinayi. Momwemonso, gulu lina linataya chigonjetso, malo agolide. - Mkonzi.)
Mpikisanowo unali wofunikanso chifukwa Evgeniy Babakhnin analandira agogo, Ivan Sosin analandira mbuye, Roman Soloviev anakhalabe agogo, Alex Parinov analandira mbuye, ndinakhala katswiri, ndipo tsopano ndine mbuye.
Kodi Quick, Draw ndi chiyani? Iyi ndi ntchito yochokera ku Google. Google inali ndi cholinga cholengeza AI ndipo ndi ntchitoyi ikufuna kuwonetsa momwe ma neural network amagwirira ntchito. Mukapita kumeneko, dinani Tiyeni jambulani, ndipo tsamba latsopano limatuluka pomwe mwauzidwa: jambulani zigzag, muli ndi masekondi 20 kuti muchite izi. Mukuyesera kujambula zigzag mumasekondi 20, monga apa, mwachitsanzo. Mukachita bwino, maukonde amati ndi zigzag ndipo mumapitilira. Pali zithunzi zisanu ndi chimodzi zokha zoterozo.
Ngati netiweki ya Google yalephera kuzindikira zomwe mwajambula, mtanda udayikidwa pa ntchitoyi. Pambuyo pake ndikuwuzani zomwe zidzatanthauze mtsogolo ngati chojambula chimadziwika ndi maukonde kapena ayi.
Utumikiwu unasonkhanitsa ogwiritsa ntchito ambiri, ndipo zithunzi zonse zomwe ogwiritsa ntchito adajambula zidalowetsedwa.
Tinatha kusonkhanitsa zithunzi pafupifupi 50 miliyoni. Kuchokera apa, tsiku la sitima ndi mayeso a mpikisano wathu zinapangidwa. Mwa njira, kuchuluka kwa data pamayeso ndi kuchuluka kwa makalasi kumawonetsedwa molimba mtima pazifukwa. Ndikuuzani za iwo posachedwa.
Mawonekedwe a data anali motere. Izi sizithunzi za RGB zokha, koma, kunena pang'ono, chipika cha zonse zomwe wogwiritsa ntchito adachita. Mawu ndiye chandamale chathu, khodi ya dziko ndi komwe wolemba doodle akuchokera, sitepe yanthawi ndi nthawi. Chizindikiro chodziwika chimangowonetsa ngati netiweki idazindikira chithunzicho kuchokera ku Google kapena ayi. Ndipo chojambulacho chokha ndikutsatizana, kuyerekezera kokhota komwe wogwiritsa ntchito amakoka ndi mfundo. Ndipo nthawi. Iyi ndi nthawi kuyambira poyambira kujambula chithunzichi.
Deta inaperekedwa m'njira ziwiri. Iyi ndi njira yoyamba, ndipo yachiwiri ndi yosavuta. Amadula nthawi kuchokera pamenepo ndikuyerekeza magawo awa ndi mfundo zazing'ono. Kwa izi adagwiritsa ntchito . Muli ndi mfundo zazikulu zomwe zimangoyerekeza mzere wowongoka, koma kwenikweni mutha kuyerekeza mzerewu ndi mfundo ziwiri zokha. Ili ndiye lingaliro la algorithm.
Deta inagawidwa motere. Zonse ndi zofanana, koma pali zina zakunja. Pamene tinathetsa vutolo, sitinayangβane. Chachikulu ndichakuti panalibe makalasi omwe anali ochepa kwenikweni, sitinayenera kuchita ma samplers olemetsa komanso kuchulukitsa kwa data.
Kodi zithunzizo zinkawoneka bwanji? Ili ndi gulu la "ndege" ndi zitsanzo kuchokera pamenepo zokhala ndi zilembo zozindikirika komanso zosazindikirika. ChiΕ΅erengero chawo chinali penapake mozungulira 1 mpaka 9. Monga mukuonera, deta imakhala yaphokoso. Ndikuganiza kuti ndi ndege. Ngati muyang'ana osadziwika, nthawi zambiri ndi phokoso chabe. Winawake adayesa kulemba "ndege," koma mwachiwonekere mu French.
Ambiri omwe adatenga nawo mbali adangotenga ma gridi, kujambula zambiri kuchokera pamndandanda uwu ngati zithunzi za RGB, ndikuziponya pa intaneti. Ndinajambula pafupifupi mofanana: ndinatenga phale lamitundu, ndinajambula mzere woyamba ndi mtundu umodzi, womwe unali kumayambiriro kwa phale ili, mzere wotsiriza ndi wina, womwe unali kumapeto kwa phale, ndi pakati pawo. Ndinamasulira paliponse pogwiritsa ntchito phale ili. Mwa njira, izi zidapereka zotsatira zabwinoko kuposa ngati mujambula ngati pazithunzi zoyambira - zakuda.
Mamembala ena amagulu, monga Ivan Sosin, adayesa njira zosiyana zojambulira. Ndi tchanelo chimodzi anangojambula chithunzi cha imvi, ndi tchanelo china anajambula sitiroko iliyonse ndi gradient kuyambira koyambirira mpaka kumapeto, kuyambira pa 32 mpaka 255, ndipo ndi njira yachitatu anajambula mikwingwirima yonse kuyambira 32 mpaka 255.
Chinthu chinanso chosangalatsa ndichakuti Alex Parinov adayika zambiri pa intaneti pogwiritsa ntchito codecode.
Metric yomwe imagwiritsidwa ntchito pampikisano ndi Mean Average Precision. Kodi tanthauzo la metric iyi pa mpikisano ndi chiyani? Mungathe kupereka ma predics atatu, ndipo ngati palibe predic yolondola mu zitatuzi, ndiye kuti mumapeza 0. Ngati pali cholondola, ndiye kuti dongosolo lake likuganiziridwa. Ndipo zotsatira zomwe mukufuna zidzawerengedwa ngati 1 yogawidwa ndi dongosolo la kulosera kwanu. Mwachitsanzo, munapanga zolosera zitatu, ndipo zolondola ndizoyamba, ndiye mumagawaniza 1 ndi 1 ndikupeza 1. Ngati cholozeracho chiri cholondola ndipo dongosolo lake ndi 2, ndiye gawani 1 ndi 2, mumapeza 0,5. Chabwino, etc.
Ndi preprocessing deta - momwe kujambula zithunzi ndi zina zotero - tasankha pang'ono. Kodi tidagwiritsa ntchito zomanga ziti? Tidayesa kugwiritsa ntchito zomanga zamafuta monga PNASNet, SENet, ndi zomanga zakale monga SE-Res-NeXt, akulowa nawo mipikisano yatsopano. Panalinso ResNet ndi DenseNet.
Tinaphunzitsa bwanji zimenezi? Mitundu yonse yomwe tidatenga idaphunzitsidwa kale pa imagenet. Ngakhale pali zambiri, zithunzi 50 miliyoni, komabe, ngati mutenga netiweki yophunzitsidwa kale pa imagenet, idawonetsa zotsatira zabwinoko kuposa mutangoyiphunzitsa kuyambira poyambira.
Kodi tinkagwiritsa ntchito njira zotani zophunzitsira? Uku ndi Cosing Annealing ndi Warm Restarts, zomwe ndikambirana pambuyo pake. Iyi ndi njira yomwe ndimagwiritsa ntchito pafupifupi mipikisano yanga yaposachedwa, ndipo ndimakhala ndikuphunzitsa ma gridi bwino, kuti ndikwaniritse zochepa.
Kenako Chepetsani Kuchuluka kwa Maphunziro pa Plateau. Mumayamba kuphunzitsa maukonde, kukhazikitsa mlingo wina wophunzirira, pitirizani kuphunzitsa, ndipo kutayika kwanu pang'onopang'ono kumasintha kukhala phindu linalake. Mumayang'ana izi, mwachitsanzo, kwa zaka khumi kutayika sikunasinthe nkomwe. Mumachepetsa kuchuluka kwa maphunziro anu ndi mtengo wake ndikupitiriza kuphunzira. Zimatsikanso pang'ono, zimasintha pang'ono, ndipo mumatsitsanso kuchuluka kwa maphunziro, ndi zina zotero, mpaka maukonde anu asintha.
Chotsatira ndi njira yosangalatsa: Osawononga kuchuluka kwa maphunziro, onjezani kukula kwa batch. Pali nkhani yomwe ili ndi dzina lomweli. Mukamaphunzitsa netiweki, simuyenera kuchepetsa kuchuluka kwa maphunziro, mutha kungowonjezera kukula kwa batch.
Njira iyi, mwa njira, idagwiritsidwa ntchito ndi Alex Parinov. Adayamba ndi gulu lofanana ndi 408, ndipo maukonde ake atafika pamtunda, adangowonjezera kukula kwa batch, ndi zina zambiri.
Ndipotu, sindikukumbukira kuti kukula kwake kwa batch kunafika pamtengo wotani, koma chochititsa chidwi ndi chakuti panali magulu a Kaggle omwe ankagwiritsa ntchito njira yomweyo, kukula kwawo kwa batch kunali pafupifupi 10000 XNUMX. Mwa njira, ndondomeko zamakono zophunzirira mozama, monga. PyTorch, mwachitsanzo, imakulolani kuti muchite izi mosavuta. Mumapanga batch yanu ndikuyipereka ku netiweki osati momwe ilili, yonse, koma mugawane m'magulu kuti igwirizane ndi khadi yanu ya kanema, kuwerengera ma gradients, ndipo mutatha kuwerengera kuchuluka kwa gulu lonse, sinthani. zolemera.
Mwa njira, kukula kwakukulu kwa batch kunkaphatikizidwabe pampikisanowu, chifukwa deta inali yaphokoso, ndipo kukula kwakukulu kwa batch kunakuthandizani kuti mufanane ndi gradient molondola.
Kulemba kwabodza kunagwiritsidwanso ntchito, komwe kumagwiritsidwa ntchito kwambiri ndi Roman Soloviev. Anayesa pafupifupi theka la data kuchokera ku mayeso m'magulu, ndikuphunzitsa gululi pamagulu oterowo.
Kukula kwa zithunzi kunali kofunikira, koma chowonadi ndi chakuti muli ndi data yambiri, muyenera kuphunzitsa kwa nthawi yayitali, ndipo ngati chithunzi chanu ndi chachikulu, ndiye kuti mudzaphunzitsidwa kwa nthawi yayitali kwambiri. Koma izi sizinawonjezeke kwambiri pagulu lanu lomaliza, chifukwa chake kunali koyenera kugwiritsa ntchito mtundu wina wamalonda. Ndipo tinangoyesa zithunzi zomwe sizinali zazikulu kwambiri kukula kwake.
Kodi zonsezi zinaphunziridwa bwanji? Choyamba, zithunzi zazing'ono zidatengedwa, nthawi zingapo zidayendetsedwa pa iwo, izi zidatenga nthawi yayitali. Kenako zithunzi zazikuluzikulu zidaperekedwa, maukondewo adaphunzitsidwa, ndiye mochulukirapo, mochulukirapo, kuti musawaphunzitse kuyambira pachiwonetsero komanso osataya nthawi yambiri.
Za optimizers. Tinagwiritsa ntchito SGD ndi Adam. Mwa njira iyi, zinali zotheka kupeza chitsanzo chimodzi, chomwe chinapereka liwiro la 0,941-0,946 pa bolodi la anthu, zomwe ziri zabwino kwambiri.
Mukaphatikiza mitundu mwanjira ina, mupeza penapake mozungulira 0,951. Mukagwiritsa ntchito njira inanso, mupeza zomaliza za 0,954 pagulu la anthu onse, monga momwe tachitira. Koma zambiri pambuyo pake. Kenako ndikuwuzani momwe tidasonkhanitsira zitsanzozo, komanso momwe tidakwanitsira kukwaniritsa liwiro lomaliza.
Chotsatira ndikufuna kunena za Cosing Annealing ndi Warm Restarts kapena Stochastic Gradient Descent with Warm Restarts. Kunena mwachidule, kwenikweni, mutha kugwiritsa ntchito chowonjezera chilichonse, koma mfundo ndi iyi: ngati mungophunzitsa netiweki imodzi ndipo pang'onopang'ono imasinthasintha pang'ono, ndiye kuti zonse zili bwino, mupeza netiweki imodzi, imapangitsa zolakwika zina, koma inu. akhoza kuphunzitsa mosiyana pang'ono. Mudzakhazikitsa mlingo wophunzirira, ndikutsitsa pang'onopang'ono molingana ndi ndondomekoyi. Mumatsitsa, maukonde anu amafika pang'onopang'ono, ndiye kuti mumasunga zolemera, ndikuyikanso mlingo wophunzirira womwe unali kumayambiriro kwa maphunziro, potero kupita kwinakwake m'mwamba kuchokera ku izi, ndikuchepetsanso maphunziro anu.
Chifukwa chake, mutha kuyendera zochepera zingapo nthawi imodzi, momwe kutayika kwanu kudzakhala, kuphatikiza kapena kuchotsera, chimodzimodzi. Koma zoona zake n'zakuti maukonde ndi zolemera izi adzapereka zolakwika zosiyanasiyana pa tsiku lanu. Mukawawerengera, mupeza mtundu wina wakuyerekeza, ndipo liwiro lanu lidzakhala lalitali.
Za momwe tinasonkhanitsira zitsanzo zathu. Kumayambiriro kwa ulaliki, ndinati kulabadira kuchuluka kwa deta mu mayeso ndi chiwerengero cha makalasi. Ngati muwonjezera 1 ku chiwerengero cha zolinga zomwe mumayesedwa ndikugawanitsa ndi chiwerengero cha makalasi, mudzapeza nambala 330, ndipo izi zinalembedwa pabwalo - kuti makalasi omwe amayesedwa ali oyenerera. Izi zitha kugwiritsidwa ntchito.
Kutengera izi, Roman Soloviev adabwera ndi metric, yomwe tidayitcha kuti Proxy Score, yomwe idalumikizana bwino ndi boardboard. Mfundo ndi yakuti: mumalosera, tengani 1 pamwamba pa zolosera zanu ndikuwerengera chiwerengero cha zinthu za kalasi iliyonse. Kenako, chotsani 330 pamtengo uliwonse ndikuwonjezera zotsatira zake.
Mfundo zotsatirazi zidapezedwa. Izi zidatithandiza kuti tisapange bolodi yofufuza, koma kutsimikizira kwanuko ndikusankha ma coefficient a magulu athu.
Ndi gulu mukhoza kupeza liwiro chotero. Ndi chiyani chinanso chomwe ndikanachita? Tiyerekeze kuti mwagwiritsa ntchito mfundo zosonyeza kuti makalasi a mayeso anu ndi abwino.
Kulinganiza kunali kosiyana. - kulinganiza kuchokera kwa anyamata omwe adatenga malo oyamba.
Kodi tinatani? Kulinganiza kwathu kunali kophweka, kunanenedwa ndi Evgeny Babakhnin. Tidakonza zolosera zathu poyambira 1 ndikusankha osankhidwa kuchokera kwa iwo - kotero kuti chiwerengero cha makalasi sichidapitirire 330. Koma m'makalasi ena mumatha kukhala ndi zolosera zosakwana 330. Chabwino, tiyeninso tikonzere 2 ndi pamwamba 3 , ndipo tidzasankhanso ofuna.
Kodi kulinganiza kwathu kunali kosiyana bwanji ndi kulinganiza kwa malo oyamba? Anagwiritsa ntchito njira yobwerezabwereza, kutenga kalasi yotchuka kwambiri ndikuchepetsa mwayi wa kalasilo ndi chiwerengero chochepa mpaka kalasilo silinali lotchuka kwambiri. Tinatenga kalasi yotsatira yotchuka kwambiri. Choncho anapitiriza kuwatsitsa mpaka chiwerengero cha magulu onse chinafanana.
Aliyense amagwiritsa ntchito njira yowonjezera kapena kuchotsera imodzi pophunzitsa maukonde, koma si onse omwe amagwiritsa ntchito kusanja. Pogwiritsa ntchito kusanja, mutha kulowa mu golidi, ndipo ngati mutakhala ndi mwayi, ndiye mu ndalama.
Momwe mungakonzeretu tsiku? Aliyense adakonzeratu tsikulo, kuphatikiza kapena kuchotsera, mwanjira yomweyo - kupanga zinthu zopangidwa ndi manja, kuyesa kuyika nthawi ndi mitundu yosiyanasiyana ya sitiroko, etc. Alexey Nozdrin-Plotnitsky, yemwe adatenga malo a 8, adalankhula za izi.
Iye anachita izo mosiyana. Anati zinthu zanu zonse zopangidwa ndi manja sizigwira ntchito, simuyenera kutero, maukonde anu ayenera kuphunzira izi okha. Ndipo m'malo mwake, adabwera ndi ma module ophunzirira omwe adasinthiratu deta yanu. Anaponyera deta yoyambirira mwa iwo popanda kukonzanso - ndondomeko ndi nthawi.
Kenako adatenga kusiyana kutengera zomwe zimagwirizanitsa, ndikuziwerengera zonse kutengera nthawi. Ndipo adabwera ndi matrix aatali. Anagwiritsa ntchito 1D convolution kwa izo kangapo kuti apeze masanjidwe a kukula kwa 64xn, pomwe n ndi chiwerengero chonse cha mfundo, ndipo 64 amapangidwa kuti adyetse matrix omwe amachokera ku gulu la convolutional network, yomwe imavomereza chiwerengero cha njira. - 64. adapeza matrix a 64xn, ndiye kuti kuchokera pa izi kunali koyenera kupanga tensor ya kukula kwina kuti chiwerengero cha ma channels chikhale chofanana ndi 64. Anasintha mfundo zonse X, Y mumtundu wa 0 mpaka 32 kuti apange kukula kwake 32x32. Sindikudziwa chifukwa chake adafuna 32x32, zidangochitika choncho. Ndipo pamakonzedwe awa adayika chidutswa cha matrix a 64xn. Chifukwa chake zidangomaliza ndi 32x32x64 tensor yomwe mutha kuyiyika patsogolo pa neural network yanu. Ndizo zonse zomwe ndimafuna kunena.
Source: www.habr.com