Ezinyangeni ezimbalwa ezedlule, ozakwethu bakwa-Google ku-Kaggle umncintiswano wokudala isigaba sezithombe ezitholwe ngendlela evusa amadlingozi "Shesha, Dweba!" Iqembu, elihlanganisa unjiniyela we-Yandex u-Roman Vlasov, wathatha indawo yesine emncintiswaneni. Ekuqeqesheni ukufundwa komshini kaJanuwari, u-Roman wabelane ngemibono yeqembu lakhe, ukuqaliswa kokugcina kokuhlelwa kwezigaba, kanye nemikhuba ethakazelisayo yabaphikisi bakhe.
- Sanibonani nonke! Igama lami ngingu-Roma Vlasov, namuhla ngizokutshela mayelana ne-Quick, Draw! I-Doodle Recognition Challenge.
Bekunabantu abahlanu eqenjini lethu. Ngijoyine ngaphambi nje komnqamulajuqu wokuhlanganisa. Saba nebhadi, sanyakaziswa kancane, kodwa sanyakaziswa endaweni yemali, futhi zanyakaziswa endaweni yegolide. Futhi sithathe indawo yesine ehloniphekile.
(Phakathi nomncintiswano, amaqembu azibonele esilinganisweni, esakhiwa ngokusekelwe emiphumeleni eboniswe engxenyeni eyodwa yesethi yedatha ehlongozwayo. Isilinganiso sokugcina, sona, sakhiwa kwenye ingxenye yedathasethi. Lokhu kwenziwa kanjalo. Ngakho-ke, kowamanqamu, lapho ushintsha phakathi kwezilinganiso, izikhundla ziyanyakaza kancane (kusuka ku-English shake up - ukuxuba): kwenye idatha, umphumela ungavela. Iqembu likaRoman libe ngowokuqala kwabathathu.Kulokhu, abathathu abaphezulu yimali, indawo yokulinganisa imali, njengoba kuphela izindawo ezintathu zokuqala ezaklonyeliswa ngomklomelo wemali.Ngemva kokuzamazama iqembu lase lingenile. indawo yesine. Ngendlela efanayo, elinye iqembu lilahlekelwe ukunqoba, indawo yegolide. - Umhleli.)
Lo mncintiswano wawubalulekile futhi ngoba u-Evgeniy Babakhnin wathola umkhulu, u-Ivan Sosin wathola i-master, u-Roman Soloviev wahlala engumkhulu, u-Alex Parinov wathola inkosi, ngaba uchwepheshe, futhi manje sengivele ngiyinkosi.
Kuyini lokhu Quick, Dweba? Lena isevisi evela kwa-Google. I-Google yayinomgomo wokwazisa i-AI futhi ngale nsizakalo yayifuna ukukhombisa ukuthi amanethiwekhi e-neural asebenza kanjani. Uya lapho, chofoza Masidwebe, bese kuvela ikhasi elisha lapho utshelwa khona: dweba i-zigzag, unemizuzwana engama-20 ukwenza lokhu. Uzama ukudweba i-zigzag kumasekhondi angu-20, njengalapha, isibonelo. Uma uphumelela, inethiwekhi ithi i-zigzag bese uqhubeka. Kunezithombe ezinjalo eziyisithupha kuphela.
Uma inethiwekhi ye-Google yehlulekile ukubona okudwebile, kubekwe isiphambano emsebenzini. Kamuva ngizokutshela ukuthi kuzosho ukuthini esikhathini esizayo ukuthi umdwebo ubonwa yinethiwekhi noma cha.
Le nsizakalo iqoqe inani elikhulu kakhulu labasebenzisi, futhi zonke izithombe ezidwetshiwe abasebenzisi zifakiwe.
Sikwazile ukuqoqa cishe izithombe eziyizigidi ezingu-50. Kusukela kulokhu, isitimela kanye nosuku lokuhlolwa komncintiswano wethu kwakhiwa. Ngendlela, inani ledatha ekuhlolweni kanye nenani lamakilasi kugqanyiswe ngokugqamile ngesizathu. Ngizokutshela ngabo ngemva kwesikhashana.
Ifomethi yedatha ibimi kanje. Lezi akuzona nje izithombe ze-RGB, kodwa, uma sikhuluma nje, ilogi lakho konke akwenzile umsebenzisi. Igama liyinhloso yethu, ikhodi yezwe yilapho umbhali we-doodle evela khona, isitembu sesikhathi yisikhathi. Ilebula elaziwayo libonisa nje ukuthi inethiwekhi isibonile isithombe esivela ku-Google noma cha. Futhi umdwebo ngokwawo uwukulandelana, ukulinganiselwa kwejika elidwetshwa umsebenzisi ngamaphoyinti. Futhi izikhathi. Lesi yisikhathi kusukela ekuqaleni kokudweba isithombe.
Idatha yethulwe ngamafomethi amabili. Lena ifomethi yokuqala, futhi eyesibili yenziwe lula. Basika izikhathi lapho futhi balinganisela le sethi yamaphuzu ngesethi encane yamaphuzu. Ukuze lokhu basebenzise . Unesethi enkulu yamaphoyinti avele acishe umugqa oqondile, kodwa empeleni ungakwazi ukulinganisa lo mugqa ngamaphoyinti amabili nje. Lona umqondo we-algorithm.
Idatha isatshalaliswe kanje. Konke kufana, kepha kukhona okuhlukile. Lapho sixazulula inkinga, asizange siyibheke. Okubalulekile ukuthi awekho amakilasi abembalwa ngempela, bekungadingeki ukuthi senze amasampula anesisindo kanye nokweqisa kwedatha.
Zazibukeka kanjani izithombe? Lesi isigaba "sendiza" nezibonelo ezisuka kuso ezinamalebula aziwayo nangaziwa. Isilinganiso sabo sasisendaweni ethile phakathi kuka-1 kuya ku-9. Njengoba ubona, idatha inomsindo impela. Ngingaqagela ukuthi yindiza. Uma ubheka kungaziwa, ezimweni eziningi kuba nomsindo nje. Othile waze wazama ukubhala βindiza,β kodwa ngokusobala ngesiFulentshi.
Iningi labahlanganyeli livele lathatha amagridi, ladonsa idatha kulokhu kulandelana kwemigqa njengezithombe ze-RGB, bese liziphonsa kunethiwekhi. Ngadweba cishe ngendlela efanayo: ngathatha iphalethi yemibala, ngadweba umugqa wokuqala ngombala owodwa, owawusekuqaleni kwaleli phalethi, umugqa wokugcina nomunye, owawusekupheleni kwephalethi, futhi phakathi kwawo. Ngihumushe yonke indawo ngisebenzisa le palette. Ngendlela, lokhu kunikeze umphumela ongcono kunalapho udweba njengaku-slide sokuqala - ngombala omnyama nje.
Amanye amalungu eqembu, njengo-Ivan Sosin, azame izindlela ezihluke kancane zokudweba. Ngesiteshi esisodwa wavele wadweba isithombe esimpunga, ngesinye isiteshi wadweba i-stroke ngayinye nge-gradient kusukela ekuqaleni kuya ekugcineni, kusukela ku-32 kuya ku-255, kanti ngesiteshi sesithathu wadweba i-gradient phezu kwayo yonke imivimbo kusukela ku-32 kuya ku-255.
Enye into ethokozisayo ukuthi u-Alex Parinov ulayishe imininingwane kunethiwekhi esebenzisa ikhodi yezwe.
Imethrikhi esetshenziswe emqhudelwaneni ithi Mean Average Precision. Iyini ingqikithi yale metric yokuncintisana? Unganikeza ama-predics amathathu, futhi uma kungekho predic elungile kulezi zintathu, khona-ke uthola u-0. Uma kukhona okulungile, khona-ke ukuhleleka kwayo kuyacatshangelwa. Futhi umphumela oqondiwe uzobalwa njengo-1 ohlukaniswe ngohlelo lokubikezela kwakho. Isibonelo, wenze izibikezelo ezintathu, futhi okulungile ngowokuqala, bese uhlukanisa u-1 ngo-1 futhi uthole u-1. Uma isibikezelo silungile futhi i-oda layo lingu-2, bese uhlukanisa 1 ngo-2, uthola u-0,5. Awu, njll.
Ngokucubungula idatha - indlela yokudweba izithombe nokunye - sinqume kancane. Iziphi izakhiwo esazisebenzisa? Sizamile ukusebenzisa izakhiwo ezinamafutha ezifana ne-PNASNet, i-SENet, kanye nezakhiwo zakudala ezinjengama-SE-Res-NeXt, ziya ngokuya zingenela imincintiswano emisha. Bekukhona neResNet neDenseNet.
Sakufundisa kanjani lokhu? Wonke amamodeli esiwathathile aqeqeshwe kusengaphambili ku-imagenet. Nakuba kunedatha eningi, izithombe eziyizigidi ezingu-50, kodwa noma kunjalo, uma uthatha inethiwekhi eqeqeshwe kusengaphambili ku-imagenet, ibonise imiphumela engcono kakhulu uma uyiqeqeshe kusukela ekuqaleni.
Iziphi izindlela zokufundisa esazisebenzisa? Lokhu Ukuhlanganisa I-Annealing nge-Warm Restarts, engizokhuluma ngakho kamuva nje. Lena indlela engiyisebenzisayo cishe kuyo yonke imiqhudelwano yami yakamuva, futhi ngayo kuvela ukuqeqesha amagridi kahle, ukufeza ubuncane obuhle.
Okulandelayo Yehlisa Izinga Lokufunda e-Plateau. Uqala ukuqeqesha inethiwekhi, usethe izinga elithile lokufunda, uqhubeke nokulifundisa, futhi ukulahlekelwa kwakho kancane kancane kuguqukela enanini elithile. Uhlola lokhu, ngokwesibonelo, ezinkathini eziyishumi ukulahlekelwa akukashintshi nhlobo. Wehlisa izinga lakho lokufunda ngenani elithile futhi uqhubeke nokufunda. Yehla kancane futhi, ihlangane okungenani, bese wehlisa futhi izinga lokufunda, njalo njalo, kuze kube yilapho inethiwekhi yakho ihlangana ekugcineni.
Okulandelayo kuyindlela ethokozisayo: Ungaboli izinga lokufunda, khulisa usayizi weqoqo. Kunesihloko esinegama elifanayo. Uma uqeqesha inethiwekhi, akumele wehlise izinga lokufunda, ungamane ukhulise usayizi weqoqo.
Le nqubo, ngendlela, yasetshenziswa u-Alex Parinov. Uqale nge-batch elingana no-408, futhi lapho inethiwekhi yakhe ifinyelela ithafa elithile, wamane waphinda kabili usayizi we-batch, njll.
Eqinisweni, angikhumbuli ukuthi usayizi weqoqo lakhe wafinyelela kuliphi inani, kodwa okuthakazelisayo ukuthi kwakukhona amaqembu e-Kaggle asebenzisa inqubo efanayo, ubukhulu beqoqo labo babungaba ngu-10000 XNUMX. Ngendlela, izinhlaka zesimanje zokufunda ngokujulile, njenge I-PyTorch, isibonelo, ikuvumela ukuthi wenze lokhu kalula. Ukhiqiza iqoqo lakho bese ulithumela kunethiwekhi hhayi njengoba linjalo, lilonke, kodwa lihlukanise libe izingcezu ukuze lingene ekhadini lakho levidiyo, ubale ama-gradient, futhi ngemva kokuba usubale ukuthambeka kweqoqo lonke, buyekeza. izisindo.
Ngendlela, osayizi beqoqo abakhulu bebesafakiwe kulo mncintiswano, ngoba idatha ibinomsindo impela, futhi usayizi weqoqo omkhulu ukusizile ukuthi usondele ngokunembe kakhulu.
Ukulebula mbumbulu nakho kwasetshenziswa, ikakhulukazi kusetshenziswa uRoman Soloviev. Wenza isampula cishe ingxenye yedatha ekuhlolweni ngamaqoqo, futhi waqeqesha igridi kumaqoqo anjalo.
Ubukhulu bezithombe bubalulekile, kodwa iqiniso liwukuthi unedatha eningi, udinga ukuqeqesha isikhathi eside, futhi uma usayizi wesithombe sakho sikhulu impela, uzoziqeqesha isikhathi eside kakhulu. Kodwa lokhu akuzange kungeze okuningi kwikhwalithi ye-classifier yakho yokugcina, ngakho-ke bekufanele usebenzise uhlobo oluthile lokuhwebelana. Futhi sazama kuphela izithombe ezazingenkulu kakhulu ngosayizi.
Kwafundwa kanjani konke? Okokuqala, izithombe ezinosayizi omncane zathathwa, izinkathi ezimbalwa zaqhutshwa kuzo, lokhu kuthathe isikhathi esiningi. Khona-ke izithombe ezinkulu zanikezwa, inethiwekhi yaqeqeshwa, ngakho-ke nakakhulu, ngisho nangaphezulu, ukuze ungayiqeqeshe kusukela ekuqaleni futhi ungachithi isikhathi esiningi.
Mayelana nama-optimizers. Sisebenzise i-SGD no-Adam. Ngale ndlela kwakungenzeka ukuthola imodeli eyodwa, eyanikeza isivinini se-0,941-0,946 ebhodini labaphambili lomphakathi, okuyinto enhle kakhulu.
Uma uhlanganisa amamodeli ngandlela thize, uzothola ndawo ezungeze u-0,951. Uma usebenzisa enye indlela, uzothola amaphuzu wokugcina angu-0,954 ebhodini lomphakathi, njengoba nje sitholile. Kodwa okuningi ngalokho kamuva. Okulandelayo ngizokutshela ukuthi siwahlanganise kanjani amamodeli, nokuthi sakwazi kanjani ukufeza isivinini esinjalo sokugcina.
Okulandelayo ngithanda ukukhuluma nge-Cosing Annealing with Warm Restarts noma Stochastic Gradient Descent with Warm Restarts. Empeleni, ungasebenzisa noma iyiphi i-optimizer, kodwa iphuzu liwukuthi: uma nje uqeqesha inethiwekhi eyodwa futhi kancane kancane iguqukela kokuncane, khona-ke konke kulungile, uzothola inethiwekhi eyodwa, yenza amaphutha athile, kodwa wena. angayiqeqesha ngokuhlukile kancane. Uzosetha izinga lokufunda lokuqala, bese ulehlisa kancane kancane ngokwale fomula. Uyayehlisa, inethiwekhi yakho ifika kokuncane, bese ulondoloza izisindo, bese uphinde usethe izinga lokufunda elalisekuqaleni kokuqeqeshwa, ngaleyo ndlela uya ndawana thize ukhuphukele kulobu buncane, uphinde wehlise izinga lakho lokufunda.
Ngakho-ke, ungavakashela ubuncane abambalwa ngesikhathi esisodwa, lapho ukulahlekelwa kwakho kuzoba khona, ukuhlanganisa noma ukususa, okufanayo. Kodwa iqiniso liwukuthi amanethiwekhi analezi zisindo azonikeza amaphutha ahlukene ngosuku lwakho. Ngokuzilinganisa, uzothola uhlobo oluthile lokuqagela, futhi isivinini sakho sizoba phezulu.
Mayelana nokuthi siwahlanganise kanjani amamodeli ethu. Ekuqaleni kwesethulo, ngithe ukunaka inani ledatha ekuhlolweni kanye nenani lamakilasi. Uma wengeza u-1 enanini lokuhloswe ngalo esivivinyweni bese uhlukanisa ngenani lamakilasi, uzothola inombolo engu-330, futhi lokhu kwabhalwa esithangamini - ukuthi amakilasi esivivinyweni abhalansile. Lokhu kungasetshenziswa.
Ngokusekelwe kulokhu, u-Roman Soloviev uqhamuke nemethrikhi, sayibiza ngokuthi I-Proxy Score, ehlotshaniswa kahle nebhodi yabaphambili. Iphuzu liwukuthi: wenza isibikezelo, thatha oku-1 okuphezulu kwezibikezelo zakho bese ubala inani lezinto zekilasi ngalinye. Okulandelayo, khipha u-330 kunani ngalinye bese uhlanganisa amanani aphelele angumphumela.
Kutholwe amanani alandelayo. Lokhu kusisize ukuthi singadali ibhodi yabaphambili ehlolayo, kodwa ukuthi siqinisekise endaweni futhi sikhethe ama-coefficients wama-ensemble ethu.
Nge-ensemble ungathola isivinini esinjalo. Yini enye engangingayenza? Ake sithi usebenzise ulwazi lokuthi amakilasi esivivinyweni sakho abhalansile.
Ukulinganisa kwakuhlukile. - ukulinganisa kusuka kubafana abathathe indawo yokuqala.
Senzeni? Ukulinganisa kwethu kwakulula, kwaphakanyiswa ngu-Evgeny Babakhnin. Siqale sahlunga izibikezelo zethu ngabaphezulu 1 futhi sakhetha amakhandidethi kubo - ukuze inani lamakilasi lingeqi ku-330. Kodwa kwamanye amakilasi ugcina unezibikezelo ezingaphansi kuka-330. Kulungile, masiphinde sihlele ngokuphezulu 2 kanye naku-3 aphezulu. , futhi sizophinde sikhethe amakhandidethi.
Ukulinganisa kwethu kwakuhluke kanjani ekulinganiseni kwendawo yokuqala? Basebenzise indlela yokuphindaphinda, bethatha ikilasi elidume kakhulu futhi behlisa amathuba alelo klasi ngenombolo ethile encane kuze kube lelo klasi lingasathandwa kakhulu. Sathatha ikilasi elilandelayo elaziwa kakhulu. Ngakho baqhubeka bewehlisa kwaze kwaba yilapho inani lazo zonke izigaba lalingana.
Wonke umuntu usebenzise indlela yokuhlanganisa noma yokususa eyodwa ukuze aqeqeshe amanethiwekhi, kodwa akubona bonke abasebenzisa ukulinganisa. Ukusebenzisa ukulinganisa, ungangena egolideni, futhi uma ube nenhlanhla, ungene emalini.
Indlela yokucubungula kusengaphambili usuku? Wonke umuntu ucubungule kusengaphambili usuku, ukuhlanganisa noma ukususa, ngendlela efanayo - ukwenza izici ezenziwe ngesandla, ezama ukufaka izikhathi ezinemibala ehlukene ye-stroke, njll. U-Alexey Nozdrin-Plotnitsky, owathatha indawo yesi-8, wakhuluma ngalokhu.
Wakwenza ngokuhlukile. Uthe zonke lezi zici zakho ezenziwe ngezandla azisebenzi, awudingi ukwenza lokho, inethiwekhi yakho kufanele ikufunde yona yodwa. Futhi esikhundleni salokho, uqhamuke namamojula okufunda acubungule ngaphambilini idatha yakho. Wajikijela idatha yokuqala kubo ngaphandle kokucubungula kusengaphambili - izixhumanisi zamaphuzu kanye nezikhathi.
Wabe esethatha umehluko ngokusekelwe kuzixhumanisi, futhi walinganisa konke ngokusekelwe esikhathini. Futhi waqhamuka ne-matrix ende. Usebenzise i-1D convolution kuyo izikhathi eziningana ukuze athole i-matrix yosayizi ongu-64xn, lapho u-n eyisamba senani lamaphuzu, futhi u-64 wenziwa ukuze kuphakelwe i-matrix ewumphumela kungqimba lwanoma iyiphi inethiwekhi yokuguqula, eyamukela inani lamashaneli. - 64. wathola i-matrix engu-64xn, ngakho-ke kusukela kulokhu kwakudingeka ukuthi udale i-tensor yobukhulu obuthile ukuze inani lamashaneli lilingane no-64. Walungisa wonke amaphuzu X, Y ebangeni ukusuka ku-0 kuya ku-32 ukuze enze i-tensor kasayizi 32x32. Angazi ukuthi kungani efuna i-32x32, kuvele kwenzeka kanjalo. Futhi kulokhu kudidiyela ubeke ucezu lwale matrix yosayizi 64xn. Ngakho-ke isanda kuthola i-tensor engu-32x32x64 ongayifaka phambili kunethiwekhi yakho ye-convolutional neural. Yilokho kuphela ebengifuna ukukusho.
Source: www.habr.com