Mwedzi mishoma yapfuura, vatinoshanda navo kubva kuGoogle paKaggle makwikwi ekugadzira classifier yemifananidzo yakawanikwa mune inonakidza "Kurumidza, Dhirowa!" Chikwata, icho chaisanganisira Yandex developer Roman Vlasov, akatora nzvimbo yechina mumakwikwi. Pakudzidziswa kwemuchina waNdira, Roman akagovera mazano echikwata chake, kuitiswa kwekupedzisira kwemugadziri, uye maitiro anonakidza evadzivisi vake.
- Mhoroi mose! Zita rangu ndinonzi Roma Vlasov, nhasi ndichakuudza nezve Quick, Draw! Doodle Recognition Dambudziko.
Pachikwata chedu paiva nevanhu vashanu. Ndakajoinha nguva yekubatanidza isati yakwana. Isu takanga tisina rombo rakanaka, takazununguswa zvishoma, asi takazununguswa kubva panzvimbo yemari, uye vakazununguswa kubva panzvimbo yegoridhe. Uye takatora nzvimbo yechina inoremekedzwa.
(Panguva yemakwikwi, zvikwata zvakazvicherechedza muchiyero, icho chakaumbwa zvichienderana nemigumisiro yakaratidzwa pane chimwe chikamu che data yakarongwa. Chiyero chekupedzisira, zvakare, chakaumbwa pane chimwe chikamu chedhata. Izvi zvinoitwa saizvozvo. kuti vatori vemakwikwi havagadzirise maalgorithms avo kune chaiyo data. Naizvozvo, mumafainari, kana uchichinja pakati pezviyero, zvinzvimbo zvinozununguka zvishoma (kubva kuChirungu shake up - kusanganisa): pane imwe data, mhedzisiro inogona kubuda. Kusiyana.Chikwata chaRoman ndicho chakatanga kupinda muzvitatu zvepamusoro.Panyaya iyi,vatatu vepamusoro imari,nzvimbo inotariswa mari,sezvo nzvimbo nhatu dzekutanga chete ndidzo dzakapihwa mubairo wemari.Mushure mekudengenyeka,chikwata chainge chatopinda. nzvimbo yechina. Nenzira imwecheteyo, chimwe chikwata chakarasikirwa nekukunda, chinzvimbo chegoridhe. - Mupepeti.)
Makwikwi aya aikoshawo pakuti Evgeniy Babakhnin akagamuchira sekuru, Ivan Sosin akagamuchira tenzi, Roman Soloviev akaramba ari mbuya, Alex Parinov akagamuchira tenzi, ndakava nyanzvi, uye ini ndatova tenzi.
Chii chinonzi Kurumidza, Dhirowa? Iri ibasa rinobva kuGoogle. Google yaive nechinangwa chekusimudzira AI uye nesevhisi iyi yaida kuratidza mashandiro anoita neural network. Unoenda ikoko, dzvanya Ngatidhirowe, uye peji idzva rinobuda kwaunoudzwa: dhirowa gomba, une masekonzi makumi maviri ekuita izvi. Iwe uri kuyedza kudhirowa zigzag mumasekonzi makumi maviri, senge pano, semuenzaniso. Kana iwe ukabudirira, network inoti iri zigzag uye iwe unoenderera mberi. Kune mitanhatu chete mifananidzo yakadaro.
Kana network yeGoogle yatadza kuziva zvawakadhirowa, muchinjikwa wakaiswa pabasa. Gare gare ini ndichakuudza zvazvichareva mune ramangwana kana mufananidzo unozivikanwa netiweki kana kwete.
Iyi sevhisi yakaunganidza nhamba yakati wandei yevashandisi, uye mapikicha ese akadhirowa nevashandisi akaiswa.
Takakwanisa kuunganidza mifananidzo inosvika mamiriyoni makumi mashanu. Kubva pane izvi, chitima uye zuva rekuyedza remakwikwi edu zvakaumbwa. Nenzira, huwandu hwe data muyedzo uye nhamba yemakirasi inosimbiswa mune bold nekuda kwechikonzero. Ndichakuudza nezvavo pava paya.
Iyo data format yaive inotevera. Iyi haingori mifananidzo yeRGB, asi, kutaura zvishoma, danda rezvese zvakaitwa nemushandisi. Shoko ndiro tarisiro yedu, kodhi yenyika ndiko kunobva munyori weiyo doodle, timestamp inguva. Iyo inozivikanwa label inongoratidza kuti network yaziva mufananidzo kubva kuGoogle kana kwete. Uye iyo dhizaini pachayo ndeyekutevedzana, fungidziro yecurve iyo mushandisi anodhirowa nemapoinzi. Uye nguva. Ino ndiyo nguva kubva pakutanga kudhirowa mufananidzo.
Data yakaratidzwa mumhando mbiri. Ichi ndicho chimiro chekutanga, uye chechipiri chakarerutswa. Ivo vakacheka nguva kubva ipapo uye vakaenzanisa iyi seti yemapoinzi nediki seti yemapoinzi. Nokuda kweizvi vakashandisa . Iwe une seti hombe yemapoinzi inongoita kunge mutsara wakatwasuka, asi kutaura zvazviri unogona kuyera mutsara uyu nemapoinzi maviri chete. Iyi ndiyo pfungwa yealgorithm.
Iyo data yakagoverwa sezvinotevera. Zvese zviri yunifomu, asi kune zvimwe zvinobuda. Patakagadzirisa dambudziko, hatina kuzvitarisa. Chinhu chikuru ndechekuti pakanga pasina makirasi aive mashoma chaizvo, isu taisafanirwa kuita huremu samplers uye data oversampling.
Mifananidzo yacho yainge yakaita sei? Iyi ndiyo kirasi ye "ndege" uye mienzaniso kubva mairi ine mavara anozivikanwa uye asingazivikanwe. Reshiyo yavo yaive pane imwe nzvimbo yakatenderedza 1 kusvika 9. Sezvauri kuona, iyo data ine ruzha. Ndinofungidzira kuti indege. Kana ukatarisa zvisingazivikanwe, kazhinji kacho ingori ruzha. Mumwe munhu akatoedza kunyora βndege,β asi sezviri pachena muchiFrench.
Vazhinji vatori vechikamu vakangotora magridi, vakadhirowa data kubva mukutevedzana kwemitsara semifananidzo yeRGB, ndokuikanda mu network. Ndakadhirowa nenzira imwecheteyo: Ndakatora palette yemavara, ndakadhirowa mutsara wekutanga nemuvara mumwechete, waive pakutanga kweiyi palette, wekupedzisira mutsara neimwe, yaive kumagumo kwepalette, uye pakati pavo. Ndakapindira kwese kwese ndichishandisa iyi palette. Nenzira, izvi zvakapa mhedzisiro iri nani pane kana iwe ukadhirowa sepaiyo yekutanga siraidhi - ingori mutema.
Dzimwe nhengo dzechikwata, dzakadai saIvan Sosin, dzakaedza nzira dzakasiyana dzekudhirowa. Nechani imwe chete aingodhirowa mufananidzo wegrey, neimwe chiteshi akadhirowa imwe neimwe sitiroko kubva kwekutanga kusvika kumagumo, kubva pa32 kusvika ku255, uye neyechitatu chiteshi akadhirowa giraidhi pamusoro pese sitiroko kubva pa32 kusvika 255.
Chimwe chinhu chinonakidza ndechekuti Alex Parinov akaisa ruzivo kunetiweki achishandisa nyikacode.
Metric inoshandiswa mumakwikwi ndeye Mean Average Precision. Chii chakakosha chemetric iyi yemakwikwi? Iwe unogona kupa zvirevo zvitatu, uye kana pasina predic yakarurama mune izvi zvitatu, zvino unowana 0. Kana pane yakarurama, saka kurongeka kwayo kunotorwa. Uye mhedzisiro yechinangwa ichaverengerwa se1 yakakamurwa nehurongwa hwekufungidzira kwako. Semuenzaniso, iwe wakaita zvirevo zvitatu, uye yakarurama ndiyo yekutanga, zvino iwe unoparadzanisa 1 ne1 uye uwane 1. Kana chirevo chacho chakarurama uye kurongeka kwayo 2, zvino kuparadzanisa 1 ne2, unowana 0,5. Zvakanaka, nezvimwewo.
Nedata preprocessing - maitiro ekudhirowa mifananidzo uye zvichingodaro - isu takasarudza zvishoma. Ndeapi mapurani atakashandisa? Takaedza kushandisa zvivakwa zvakakora zvakaita sePNASNet, SENet, uye zvivakwa zvekare zvekare seSE-Res-NeXt, vari kuwedzera kupinda mumakwikwi matsva. Paivewo neResNet uye DenseNet.
Takadzidzisa sei izvi? Ese mamodheru atakatora anga atodzidziswa pa imagenet. Kunyangwe paine data rakawanda, mamirioni makumi mashanu emifananidzo, asi zvakadaro, kana iwe ukatora inetiweki yakadzidziswa pa imagenet, yakaratidza mhedzisiro iri nani pane kana iwe wakangoidzidzisa kubva pakutanga.
Madzidzisiro api atakashandisa? Uku ndiko Cosing Annealing neWarm Restarts, izvo zvandichataura nezvazvo zvishoma gare gare. Iyi inzira yandinoshandisa mune angangoita ese emakwikwi angu achangoburwa, uye navo zvinobuda kudzidzisa magidhi zvakanaka, kuti vawane hushoma hwakanaka.
Next Deredza Kudzidza Mwero paPlateau. Iwe unotanga kudzidzisa network, isa imwe yero yekudzidza, ramba uchiidzidzisa, uye kurasikirwa kwako zvishoma nezvishoma kunochinja kune imwe kukosha. Iwe tarisa izvi, semuenzaniso, kune gumi epoch kurasikirwa hakuna kumbochinja. Iwe unoderedza mwero wako wekudzidza neimwe kukosha uye woenderera mberi nekudzidza. Inodonha zvishoma zvakare, inoshanduka pane imwe shoma, uye iwe zvakare unodzikisa mwero wekudzidza, uye zvichingodaro, kudzamara network yako yachinja.
Inotevera inzira inonakidza: Usaora mwero wekudzidza, wedzera saizi yebatch. Pane chinyorwa chine zita rimwe chete. Paunodzidzisa network, haufanirwe kudzikisa mwero wekudzidza, unogona kungowedzera saizi yebatch.
Iyi nzira, nenzira, yakashandiswa naAlex Parinov. Akatanga nebatch yakaenzana ne408, uye network yake payakasvika pane imwe nzvimbo, aingopeta kaviri saizi yebatch, nezvimwe.
Muchokwadi, handiyeuki kukosha kwehukuru hwebatch yake kwakasvika, asi chinonakidza ndechekuti pakanga paine zvikwata paKaggle zvakashandisa nzira imwechete, saizi yavo yebatch yaive inenge 10000 XNUMX. Nenzira, maitiro emazuva ano ekudzidza kwakadzama, senge. PyTorch, semuenzaniso, inobvumidza iwe kuti uite izvi zviri nyore. Iwe unogadzira batch yako uye woiendesa kune network kwete sezvairi, yakazara, asi ikamure kuita machunks kuti ikwane mukadhi rako revhidhiyo, verenga magradients, uye mushure mekunge waverenga gradient yebatch rese, gadziridza. zviyereso.
Nenzira, hukuru hukuru hwebhechi hwakanga huchiri kuverengerwa mumakwikwi aya, nekuti data racho raive neruzha, uye saizi hombe yebhechi yakakubatsira iwe kunyatso kuenzanisa gradient.
Pseudo-labeling yakashandiswa zvakare, yakanyanya kushandiswa naRoman Soloviev. Akatora inenge hafu yedata kubva muyedzo mumabhechi, uye akadzidzisa gidhi pamabheji akadaro.
Saizi yemifananidzo ine basa, asi chokwadi ndechekuti une data rakawanda, unofanirwa kudzidzisa kwenguva yakareba, uye kana saizi yako yemifananidzo yakakura, ipapo iwe unozodzidzisa kwenguva yakareba. Asi izvi hazvina kuwedzera zvakanyanya kumhando yeyekupedzisira classifier, saka zvaive zvakakodzera kushandisa imwe mhando yekutengeserana-off. Uye takaedza chete mifananidzo yakanga isina kunyanya kukura muhukuru.
Zvose zvakadzidzwa sei? Kutanga, mifananidzo midiki-diki yakatorwa, nguva yakati wandei yakamhanyiswa pavari, izvi zvakatora nguva yakawanda. Zvadaro mapikicha akakura akapiwa, network yakadzidziswa, uye kunyanya, kunyanya, kuitira kuti urege kuidzidzisa kubva pakutanga uye kusaparadza nguva yakawanda.
Nezve optimizers. Takashandisa SGD naAdam. Nenzira iyi zvaive zvichikwanisika kuwana imwe modhi, iyo yakapa kumhanya kwe0,941-0,946 paruzhinji leaderboard, izvo zvakanaka chaizvo.
Kana iwe ukabatanidza iwo modhi neimwe nzira, iwe unowana kumwe kwakatenderedza 0,951. Kana iwe ukashandisa imwe nzira, iwe unowana yekupedzisira mamakisi 0,954 paruzhinji bhodhi, sezvatakawana. Asi zvakawanda pamusoro pazvo gare gare. Tevere ini ndichakuudza kuti takaunganidza sei mamodheru, uye kuti takakwanisa sei kuwana iyo yekupedzisira kumhanya.
Tevere ndinoda kutaura nezve Cosing Annealing neWarm Restarts kana Stochastic Gradient Descent ine Warm Restarts. Zvichireva kutaura, musimboti, unogona kushandisa chero optimizer, asi poindi ndeiyi: kana iwe ukangodzidzisa imwe network uye zvishoma nezvishoma inoshanduka kune imwe shoma, saka zvese zvakanaka, iwe uchawana network imwe, inoita zvikanganiso, asi iwe. anogona kuidzidzisa zvishoma zvakasiyana. Iwe unozoisa imwe yekutanga chiyero chekudzidza, uye zvishoma nezvishoma ichidzikisa zvinoenderana neiyi fomula. Iwe unoidzikisa, network yako inosvika kune imwe shoma, wobva wachengetedza uremu, uye zvakare woisa mwero wekudzidza wanga uri pakutanga kwekudzidziswa, nekudaro uchienda kumwe kukwira kubva paushoma uhwu, uye zvakare kudzikisa mwero wako wekufunda.
Nekudaro, iwe unogona kushanyira akati wandei kamwechete, umo kurasikirwa kwako kuchave, kuwedzera kana kubvisa, zvakafanana. Asi chokwadi ndechekuti ma network ane huremu aya achapa zvikanganiso zvakasiyana pazuva rako. Nekuvaenzanisa, iwe uchawana imwe mhando yekufungidzira, uye kumhanya kwako kuchakwira.
Nezve kuti takaunganidza sei mamodheru edu. Pakutanga kwemharidzo, ndakati nditeerere kuwanda kwe data muyedzo uye nhamba yemakirasi. Kana iwe ukawedzera 1 kune nhamba yezvinangwa mumuedzo wakaiswa uye kupatsanura nehuwandu hwemakirasi, iwe uchawana nhamba 330, uye izvi zvakanyorwa paforamu - kuti makirasi ari muyedzo akaenzana. Izvi zvinogona kushandiswa.
Zvichienderana neizvi, Roman Soloviev akauya nemetric, isu takaidaidza kuti Proxy Score, iyo yakanyatso enderana neiyo leaderboard. Pfungwa ndeyokuti: iwe unoita kufanotaura, tora yepamusoro 1 yevanofanotaura uye uverenge nhamba yezvinhu zvekirasi yega yega. Tevere, bvisa 330 kubva pane imwe neimwe kukosha uye wedzera iyo inokonzeresa tsika dzakakwana.
Zvinotevera zvakakosha zvakawanikwa. Izvi zvakatibatsira kuti tisagadzire bhodhi rekuongorora, asi kusimbisa munharaunda uye kusarudza coefficients ema ensembles edu.
Ne ensemble iwe unogona kuwana kumhanya kwakadaro. Chii chimwe chandaigona kuita? Ngatiti washandisa ruzivo rwekuti makirasi ebvunzo yako akaenzana.
Kuenzanisa kwaive kwakasiyana. - kuenzanisa kubva kune vakomana vakatora nzvimbo yekutanga.
Takaitei? Kuenzanisa kwedu kwaive nyore, zvakataurwa naEvgeny Babakhnin. Takatanga taronga fungidziro dzedu nepamusoro 1 uye takasarudza vachada kubva kwavari - kuitira kuti nhamba yemakirasi isapfuure 330. Asi kune mamwe makirasi unopedzisira wava neanofungidzira asingasviki 330. Zvakanaka, ngatirongeiwo nepamusoro 2 uye pamusoro 3 , uye tichasarudzawo vanoda kukwikwidza.
Kuenzanisa kwedu kwakasiyana sei nokudzikamisa nzvimbo yokutanga? Vakashandisa nzira yekudzokorora, kutora kirasi inonyanya kufarirwa uye kuderedza zvingangoitika zvekirasi iyoyo neimwe nhamba diki kusvika kirasi iyoyo isati yanyanya kufarirwa. Takatora kirasi yaitevera yakakurumbira zvikuru. Saka vakaramba vachidzidzikisa kusvikira nhamba yemapoka ose yaenzana.
Wese munhu akashandisa kuwedzera kana kubvisa imwe nzira yekudzidzisa network, asi havasi vese vakashandisa kuenzanisa. Uchishandisa kuenzanisa, unogona kupinda mugoridhe, uye kana uine rombo rakanaka, wobva wapinda mumari.
Nzira yekugadzirisa sei zuva? Munhu wose akafanogadzirisa zuva, kuwedzera kana kubvisa, nenzira imwechete - kugadzira zvinhu zvakagadzirwa nemaoko, kuedza kuvhara nguva nemavara akasiyana-siyana ekurohwa, nezvimwewo Alexey Nozdrin-Plotnitsky, uyo akatora nzvimbo yechisere, akataura pamusoro peizvi.
Akazviita zvakasiyana. Akataura kuti zvese izvi zvakagadzirwa nemaoko maficha ako haashande, haufanire kudaro, network yako inofanirwa kudzidza zvese izvi wega. Uye panzvimbo pacho, akauya nemamodule ekudzidza akafanogadzirisa data rako. Akakanda iyo yekutanga data mavari pasina preprocessing - mapoinzi coordinates uye nguva.
Akabva atora mutsauko zvichienderana nemakonisheni, uye akazvienzanisa zvese zvichienderana nenguva. Uye akauya nematrix akareba. Akashandisa 1D convolution kwairi kakawanda kuti awane matrix yehukuru 64xn, apo n ndiyo nhamba yese yemapoinzi, uye 64 inogadzirwa kuitira kudyisa iyo inoguma matrix kune layer yechero convolutional network, iyo inogamuchira huwandu hwematanho. - 64. akawana 64xn matrix, zvino kubva pane izvi zvaive zvakakodzera kugadzira tensor yehumwe hukuru kuitira kuti nhamba yematanho ienzane ne 64. Akagadzirisa pfungwa dzose X, Y muhuwandu kubva pa0 kusvika ku32 kugadzira tensor yehukuru 32x32. Handizivi kuti sei aida 32x32, zvakangoitika saizvozvo. Uye pakurongeka uku akaisa chidimbu cheiyi matrix yehukuru 64xn. Saka zvakazongoguma ne 32x32x64 tensor yaunokwanisa kuisa mberi mune yako convolutional neural network. Ndizvo chete zvandaida kutaura.
Source: www.habr.com