Ngaphambi kwakho futhi kuwumsebenzi wokubona izinto. Okubalulekile isivinini sokusebenza ngokunemba okwamukelekayo. Uthatha i-YOLOv3 yezakhiwo futhi uqhubeke uyiqeqeshe. Ukunemba(mAp75) kukhulu kuno-0.95. Kodwa izinga lokugijima lisephansi. Crap.
Namuhla sizodlula i-quantization. Futhi ngaphansi kokusika sizobheka Ukuthena Imodeli β ukusika izingxenye ezingadingekile zenethiwekhi ukuze kusheshiswe i-Inference ngaphandle kokulahlekelwa ukunemba. Kuyacaca lapho, malini futhi kanjani ukusika. Ake sithole ukuthi sikwenza kanjani lokhu ngokwenza nokuthi ungakwenza kanjani ngokuzenzakalelayo. Ekugcineni kukhona inqolobane kuma-keras.
Isingeniso
Endaweni yami yokusebenza yangaphambilini, iMacroscop ePerm, ngathola umkhuba owodwa - ukuhlale ngiqapha isikhathi sokwenziwa kwama-algorithms. Futhi hlola njalo isikhathi sokusebenza senethiwekhi ngesihlungi sokwanela. Imvamisa ukukhiqizwa kwesimanjemanje akudluli lesi sihlungi, okungiholele kuPruning.
Ukuthena yisihloko esidala okwaxoxwa ngaso ngo-2017. Umqondo oyinhloko ukunciphisa ubukhulu benethiwekhi eqeqeshiwe ngaphandle kokulahlekelwa ukunemba ngokususa ama-node ahlukahlukene. Kuzwakala kupholile, kodwa angivamile ukuzwa ngokusetshenziswa kwayo. Mhlawumbe, akukho ukuqaliswa okwanele, azikho izindatshana zolimi lwesiRashiya, noma wonke umuntu ubheka njengokuthena ulwazi futhi athule.
Kodwa masiyihlukanise
Ukuthi shazi kubhayoloji
Ngiyathanda uma i-Deep Learning ibheka imibono evela kubhayoloji. Bona, njengokuziphendukela kwemvelo, bangathenjwa (bewazi ukuthi i-ReLU ifana kakhulu ne ?)
Inqubo ye-Model Pruning nayo iseduze nesayensi yezinto eziphilayo. Impendulo yenethiwekhi lapha ingaqhathaniswa nepulasitiki yobuchopho. Kunezibonelo ezimbalwa ezithakazelisayo encwadini. :
- Ubuchopho bowesifazane owazalwa nengxenye eyodwa kuphela buzihlele kabusha ukuze zenze imisebenzi yengxenye engekho.
- Umfana wadubula ingxenye yobuchopho bakhe ebhekele ukubona. Ngokuhamba kwesikhathi, ezinye izingxenye zobuchopho zathatha le misebenzi. (asizami ukuphinda)
Ngokufanayo, ungasika amanye ama-convolutions abuthakathaka kumodeli yakho. Njengendlela yokugcina, izinyanda ezisele zizosiza esikhundleni salezo ezisikiwe.
Ingabe uthanda i-Transfer Learning noma ufunda kusukela ekuqaleni?
Inketho yokuqala. Usebenzisa i-Transfer Learning ku-Yolov3. I-Retina, Mask-RCNN noma i-U-Net. Kodwa isikhathi esiningi asidingi ukubona amakilasi ezinto ezingama-80 njengaku-COCO. Ngokwenza kwami, yonke into ikhawulelwe kumabanga 1-2. Umuntu angase acabange ukuthi izakhiwo zamakilasi angu-80 azisasebenzi lapha. Lokhu kusikisela ukuthi i-architecture idinga ukwenziwa ibe mncane. Ngaphezu kwalokho, ngingathanda ukwenza lokhu ngaphandle kokulahlekelwa izisindo ezikhona eziqeqeshwe ngaphambilini.
Inombolo yenketho yesibili. Mhlawumbe unedatha eningi nezinsiza zekhompuyutha, noma udinga nje i-super-custom architecture. Akunandaba. Kodwa ufunda inethiwekhi kusukela ekuqaleni. Inqubo evamile iwukubheka ukwakheka kwedatha, ukhethe i-architecture enamandla ABANGAPHEZULU, bese ucindezela abashiya phansi ekuqeqeshweni kabusha. Ngibone abashiya isikolo abangu-0.6, uKarl.
Kuzo zombili izimo, inethiwekhi ingancishiswa. Ekhuthazekile. Manje ake sihlole ukuthi kuyini ukuthenwa kokusoka
I-algorithm evamile
Sinqume ukuthi singawasusa ama-bundle. Kubukeka kulula kakhulu:
Ukususa noma yikuphi ukuguquguquka kuyacindezela inethiwekhi, okuvame ukuholela ekwandeni kwephutha. Ngakolunye uhlangothi, lokhu kwanda kwephutha kuyinkomba yokuthi sisusa kanjani ama-convolutions ngendlela efanele (isibonelo, ukwanda okukhulu kubonisa ukuthi senza okuthile okungalungile). Kodwa ukukhuphuka okuncane kuyamukeleka futhi kuvame ukuqedwa ngokuqeqeshwa okungeziwe kokukhanya okulandelayo nge-LR encane. Engeza isinyathelo esingeziwe sokuqeqesha:
Manje sidinga ukuthola ukuthi sifuna ukumisa nini i-Learning<->Pruning loop yethu. Kungase kube nezinketho ezingavamile lapha lapho sidinga ukwehlisa inethiwekhi ibe usayizi nesivinini esithile (isibonelo, kumadivayisi eselula). Kodwa-ke, inketho evame kakhulu ukuqhubeka nomjikelezo kuze kube yilapho iphutha liba phezulu kunokwamukelekayo. Engeza umbandela:
Ngakho, i-algorithm iyacaca. Kusele ukuthola ukuthi unganquma kanjani ama-convolutions asusiwe.
Sesha amaphakheji asusiwe
Kudingeka sisuse ama-convolutions athile. Ukugijimela phambili futhi "ukudubula" noma ubani kuwumbono omubi, nakuba kuzosebenza. Kodwa njengoba unekhanda, ungacabanga futhi uzame ukukhetha ama-convolutions βabuthakaβ ukuze asuswe. Kunezinketho ezimbalwa:
- . Umbono wokuthi ama-convolutions anesisindo esincane enza umnikelo omncane esinqumweni sokugcina
- Isilinganiso esincane kunazo zonke se-L1 ngokucabangela incazelo kanye nokuchezuka okujwayelekile. Sihlanganisa nokuhlola uhlobo lokusabalalisa.
- . Ukunqunywa okunembe kakhudlwana kokuguquguquka okungabalulekile, kodwa kudla isikhathi kakhulu futhi kudla izinsiza.
- Okunye
Ngayinye yezinketho inelungelo lokuphila kanye nezici zayo zokusebenzisa. Lapha sibheka inketho ngesilinganiso esincane kunazo zonke se-L1
Inqubo eyenziwa mathupha ye-YOLOv3
I-architecture yasekuqaleni iqukethe amabhlogo ayinsalela. Kodwa kungakhathaliseki ukuthi zipholile kangakanani kumanethiwekhi ajulile, azosivimbela ngandlela-thile. Ubunzima ukuthi awukwazi ukususa ukubuyisana ngezinkomba ezihlukile kulezi zendlalelo:
Ngakho-ke, ake sikhethe izendlalelo esingasusa kuzo ngokukhululekile ukubuyisana:
Manje ake sakhe umjikelezo womsebenzi:
- Ilayisha ukwenza kusebenze
- Ukuthola ukuthi kunqunywa malini
- Sika
- Ukufunda izinkathi eziyi-10 nge-LR=1e-4
- Ukuhlola
Ukukhipha ama-convolution kuyasiza ukulinganisa ukuthi ingakanani ingxenye esingayisusa esinyathelweni esithile. Izibonelo zokulayisha:
Siyabona ukuthi cishe yonke indawo u-5% wama-convolutions ane-L1-normal ephansi kakhulu futhi singawasusa. Esinyathelweni ngasinye, lokhu kuthululwa kwakuphindwa futhi kwahlolwa ukuthi yiziphi izendlalelo nokuthi zingaki ezingasikwa.
Yonke inqubo yaqedwa ngezinyathelo ezi-4 (izinombolo lapha nakuyo yonke indawo ye-RTX 2060 Super):
| Isinyathelo | mp75 | Inani lamapharamitha, isigidi | Usayizi wenethiwekhi, mb | Kusukela ekuqaleni, % | Isikhathi sokugijima, Nks | Isimo sokusoka |
|---|---|---|---|---|---|---|
| 0 | 0.9656 | 60 | 241 | 100 | 180 | - |
| 1 | 0.9622 | 55 | 218 | 91 | 175 | 5% yazo zonke |
| 2 | 0.9625 | 50 | 197 | 83 | 168 | 5% yazo zonke |
| 3 | 0.9633 | 39 | 155 | 64 | 155 | 15% ngezendlalelo ezinama-convolutions angu-400+ |
| 4 | 0.9555 | 31 | 124 | 51 | 146 | 10% ngezendlalelo ezinama-convolutions angu-100+ |
Omunye umphumela omuhle wengezwe esinyathelweni sesi-2 - usayizi weqoqo 4 ulingana enkumbulo, okusheshise kakhulu inqubo yokuqeqeshwa okwengeziwe.
Esinyathelweni sesi-4, inqubo yamiswa ngoba ngisho nokuqeqeshwa okwengeziwe kwesikhathi eside akuzange kukhuphule i-mAp75 kumanani amadala.
Ngenxa yalokho, sikwazile ukusheshisa ukucatshangelwa 15%, nciphisa usayizi nge 35% futhi ungalahlekelwa ncamashi.
I-automation yezakhiwo ezilula
Ngezakhiwo zenethiwekhi ezilula (ngaphandle kwesengezo esinemibandela, i-concaternate kanye namabhulokhi ayizinsalela), kungenzeka ukuthi ugxile ekucubunguleni zonke izendlalelo ze-convolutional futhi wenze inqubo yokusika ama-convolutions ngokuzenzakalelayo.
Ngisebenzise le nketho .
Kulula: udinga kuphela umsebenzi wokulahlekelwa, i-optimizer kanye namajeneretha e-batch:
import pruning
from keras.optimizers import Adam
from keras.utils import Sequence
train_batch_generator = BatchGenerator...
score_batch_generator = BatchGenerator...
opt = Adam(lr=1e-4)
pruner = pruning.Pruner("config.json", "categorical_crossentropy", opt)
pruner.prune(train_batch, valid_batch)Uma kunesidingo, ungashintsha amapharamitha wokumisa:
{
"input_model_path": "model.h5",
"output_model_path": "model_pruned.h5",
"finetuning_epochs": 10, # the number of epochs for train between pruning steps
"stop_loss": 0.1, # loss for stopping process
"pruning_percent_step": 0.05, # part of convs for delete on every pruning step
"pruning_standart_deviation_part": 0.2 # shift for limit pruning part
}Ukwengeza, umkhawulo osuselwe ekuchezukeni okujwayelekile uyasetshenziswa. Umgomo uwukukhawulela ingxenye esusiwe, ngaphandle kokuguqulwa okunezinyathelo ezivele βzaneleβ ze-L1:
Ngakho-ke, sikuvumela ukuthi ususe kuphela ama-convolutions abuthaka ekusabalaliseni okufana nesokudla futhi ungaphazamisi ukukhishwa ekusabalaliseni okufana nokwesobunxele:
Uma ukusatshalaliswa kusondela kokujwayelekile, i-pruning_standart_deviation_part coefficient ingakhethwa kulokhu:
Ngincoma ukucatshangwa kwe-2 sigma. Noma ungaziba lesi sici, ushiye inani <1.0.
Okukhiphayo igrafu yosayizi wenethiwekhi, ukulahleka, nesikhathi sokusebenza senethiwekhi salo lonke uhlolo, olujwayeleke lube ngu-1.0. Isibonelo, lapha usayizi wenethiwekhi wehliswe cishe izikhathi ezi-2 ngaphandle kokulahlekelwa ikhwalithi (inethiwekhi encane yokuguqulwa enesisindo esingu-100k):
Ijubane lokugijima lingaphansi kokuguquguquka okuvamile futhi lihlala lingashintshile. Kunencazelo yalokhu:
- Inani lama-convolutions liyashintsha lisuka kokulula (32, 64, 128) liye kumakhadi wevidiyo alungele kakhulu - 27, 51, njll. Ngingaba nephutha lapha, kodwa cishe kunomphumela.
- I-architecture ayibanzi, kodwa iyahambisana. Ngokunciphisa ububanzi, asithinti ukujula. Ngakho, sinciphisa umthwalo, kodwa ungashintshi isivinini.
Ngakho-ke, ukuthuthukiswa kuboniswe ekunciphiseni umthwalo we-CUDA ngesikhathi sokugijima ngo-20-30%, kodwa hhayi ekunciphiseni isikhathi sokugijima.
Imiphumela
Ake sicabange. Sicabangele izinketho ezi-2 zokuthena - ze-YOLOv3 (uma kufanele usebenze ngezandla zakho) kanye namanethiwekhi anezakhiwo ezilula. Kungabonakala ukuthi kuzo zombili izimo kungenzeka ukufeza ukunciphisa usayizi wenethiwekhi nokusheshisa ngaphandle kokulahlekelwa ukunemba. Imiphumela:
- Ukunciphisa usayizi
- Ukusheshisa ukugijima
- Ukunciphisa Umthwalo we-CUDA
- Ngenxa yalokho, ubungane bemvelo (Sithuthukisa ukusetshenziswa kwesikhathi esizayo kwezinsiza zekhompyutha. Kwenye indawo umuntu uyajabula )
isithasiselo
- Ngemuva kwesinyathelo sokuthena, ungangeza ukulinganisa (isibonelo, nge-TensorRT)
- I-Tensorflow ihlinzeka ngamakhono we . Iyasebenza.
- Ngifuna ukuthuthukisa futhi ngizokujabulela ukusiza
Source: www.habr.com