Ngaphambi kwakho kwakhona ngumsebenzi wokukhangela izinto. Okuphambili kukukhawuleza kokusebenza ngokuchaneka okwamkelekileyo. Uthatha iYOLOv3 uyilo kwaye uyiqeqeshe ngakumbi. Ukuchaneka(mAp75) kukhulu kuno-0.95. Kodwa izinga lokubaleka lisephantsi. Crap.
Namhlanje siza kudlula i-quantization. Kwaye phantsi kokusikwa siya kujonga Umzekelo wokuPruna - ukucheba iindawo ezingafunekiyo zenethiwekhi ukukhawulezisa i-Inference ngaphandle kokulahleka kokuchaneka. Kucacile apho, kangakanani kwaye njani ukusika. Makhe sijonge indlela yokwenza oku ngesandla kwaye apho ungayenza ngokuzenzekelayo. Ekugqibeleni kukho indawo yokugcina kwi-keras.
Intshayelelo
Kwindawo yam yangaphambili yokusebenza, iMacroscop ePerm, ndafumana umkhwa omnye-ukuhlala ndibeke iliso ixesha lokwenziwa kwe-algorithms. Kwaye soloko ujonge ixesha lokuqhuba umsebenzi womnatha ngokusebenzisa isihluzo esaneleyo. Ngokuqhelekileyo i-state-of-the-art kwimveliso ayidluli le filter, eyandikhokelela kwiPruning.
Ukuthena sisihloko esidala ebekuxoxwa ngaso ngo-2017. Ingcamango ephambili kukunciphisa ubungakanani benethiwekhi eqeqeshiweyo ngaphandle kokulahlekelwa ukuchaneka ngokususa iindawo ezahlukeneyo. Kuvakala kupholile, kodwa andifane ndive malunga nokusetyenziswa kwayo. Mhlawumbi, akukho ukuphunyezwa okwaneleyo, akukho manqaku olwimi lwesiRashiya, okanye wonke umntu uthatha nje ukuthena ulwazi kwaye uhlala ethule.
Kodwa masiyahlule
Ukukrola kwibhayoloji
Ndiyayithanda into yokuba i-Deep Learning ijonga imibono evela kwibhayoloji. Bona, njengendaleko, banokuthenjwa (ngaba ubusazi ukuba i-ReLU iyafana kakhulu ?)
Inkqubo yokuPrayina uMzekelo nayo isondele kwibhayoloji. Impendulo yenethiwekhi apha inokuthelekiswa neplastiki yengqondo. Kukho imizekelo embalwa enomdla kule ncwadi. :
- Ingqondo yomfazi owazalwa enesiqingatha esinye kuphela iye yazilungiselela ukwenza imisebenzi yesiqingatha esilahlekileyo.
- Lo mfo wadubula inxalenye yengqondo yakhe enoxanduva lokubona. Ekuhambeni kwexesha, amanye amalungu engqondo athatha le misebenzi. (asizami ukuphinda)
Ngokukwanjalo, unokusika ezinye zeeconvolutions ezibuthathaka kwimodeli yakho. Njengendlela yokugqibela, ama-bundle asele aya kunceda ukutshintshela abasikiweyo.
Ngaba uyayithanda iTransfer Learning okanye ufunda ukusuka ekuqaleni?
Inombolo yokhetho enye. Usebenzisa iTransfer Learning kwiYolov3. I-Retina, iMask-RCNN okanye i-U-Net. Kodwa ixesha elininzi akufuneki ukuba siqaphele iiklasi zezinto ezingama-80 ezifana neCOCO. Kwinkqubo yam, yonke into ilinganiselwe kumabakala 1-2. Umntu usenokucinga ukuba uyilo lweeklasi ezingama-80 alunamsebenzi apha. Oku kuthetha ukuba ulwakhiwo kufuneka lwenziwe luncinci. Ngaphezu koko, ndingathanda ukwenza oku ngaphandle kokulahlekelwa ubunzima obusele buqeqeshwe kwangaphambili.
Inombolo yokhetho yesibini. Mhlawumbi unedatha eninzi kunye nezixhobo zekhompyutha, okanye ufuna nje i-architecture ye-super-custom. Ayinamsebenzi. Kodwa ufunda inethiwekhi ukusuka ekuqaleni. Inkqubo yesiqhelo kukujonga ulwakhiwo lwedatha, khetha ulwakhiwo OLUGQIWEYO kumandla, kwaye utyhale abayekayo ekuziqeqesheni kwakhona. Ndabona i-0.6 eyehlayo, uKarl.
Kuzo zombini iimeko, inethiwekhi inokuncitshiswa. Ukhuthazekile. Ngoku makhe sijonge ukuba luhlobo luni lokuthenwa kolwaluko
I-algorithm ngokubanzi
Sagqiba kwelokuba sizisuse ezi bundles. Ikhangeleka ilula kakhulu:
Ukususa nayiphi na i-convolution kuxinzelelo kuthungelwano, oluhlala lukhokelela ekwandeni kwempazamo. Ngakolunye uhlangothi, oku kunyuka kwempazamo kusalathiso sendlela esisusa ngayo ngokuchanekileyo i-convolutions (umzekelo, ukwanda okukhulu kubonisa ukuba senza into engalunganga). Kodwa ukonyuka okuncinci kwamkelekile kwaye kudla ngokupheliswa kukukhanya okulandelayo koqeqesho olongezelelweyo kunye ne-LR encinci. Yongeza inyathelo elongezelelweyo loqeqesho:
Ngoku kufuneka siqonde xa sifuna ukuyeka ukufunda kwethuPruning loop. Kusenokubakho iinketho ezingaqhelekanga apha xa sifuna ukunciphisa inethiwekhi ukuya kubungakanani obuthile kunye nesantya (umzekelo, kwizixhobo eziphathwayo). Nangona kunjalo, olona khetho luqhelekileyo kukuqhubeka nomjikelo de impazamo ibe phezulu kunokwamkelekileyo. Yongeza imeko:
Ke, i-algorithm iyacaca. Kuhlala ukufumanisa indlela yokumisela i-convolutions ecinyiweyo.
Khangela imiqulu ecinyiweyo
Kufuneka sisuse ezinye iiconvolutions. Ukukhawuleza phambili kunye "nokudubula" nabani na ingcamango embi, nangona iya kusebenza. Kodwa ekubeni unentloko, unokucinga kwaye uzame ukukhetha ii-convolutions "ezibuthathaka" zokususwa. Kukho iindlela ezininzi onokukhetha kuzo:
- . Ingcamango yokuba i-convolutions enobunzima obuncinci yenza igalelo elincinci kwisigqibo sokugqibela
- Owona mlinganiselo umncinci we-L1 uthathela ingqalelo intsingiselo kunye nokutenxa okusemgangathweni. Sincedisa ngovavanyo lohlobo losasazo.
- . Ukuzimisela okuchanekileyo ngakumbi kwee-convolutions ezingabalulekanga, kodwa kuthatha ixesha elininzi kunye nobutyebi.
- Okunye
Nganye yeenketho inelungelo lobomi kunye neempawu zayo zokuphunyezwa. Apha siqwalasela ukhetho ngomlinganiselo omncinci we-L1
Inkqubo yeManuwali ye-YOLOv3
I-architecture yokuqala iqulethe iibhloko ezishiyekileyo. Kodwa nokuba zipholile kangakanani na kwiinethiwekhi ezinzulu, ziya kusithintela ngandlel 'ithile. Ubunzima kukuba awukwazi ukucima ulungelelwaniso kunye nezalathisi ezahlukeneyo kolu maleko:
Ke ngoko, makhe sikhethe iileya apho sinokulucima ngokukhululekileyo ulungelelwaniso:
Ngoku makhe umjikelo womsebenzi:
- Kulayishwa ukusebenza
- Ukufumana ukuba yimalini yokusika
- Yisike ngaphandle
- Ukufunda ii-epochs ezili-10 nge-LR=1e-4
- Uvavanyo
Ukothula iiconvolutions kuluncedo ukuqikelela ukuba ingakanani inxalenye esinokuyisusa kwinqanaba elithile. Imizekelo yokukhulula:
Siyabona ukuba phantse yonke indawo i-5% ye-convolutions ine-L1-isiqhelo esisezantsi kwaye sinokuyisusa. Kwinqanaba ngalinye, oku kothulwa kwaphindwa kwaza kwavavanywa ukuba yeyiphi na imigangatho kunye nokuba mingaphi na enokusikwa.
Yonke inkqubo yagqitywa ngamanyathelo ama-4 (amanani apha nakuyo yonke indawo ye-RTX 2060 Super):
| Inyathelo | mAp75 | Inani leeparamitha, izigidi | Ubungakanani bothungelwano, mb | Ukususela ekuqaleni, % | Ixesha lokubaleka, Nksk | Imeko yolwaluko |
|---|---|---|---|---|---|---|
| 0 | 0.9656 | 60 | 241 | 100 | 180 | - |
| 1 | 0.9622 | 55 | 218 | 91 | 175 | 5% yazo zonke |
| 2 | 0.9625 | 50 | 197 | 83 | 168 | 5% yazo zonke |
| 3 | 0.9633 | 39 | 155 | 64 | 155 | I-15% yeeleya ezine-400+ convolutions |
| 4 | 0.9555 | 31 | 124 | 51 | 146 | I-10% yeeleya ezine-100+ convolutions |
Esinye isiphumo esihle songezwa kwisinyathelo sesi-2 - i-batch size 4 ingene kwimemori, eyakhawuleza kakhulu inkqubo yoqeqesho olongezelelweyo.
Kwinqanaba lesi-4, inkqubo yayekwa kuba noqeqesho olongezelelweyo lwexesha elide aluzange lunyuse i-mAp75 kumaxabiso amadala.
Ngenxa yoko, sikwazile ukukhawulezisa i-inference by 15%, nciphisa ubungakanani nge 35% kwaye ungalahlekelwa ncam.
Ukuzenzekela kwizakhiwo ezilula
Ulwakhiwo olulula lwenethiwekhi (ngaphandle kokudibanisa okunemiqathango, i-concaternate kunye neebhloko ezishiyekileyo), kuyenzeka ukuba kugxininise ekuqhubeni zonke iileya ze-convolution kunye nokuzenzekela inkqubo yokusika ii-convolutions.
Ndiphumeze olu khetho .
Ilula: ufuna kuphela umsebenzi welahleko, i-optimizer kunye neejeneretha zebhetshi:
import pruning
from keras.optimizers import Adam
from keras.utils import Sequence
train_batch_generator = BatchGenerator...
score_batch_generator = BatchGenerator...
opt = Adam(lr=1e-4)
pruner = pruning.Pruner("config.json", "categorical_crossentropy", opt)
pruner.prune(train_batch, valid_batch)Ukuba kuyimfuneko, unokutshintsha iiparamitha zoqwalaselo:
{
"input_model_path": "model.h5",
"output_model_path": "model_pruned.h5",
"finetuning_epochs": 10, # the number of epochs for train between pruning steps
"stop_loss": 0.1, # loss for stopping process
"pruning_percent_step": 0.05, # part of convs for delete on every pruning step
"pruning_standart_deviation_part": 0.2 # shift for limit pruning part
}Ukongezelela, umda osekelwe ekuphambukeni okusemgangathweni kuphunyezwa. Injongo kukunciphisa indawo esusiweyo, ngaphandle kokudibanisa kunye nemilinganiselo ye-L1 "eyaneleyo":
Ke, sikuvumela ukuba ususe kuphela ii-convolutions ezibuthathaka kunikezelo olufana nelo lasekunene kwaye ungachaphazeli ukususwa kunikezelo olufana nelo lwasekhohlo:
Xa unikezelo lusondela kwisiqhelo, i-pruning_standart_deviation_part coefficient inokukhethwa kuyo:
Ndincoma ingqikelelo ye-2 sigma. Okanye ungalihoya eli nqaku, ushiye ixabiso <1.0.
Imveliso yigrafu yobungakanani bothungelwano, ilahleko, kunye nexesha lokuqhuba inethiwekhi kulo lonke uvavanyo, oluqhelekileyo ukuya kwi-1.0. Umzekelo, apha isayizi yenethiwekhi yancitshiswa phantse amaxesha ama-2 ngaphandle kokulahleka komgangatho (inethiwekhi encinci ye-convolutional ene-100k weights):
Isantya sokubaleka sixhomekeke ekuguquguqukeni okuqhelekileyo kwaye sihlala singatshintshi. Kukho inkcazo malunga noku:
- Inani le-convolutions litshintsha ukusuka kwizinto ezifanelekileyo (i-32, i-64, i-128) ukuba ingabi yinto efanelekileyo kakhulu kumakhadi evidiyo - 27, 51, njl. Ndingaphazama apha, kodwa kusenokwenzeka ukuba inesiphumo.
- I-architecture ayibanzi, kodwa iyahambelana. Ngokunciphisa ububanzi, asichaphazeli ubunzulu. Ngaloo ndlela, sinciphisa umthwalo, kodwa ungatshintshi isantya.
Ngoko ke, ukuphuculwa kubonakaliswe ekunciphiseni umthwalo weCUDA ngexesha lokuqhuba nge-20-30%, kodwa kungekhona ekunciphiseni ixesha lokuhamba.
Iziphumo
Makhe sicamngce. Siqwalasele iinketho ezi-2 zokuthena - kwi-YOLOv3 (xa kufuneka usebenze ngezandla zakho) kunye neenethiwekhi ezinezakhiwo ezilula. Ingabonwa ukuba kuzo zombini iimeko kunokwenzeka ukufezekisa ukunciphisa ubungakanani benethiwekhi kunye nokukhawuleza ngaphandle kokulahlekelwa kokuchaneka. Iziphumo:
- Ukunciphisa ubungakanani
- Ukukhawulezisa ukubaleka
- Ukunciphisa Umthwalo weCUDA
- Ngenxa yoko, ubuhlobo bokusingqongileyo (Sikhulisa ukusetyenziswa kwezixhobo zekhompyutha kwixesha elizayo. Kwenye indawo umntu uyonwabile )
isihlomelo
- Emva kwenyathelo lokuthena, unokongeza ubungakanani (umzekelo, ngeTensorRT)
- I-Tensorflow ibonelela ngezakhono . Iyasebenza.
- Ndifuna ukuphuhlisa kwaye ndiya kukuvuyela ukunceda
umthombo: www.habr.com