Pamberi pako zvakare ibasa rekuona zvinhu. Chinonyanya kukosha ndechekumhanya kwekushanda nekururama kunogamuchirwa. Iwe unotora iyo YOLOv3 dhizaini uye woidzidzisa zvakare. Kururama(mAp75) kunopfuura 0.95. Asi chiyero chekumhanya chichiri chakaderera. Crap.
Nhasi tichapfuura quantization. Uye pasi pekucheka tichatarisa Model Pruning - kucheka zvisingaite zvikamu zvetiweki kuti ikurumidze Inference pasina kurasikirwa kwechokwadi. Zviri pachena kupi, yakawanda sei uye sei kucheka. Ngationei maitiro ekuita izvi nemaoko uye kwaungaite otomatiki. Kumagumo kune repository pane keras.
Nhanganyaya
Panzvimbo yangu yekare yebasa, Macroscop muPerm, ndakawana tsika imwe - kugara ndichitarisisa nguva yekuitwa kwealgorithms. Uye gara uchitarisa iyo network yekumhanyisa nguva kuburikidza neyakakwana sefa. Kazhinji mamiriro-e-iyo-anyanzvi mukugadzirwa haapfuure iyi sefa, izvo zvakanditungamira kuPruning.
Kuchekerera inyaya yekare yakakurukurwa mukati muna 2017. Pfungwa huru ndeyekudzikisa saizi yetiweki yakadzidziswa pasina kurasikirwa nemazvo nekubvisa node dzakasiyana. Zvinonzwika zvakanaka, asi ini handiwanzonzwa nezvekushandiswa kwayo. Pamwe, hapana mashandisirwo akakwana, hapana zvinyorwa zvemutauro weRussia, kana munhu wese anozviona sekuchekerera kuziva-kuita uye anoramba akanyarara.
Asi ngatizviparadzanise
Chidzidzo chebiology
Ndinozvifarira kana Kudzidza Kwakadzika kunotarisa mazano anobva kubiology. Ivo, sekushanduka-shanduka, vanogona kuvimbwa (waizviziva here kuti ReLU yakafanana chaizvo ne ?)
Iyo Model Pruning process iri padyo nebiology. Mhinduro yemambure pano inogona kufananidzwa nepurasitiki yehuropi. Pane mienzaniso miviri inofadza mubhuku. :
- Uropi hwemukadzi akaberekwa aine hafu imwe chete yakazvigadzirisa kuti iite mabasa ehafu yakashaikwa.
- Mukomana akapfura chikamu cheuropi hwake chaikonzera kuona. Nokufamba kwenguva, dzimwe nhengo dzouropi dzakatora mabasa aya. (hatisi kuedza kudzokorora)
Saizvozvo, iwe unogona kucheka mamwe asina kusimba convolutions kubva kune yako modhi. Sechisarudzo chekupedzisira, masumbu akasara achabatsira kutsiva akachekwa.
Iwe unoda Transfer Kudzidza kana uri kudzidza kubva kutanga?
Sarudzo nhamba yekutanga. Iwe unoshandisa Transfer Kudzidza paYolov3. Retina, Mask-RCNN kana U-Net. Asi kazhinji yenguva isu hatidi kuziva makumi masere echinhu makirasi senge muCOCO. Mukuita kwangu, zvese zvinongogumira kumagiredhi 80-1. Mumwe anogona kufunga kuti dhizaini yemakirasi makumi masere haina basa pano. Izvi zvinoratidza kuti chivakwa chinoda kuitwa chidiki. Uyezve, ndinoda kuita izvi pasina kurasikirwa nehuremu huripo hwakadzidziswa.
Chisarudzo chechipiri. Pamwe iwe une data rakawanda uye zviwanikwa zvekombuta, kana kungoda super-custom architecture. Hazvina basa. Asi iwe uri kudzidza network kubva pakutanga. Iyo yakajairika maitiro ndeyekutarisa iyo data chimiro, sarudza chivakwa chiri EXCESSIVE musimba, uye kusundira vanodonha kubva pakudzidzirazve. Ndakaona 0.6 vakadonha, Karl.
Muzviitiko zviviri izvi, network inogona kuderedzwa. Kukurudzirwa. Zvino handei tinoona kuti rudzii rwekuchekerera
General algorithm
Takasarudza kuti tinogona kubvisa mabundles. Zvinotaridzika zviri nyore:
Kubvisa chero convolution kunonetsa kune network, izvo zvinowanzo tungamira kune kumwe kuwedzera kwekukanganisa. Kune rumwe rutivi, kuwedzera uku mukukanganisa chiratidzo chekuti isu tinobvisa sei convolutions (semuenzaniso, kuwedzera kukuru kunoratidza kuti tiri kuita chimwe chinhu chakaipa). Asi kuwedzera kudiki kunogamuchirwa uye kazhinji kunobviswa nechiedza chinotevera kudzidziswa neLR diki. Wedzera imwe nhanho yekudzidzira:
Zvino isu toda kufunga patinenge tichida kumisa Kudzidza kwedu<->Pruning loop. Panogona kunge paine ekunze sarudzo pano patinenge tichida kudzikisa network kune imwe saizi uye nekumhanya (semuenzaniso, kune nharembozha). Nekudaro, iyo yakajairika sarudzo ndeyekuenderera mberi kutenderera kusvika kukanganisa kwave kwepamusoro pane kugamuchirwa. Wedzera chimiro:
Saka, iyo algorithm inova yakajeka. Izvo zvinosara kufunga kuti ungaziva sei zvakadzimwa convolutions.
Tsvaga mabundle akadzimwa
Tinofanira kubvisa mamwe maconvolutions. Kumhanyira mberi uye "kupfura" chero munhu ipfungwa yakaipa, kunyange zvazvo ichashanda. Asi sezvo uine musoro, unogona kufunga uye kuedza kusarudza "isina simba" convolutions yekubvisa. Pane zvakawanda zvingasarudzwa:
- . Pfungwa yekuti convolutions ine huremu hudiki inopa mupiro mudiki kune yekupedzisira sarudzo
- Dikidiki L1-chiyero uchifunga nezve zvinoreva uye yakajairwa kutsauka. Isu tinowedzera nekuongororwa kwemhando yekugovera.
- . Kuwedzera kwakaringana kutsunga kwezvisingakoshi convolutions, asi zvakanyanya-kutora nguva uye kushandisa zviwanikwa.
- Zvimwe
Imwe neimwe yesarudzo ine kodzero yehupenyu uye yayo yega kuita maficha. Pano isu tinofunga sarudzo nediki L1-chiyero
Manual maitiro eYOLOv3
Iyo yekutanga dhizaini ine mabhuroko akasara. Asi zvisinei nekuti vanotonhorera sei kune akadzika network, ivo vanotitadzisa. Chinonetsa ndechekuti haugone kudzima kuyananiswa nemaindex akasiyana muzvikamu izvi:
Naizvozvo, ngatisarudzei mataira kubva kwatinogona kudzima zvakasununguka kuyanana:
Zvino ngativake kutenderera kwebasa:
- Uploading activations
- Kuziva kuti ingani yekucheka
- Cheka kunze
- Kudzidza epoch gumi neLR=10e-1
- Testing
Kuburutsa convolutions kunobatsira kufungidzira kuti chikamu chakawanda sei chatinogona kubvisa pane imwe nhanho. Kuburutsa mienzaniso:
Isu tinoona kuti pese pese 5% yemaconvolutions ane yakaderera L1-yakajairwa uye tinogona kuvabvisa. Panhanho imwe neimwe, kuburutswa uku kwaidzokororwa uye ongororo yakaitwa yezvikamu zvipi uye kuti zvingani zvingachekwa.
Maitiro ese akapedzwa mumatanho mana (nhamba pano uye kwese kweiyo RTX 4 Super):
| Nhanho | mAp75 | Nhamba yemiganhu, mamiriyoni | Saizi yetiweki, mb | Kubva pakutanga,% | Nguva yekumhanya, ms | Mamiriro ekudzingiswa |
|---|---|---|---|---|---|---|
| 0 | 0.9656 | 60 | 241 | 100 | 180 | - |
| 1 | 0.9622 | 55 | 218 | 91 | 175 | 5% yezvose |
| 2 | 0.9625 | 50 | 197 | 83 | 168 | 5% yezvose |
| 3 | 0.9633 | 39 | 155 | 64 | 155 | 15% yezvikamu zvine 400+ convolutions |
| 4 | 0.9555 | 31 | 124 | 51 | 146 | 10% yezvikamu zvine 100+ convolutions |
Imwe mhedzisiro yakanaka yakawedzerwa kune nhanho 2 - batch saizi 4 inokodzera mundangariro, iyo yakamhanyisa zvakanyanya maitiro ekuwedzera kudzidziswa.
Padanho rechina, maitiro akamiswa nekuti kunyangwe kudzidziswa kwenguva refu hakuna kusimudza mAp4 kune yekare tsika.
Nekuda kweizvozvo, takakwanisa kukurumidza kufungidzira 15%, kuderedza saizi ne 35% uye kwete kurasikirwa chaizvo.
Automation yezvivakwa zviri nyore
Kune akareruka magadzirirwo etiweki (pasina zvirevo zvekuwedzera, concaternate uye zvakasara zvidhinha), zvinokwanisika kutarisa pakugadzirisa ese convolutional layer uye otomatiki maitiro ekucheka maconvolutions.
Ndakashandisa sarudzo iyi .
Zviri nyore: iwe unongoda basa rekurasikirwa, optimizer uye batch jenareta:
import pruning
from keras.optimizers import Adam
from keras.utils import Sequence
train_batch_generator = BatchGenerator...
score_batch_generator = BatchGenerator...
opt = Adam(lr=1e-4)
pruner = pruning.Pruner("config.json", "categorical_crossentropy", opt)
pruner.prune(train_batch, valid_batch)Kana zvichidikanwa, unogona kushandura config parameters:
{
"input_model_path": "model.h5",
"output_model_path": "model_pruned.h5",
"finetuning_epochs": 10, # the number of epochs for train between pruning steps
"stop_loss": 0.1, # loss for stopping process
"pruning_percent_step": 0.05, # part of convs for delete on every pruning step
"pruning_standart_deviation_part": 0.2 # shift for limit pruning part
}Pamusoro pezvo, kuganhurirwa kunoenderana neyakajairwa kutsauka kunoitwa. Chinangwa ndechekudzikamisa chikamu chinobviswa, kusasanganisa convolutions ine "yakakwana" L1 zviyero:
Nekudaro, isu tinokutendera iwe kuti ubvise chete isina kusimba convolutions kubva mukugovaniswa yakafanana neiyo yekurudyi uye kwete kukanganisa kubviswa kubva mukugovera kwakafanana nekuruboshwe:
Kana kugovera kwasvika kwakajairika, iyo pruning_standart_deviation_part coefficient inogona kusarudzwa kubva:
Ini ndinokurudzira fungidziro ye2 sigma. Kana iwe unogona kufuratira chimiro ichi, uchisiya kukosha <1.0.
Iyo inobuda igirafu yehukuru hwetiweki, kurasikirwa, uye network yekumhanyisa nguva yebvunzo yese, yakajairwa kusvika 1.0. Semuenzaniso, pano saizi yetiweki yakaderedzwa nekanenge ka2 pasina kurasikirwa kwemhando (diki convolutional network ine 100k uremu):
Iyo yekumhanya inomhanya inoenderana neyakajairwa kuchinja uye inoramba isina kushanduka. Pane tsananguro yeizvi:
- Huwandu hwemaconvolutions hunochinja kubva pazviri nyore (32, 64, 128) kuenda kusiri nyore kune makadhi evhidhiyo - 27, 51, nezvimwe. Ndinogona kunge ndakakanganisa pano, asi kazhinji zvine chekuita.
- Iyo dhizaini haina kupamhama, asi inowirirana. Nokuderedza hupamhi, hatikanganisi kudzika. Nokudaro, tinoderedza mutoro, asi usashandura kukurumidza.
Nokudaro, kuvandudzwa kwakaratidzwa mukuderedza mutoro weCUDA panguva yekumhanya ne20-30%, asi kwete mukuderedza nguva yekumhanya.
Migumisiro
Ngatitarisei. Takafunga 2 sarudzo dzekuchekerera - yeYOLOv3 (kana iwe uchifanira kushanda nemaoko ako) uye kune network ine zvivakwa zviri nyore. Zvinogona kuoneka kuti mune ese ari maviri zviitiko zvinokwanisika kuwana network kuderedza saizi uye kukurumidza pasina kurasikirwa kwechokwadi. Mhinduro:
- Kuderedza ukuru
- Kumhanyisa kumhanya
- Kuderedza CUDA Mutoro
- Nekuda kweizvozvo, hushamwari hwezvakatipoteredza (We optimize the future use of computing resources. Somewhere one is happy )
chinamatidzwa
- Mushure medanho rekuchekerera, unogona kuwedzera quantization (semuenzaniso, neTensorRT)
- Tensorflow inopa kugona kwe . Mabasa.
- Ndinoda kukudziridza uye ndichafara kubatsira
Source: www.habr.com