Njira ya Jedi yochepetsera maukonde a convolutional - kudulira

Pamaso panu kachiwiri ndi ntchito yozindikira zinthu. Chofunika kwambiri ndi kuthamanga kwa ntchito ndi kulondola kovomerezeka. Mumatenga zomanga za YOLOv3 ndikuziphunzitsanso. Kulondola (mAp75) ndikokulirapo kuposa 0.95. Koma liwiro lothamanga likadali lotsika. Zopusa.

Lero tidzalambalala quantization. Ndipo m'munsimu tidzayang'ana Kudulira Chitsanzo - kudula magawo osafunikira a netiweki kuti mufulumizitse Inference popanda kutaya kulondola. Zimadziwika kuti, zingati komanso momwe mungadulire. Tiyeni tiwone momwe tingachitire izi pamanja ndi komwe mungapangire zokha. Pamapeto pake pali posungira pa keras.

Mau oyamba

Kumalo anga akale omwe ndinkagwira ntchito, Macroscop ku Perm, ndinakhala ndi chizolowezi chimodzi - kuyang'anira nthawi zonse zogwiritsira ntchito ma algorithms. Ndipo nthawi zonse fufuzani nthawi yogwiritsira ntchito maukonde kudzera pa fyuluta yokwanira. Kawirikawiri zamakono pakupanga sizidutsa fyuluta iyi, yomwe inanditsogolera ku Kudulira.

Kudulira ndi nkhani yakale yomwe idakambidwamo Maphunziro a Stanford mu 2017. Lingaliro lalikulu ndi kuchepetsa kukula kwa maukonde ophunzitsidwa popanda kutaya kulondola mwa kuchotsa mfundo zosiyanasiyana. Zimamveka bwino, koma nthawi zambiri sindimva za kugwiritsidwa ntchito kwake. Mwinamwake, palibe njira zokwanira, palibe zolemba za chinenero cha Chirasha, kapena aliyense amangoona ngati kudulira luso ndipo amakhala chete.
Koma tiyeni tisiyanitse

Chidziwitso cha biology

Ndimakonda pamene Kuphunzira Mwakuya kumayang'ana malingaliro omwe amachokera ku biology. Iwo, monga chisinthiko, akhoza kudaliridwa (kodi mumadziwa kuti ReLU ndi yofanana kwambiri ndi ntchito ya neuron activation mu ubongo?)

Njira Yodulira Model ilinso pafupi ndi biology. Mayankho a netiweki apa angayerekezedwe ndi pulasitiki ya ubongo. Pali zitsanzo zingapo zosangalatsa m'bukuli. Norman Doidge:

1. Ubongo wa mkazi yemwe anabadwa ndi theka limodzi lokha wadzikonzekeretsa kuti agwire ntchito za theka lomwe likusowa.
2. Mnyamatayo anawombera mbali ya ubongo wake yomwe imayang'anira masomphenya. Patapita nthawi, mbali zina za ubongo zinayamba kugwira ntchito zimenezi. (sitikuyesera kubwereza)

Momwemonso, mutha kudula ma convolutions ofooka kuchokera ku chitsanzo chanu. Monga njira yomaliza, mitolo yotsalayo idzathandizira m'malo odulidwawo.

Kodi mumakonda Transfer Learning kapena mukuphunzira kuyambira poyambira?

Njira nambala wani. Mumagwiritsa ntchito Transfer Learning pa Yolov3. Retina, Mask-RCNN kapena U-Net. Koma nthawi zambiri sitifunika kuzindikira makalasi 80 ngati COCO. M'zochita zanga, zonse zimangokhala magiredi 1-2. Wina angaganize kuti zomanga zamakalasi 80 ndizosowa pano. Lingaliro limadziwonetsera lokha kuti zomangamanga ziyenera kuchepetsedwa. Komanso, ndikufuna kuchita izi popanda kutaya zolemera zomwe zidaphunzitsidwa kale.

Njira yachiwiri. Mwinamwake muli ndi deta yambiri ndi zipangizo zamakompyuta, kapena mumangofunika zomangamanga zapamwamba kwambiri. Zilibe kanthu. Koma mukuphunzira maukonde kuyambira pachiyambi. Zomwe zimachitika nthawi zonse ndikuyang'ana mawonekedwe a data, kusankha kamangidwe kamene kali ndi mphamvu ZOPHUNZITSA, ndikukankhira osiya maphunziro. Ndidawona osiya 0.6, Karl.

Muzochitika zonsezi, maukonde amatha kuchepetsedwa. Kulimbikitsidwa. Tsopano tiyeni tiwone mtundu wa mtundu wa kudulira

General algorithm

Tinaganiza kuti titha kuchotsa mitolo. Zikuwoneka zosavuta:

Kuchotsa convolution kulikonse kumakhala kovutirapo pa netiweki, zomwe nthawi zambiri zimabweretsa kuwonjezereka kwa zolakwika. Kumbali imodzi, kuwonjezeka kwa zolakwika ndi chizindikiro cha momwe timachotsera molondola ma convolutions (mwachitsanzo, kuwonjezeka kwakukulu kumasonyeza kuti tikuchita zolakwika). Koma kuwonjezeka pang'ono ndikovomerezeka ndipo nthawi zambiri kumachotsedwa ndi maphunziro owonjezera owonjezera ndi LR yaying'ono. Onjezani sitepe yowonjezera yophunzitsira:

Tsopano tiyenera kudziwa pamene tikufuna kusiya Kuphunzira kwathu<->Kudulira loop. Pakhoza kukhala zosankha zachilendo pano pamene tikufuna kuchepetsa maukonde kukula ndi liwiro linalake (mwachitsanzo, pazida zam'manja). Komabe, njira yodziwika kwambiri ndikupitilira kuzungulira mpaka cholakwikacho chikhale chachikulu kuposa chovomerezeka. Onjezani chikhalidwe:

Chifukwa chake, algorithm imamveka bwino. Kutsala kudziwa mmene kudziwa zichotsedwa convolutions.

Sakani mitolo zichotsedwa

Tiyenera kuchotsa ma convolutions. Kuthamangira patsogolo ndi "kuwombera" aliyense ndi lingaliro loipa, ngakhale lidzagwira ntchito. Koma popeza muli ndi mutu, mutha kuganiza ndikuyesera kusankha ma convolutions "ofooka" kuti muchotse. Pali zingapo zomwe mungachite:

1. Laling'ono kwambiri L1-mulingo kapena low_magnitude_pruning. Lingaliro lakuti ma convolutions ndi zolemera zazing'ono amapereka zochepa pa chisankho chomaliza
2. Laling'ono kwambiri L1-muyezo poganizira tanthauzo ndi kupatuka wamba. Timawonjezera ndikuwunika momwe amagawira.
3. Kuyika ma convolutions ndikupatula zomwe sizimakhudza kulondola komaliza. Kutsimikiza kolondola kwa ma convolutions osafunikira, koma kuwononga nthawi komanso kugwiritsa ntchito zinthu.
4. Zina

Chilichonse mwazosankha chili ndi ufulu wokhala ndi moyo komanso mawonekedwe ake ake. Apa tikuwona njirayo ndi muyeso wocheperako wa L1

Ndondomeko yamanja ya YOLOv3

Zomangamanga zoyambirira zimakhala ndi midadada yotsalira. Koma ziribe kanthu kuti ali ozizira bwanji pa maukonde akuya, adzatilepheretsa penapake. Vuto ndiloti simungathe kuchotsa kuyanjanitsa ndi ma index osiyanasiyana m'magawo awa:

Chifukwa chake, tiyeni tisankhe zigawo zomwe titha kuchotsamo mwaulere kuyanjanitsa:

Tsopano tiyeni tipange kuzungulira kwa ntchito:

1. Kutsegula ma activation
2. Kuzindikira kuchuluka kwa kudula
3. Dula
4. Kuphunzira ma epoch 10 ndi LR=1e-4
5. Kuyesa

Kutsitsa ma convolutions ndikothandiza kuyerekeza kuchuluka kwa gawo lomwe tingachotse pa sitepe inayake. Zitsanzo zotsitsa:

Tikuwona kuti pafupifupi kulikonse 5% ya convolutions ali otsika kwambiri L1-chizoloŵezi ndipo tikhoza kuwachotsa. Pa sitepe iliyonse, kutsitsa uku kunkabwerezedwanso ndipo kuwunika kunapangidwa kuti ndi zigawo ziti ndi zingati zomwe zingadulidwe.

Ntchito yonseyo idamalizidwa mu masitepe 4 (manambala pano ndi kulikonse kwa RTX 2060 Super):

Khwerero	mp75	Chiwerengero cha magawo, miliyoni	Network size, mb	Kuyambira pachiyambi,%	Nthawi yothamanga, ms	Mdulidwe chikhalidwe
0	0.9656	60	241	100	180	-
1	0.9622	55	218	91	175	5% ya onse
2	0.9625	50	197	83	168	5% ya onse
3	0.9633	39	155	64	155	15% pamagulu okhala ndi 400+ convolutions
4	0.9555	31	124	51	146	10% pamagulu okhala ndi 100+ convolutions

Chotsatira chimodzi chabwino chinawonjezeredwa ku sitepe 2 - kukula kwa batch 4 kulowa mu kukumbukira, komwe kunapititsa patsogolo maphunziro owonjezera.
Pa sitepe 4, ndondomekoyi inaimitsidwa chifukwa ngakhale maphunziro owonjezera a nthawi yayitali sanakweze mAp75 kuzinthu zakale.
Zotsatira zake, tinakwanitsa kufulumizitsa zomwe tafotokozazo 15%, kuchepetsa kukula ndi 35% ndipo osataya ndendende.

Makina opangira mamangidwe osavuta

Pamamangidwe osavuta a netiweki (popanda kuwonjezera zokhazikika, zolumikizirana ndi zotsalira), ndizotheka kuyang'ana kwambiri pakukonza zigawo zonse zosinthira ndikusinthiratu njira yodula ma convolutions.

Ndinagwiritsa ntchito njirayi apa.
Ndi zophweka: mumangofunika ntchito yotayika, chowonjezera ndi majenereta a batch:

import pruning
from keras.optimizers import Adam
from keras.utils import Sequence

train_batch_generator = BatchGenerator...
score_batch_generator = BatchGenerator...

opt = Adam(lr=1e-4)
pruner = pruning.Pruner("config.json", "categorical_crossentropy", opt)

pruner.prune(train_batch, valid_batch)

Ngati ndi kotheka, mutha kusintha magawo a config:

{
    "input_model_path": "model.h5",
    "output_model_path": "model_pruned.h5",
    "finetuning_epochs": 10, # the number of epochs for train between pruning steps
    "stop_loss": 0.1, # loss for stopping process
    "pruning_percent_step": 0.05, # part of convs for delete on every pruning step
    "pruning_standart_deviation_part": 0.2 # shift for limit pruning part
}

Kuonjezera apo, malire ozikidwa pa kupatuka kokhazikika amakhazikitsidwa. Cholinga ndikuchepetsa gawo lomwe lachotsedwa, kupatula ma convolution okhala ndi "zokwanira" L1 miyeso:

Chifukwa chake, tikukulolani kuti muchotse ma convolutions ofooka okha pamagawidwe ofanana ndi oyenera komanso osakhudza kuchotsedwa kwa magawo ofanana ndi kumanzere:

Kugawa kukafika bwino, pruning_standart_deviation_part coefficient ingasankhidwe kuchokera:

Ndikupangira lingaliro la 2 sigma. Kapena mutha kunyalanyaza izi, ndikusiya mtengo <1.0.

Zotsatira zake ndi graph ya kukula kwa netiweki, kutayika, ndi nthawi yothamanga pamaneti pamayeso onse, osinthidwa kukhala 1.0. Mwachitsanzo, apa kukula kwa netiweki kudachepetsedwa pafupifupi ka 2 popanda kutayika bwino (ma network ang'onoang'ono a convolutional okhala ndi zolemera 100k):

Kuthamanga kwa liwiro kumakhala kusinthasintha kwabwinobwino ndipo kumakhala kosasinthika. Pali kufotokozera kwa izi:

1. Kuchuluka kwa ma convolutions kumasintha kuchokera pazabwino (32, 64, 128) kukhala osakhala bwino kwambiri pamakhadi apakanema - 27, 51, ndi zina. Ndikhoza kukhala ndikulakwitsa apa, koma mwina zili ndi zotsatira zake.
2. Zomangamanga si zazikulu, koma zogwirizana. Pochepetsa m'lifupi, sitikhudza kuya. Choncho, timachepetsa katundu, koma osasintha liwiro.

Chifukwa chake, kuwongolerako kudawonetsedwa pakuchepetsa katundu wa CUDA pakuthamanga ndi 20-30%, koma osati pakuchepetsa nthawi yothamanga.

Zotsatira

Tiyeni tilingalire. Taganizirani njira ziwiri zodulira - za YOLOv2 (pamene muyenera kugwira ntchito ndi manja anu) ndi maukonde okhala ndi zomangamanga zosavuta. Zitha kuwoneka kuti muzochitika zonsezi ndizotheka kukwaniritsa kuchepetsa kukula kwa maukonde ndi kuthamanga popanda kutaya kulondola. Zotsatira:

• Kuchepetsa kukula
• Kuthamanga kuthamanga
• Kuchepetsa Katundu wa CUDA
• Zotsatira zake, kuyanjana ndi chilengedwe (Timakulitsa kugwiritsa ntchito mtsogolo kwazinthu zamakompyuta. Penapake munthu amakhala wokondwa Greta Thunberg)

Zakumapeto

• Pambuyo podulira, mutha kuwonjezera kuchuluka (mwachitsanzo, ndi TensorRT)
• Tensorflow imapereka mwayi kwa kutsika_kudula_kudulira. Ntchito.
• posungira Ndikufuna kukulitsa ndipo ndidzakhala wokondwa kuthandiza

Source: www.habr.com