Dabarar Jedi don rage hanyoyin sadarwa na juyin juya hali - pruning

Kafin ku kuma shine aikin gano abubuwa. fifiko shine saurin aiki tare da daidaito karbuwa. Kuna ɗaukar gine-ginen YOLOv3 kuma ku ƙara horar da shi. Daidaiton (mAp75) ya fi 0.95. Amma har yanzu adadin gudu yana da ƙasa. Abin banza.

A yau za mu ketare ƙididdiga. Kuma a ƙarƙashin yanke za mu duba Samfurin Yankewa - datsa sassa na cibiyar sadarwa don hanzarta Inference ba tare da asarar daidaito ba. Ya bayyana a ina, nawa da yadda za a yanke. Bari mu gano yadda ake yin wannan da hannu da kuma inda zaku iya sarrafa shi ta atomatik. A ƙarshe akwai wurin ajiya akan keras.

Gabatarwar

A wurin aikina na baya, Macroscop a Perm, Na sami ɗabi'a ɗaya - don koyaushe saka idanu lokacin aiwatar da algorithms. Kuma ko da yaushe duba lokacin aiki na cibiyar sadarwa ta hanyar isasshiyar tacewa. Yawancin lokaci na zamani a samarwa ba ya wuce wannan tacewa, wanda ya kai ni ga Pruning.

Tsoho wani batu ne da aka tattauna a ciki Stanford laccoci a shekarar 2017. Babban ra'ayi shine a rage girman cibiyar sadarwar da aka horar ba tare da rasa daidaito ba ta hanyar cire nodes daban-daban. Yana da kyau, amma ba na jin labarin amfani da shi. Wataƙila, babu isassun abubuwan aiwatarwa, babu labarin yaren Rashanci, ko kuma kawai kowa yana ɗaukan yadda ake yin shuru kuma ya yi shuru.
Amma bari mu ware

A hango cikin ilmin halitta

Ina son shi lokacin da zurfin koyo ya dubi ra'ayoyin da suka fito daga ilmin halitta. Su, kamar juyin halitta, ana iya amincewa da su (shin kun san cewa ReLU yayi kama da aikin kunna neuron a cikin kwakwalwa?)

Tsarin Shuke Model shima yana kusa da ilmin halitta. Ana iya kwatanta martanin hanyar sadarwa a nan da filastik na kwakwalwa. Akwai misalai biyu masu ban sha'awa a cikin littafin. Norman Doidge:

1. Kwakwalwar macen da aka haifa da rabi kawai ta sake tsara kanta don yin ayyukan rabin da ya ɓace.
2. Mutumin ya harbe sashin kwakwalwar sa da ke da alhakin gani. Bayan lokaci, sauran sassan kwakwalwa sun ɗauki waɗannan ayyuka. (ba muna ƙoƙarin maimaitawa ba)

Hakanan, zaku iya yanke wasu rarraunawar rikice-rikice daga ƙirar ku. A matsayin maƙasudin ƙarshe, ragowar daure zai taimaka maye gurbin da aka yanke.

Kuna son Canja wurin Learning ko kuna koyo daga karce?

Zabi lamba daya. Kuna amfani da Koyon Canja wurin akan Yolov3. Retina, Mask-RCNN ko U-Net. Amma mafi yawan lokuta ba ma buƙatar gane nau'ikan abubuwa 80 kamar na COCO. A cikin aikina, komai yana iyakance ga maki 1-2. Mutum na iya ɗauka cewa gine-gine na azuzuwan 80 ba su da yawa a nan. Wannan yana nuna cewa gine-ginen yana buƙatar ƙarami. Bugu da ƙari, Ina so in yi wannan ba tare da rasa ma'aunin da aka riga aka horar da shi ba.

Zabi na biyu. Wataƙila kuna da bayanai da yawa da albarkatun kwamfuta, ko kawai kuna buƙatar babban gine-gine na al'ada. Ba komai. Amma kuna koyon hanyar sadarwa daga karce. Hanyar da aka saba ita ce duba tsarin bayanai, zaɓi tsarin gine-ginen da yake da WUYA a cikin iko, da kuma tura waɗanda suka fice daga sake horarwa. Na ga raguwar 0.6, Karl.

A cikin lokuta biyu, ana iya rage cibiyar sadarwa. Ƙarfafawa. Yanzu bari mu je mu gano ko wane irin yankan kaciya ne

Gabaɗaya algorithm

Mun yanke shawarar cewa za mu iya cire dauren. Yana kama da sauki:

Cire duk wani rikice-rikice yana da damuwa ga hanyar sadarwa, wanda yawanci yana haifar da ƙarin kuskure. A gefe guda, wannan haɓakar kuskure yana nuna daidai yadda muke cire rikice-rikice (misali, haɓaka mai girma yana nuna cewa muna yin wani abu ba daidai ba). Amma ƙaramin haɓaka yana da karɓuwa sosai kuma galibi ana kawar da shi ta hanyar ƙarin horo na gaba tare da ƙaramin LR. Ƙara ƙarin matakin horo:

Yanzu muna buƙatar gano lokacin da muke son dakatar da Koyonmu<->Pruning madauki. Ana iya samun zaɓuka masu ban sha'awa anan lokacin da muke buƙatar rage hanyar sadarwar zuwa takamaiman girman da sauri (misali, na na'urorin hannu). Koyaya, zaɓin gama gari shine ci gaba da zagayowar har sai kuskuren ya zama mafi girma fiye da karɓuwa. Ƙara sharadi:

Don haka, algorithm ya zama bayyananne. Ya rage don gano yadda za a tantance rikice-rikicen da aka goge.

Nemo fakitin da aka goge

Muna buƙatar cire wasu rikice-rikice. Gudun gaba da "harbi" kowa mummunan ra'ayi ne, ko da yake zai yi aiki. Amma tunda kuna da kai, zaku iya yin tunani kuma kuyi ƙoƙarin zaɓar juzu'in "rauni" don cirewa. Akwai zaɓuɓɓuka da yawa:

1. Mafi ƙarancin ma'auni na L1 ko ƙaramin_magnitude_pruning. Tunanin cewa rikice-rikice tare da ƙananan ma'auni suna ba da gudummawa kaɗan ga yanke shawara na ƙarshe
2. Mafi ƙarancin ma'auni na L1 yana la'akari da ma'ana da daidaitattun sabani. Muna kari tare da kimanta yanayin rarraba.
3. Yin jujjuyawar rufe fuska da ban da waɗanda ba su da tasiri a kan daidaiton ƙarshe. Ingantattun ƙayyadaddun ƙayyadaddun rikice-rikice marasa mahimmanci, amma mai ɗaukar lokaci da cin albarkatu.
4. Sauran

Kowane ɗayan zaɓuɓɓuka yana da haƙƙin rayuwa da fasalin aiwatar da kansa. Anan muna la'akari da zaɓi tare da mafi ƙarancin L1-ma'auni

Tsarin hannu don YOLOv3

Gine-gine na asali ya ƙunshi ragowar tubalan. Amma duk yadda suke da kyau don hanyoyin sadarwa masu zurfi, za su ɗan yi mana cikas. Matsalar ita ce ba za ku iya share sulhu tare da fihirisa daban-daban a cikin waɗannan yadudduka ba:

Don haka, bari mu zaɓi yadudduka waɗanda za mu iya share sulhu cikin yardar kaina:

Yanzu bari mu gina tsarin aiki:

1. Ana loda kunnawa
2. Gano nawa za a yanke
3. Yanke shi
4. Koyan zamanin 10 tare da LR=1e-4
5. Gwaji

Zazzage rikice-rikice yana da amfani don ƙididdige sashi nawa za mu iya cirewa a wani mataki. Ana saukewa misalai:

Mun ga cewa kusan ko'ina 5% na convolutions suna da ƙananan L1-al'ada kuma za mu iya cire su. A kowane mataki, an sake sauke wannan saukewa kuma an yi kima akan wane yadudduka da nawa za a iya yanke.

An kammala dukkan tsarin a cikin matakai 4 (lambobi anan da ko'ina don RTX 2060 Super):

An ƙara sakamako ɗaya mai kyau zuwa mataki na 2 - girman batch 4 ya dace da ƙwaƙwalwar ajiya, wanda ya haɓaka aiwatar da ƙarin horo.
A mataki na 4, an dakatar da tsarin saboda hatta ƙarin horo na dogon lokaci bai ɗaga mAp75 zuwa tsoffin ƙima ba.
A sakamakon haka, mun gudanar da hanzarin ƙaddamarwa ta hanyar 15%, rage girman ta 35% kuma ba a rasa daidai ba.

Yin aiki da kai don mafi sauƙin gine-gine

Don mafi sauƙin tsarin gine-ginen cibiyar sadarwa (ba tare da ƙara sharadi ba, haɗaka da sauran tubalan), yana yiwuwa a mai da hankali kan sarrafa duk yadudduka na juyi da sarrafa sarrafa tsarin yanke juzu'ai.

Na aiwatar da wannan zaɓi a nan.
Abu ne mai sauƙi: kawai kuna buƙatar aikin asara, mai ingantawa da janareta batch:

import pruning
from keras.optimizers import Adam
from keras.utils import Sequence

train_batch_generator = BatchGenerator...
score_batch_generator = BatchGenerator...

opt = Adam(lr=1e-4)
pruner = pruning.Pruner("config.json", "categorical_crossentropy", opt)

pruner.prune(train_batch, valid_batch)

Idan ya cancanta, zaku iya canza sigogin saiti:

{
    "input_model_path": "model.h5",
    "output_model_path": "model_pruned.h5",
    "finetuning_epochs": 10, # the number of epochs for train between pruning steps
    "stop_loss": 0.1, # loss for stopping process
    "pruning_percent_step": 0.05, # part of convs for delete on every pruning step
    "pruning_standart_deviation_part": 0.2 # shift for limit pruning part
}

Ƙari ga haka, ana aiwatar da ƙayyadaddun ƙayyadaddun ƙayyadaddun sabawa. Manufar ita ce iyakance ɓangaren da aka cire, ban da rikice-rikice tare da “isasshen” matakan L1:

Don haka, muna ba ku damar cire rikice-rikice masu rauni kawai daga rabawa masu kama da na dama kuma ba za su shafi cirewa daga rabo mai kama da na hagu ba:

Lokacin da rabon ya kusanci al'ada, ana iya zaɓar madaidaicin pruning_standart_deviation_part daga:

Ina ba da shawarar zato na sigma 2. Ko kuna iya watsi da wannan fasalin, barin ƙimar <1.0.

Fitowar jadawali ne na girman cibiyar sadarwa, asara, da lokacin aikin cibiyar sadarwa don ɗaukacin gwajin, wanda aka daidaita zuwa 1.0. Misali, a nan an rage girman cibiyar sadarwa da kusan sau 2 ba tare da asarar inganci ba (ƙaramin cibiyar sadarwa mai jujjuyawa tare da ma'aunin 100k):

Gudun gudu yana ƙarƙashin sauye-sauye na al'ada kuma ya kasance kusan baya canzawa. Akwai bayani akan haka:

1. Yawan juzu'i yana canzawa daga dacewa (32, 64, 128) zuwa ba mafi dacewa don katunan bidiyo - 27, 51, da sauransu. Zan iya yin kuskure a nan, amma mai yiwuwa yana da tasiri.
2. Gine-ginen ba mai faɗi ba ne, amma daidaitacce. Ta hanyar rage nisa, ba za mu shafi zurfin ba. Don haka, muna rage nauyi, amma ba mu canza saurin ba.

Sabili da haka, an bayyana haɓakawa a cikin raguwa a cikin nauyin CUDA a lokacin gudu ta hanyar 20-30%, amma ba a cikin raguwa a lokacin gudu ba.

Sakamakon

Mu yi tunani. Mun yi la'akari da zaɓuɓɓukan 2 don pruning - don YOLOv3 (lokacin da za ku yi aiki da hannuwanku) da kuma hanyoyin sadarwa tare da gine-gine masu sauƙi. Ana iya ganin cewa a cikin duka biyun yana yiwuwa a cimma raguwar girman cibiyar sadarwa da sauri ba tare da asarar daidaito ba. Sakamako:

• Rage girma
• Gudun hanzari
• Rage lodin CUDA
• A sakamakon haka, abokantaka na muhalli (Muna inganta amfani da albarkatun kwamfuta a nan gaba. Wani wuri yana farin ciki Greta Thunberg)

shafi

• Bayan matakin pruning, zaku iya ƙara ƙididdigewa (misali, tare da TensorRT)
• Tensorflow yana ba da damar iyawa don low_magnitude_pruning. Ayyuka.
• wurin ajiya Ina so in haɓaka kuma zan yi farin cikin taimakawa

source: www.habr.com