Indlela ye-Jedi yokunciphisa amanethiwekhi e-convolutional - ukuthena

Ubhekene nomsebenzi wokutholwa kwento futhi. Okuhamba phambili yijubane elinokunemba okwamukelekayo. Uthatha isakhiwo se-YOLOv3 bese usiqeqesha kabusha. Ukunemba (mAp75) kukhulu kuno-0.95. Kodwa isikhathi sokusebenza sisephansi. Hewu.

Namuhla sizokweqa i-quantization. Kodwa ngezansi sizobheka Ukuthena Imodeli — ukusika izingxenye ezingadingekile zenethiwekhi ukusheshisa ukucabanga ngaphandle kokulahlekelwa ukunemba. Lokhu kukhombisa ngokusobala ukuthi kunqunywa kuphi, kungakanani, futhi kanjani. Sizobheka ukuthi sikwenza kanjani lokhu mathupha nalapho kungenziwa khona ngokuzenzakalelayo. Indawo yokugcina ye-Keras iyatholakala ekugcineni.

Isingeniso

Emsebenzini wami wangaphambilini, iMacroscop ePerm, ngaba nomkhuba: ngihlale ngiqapha izikhathi zokubulawa kwe-algorithm. Izikhathi zokusebenza zenethiwekhi zihlala zihlolwa ngesihlungi esanele. Idatha yezinga eliphezulu ekukhiqizweni ngokuvamile ayidluli kulesi sihlungi, okuyikhona okungiholele ekutheneni.

Ukuthena kuyisihloko esidala okuxoxwe ngaso Izifundo zaseStanford ngo-2017. Umqondo oyisisekelo ukunciphisa ubukhulu benethiwekhi eqeqeshiwe ngaphandle kokulahlekelwa ukunemba ngokususa ama-node ahlukahlukene. Kuzwakala kupholile, kodwa angivamile ukuzwa ngokusetshenziswa kwayo. Mhlawumbe akukho ukuqaliswa okwanele, azikho izindatshana zolimi lwesiRashiya, noma wonke umuntu uvele acabangele ukuthena ulwazi futhi athule.
Kodwa kunzima ukuqonda.

Ukubheka kubhayoloji

Ngiyathanda lapho imibono evela kubhayoloji ingena ekuFundeni Okujulile. Bona, njengokuziphendukela kwemvelo, bangathenjwa (bewazi ukuthi i-ReLU ifana kakhulu ne umsebenzi we-neuronal activation ebuchosheni?)

Inqubo yokuthena okuyimodeli nayo iseduze nesayensi yezinto eziphilayo. Impendulo yenethiwekhi lapha ingaqhathaniswa nepulasitiki yobuchopho. Le ncwadi iqukethe izibonelo ezimbalwa ezithakazelisayo. UNorman Doidge:

1. Ubuchopho bowesifazane owazalwa nengxenye eyodwa kuphela buzihlele kabusha ukwenza imisebenzi yengxenye engekho.
2. Umfana wadubula ingxenye yobuchopho bakhe ebhekele ukubona. Ngokuhamba kwesikhathi, ezinye izingxenye zobuchopho zathatha le misebenzi. (Ngeke sizame ukuphinda lokhu.)

Ngokufanayo, ungakwazi ukusika ezinye zezinqwaba ezibuthakathaka kumodeli yakho. Ngengcosana, izinyanda ezisele zingasetshenziswa ukufaka esikhundleni salezo ezisikiwe.

Ingabe uthanda i-Transfer Learning noma ufunda kusukela ekuqaleni?

Inketho yokuqala. Usebenzisa i-Transfer Learning ku-Yolov3, Retina, Mask-RCNN, noma i-U-Net. Kodwa ezikhathini eziningi, asikho isidingo sokuqaphela amakilasi ezinto ezingama-80, njengaku-COCO. Ngokuhlangenwe nakho kwami, kukhawulelwe kumakilasi angu-1-2. Kungase kulingeke ukucabanga ukuthi i-architecture yamakilasi angu-80 idlula ngokweqile. Iphakamisa ukuthi i-architecture idinga ukuncishiswa. Ngaphezu kwalokho, singathanda ukwenza lokhu ngaphandle kokulahlekelwa izisindo ezikhona kakade.

Inombolo yenketho yesibili. Mhlawumbe unedatha eningi nezinsiza zekhompuyutha, noma udinga nje ukwakheka okungokwezifiso okuphezulu. Akunandaba. Kodwa uqeqesha inethiwekhi kusukela ekuqaleni. Inqubo evamile iwukubheka ukwakheka kwedatha, ukhethe i-architecture enamandla KAKHULU, bese ucindezela abashiyayo ukuze ugweme ukufingqa. Ngibone abangu-0.6 abashiya phakathi, Karl.

Kuzo zombili izimo, inethiwekhi ingancishiswa. Ungikhuthazile. Manje ake sithole ukuthi kuyini ukuthena.

I-algorithm evamile

Sinqume ukuthi singawasusa amafolda. Kubukeka kulula kakhulu:

Ukususa noma yikuphi ukuguqulwa kugcizelela inethiwekhi, okuvame ukuholela ekwandeni kwamaphutha athile. Ngakolunye uhlangothi, lokhu kwanda kwephutha kuyinkomba yokuthi siwasusa kahle kangakanani ama-convolutions (isibonelo, ukwanda okukhulu kubonisa ukuthi senza okuthile okungalungile). Kodwa-ke, ukwenyuka kwephutha okuncane kwamukeleka ngokuphelele futhi kungalungiswa ngokuqeqeshwa kabusha kokukhanya okulandelayo nge-LR encane. Ake sengeze isinyathelo sokuqeqesha kabusha:

Manje sidinga ukuthola ukuthi sifuna ukumisa nini i-Learning<->Pruning loop yethu. Kungase kube nezimo ezithile ezingavamile lapha, njengalapho sidinga ukukala inethiwekhi yehlisele kusayizi othize kanye nezinga lokuqalisa (isibonelo, kumadivayisi eselula). Nokho, isimo esivame kakhulu ukuqhubeka neluphu kuze kube yilapho iphutha seliphakeme kunevelu eyamukelekayo. Masingeze umbandela:

Ngakho-ke, i-algorithm icacile. Manje ake sithole ukuthi singanquma kanjani ukuthi yimaphi ama-convolutions okufanele uwasuse.

Sesha okugoqiwe okungasuswa

Sidinga ukususa amaphakheji athile. Ukugijimela futhi "ukudubula" konke kuwumbono omubi, nakuba kuzosebenza. Kodwa uma unamandla obuchopho, ungacabanga ngakho bese uzama ukuhlonza amaphakheji "abuthaka" ukuze asuswe. Kunezinketho ezimbalwa:

1. Isilinganiso esiphansi se-L1 noma ukuthenwa_kobukhulu_okuphansiUmbono wokuthi ukuguquguquka ngamavelu esisindo esincane kunomthelela omncane esinqumweni sokugcina
2. Isikolo esincane kakhulu se-L1 sinikezwe isilinganiso nokuchezuka okujwayelekile. Kungezwe ngesilinganiso semvelo yokusabalalisa.
3. I-convolution masking futhi ingabandakanyi lezo ezinomthelela omncane ekunembeni kokugcinaUkutholwa okunembe kakhudlwana kokugoqa okungabalulekile, kodwa kudla isikhathi esiningi nezinsiza.
4. Okunye

Inketho ngayinye inokusebenza kwayo kanye nokucatshangelwa kokuqaliswa. Lapha, sizocubungula inketho enamaphuzu aphansi we-L1.

Inqubo eyenziwa mathupha ye-YOLOv3

I-architecture yasekuqaleni iqukethe amabhlogo ayinsalela. Kodwa njengoba epholile kumanethiwekhi ajulile, ayisithiyo ngandlela thize. Ubunzima ukuthi asikwazi ukususa ukuqhathanisa nezinkomba ezihlukene kulezi zendlalelo:

Ngakho-ke, ake sikhethe izendlalelo esingasusa kuzo ngokukhululekile ukubuyisana:

Manje ake sakhe umjikelezo womsebenzi:

1. Ikhipha ukusebenza
2. Ake sithole ukuthi kungakanani okumele sikunqume
3. Sinqamule
4. Sifunda izinkathi eziyi-10 nge-LR=1e-4
5. Ukuhlola

Ukukhipha amafolda kuyasiza ekuhloleni ukuthi kungakanani okungasuswa esinyathelweni esinikeziwe. Izibonelo zokukhipha:

Siyabona ukuthi cishe yonke indawo, u-5% wama-convolutions anesimiso esiphansi kakhulu se-L1, futhi singawasusa. Esinyathelweni ngasinye, lokhu kuthululwa kwakuphindwa, futhi kwahlolwa ukuthi yiziphi izendlalelo nokuthi zingaki ezingakhishwa.

Yonke le nqubo ithathe izinyathelo ezine (izinombolo lapha nakwezinye izindawo eze-RTX 2060 Super):

Isinyathelo sesi-2 sengeze umphumela owodwa omuhle: usayizi we-batch 4 ungene kumemori, okusheshise kakhulu inqubo yokuqeqeshwa kabusha.
Esinyathelweni sesi-4, inqubo imisiwe, njengoba ngisho nokuqeqeshwa okwengeziwe kwesikhathi eside akuzange kukhuphule i-mAp75 kumanani amadala.
Ngenxa yalokho, sikwazile ukusheshisa lokho okucatshangwayo 15%, nciphisa usayizi nge 35% futhi ungalahlekelwa ukunemba.

I-automation yezakhiwo ezilula

Ngezakhiwo zenethiwekhi ezilula (ngaphandle kokwengeza okunemibandela, ukuhlanganisa, namabhulokhi ayizinsalela), kungenzeka ukuthi ugxile ekucubunguleni zonke izendlalelo ze-convolutional futhi wenze inqubo yokusika ama-convolutions ngokuzenzakalelayo.

Ngisebenzise le nketho. lapha.
Kulula: okudingayo nje umsebenzi wokulahlekelwa, i-optimizer, namajeneretha enqwaba:

import pruning
from keras.optimizers import Adam
from keras.utils import Sequence

train_batch_generator = BatchGenerator...
score_batch_generator = BatchGenerator...

opt = Adam(lr=1e-4)
pruner = pruning.Pruner("config.json", "categorical_crossentropy", opt)

pruner.prune(train_batch, valid_batch)

Uma kunesidingo, ungashintsha amapharamitha wokumisa:

{
    "input_model_path": "model.h5",
    "output_model_path": "model_pruned.h5",
    "finetuning_epochs": 10, # the number of epochs for train between pruning steps
    "stop_loss": 0.1, # loss for stopping process
    "pruning_percent_step": 0.05, # part of convs for delete on every pruning step
    "pruning_standart_deviation_part": 0.2 # shift for limit pruning part
}

Ukwengeza, umkhawulo osuselwe ekuchezukeni ojwayelekile usetshenzisiwe. Umgomo uwukukhawulela ingxenye yedatha ezosuswa, ngaphandle kokuguqulwa okunezinyathelo ezivele "zanele" ze-L1:

Ngale ndlela, sisusa kuphela ama-convolutions abuthaka ekusabalaliseni okufana nesokudla futhi akuthinti ukususwa ekusabalaliseni okufana nokwesobunxele:

Uma ukusatshalaliswa kusondela kokujwayelekile, i-coefficient pruning_standard_deviation_part ingakhethwa kusukela:

Ngincoma ukubekezelelana oku-2 kwe-sigma. Kungenjalo, ungakwazi ukuziba lesi sici futhi ushiye inani <1.0.

Okukhiphayo igrafu yosayizi wenethiwekhi, ukulahleka, nesikhathi sokusebenza senethiwekhi salo lonke uhlolo, olwejwayelekile ukuze lube ngu-1.0. Isibonelo, lapha usayizi wenethiwekhi wehliswe cishe ngesici esingu-2 ngaphandle kokulahlekelwa yikhwalithi (inethiwekhi encane yokuguqulwa enesisindo esingu-100):

Ijubane lokugijima lingaphansi kokuguquguquka okuvamile futhi lihlale lingashintshile. Kunencazelo yalokhu:

1. Inani lama-convolutions liyashintsha lisuka ku-kulula (32, 64, 128) liye kulawo angakulungele kangako amakhadi evidiyo - 27, 51, njll. Ngingaba nephutha lapha, kodwa cishe lokhu kunomphumela.
2. I-architecture ayibanzi, kodwa iyahambisana. Ngokunciphisa ububanzi, sishiya ukujula kuphelele. Lokhu kunciphisa umthwalo ngaphandle kokuphazamisa isivinini.

Ngakho-ke, ukuthuthukiswa kuholele ekunciphiseni kwe-20-30% kumthwalo we-CUDA ngesikhathi sokugijima, kodwa hhayi ekunciphiseni isikhathi sokugijima.

Imiphumela

Ake sicabange. Sicabangele izinketho ezimbili zokuthena: eyodwa ye-YOLOv3 (edinga ukucutshungulwa ngesandla) kanye neyenethiwekhi enezakhiwo ezilula. Kuyacaca ukuthi kuzo zombili izimo, usayizi wenethiwekhi ungancishiswa futhi ukusheshisa kufinyelelwe ngaphandle kokudela ukunemba. Imiphumela:

• Ukunciphisa usayizi
• Ukusheshisa ukugijima
• Ukunciphisa umthwalo we-CUDA
• Njengomphumela, i-eco-friendly (Sithuthukisa ukusetshenziswa kwesikhathi esizayo kwezinsiza zekhompyutha. Kwenye indawo umuntu uyajabula Greta Thunberg)

isithasiselo

• Ngemva kwesinyathelo sokuthena, ungakwazi futhi ukulungisa ubuningi (isibonelo, nge-TensorRT)
• I-Tensorflow ihlinzeka ngamakhono we ukuthena_okuphansi_kobukhulu. Iyasebenza.
• inqolobane Ngifuna ukuthuthukisa futhi ngingajabula ukuthola usizo

Source: www.habr.com