Ua ntej koj dua yog txoj hauj lwm ntawm kev ntes cov khoom. Qhov tseem ceeb tshaj plaws yog kev ceev ntawm kev ua haujlwm nrog qhov ua tau raug. Koj coj YOLOv3 architecture thiab cob qhia nws ntxiv. Qhov tseeb (mAp75) yog ntau dua 0.95. Tab sis tus nqi khiav tseem tsawg. Crap.
Niaj hnub no peb yuav bypass quantization. Thiab nyob rau hauv lub txiav peb yuav saib Qauv Pruning - trimming redundant seem ntawm lub network kom ceev Inference yam tsis poob qhov raug. Nws yog qhov tseeb qhov twg, ntau npaum li cas thiab yuav txiav li cas. Cia peb xav txog yuav ua li cas ua qhov no manually thiab qhov twg koj tuaj yeem automate nws. Thaum kawg muaj ib lub repository ntawm keras.
Taw qhia
Ntawm kuv qhov chaw ua haujlwm dhau los, Macroscop hauv Perm, Kuv tau txais ib tus cwj pwm - los saib xyuas lub sijhawm ua tiav ntawm algorithms. Thiab nco ntsoov xyuas lub sijhawm ua haujlwm network los ntawm kev lim dej txaus. Feem ntau lub xeev-ntawm-tus-kos duab hauv ntau lawm tsis dhau qhov lim no, uas coj kuv mus rau Pruning.
Pruning yog ib lub ntsiab lus qub uas tau tham hauv hauv 2017. Lub tswv yim tseem ceeb yog txo qhov loj ntawm cov kev cob qhia network yam tsis tau poob qhov tseeb los ntawm kev tshem tawm ntau cov nodes. Nws suab txias, tab sis kuv tsis tshua hnov ββββtxog nws siv. Tej zaum, tsis muaj kev siv txaus, tsis muaj cov lus hais lus Lavxias, lossis tsuas yog txhua tus txiav txim siab nws pruning paub thiab nyob twj ywm.
Tab sis cia wb sib cais
Ib glimpse rau biology
Kuv nyiam nws thaum Deep Learning saib cov tswv yim los ntawm biology. Lawv, zoo li evolution, tuaj yeem ntseeg tau (koj puas paub tias ReLU zoo ib yam li ?)
Tus Qauv Pruning txheej txheem kuj nyob ze rau biology. Lub network cov lus teb ntawm no tuaj yeem muab piv rau plasticity ntawm lub hlwb. Muaj ob peb qhov piv txwv nthuav hauv phau ntawv. :
- Lub paj hlwb ntawm ib tug poj niam uas yug los tsuas yog ib nrab tau reprogrammed nws tus kheej los ua cov haujlwm ntawm ib nrab uas ploj lawm.
- Tus txiv neej tua ib feem ntawm nws lub hlwb lub luag hauj lwm rau kev pom. Nyob rau tib lub sijhawm, lwm qhov ntawm lub paj hlwb tau siv cov haujlwm no. (peb tsis sim rov hais dua)
Ib yam li ntawd, koj tuaj yeem txiav tawm qee qhov tsis muaj zog convolutions ntawm koj tus qauv. Raws li qhov chaw kawg, cov pob khoom seem yuav pab hloov cov txiav.
Koj puas nyiam Hloov Kev Kawm lossis koj puas kawm los ntawm kos?
Xaiv tus naj npawb ib. Koj siv Hloov Kev Kawm ntawm Yolov3. Retina, Mask-RCNN lossis U-Net. Tab sis feem ntau peb tsis tas yuav paub txog 80 yam khoom xws li hauv COCO. Hauv kuv qhov kev xyaum, txhua yam tsuas yog txwv rau qib 1-2. Ib tug yuav xav hais tias lub architecture rau 80 chav kawm yog redundant ntawm no. Qhov no qhia tias architecture yuav tsum tau ua kom me me. Ntxiv mus, kuv xav ua qhov no yam tsis tau poob qhov hnyav uas twb tau kawm ua ntej lawm.
Kev xaiv thib ob. Tej zaum koj muaj ntau cov ntaub ntawv thiab suav cov peev txheej, lossis tsuas yog xav tau ib qho kev cai super-kev cai. Tsis muaj teeb meem. Tab sis koj tab tom kawm lub network los ntawm kos. Cov txheej txheem ib txwm yog saib cov ntaub ntawv qauv, xaiv ib qho kev tsim qauv uas yog EXCESSIVE nyob rau hauv lub hwj chim, thiab thawb tawm los ntawm retraining. Kuv pom 0.6 tso tseg, Karl.
Hauv ob qho tib si, lub network tuaj yeem txo qis. Txhawb siab. Tam sim no cia peb mus saib seb hom kev txiav plaub hau yog dab tsi
General algorithm
Peb txiav txim siab tias peb tuaj yeem tshem cov pob khoom. Nws zoo nkaus li yooj yim heev:
Tshem tawm ib qho kev sib cav yog kev ntxhov siab rau lub network, uas feem ntau ua rau qee qhov yuam kev. Ntawm qhov tod tes, qhov kev ua yuam kev no nce ntxiv yog qhov qhia tau tias yog peb tshem tawm cov kev sib cav li cas (piv txwv li, qhov nce loj qhia tias peb tab tom ua qee yam tsis raug). Tab sis qhov nce me me yog qhov zoo heev thiab feem ntau raug tshem tawm los ntawm kev cob qhia ntxiv tom ntej nrog LR me me. Ntxiv ib kauj ruam qhia ntxiv:
Tam sim no peb yuav tsum paub seb thaum twg peb xav nres peb Kev Kawm <-> Pruning voj. Tej zaum yuav muaj cov kev xaiv txawv ntawm no thaum peb yuav tsum tau txo lub network rau qee qhov loj thiab ceev (piv txwv li, rau cov khoom siv txawb). Txawm li cas los xij, qhov kev xaiv ntau tshaj plaws yog mus txuas ntxiv mus kom txog thaum qhov kev ua yuam kev dhau los ua siab tshaj qhov tau txais. Ntxiv ib qho xwm txheej:
Yog li, algorithm ua kom pom tseeb. Nws tseem yuav paub txog yuav ua li cas los txiav txim siab cov teeb meem deleted.
Tshawb nrhiav cov pob khoom deleted
Peb yuav tsum tshem tawm qee qhov kev sib cav. Rushing ua ntej thiab "tua" leej twg yog lub tswv yim phem, txawm tias nws yuav ua haujlwm. Tab sis txij li thaum koj muaj lub taub hau, koj tuaj yeem xav thiab sim xaiv "tsis muaj zog" convolutions rau kev tshem tawm. Muaj ntau txoj kev xaiv:
- . Lub tswv yim hais tias convolutions nrog qhov hnyav me me ua rau me me rau qhov kev txiav txim zaum kawg
- Qhov tsawg tshaj plaws L1-kev ntsuas coj mus rau hauv tus account txhais tau tias thiab tus qauv sib txawv. Peb ntxiv nrog rau kev ntsuam xyuas ntawm qhov xwm txheej ntawm kev faib khoom.
- . Kev txiav txim siab ntau dua ntawm qhov tsis tseem ceeb convolutions, tab sis siv sij hawm ntau thiab siv cov peev txheej.
- Lwm yam
Txhua qhov kev xaiv muaj txoj cai rau lub neej thiab nws txoj kev siv cov yam ntxwv. Ntawm no peb xav txog qhov kev xaiv nrog qhov tsawg tshaj plaws L1- ntsuas
Phau ntawv txheej txheem rau YOLOv3
Tus thawj architecture muaj cov seem seem. Tab sis txawm tias lawv txias npaum li cas rau kev sib sib zog nqus sib sib zog nqus, lawv yuav cuam tshuam peb qee yam. Qhov nyuaj yog tias koj tsis tuaj yeem tshem tawm cov kev sib raug zoo nrog cov indexes sib txawv hauv cov khaubncaws sab nraud povtseg:
Yog li ntawd, cia peb xaiv cov khaubncaws sab nraud povtseg uas peb tuaj yeem tshem tawm kev sib raug zoo:
Tam sim no cia peb tsim lub voj voog ua haujlwm:
- Uploading activations
- Nrhiav seb yuav txiav ntau npaum li cas
- Txiav tawm
- Kawm 10 lub sijhawm nrog LR = 1e-4
- Kev sim
Unloading convolutions muaj txiaj ntsig los kwv yees ntau npaum li cas peb tuaj yeem tshem tawm ntawm ib kauj ruam. Unloading piv txwv:
Peb pom tias yuav luag txhua qhov chaw 5% ntawm kev sib cav muaj qhov qis L1-tus qauv thiab peb tuaj yeem tshem tawm lawv. Ntawm txhua kauj ruam, qhov kev tshem tawm no tau rov ua dua thiab kev ntsuam xyuas tau ua los ntawm cov txheej txheem twg thiab muaj pes tsawg tus tuaj yeem txiav tawm.
Tag nrho cov txheej txheem tau ua tiav hauv 4 kauj ruam (tus lej ntawm no thiab txhua qhov chaw rau RTX 2060 Super):
| Kauj ruam | mAp 75 | Tus naj npawb ntawm cov tsis muaj, lab | Network loj, mb | Los ntawm qhov pib, % | Lub sijhawm ua haujlwm, ms | Kev txiav txiav |
|---|---|---|---|---|---|---|
| 0 | 0.9656 | 60 | 241 | 100 | 180 | - |
| 1 | 0.9622 | 55 | 218 | 91 | 175 | 5% ntawm tag nrho cov |
| 2 | 0.9625 | 50 | 197 | 83 | 168 | 5% ntawm tag nrho cov |
| 3 | 0.9633 | 39 | 155 | 64 | 155 | 15% rau txheej nrog 400+ convolutions |
| 4 | 0.9555 | 31 | 124 | 51 | 146 | 10% rau txheej nrog 100+ convolutions |
Ib qho txiaj ntsig zoo tau ntxiv rau qib 2 - batch loj 4 haum rau hauv kev nco, uas ua rau kom cov txheej txheem ntawm kev cob qhia ntxiv.
Nyob rau theem 4, cov txheej txheem raug tso tseg vim txawm tias mus sij hawm ntev ntxiv kev cob qhia tsis tau tsa mAp75 rau cov nqi qub.
Raws li qhov tshwm sim, peb tswj kom nrawm qhov kev xav los ntawm 15%, txo qhov loj los ntawm 35% thiab tsis poob raws nraim.
Automation rau yooj yim architectures
Rau kev yooj yim network architectures (tsis muaj kev cai ntxiv, concatenate thiab residual blocks), nws yog ib qho ua tau kom tsom mus rau kev ua tiav tag nrho cov txheej txheem convolutional thiab automate cov txheej txheem ntawm kev txiav tawm convolutions.
Kuv tau siv qhov kev xaiv no .
Nws yog qhov yooj yim: koj tsuas yog xav tau lub luag haujlwm poob, qhov zoo tshaj plaws thiab cov tshuab hluav taws xob batch:
import pruning
from keras.optimizers import Adam
from keras.utils import Sequence
train_batch_generator = BatchGenerator...
score_batch_generator = BatchGenerator...
opt = Adam(lr=1e-4)
pruner = pruning.Pruner("config.json", "categorical_crossentropy", opt)
pruner.prune(train_batch, valid_batch)Yog tias tsim nyog, koj tuaj yeem hloov pauv cov kev txwv tsis pub dhau:
{
"input_model_path": "model.h5",
"output_model_path": "model_pruned.h5",
"finetuning_epochs": 10, # the number of epochs for train between pruning steps
"stop_loss": 0.1, # loss for stopping process
"pruning_percent_step": 0.05, # part of convs for delete on every pruning step
"pruning_standart_deviation_part": 0.2 # shift for limit pruning part
}Tsis tas li ntawd, ib qho kev txwv raws li tus qauv sib txawv yog siv. Lub hom phiaj yog txhawm rau txwv qhov uas raug tshem tawm, tsis suav nrog kev sib koom ua ke nrog kev ntsuas "tsim" L1:
Yog li, peb tso cai rau koj tshem tawm tsuas yog tsis muaj zog convolutions los ntawm kev faib khoom zoo ib yam li txoj cai thiab tsis cuam tshuam rau kev tshem tawm ntawm kev faib tawm zoo ib yam li sab laug:
Thaum qhov kev faib tawm mus txog qhov qub, pruning_standart_deviation_part coefficient tuaj yeem xaiv los ntawm:
Kuv pom zoo qhov kev xav ntawm 2 sigma. Lossis koj tuaj yeem tsis quav ntsej qhov no, tawm hauv tus nqi <1.0.
Cov zis yog ib daim duab ntawm lub network loj, poob, thiab lub sijhawm ua haujlwm network rau tag nrho cov kev xeem, normalized rau 1.0. Piv txwv li, ntawm no lub network loj tau txo los ntawm yuav luag 2 zaug yam tsis tau poob zoo (me me convolutional network nrog 100k hnyav):
Kev khiav ceev yog nyob rau hauv ib txwm hloov pauv thiab tseem zoo li tsis hloov. Muaj lus piav qhia rau qhov no:
- Tus naj npawb ntawm convolutions hloov los ntawm qhov yooj yim (32, 64, 128) kom tsis yooj yim tshaj rau daim npav video - 27, 51, thiab lwm yam. Kuv tuaj yeem ua yuam kev ntawm no, tab sis feem ntau nws yuav muaj txiaj ntsig.
- Lub architecture tsis dav, tab sis zoo ib yam. Los ntawm kev txo qhov dav, peb tsis cuam tshuam qhov tob. Yog li, peb txo cov load, tab sis tsis txhob hloov qhov ceev.
Yog li ntawd, qhov kev txhim kho tau nthuav tawm hauv kev txo qis hauv CUDA load thaum lub sij hawm khiav los ntawm 20-30%, tab sis tsis nyob rau hauv ib tug txo nyob rau hauv lub sij hawm khiav.
Cov txiaj ntsim tau los
Cia peb xav txog. Peb txiav txim siab 2 txoj kev xaiv rau pruning - rau YOLOv3 (thaum koj yuav tsum ua haujlwm nrog koj txhais tes) thiab rau tes hauj lwm nrog cov qauv yooj yim. Nws tuaj yeem pom tau tias nyob rau hauv ob qho tib si nws muaj peev xwm ua tiav kev sib txuas ntawm qhov loj me thiab kev nrawm yam tsis muaj qhov raug. Cov txiaj ntsig:
- Txo qhov loj me
- Acceleration khiav
- Txo CUDA Load
- Yog li ntawd, ib puag ncig tus phooj ywg (Peb optimize lub neej yav tom ntej siv cov kev pab cuam xam. Qhov chaw ib tug zoo siab )
Lus Qhia Ntxiv
- Tom qab pruning kauj ruam, koj tuaj yeem ntxiv quantization (piv txwv li, nrog TensorRT)
- Tensorflow muab peev xwm rau . Ua haujlwm.
- Kuv xav txhim kho thiab yuav zoo siab pab
Tau qhov twg los: www.hab.com