Ma mua o ʻoe ka hana o ka ʻimi ʻana i nā mea. ʻO ka mea nui ka wikiwiki o ka hana me ka pololei kūpono. Lawe ʻoe i ka hoʻolālā YOLOv3 a hoʻomaʻamaʻa hou iā ia. ʻOi aku ka pololei (mAp75) ma mua o 0.95. Akā, haʻahaʻa ka holo. Crap.
I kēia lā e kāpae mākou i ka quantization. A ma lalo o ka ʻoki e nānā mākou ʻOki ʻana hoʻohālike - ka ʻoki ʻana i nā ʻāpana ʻokoʻa o ka pūnaewele e wikiwiki i ka Inference me ka nalowale ʻole o ka pololei. Ua maopopo kahi, pehea ka nui a pehea e ʻoki ai. E noʻonoʻo kākou pehea e hana ai i kēia me ka lima a ma kahi e hiki ai iā ʻoe ke hoʻokaʻawale iā ia. Ma ka hope aia kahi waihona ma keras.
Hōʻike
Ma kaʻu wahi hana ma mua, ʻo Macroscop ma Perm, loaʻa iaʻu kahi maʻamau - e nānā mau i ka manawa hoʻokō o nā algorithms. A e nānā mau i ka manawa holo pūnaewele ma o kahi kānana kūpono. ʻO ka mea maʻamau, ʻaʻole i hala kēia kānana i kēia kānana, ka mea i alakaʻi iaʻu i ka Pruning.
ʻO ka ʻoki ʻana he kumuhana kahiko i kūkākūkā ʻia ma i ka makahiki 2017. ʻO ka manaʻo nui e hōʻemi i ka nui o ka pūnaewele i aʻo ʻia me ka nalowale ʻole o ka pololei ma ka wehe ʻana i nā nodes like ʻole. He leo ʻoluʻolu, akā ʻaʻole au i lohe e pili ana i kona hoʻohana ʻana. Malia paha, ʻaʻole lawa ka hoʻokō ʻana, ʻaʻohe ʻatikala ʻōlelo Lūkini, a i ʻole nā mea a pau e noʻonoʻo i ka ʻike ʻana i ka ʻike a noho mālie.
Akā, e wehe kākou
He ʻike i ka biology
Aloha au i ka nānā ʻana o Deep Learning i nā manaʻo mai ka biology. Hiki iā lākou, e like me ka evolution, ke hilinaʻi ʻia (ua ʻike paha ʻoe ua like like ʻo ReLU me ?)
ʻO ke kaʻina hana ʻoki kumu hoʻohālike e pili pū ana i ka biology. Hiki ke hoʻohālikelike ʻia ka pane o ka ʻupena ma ʻaneʻi me ka palaki o ka lolo. Aia kekahi mau hiʻohiʻona hoihoi i loko o ka puke. :
- ʻO ka lolo o kahi wahine i hānau ʻia me hoʻokahi hapa wale nō ua hoʻonohonoho hou iā ia iho e hana i nā hana o ka hapa nalowale.
- Ua pana ke kanaka i ka ʻāpana o kona lolo nona ka ʻike. I ka wā lōʻihi, ua lawe nā ʻāpana ʻē aʻe o ka lolo i kēia mau hana. (ʻaʻole mākou e hoʻāʻo e hana hou)
Pēlā nō, hiki iā ʻoe ke ʻoki i kekahi o nā convolutions nāwaliwali mai kāu kumu hoʻohālike. Ma keʻano he hopena hope loa, e kōkua nā pūpū i koe e pani i nā mea iʻokiʻia.
Makemake ʻoe i ka Transfer Learning a i ʻole ke aʻo ʻana mai ka ʻōpala?
Helu helu ekahi. Hoʻohana ʻoe i ka Transfer Learning ma Yolov3. Retina, Mask-RCNN a i ʻole U-Net. Akā ʻo ka hapa nui o ka manawa ʻaʻole pono mākou e ʻike i nā papa mea 80 e like me COCO. I kaʻu hoʻomaʻamaʻa, ua kaupalena ʻia nā mea a pau i nā papa 1-2. Manaʻo paha kekahi he mea ʻole ka hoʻolālā no nā papa 80 ma aneʻi. Hōʻike kēia e pono ke hoʻoemi ʻia ka hale hoʻolālā. Eia kekahi, makemake wau e hana i kēia me ka nalowale ʻole o nā paona i hoʻomaʻamaʻa mua ʻia.
Koho helu ʻelua. Loaʻa paha iā ʻoe ka nui o ka ʻikepili a me nā kumuwaiwai helu, a i ʻole pono i kahi hoʻolālā super-custom. He mea ʻole. Akā ke aʻo nei ʻoe i ka ʻupena mai ka wā kahiko. ʻO ke kaʻina hana maʻamau, ʻo ia ka nānā ʻana i ke ʻano o ka ʻikepili, koho i kahi hoʻolālā i ʻoi aku ka mana, a hoʻokuʻu i nā haʻalele mai ke aʻo hou ʻana. Ua ʻike au he 0.6 poʻe haʻalele, Karl.
I nā hihia ʻelua, hiki ke hoʻemi ʻia ka pūnaewele. Hoʻoikaika ʻia. I kēia manawa, e noʻonoʻo kākou i ke ʻano o ke ʻoki poepoe ʻana
ʻO ka algorithm maʻamau
Ua hoʻoholo mākou e hiki iā mākou ke wehe i nā pūʻolo. He mea maʻalahi loa ia:
He mea koʻikoʻi ka wehe ʻana i kekahi convolution no ka pūnaewele, e alakaʻi pinepine ai i kekahi piʻi ʻana o ka hewa. Ma kekahi ʻaoʻao, ʻo kēia piʻi ʻana o ka hewa he hōʻailona ia o ka wehe ʻana i nā convolutions (no ka laʻana, ʻo ka piʻi nui e hōʻike ana e hana hewa ana mākou). Akā ʻae ʻia kahi hoʻonui liʻiliʻi a hoʻopau pinepine ʻia e ka hoʻomaʻamaʻa hou ʻana me kahi LR liʻiliʻi. Hoʻohui i kahi ʻanuʻu aʻo hou:
I kēia manawa pono mākou e noʻonoʻo i ka wā e makemake ai mākou e hoʻōki i kā mākou Learning<->Pruning loop. Aia paha nā koho exotic ma aneʻi inā pono mākou e hōʻemi i ka pūnaewele i kahi nui a me ka wikiwiki (no ka laʻana, no nā polokalamu kelepona). Eia naʻe, ʻo ka koho maʻamau e hoʻomau i ka pōʻai a hiki i ka piʻi ʻana o ka hewa ma mua o ka ʻae ʻia. Hoʻohui i kahi kūlana:
No laila, maopopo ka algorithm. Ke hoʻomau nei e noʻonoʻo pehea e hoʻoholo ai i nā convolutions i holoi ʻia.
Huli i nā pūʻolo i holoi ʻia
Pono mākou e wehe i kekahi mau convolutions. ʻO ka holo wikiwiki a "pana" i kekahi he manaʻo maikaʻi ʻole, ʻoiai e hana ia. Akā, no kou poʻo, hiki iā ʻoe ke noʻonoʻo a hoʻāʻo e koho i nā convolutions "nawaliwali" no ka wehe ʻana. Nui nā koho:
- . ʻO ka manaʻo e hāʻawi liʻiliʻi ka convolutions me nā paona liʻiliʻi i ka hoʻoholo hope
- ʻO ka liʻiliʻi loa L1-ana e noʻonoʻo ana i ka mean a me ka haʻalele maʻamau. Hoʻohui mākou me kahi loiloi o ke ʻano o ka puʻunaue.
- . ʻOi aku ka pololei o ka hoʻoholo ʻana i nā convolutions liʻiliʻi, akā nui ka manawa a me ka waiwai.
- ʻO kekahi
Loaʻa i kēlā me kēia o nā koho ke kuleana i ke ola a me kāna mau hiʻohiʻona hoʻokō ponoʻī. Maanei mākou e noʻonoʻo ai i ke koho me ka liʻiliʻi L1-ana
Kaʻina hana lima no YOLOv3
Aia i loko o ka hale hana kumu nā poloka koena. Akā ʻaʻohe mea ʻoluʻolu o lākou no nā pūnaewele hohonu, e keakea lākou iā mākou. ʻO ka paʻakikī, ʻaʻole hiki iā ʻoe ke hoʻopau i nā hoʻolauleʻa me nā ʻōlelo kuhikuhi like ʻole i kēia mau papa:
No laila, e koho mākou i nā papa kahi e hiki ai iā mākou ke hoʻopau manuahi i nā hoʻolauleʻa:
I kēia manawa, e kūkulu kākou i ka pōʻaiapili hana:
- Hoʻouka ʻana i nā hana
- E noʻonoʻo ana i ka nui e ʻoki ai
- ʻoki ʻoki
- E aʻo ana i 10 mau manawa me LR=1e-4
- Hoʻāʻo
He mea pono ka wehe ʻana i nā convolutions e hoʻohālikelike i ka nui o ka ʻāpana hiki iā mākou ke wehe i kekahi ʻanuʻu. Ke wehe nei i nā laʻana:
ʻIke mākou kokoke i nā wahi āpau he 5% o nā convolutions he haʻahaʻa L1-maʻamau a hiki iā mākou ke wehe iā lākou. I kēlā me kēia ʻanuʻu, ua hana hou ʻia kēia wehe ʻana a ua hana ʻia kahi loiloi o nā papa a me ka nui e hiki ke ʻoki ʻia.
Ua hoʻopau ʻia ke kaʻina holoʻokoʻa ma 4 mau ʻanuʻu (helu ma aneʻi a ma nā wahi āpau no ka RTX 2060 Super):
| Mea hana | mAp75 | Ka helu o nā palena, miliona | Nui pūnaewele, mb | Mai ka mua, % | Ka manawa holo, ms | Ke ʻoki poepoe |
|---|---|---|---|---|---|---|
| 0 | 0.9656 | 60 | 241 | 100 | 180 | - |
| 1 | 0.9622 | 55 | 218 | 91 | 175 | 5% o nā mea a pau |
| 2 | 0.9625 | 50 | 197 | 83 | 168 | 5% o nā mea a pau |
| 3 | 0.9633 | 39 | 155 | 64 | 155 | 15% no nā papa me 400+ convolutions |
| 4 | 0.9555 | 31 | 124 | 51 | 146 | 10% no nā papa me 100+ convolutions |
Hoʻokahi hopena maikaʻi i hoʻohui ʻia i ka ʻanuʻu 2 - ka nui o ka pūʻulu 4 kūpono i ka hoʻomanaʻo, ka mea i hoʻolalelale i ke kaʻina hana o ka hoʻomaʻamaʻa hou.
Ma ka ʻanuʻu 4, ua kāpae ʻia ke kaʻina hana no ka mea ʻaʻole i hoʻonui i ka mAp75 i nā waiwai kahiko.
ʻO ka hopena, ua hoʻokō mākou i ka wikiwiki i ka inference e 15%, hoemi i ka nui ma 35% ʻaʻole e nalowale pololei.
ʻOtomation no nā hale hana maʻalahi
No nā hale hana pūnaewele maʻalahi (me ka hoʻohui ʻole, concaternate a me nā poloka koena), hiki ke nānā pono i ka hoʻoponopono ʻana i nā papa convolutional āpau a hoʻokaʻawale i ke kaʻina o ka ʻoki ʻana i nā convolutions.
Ua hoʻokō au i kēia koho .
He mea maʻalahi: pono wale ʻoe i kahi hana poho, kahi mea hoʻonui a me nā mea hana pū:
import pruning
from keras.optimizers import Adam
from keras.utils import Sequence
train_batch_generator = BatchGenerator...
score_batch_generator = BatchGenerator...
opt = Adam(lr=1e-4)
pruner = pruning.Pruner("config.json", "categorical_crossentropy", opt)
pruner.prune(train_batch, valid_batch)Inā pono, hiki iā ʻoe ke hoʻololi i nā ʻāpana config:
{
"input_model_path": "model.h5",
"output_model_path": "model_pruned.h5",
"finetuning_epochs": 10, # the number of epochs for train between pruning steps
"stop_loss": 0.1, # loss for stopping process
"pruning_percent_step": 0.05, # part of convs for delete on every pruning step
"pruning_standart_deviation_part": 0.2 # shift for limit pruning part
}Eia hou, ua hoʻokō ʻia kahi palena e pili ana i ka deviation maʻamau. ʻO ka pahuhopu e kaupalena i ka ʻāpana i wehe ʻia, me ka ʻole o nā convolutions me nā ana L1 " lawa ":
No laila, ʻae mākou iā ʻoe e wehe wale i nā convolutions nāwaliwali mai nā puʻunaue like me ka mea ʻākau a ʻaʻole pili i ka wehe ʻana mai nā puʻunaue like me ka mea hema:
Ke hoʻokokoke ka puʻunaue maʻamau, hiki ke koho ʻia ka coefficient pruning_standart_deviation_part:
Manaʻo wau i kahi manaʻo o 2 sigma. A i ʻole hiki iā ʻoe ke haʻalele i kēia hiʻohiʻona, e waiho ana i ka waiwai <1.0.
ʻO ka hoʻopuka he pakuhi o ka nui o ka pūnaewele, ka nalowale, a me ka manawa holo pūnaewele no ka hoʻāʻo holoʻokoʻa, hoʻohālikelike ʻia i ka 1.0. No ka laʻana, ma aneʻi ua hoʻemi ʻia ka nui o ka pūnaewele ma kahi o 2 mau manawa me ka ʻole o ka nalowale o ka maikaʻi (ka pūnaewele convolutional liʻiliʻi me 100k mau paona):
ʻO ka māmā holo e pili ana i nā loli maʻamau a ʻaʻole i loli. Aia kahi wehewehe no kēia:
- Hoʻololi ka helu o nā convolutions mai ka maʻalahi (32, 64, 128) a ʻaʻole maʻalahi no nā kāleka wikiō - 27, 51, etc. Hiki paha iaʻu ke hewa ma ʻaneʻi, akā he hopena paha ia.
- ʻAʻole ākea ka hoʻolālā, akā kūlike. Ma ka ho'ēmiʻana i ka laulā,ʻaʻole mākou e pili i ka hohonu. No laila, ho'ēmi mākou i ka ukana, akā,ʻaʻole e hoʻololi i ka wikiwiki.
No laila, ua hōʻike ʻia ka hoʻomaikaʻi ʻana i ka hoʻemi ʻana i ka ukana CUDA i ka wā o ka holo ʻana e 20-30%, akā ʻaʻole i ka hōʻemi ʻana i ka manawa holo.
Nā hopena
E noonoo kakou. Ua noʻonoʻo mākou i nā koho 2 no ka ʻoki ʻana - no YOLOv3 (inā pono ʻoe e hana me kou mau lima) a no nā pūnaewele me nā hale hana maʻalahi. Hiki ke ʻike ʻia ma nā hihia ʻelua hiki ke hoʻokō i ka hōʻemi ʻana o ka nui o ka pūnaewele a me ka wikiwiki me ka nalowale ʻole o ka pololei. Nā hualoaʻa:
- E hoemi ana i ka nui
- Holo wikiwiki
- Hoemi CUDA Haawe
- ʻO ka hopena, ʻo ke aloha kaiapuni (We optimize the future use of computing resources. Somewhere one is happy )
Pākuʻi
- Ma hope o ka ʻoki ʻana, hiki iā ʻoe ke hoʻohui i ka quantization (no ka laʻana, me TensorRT)
- Hāʻawi ʻo Tensorflow i nā hiki no . Nā hana.
- Makemake wau e hoʻomohala a hauʻoli wau e kōkua
Source: www.habr.com