Cov cai ntawm kev kawm tshuab software khoom feem ntau nyuaj thiab tsis meej pem. Tshawb nrhiav thiab tshem tawm cov kab hauv nws yog ib txoj haujlwm siv nyiaj ntau. Txawm yog qhov yooj yim tshaj plaws xav tau ib txoj hauv kev loj rau network architecture, pib qhov hnyav, thiab kev ua kom zoo dua hauv network. Qhov yuam kev me me tuaj yeem ua rau muaj teeb meem tsis zoo.
Kab lus no yog hais txog ib qho algorithm rau kev debugging koj neural networks.
Skillbox pom zoo: Cov chav kawm siv tau .
Peb nco qab: rau txhua tus neeg nyeem Habr - 10 ruble luv nqi thaum tso npe rau hauv ib chav kawm Skillbox siv Habr promo code.
Lub algorithm muaj tsib theem:
- yooj yim pib;
- kev lees paub ntawm kev poob;
- tshuaj xyuas cov txiaj ntsig nruab nrab thiab kev sib txuas;
- parameter diagnostics;
- tswj kev ua haujlwm.
Yog tias ib yam dab tsi zoo li nthuav rau koj ntau dua li lwm tus, koj tuaj yeem hla mus rau cov ntu no tam sim ntawd.
Yooj yim pib
Lub network neural nrog cov txheej txheem nyuaj, kev ua haujlwm tsis tu ncua, thiab tus nqi kawm ntawv yog qhov nyuaj rau kev debug dua li ib txwm muaj. Peb tab tom ua me ntsis lo lus nug ntawm no, txij li lub ntsiab lus nws tus kheej tsis ncaj qha cuam tshuam nrog kev debugging, tab sis qhov no tseem yog qhov kev pom zoo tseem ceeb.
Ib qho yooj yim pib yog los tsim ib qho qauv yooj yim thiab cob qhia nws ntawm ib pawg (point) ntawm cov ntaub ntawv.
Ua ntej peb tsim cov qauv yooj yim
Txhawm rau pib sai, peb tsim ib lub network me me nrog ib txheej zais thiab xyuas tias txhua yam ua haujlwm raug. Tom qab ntawd peb maj mam ua rau tus qauv nyuaj, kuaj xyuas txhua yam tshiab ntawm nws cov qauv (ntxiv txheej, parameter, thiab lwm yam), thiab txav mus.
Peb cob qhia tus qauv ntawm ib pawg (point) ntawm cov ntaub ntawv
Raws li kev soj ntsuam ceev ceev rau koj qhov project, koj tuaj yeem siv ib lossis ob qhov kev qhia cov ntaub ntawv kom paub meej tias lub kaw lus puas ua haujlwm raug. Lub neural network yuav tsum qhia 100% qhov tseeb hauv kev cob qhia thiab kev sim. Yog tias qhov no tsis yog, ces tus qauv me dhau los yog koj twb muaj kab mob.
Txawm hais tias txhua yam zoo, npaj tus qauv rau ib lossis ntau lub sijhawm ua ntej yuav mus.
Kev ntsuam xyuas poob
Kev kwv yees poob yog txoj hauv kev tseem ceeb los kho cov qauv kev ua tau zoo. Koj yuav tsum xyuas kom meej tias qhov poob yog qhov tsim nyog rau qhov teeb meem thiab qhov kev poob haujlwm raug ntsuas ntawm qhov raug ntsuas. Yog tias koj siv ntau tshaj ib hom kev poob, xyuas kom tseeb tias lawv yog tag nrho cov kev txiav txim tib yam thiab ntsuas kom raug.
Nws yog ib qho tseem ceeb uas yuav tsum tau ua tib zoo saib xyuas thaum pib poob. Xyuas seb qhov txiaj ntsig tiag tiag yog li cas rau qhov kev xav tau yog tias tus qauv pib nrog kev kwv yees random. IN : βNco ntsoov tias koj tau txais cov txiaj ntsig koj xav tau thaum koj pib nrog qee qhov ntsuas me me. Nws yog qhov zoo dua los tshawb xyuas cov ntaub ntawv poob tam sim ntawd (nrog rau qib ntawm kev ua haujlwm tsis tu ncua rau xoom). Piv txwv li, rau CIFAR-10 nrog Softmax classifier, peb cia siab tias qhov pib poob yog 2.302 vim tias qhov kev cia siab diffuse qhov tshwm sim yog 0,1 rau txhua chav kawm (vim tias muaj 10 chav kawm) thiab Softmax poob yog qhov tsis zoo ntawm cov chav kawm raug. as β ln (0.1) = 2.302.
Rau qhov piv txwv binary, qhov kev suav zoo sib xws tsuas yog ua rau txhua chav kawm. Ntawm no, piv txwv li, yog cov ntaub ntawv: 20% 0's thiab 80% 1's. Qhov kev xav tau pib poob yuav nce mus txog -0,2ln (0,5) -0,8ln (0,5) = 0,693147. Yog tias qhov tshwm sim ntau dua 1, nws yuav qhia tau tias qhov hnyav ntawm neural network tsis zoo sib npaug lossis cov ntaub ntawv tsis zoo li qub.
Tshawb xyuas cov txiaj ntsig nruab nrab thiab kev sib txuas
Txhawm rau debug lub neural network, nws yog ib qho tsim nyog yuav tsum nkag siab txog qhov kev hloov pauv ntawm cov txheej txheem hauv lub network thiab lub luag haujlwm ntawm cov txheej txheem nruab nrab ntawm tus kheej thaum lawv txuas nrog. Nov yog cov yuam kev uas koj yuav ntsib:
- cov kab lus tsis raug rau kev hloov kho gradle;
- tsis siv qhov hnyav hloov tshiab;
- tawg gradients.
Yog tias qhov gradient qhov tseem ceeb yog xoom, qhov no txhais tau hais tias qhov kev kawm hauv qhov optimizer qeeb dhau, lossis koj tab tom ntsib cov lus qhia tsis raug rau kev hloov kho gradient.
Tsis tas li ntawd, nws yog ib qho tsim nyog los saib xyuas cov txiaj ntsig ntawm kev ua kom muaj zog, qhov hnyav thiab hloov tshiab ntawm txhua txheej. Piv txwv li, qhov loj ntawm qhov hloov tshiab parameter (qhov hnyav thiab kev tsis ncaj ncees) .
Muaj ib qho tshwm sim hu ua "Dying ReLU" lossis , thaum ReLU neurons yuav tso tawm xoom tom qab kawm txog qhov tsis zoo ntawm qhov tsis zoo rau nws qhov hnyav. Cov neurons no yeej tsis raug rho tawm haujlwm ib zaug ntxiv ntawm cov ntaub ntawv.
Koj tuaj yeem siv gradient checking los txheeb xyuas cov kev ua yuam kev no los ntawm kwv yees qhov gradient siv tus lej mus kom ze. Yog hais tias nws nyob ze rau cov gradients xam, ces backpropagation raug siv kom raug. Txhawm rau tsim qhov ntsuas gradient, tshawb xyuas cov peev txheej zoo ntawm CS231 ΠΈ thiab nrog Andrew Nga on this topic.
qhia peb txoj hauv kev tseem ceeb rau kev pom lub network neural:
- Preliminaries yog cov txheej txheem yooj yim uas qhia peb txog cov qauv ntawm cov qauv kev cob qhia. Cov no suav nrog cov zis ntawm cov duab lossis cov ntxaij lim dej ntawm cov txheej txheem ntawm cov neural network thiab cov tsis nyob hauv txhua txheej.
- Ua kom raws li. Hauv lawv, peb txiav txim siab qhov kev ua haujlwm ntawm ib tus neeg neurons lossis pab pawg ntawm cov neurons kom nkag siab txog lawv txoj haujlwm.
- Raws li gradients. Cov txheej txheem no zoo li los tswj cov gradients uas yog tsim los ntawm kev mus rau pem hauv ntej thiab rov qab dhau ntawm cov qauv kev cob qhia (nrog rau daim ntawv qhia kev noj qab haus huv thiab daim ntawv qhia ua haujlwm hauv chav kawm).
Muaj ntau ntau yam cuab yeej siv tau los ua kom pom kev ua haujlwm thiab kev sib txuas ntawm cov txheej txheem ib leeg, xws li. ΠΈ .
Parameter diagnostics
Neural networks muaj ntau qhov tsis sib xws, uas cuam tshuam rau kev ua kom zoo dua qub. Qhov tseeb, ntu no yog qhov kev tshawb fawb nquag los ntawm cov kws tshaj lij, yog li cov lus qhia hauv qab no yuav tsum raug suav hais tias tsuas yog cov lus qhia, pib cov ntsiab lus los ntawm kev tsim.
Pob loj (batch loj) - Yog tias koj xav kom cov batch loj txaus kom tau txais qhov tseeb qhov yuam kev gradient kwv yees, tab sis me me txaus rau stochastic gradient qhovntsej thiaj tsis mob (SGD) los ua kom koj lub network tsis tu ncua. Cov batch me me yuav ua rau kev sib koom ua ke sai vim yog lub suab nrov thaum lub sijhawm kev cob qhia thiab tom qab ntawd rau kev ua kom zoo dua qub. Qhov no tau piav qhia ntau ntxiv .
Kev kawm tus nqi - tsawg dhau yuav ua rau kev sib tshuam qeeb lossis kev pheej hmoo ntawm kev daig hauv minima hauv zos. Nyob rau tib lub sijhawm, qhov kev kawm siab yuav ua rau muaj kev sib txawv ntawm qhov zoo vim tias koj pheej hmoo dhia mus rau qhov tob tab sis nqaim ib feem ntawm kev poob haujlwm. Sim siv lub sijhawm teem sijhawm kom txo tau nws thaum cob qhia neural network. Txog hnub tim nrog CS231n .
Gradient clippingβ - trimming parameter gradients thaum lub sij hawm backpropagation ntawm tus nqi siab tshaj plaws los yog marginal qauv. Pab tau rau kev daws teeb meem txhua qhov tawg gradients koj yuav ntsib nyob rau hauv point peb.
Batch normalization - siv los normalize cov ntaub ntawv tawm tswv yim ntawm txhua txheej, uas tso cai rau peb los daws qhov teeb meem ntawm kev hloov hauv covariate. Yog tias koj siv Dropout thiab Batch Norma ua ke, .
Stochastic gradient qhovntsej thiaj tsis mob (SGD) - muaj ntau ntau yam ntawm SGD uas siv lub zog, hloov pauv kev kawm thiab Nesterov txoj kev. Txawm li cas los xij, tsis muaj leej twg muaj qhov zoo dua ntawm kev kawm thiab kev nthuav dav ().
Regularization - Nws yog ib qho tseem ceeb rau kev tsim cov qauv dav dav, txij li nws ntxiv qhov nplua rau cov qauv kev nyuaj lossis qhov ntsuas qhov ntsuas huab cua. Qhov no yog ib txoj hauv kev los txo cov qauv variance yam tsis muaj qhov ua rau nws tsis ncaj ncees lawm. Ntau .
Txhawm rau ntsuas txhua yam ntawm koj tus kheej, koj yuav tsum lov tes taw tsis tu ncua thiab xyuas cov ntaub ntawv poob gradient koj tus kheej.
Tawm tawm yog lwm txoj hauv kev los txhim kho koj lub network kom tsis txhob muaj teeb meem. Thaum lub sijhawm kev cob qhia, kev tso tawm tsuas yog ua los ntawm kev tswj hwm cov haujlwm ntawm cov neuron nrog qee qhov tshwm sim p (hyperparameter) lossis teeb tsa nws mus rau xoom hauv qhov sib txawv. Yog li ntawd, lub network yuav tsum siv qhov sib txawv ntawm qhov tsis sib xws rau txhua qhov kev cob qhia batch, uas txo cov kev hloov pauv hauv qee qhov kev txwv uas dhau los ua qhov tseem ceeb.
Tseem ceeb: Yog tias koj siv ob qho tib si tso tawm thiab batch normalization, ceev faj txog qhov kev txiav txim ntawm cov haujlwm no lossis txawm siv ua ke. Tag nrho cov no tseem tab tom sib tham thiab ntxiv. Nov yog ob qhov kev sib tham tseem ceeb ntawm lub ncauj lus no ΠΈ .
Tswj haujlwm
Nws yog hais txog cov ntaub ntawv ua haujlwm thiab kev sim. Yog tias koj tsis sau ib yam dab tsi, tej zaum koj yuav hnov ββββqab, piv txwv li, qhov kev kawm twg los yog siv qhov hnyav hauv chav kawm. Ua tsaug rau kev tswj hwm, koj tuaj yeem yooj yim saib thiab rov tsim dua cov kev sim yav dhau los. Qhov no tso cai rau koj los txo tus naj npawb ntawm cov kev sim dua.
Txawm li cas los xij, phau ntawv cov ntaub ntawv tuaj yeem dhau los ua haujlwm nyuaj nyob rau hauv rooj plaub ntawm kev ua haujlwm loj. Qhov no yog qhov uas cov cuab yeej zoo li Comet.ml tuaj yeem pab koj tau txais cov ntaub ntawv teev tseg, cov lej hloov pauv, kev sim keeb kwm, thiab cov qauv tsim, suav nrog cov ntaub ntawv tseem ceeb ntawm koj tus qauv (hyperparameters, qauv ntsuas kev ua tau zoo, thiab cov ntaub ntawv ib puag ncig).
Lub network neural tuaj yeem nkag siab zoo rau cov kev hloov pauv me me, thiab qhov no yuav ua rau poob qis hauv cov qauv kev ua haujlwm. Taug qab thiab sau koj cov ntaub ntawv ua haujlwm yog thawj kauj ruam uas koj tuaj yeem ua rau tus qauv tsim koj ib puag ncig thiab ua qauv.
Kuv vam tias cov ntawv tshaj tawm no tuaj yeem yog qhov pib rau koj pib debugging koj lub neural network.
Skillbox pom zoo:
- Ob xyoos xyaum xyaum .
- Hoob kawm online .
- Cov chav kawm xyaum ua ib xyoos .
Tau qhov twg los: www.hab.com