Iyo OpenAI purojekiti, iyo inovandudza mapurojekiti eruzhinji mumunda wehungwaru hwekunyepedzera, yakaburitsa zviitiko zvine chekuita neiyo Whisper kutaura kwekuziva system. Zvinonzi pakutaura muChirungu hurongwa hunopa mwero wekuvimbika uye kurongeka kwekuzivikanwa kweotomatiki pedyo nekuzivikanwa kwevanhu. Iyo kodhi yekushandiswa kwereferensi yakavakirwa paPyTorch chimiro uye seti yemamodhi akadzidziswa, akagadzirira kushandiswa, akavhurwa. Iyo kodhi yakavhurika pasi peMIT rezinesi.
Kudzidzisa modhi, zviuru mazana matanhatu nemakumi masere ezviuru zvemaawa edhata rekutaura akashandiswa, akaunganidzwa kubva muunganidzwa akati wandei unofukidza mitauro yakasiyana nenzvimbo dzezvidzidzo. Inenge 680/1 yedata yekutaura inobatanidzwa mukudzidziswa iri mumitauro isiri yeChirungu. Iyo yakarongwa sisitimu inobata nemazvo mamiriro akadai semataurirwo akaitwa, ruzha rwekumashure, uye kushandiswa kwehunyanzvi jargon. Pamusoro pekunyora kutaura kuita zvinyorwa, sisitimu inogona zvakare kushandura kutaura kubva kune chero mutauro kuenda kuChirungu uye kuona kutaridzika kwekutaura murukova rweodhiyo.
Mienzaniso inoumbwa mumienzaniso miviri: muenzaniso wemutauro weChirungu uye mhando yemitauro yakawanda, iyo inotsigirawo mitauro yeRussia, Ukrainian uye Belarusian. Nekudaro, imwe neimwe inomiririra yakakamurwa kuita 5 sarudzo, dzakasiyana muhukuru uye nhamba yemaparamita akafukidzwa mumuenzaniso. Iyo yakakura saizi, iyo yakakura kurongeka uye kunaka kwekuzivikanwa, asi zvakare kukwirisa zvinodiwa zvehukuru hweGPU vhidhiyo yekuyeuka uye kuderera kwekuita. Semuenzaniso, iyo yakaderera sarudzo inosanganisira 39 miriyoni paramita uye inoda 1 GB yevhidhiyo memory, uye iyo yepamusoro inosanganisira 1550 miriyoni paramita uye inoda 10 GB yevhidhiyo memory. Iyo yakaderera sarudzo ndeye 32 times nekukurumidza kupfuura iyo yakanyanya.
Iyo sisitimu inoshandisa iyo Transformer neural network architecture, iyo inosanganisira encoder uye decoder inosangana nemumwe. Odhiyo yakaputswa kuita makumi matatu-yechipiri chunks, iyo inoshandurwa kuita log-Mel spectrogram uye inotumirwa kune encoder. Kubuda kwe encoder kunotumirwa kudhikodha, iyo inofanotaura chinomiririra mavara akasanganiswa neakakosha tokens anobvumira, mune imwe chete modhi, kugadzirisa matambudziko akadai sekuziva mutauro, accounting yenguva yekududzwa kwemitsara, kududzirwa kwekutaura mu. mitauro yakasiyana, uye kududzira muchiRungu.
Source: opennet.ru