Iyo OpenAI purojekiti, iyo inovandudza mapurojekiti anowanikwa pachena mumunda wehungwaru hwekugadzira, yakaburitsa zviitiko zvine chekuita neiyo Whisper kutaura kwekuziva system. Zvinonzi pakutaura muChirungu, sisitimu iyi inopa mwero wekuvimbika nekururama kwekuzivikanwa kweotomatiki pedyo nekuzivikanwa kwevanhu. Iyo kodhi yekumisikidzwa kwereferenzi yakavakirwa paPyTorch chimiro uye seti yemamodhi akadzidziswa akagadzirira kushandiswa akavhurika. Iyo kodhi yakavhurika pasi peMIT rezinesi.
Iyo modhi yakadzidziswa ichishandisa 680 maawa ekutaura data yakaunganidzwa kubva kune akati wandei akaunganidzwa anofukidza mitauro yakasiyana nenzvimbo dzezvidzidzo. Inenge 1/3 yedata rekutaura rinoshandiswa mukudzidziswa rinobva kune mitauro isiri yeChirungu. Iyo yakarongwa sisitimu inobata nemazvo mamiriro akadai semataurirwo akaitwa, kuvapo kweruzha rwekumashure, uye kushandiswa kwehunyanzvi jargon. Pamusoro pekunyora kutaura kuita zvinyorwa, sisitimu inogona zvakare kushandura kutaura kubva kune chero mutauro kuenda kuChirungu uye kuona kuitika kwekutaura murukova rweodhiyo.
Iwo mamodheru akaumbwa mumienzaniso miviri: modhi yemutauro weChirungu uye mhando yemitauro yakawanda inotsigira Russian, Ukrainian, uye Belarusian. Nekudaro, chinomiririra chega chega chakakamurwa kuita mashanu akasiyana muhukuru uye huwandu hwema paramita akafukidzwa nemuenzaniso. Iyo yakakura saizi, iyo yakakura kurongeka uye hunhu hwekuzivikanwa, asiwo zvakakwirira zvinodikanwa zvehukuru hweGPU vhidhiyo ndangariro uye yakaderera kuita. Semuenzaniso, iyo yakaderera musiyano inosanganisira 5 miriyoni paramita uye inoda 39 GB yevhidhiyo memory, uye iyo yepamusoro inosanganisira 1 miriyoni paramita uye inoda 1550 GB yevhidhiyo memory. Musiyano wakaderera ndiwo 10 times nekukurumidza kupfuura iyo yakanyanya.
Iyo sisitimu inoshandisa iyo Transformer neural network architecture, iyo inosanganisira inodyidzana encoder uye decoder. Odhiyo yakaputswa kuita makumi matatu-yechipiri chunks, iyo inoshandurwa kuita log-Mel spectrogram uye inopfuudzwa kune encoder. Kubuda kwekodha kunotumirwa kudhikodha, iyo inofanotaura zvinomiririra mavara akasanganiswa neakakosha tokens anobvumira modhi imwe chete yakajairika kugadzirisa matambudziko akadai sekuziva mutauro, kuverenga nguva yekududzwa kwezvirevo, kunyorwa kwekutaura mumitauro yakasiyana, uye kududzira kuChirungu.
Source: opennet.ru
