Morero oa OpenAI, o hlahisang merero ea sechaba lefapheng la bohlale ba maiketsetso, o phatlalalitse tsoelo-pele e amanang le mokhoa oa ho lemoha puo ea Whisper. Ho boleloa hore bakeng sa puo ea Senyesemane tsamaiso e fana ka maemo a ho tšepahala le ho nepahala ha kananelo ea othomathiki e haufi le ho elelloa batho. Khoutu ea ts'ebetsong ea litšupiso e thehiloeng ho moralo oa PyTorch le sete ea mefuta e seng e koetlisitsoe, e seng e loketse ho sebelisoa, e butsoe. Khoutu e butsoe tlasa laesense ea MIT.
Ho koetlisa mohlala, ho sebelisitsoe lihora tse likete tse 680 tsa data ea puo, e bokelletsoeng ho tsoa likolekeng tse 'maloa tse buang ka lipuo tse fapaneng le libaka tsa sehlooho. Hoo e ka bang 1/3 ea data ea puo e amehang thupelong e ka lipuo tse ling ntle le Senyesemane. Sistimi e reriloeng e sebetsana ka nepo le maemo a kang ho bitsa mantsoe ka mokhoa o hlakileng, lerata le ka morao, le tšebeliso ea mantsoe a setegeniki. Ntle le ho fetolela puo ho sengoloa, sistimi e ka fetolela puo ho tsoa puong efe kapa efe ho ea Senyesemaneng le ho lemoha ponahalo ea puo molapong oa molumo.
Mehlala e entsoe ka liemeli tse peli: mohlala oa puo ea Senyesemane le mohlala oa lipuo tse ngata, o tšehetsang lipuo tsa Serussia, Seukraine le Sebelarusi. Ka lehlakoreng le leng, setšoantšo se seng le se seng se arotsoe ka likhetho tse 5, tse fapaneng ka boholo le palo ea li-parameter tse koahetsoeng ka mohlala. Ha boholo bo le boholo, ho nepahala le boleng ba kananelo, empa hape ho ba le litlhoko tse phahameng tsa boholo ba memori ea video ea GPU le ho fokotsa ts'ebetso. Ka mohlala, khetho e fokolang e kenyelletsa li-parameter tse limilione tse 39 'me e hloka 1 GB ea memori ea video,' me boholo bo kenyelletsa li-parameter tse limilione tse 1550 'me bo hloka 10 GB ea memori ea video. Khetho e fokolang ke makhetlo a 32 ka potlako ho feta boholo.
Sistimi e sebelisa meralo ea marang-rang ea Transformer neural network, e kenyelletsang encoder le decoder tse sebelisanang. Molumo o arotsoe ka likaroloana tsa metsotsoana e 30, tse fetoloang ho log-Mel spectrogram ebe e romeloa ho encoder. Tlhahiso ea khouto e romelloa ho sehatisi, se bolelang esale pele kemelo ea mongolo e kopantsoeng le matšoao a ikhethileng a lumellang, ka mokhoa o le mong, ho rarolla mathata a kang ho lemoha puo, tlaleho ea tatellano ea liketsahalo tsa mantsoe, ho ngoloa ha puo ho lipuo tse fapaneng, le phetolelo ea Senyesemane.
Source: opennet.ru