Kwethulwe i-RedPajama, iphrojekthi ehlanganyelwayo ehloselwe ukudala amamodeli okufunda omshini ovulekile kanye nokokufaka okuhambisana nokuqeqeshwa okungasetshenziswa ukudala abasizi abahlakaniphile abaqhudelana nemikhiqizo yezentengiso efana ne-ChatGPT. Ukutholakala kwedatha yomthombo ovulekile namamodeli olimi amakhulu kulindeleke ukuthi kukhulule amaqembu azimele ocwaningo lokufunda ngomshini futhi kwenze kube lula ukwakha amasistimu engxoxo angokwezifiso. Izinhlangano nemiphakathi efana ne-Together, Ontocord.ai, ETH DS3Lab, Stanford CRFM, Hazy Research kanye ne-MILA Québec AI Institute bajoyine iphrojekthi.
Isinyathelo sokuqala kwaba ukushicilelwa kwedathasethi ye-RedPajama-Data-1T yokuqeqesha amamodeli ezingxoxo, aqukethe amathokheni we-1.2 trillion. I-RedPajama suite ikhiqiza kabusha idatha etholakala esidlangalaleni esetshenziswa yi-Facebook ukudala imodeli yayo ye-LLaMA (ebiza amathokheni ayizigidi eziyizinkulungwane ezingu-1.25), kodwa inikezwa ngaphansi kwelayisensi evulekile, yomthombo ovulekile (idatha ye-LLaMA namamodeli enziwa atholakale kubacwaningi kuphela ngesicelo esikhethekile kwabangewona. - ukusetshenziswa kwezohwebo). Isethi elandekayo ye-RedPajama-Data-1T ingu-2.67 TB ngobukhulu futhi ihlanganisa ulwazi oluvela kumakhasi ewebhu anenkomba e-Common Crawl, izingobo zomlando ze-Wikipedia, ikhodi yomthombo evela ku-GitHub, izincwadi zesizinda somphakathi ezivela kumtapo wezincwadi wakwa-Gutenberg, izindatshana zesayensi ezivela kungobo yomlando ye-ArXiv, nezingxoxo ezivela Ukuchichima Kwesitaki namanye amasayithi e-Stack Exchange.
Amamodeli enziwe ngomumo, aqeqeshwe ngesisekelo sesethi yedatha elungisiwe futhi ethuthukisiwe kusetshenziswa izibonelo esezilungile zezingxoxo ngendlela yokwenziwa kwemiyalelo evela kumaphrojekthi we-Alpaca kanye ne-OpenChatKit, ahlelelwe ukuthi akheke emasontweni ambalwa ezayo. Izinyathelo ezifanayo zemodeli yolimi zifaka phakathi amaphrojekthi omthombo ovulekile kancane i-LLaMA, i-Alpaca, i-Vicuna, ne-Koala, kanye nezinhlelo zomthombo ovulekile ngokugcwele we-Pythia, i-OpenChatKit, i-Open Assistant, no-Dolly.
Ukwengeza, amaphrojekthi amasha amaningana ahlobene nokufunda komshini angaqashelwa:
- I-MiniGPT-4 - inweba ama-chatbots endabuko asebenzisanayo ngamakhono acabangela imininingwane ebonakalayo, ekuvumela ukuthi uhlaziye izithombe futhi ucabangele umbhalo obhalwe ngesandla lapho usebenzisana nohlelo (ngokwesibonelo, ungabuza ukuthi hlobo luni lwento eboniswa esithombeni. , cela i-bot ukuthi ibhale indaba esekelwe kuleyo eboniswe esithombeni, noma ngokusekelwe kumdwebo odwetshiwe, cela ukudala iwebhusayithi). Ukuqaliswa kweMiniGPT-4 kubhalwe ngePython futhi kusatshalaliswa ngaphansi kwelayisensi ye-BSD.
- I-Facebook ishicilele amathuluzi kanye nokuzifundela (SSL, Ukufunda Okuziqondisayo, akusebenzisi amalebula alungiselelwe abantu nezichasiselo ngesikhathi sokuqeqeshwa) imodeli yombono wekhompyutha i-DINOv2, elungele ukuxazulula izinkinga zokucubungula idatha ebonakalayo okuvamile (ukuhlukaniswa kwesithombe, ukukhipha ulwazi mayelana izinto ezisezithombeni, ukuqonda okwenzeka kuvidiyo) kanye nokukhohlisa ezingeni lephikseli (ukubikezela okujulile, ukuhlukaniswa). Imodeli yaqeqeshwa eqoqweni lezithombe eziyizigidi ezingu-142. Ukuqaliswa kubhalwe nge-Python futhi kusakazwa ngaphansi kwelayisensi ye-Creative Commons Attribution-NonCommerce 4.0, evumela ukusetshenziswa okungekhona okokuhweba.
- I-GPT4All iyikhithi yamathuluzi yokwethula ngokushesha ama-chatbots azimele wodwa ku-hardware yakho (awafinyeleli kumasevisi angaphandle futhi asebenzisa i-CPU esekelwa i-AVX2 ukuze ayenze). Isekela ukuxhumeka kwamamodeli amakhulu olimi asekelwe ku-GPT-J ne-LLaMa. Ikhodi ibhalwe ngePython futhi isatshalaliswa ngaphansi kwelayisense ye-MIT.
Source: opennet.ru