Inkampani i-xAI, eyasungulwa ngu-Elon Musk futhi esithole cishe isigidi sezigidi zamaRandi ukuze kuthuthukiswe ubuchwepheshe obuhlobene nobuhlakani bokwenziwa, imemezele ukutholakala kwemodeli enkulu yolimi lwesiGrok esetshenziswa ku-chatbot ehlanganiswe nenkundla yezokuxhumana X (Twitter). Isethi yama-coefficients anesisindo, i-neural network Architecture, namacala okusetshenziswa ashicilelwe ngaphansi kwelayisensi ye-Apache 2.0. Ingobo yomlando esilungele ukusetshenziswa enemodeli, engu-296 GB ngosayizi (uzibuthe), iyatholakala ukuze ilandwe.
Imodeli ye-Grok iqeqeshwe kusengaphambili eqoqweni elikhulu ledatha yombhalo kusetshenziswa isitaki sokufunda sobunikazi se-xAI futhi ihlanganisa cishe amapharamitha ayizigidi eziyizinkulungwane ezingu-314, okuyenza ibe imodeli enkulu yolimi oluvulekile enkulu kunazo zonke etholakalayo. Ukuze uqhathanise, imodeli ye-Gemma esanda kuvulwa yi-Google inamapharamitha ayizigidi eziyizinkulungwane ezingu-7, i-Sber GigaChat - imingcele eyizigidi eziyizinkulungwane ezingu-29, i-Meta LLaMA - i-65 billion, i-Yandex YaLM - i-100 billion, i-OpenAI GPT-3.5 - 175 billion, nomholi wemakethe, i-GPT- 4, okuthiwa ihlanganisa amapharamitha we-1.76 trillion.
Inguqulo evuliwe yemodeli ye-Grok-1 ishicilelwe isethulo esiyisisekelo futhi ayifaki ukulungiselelwa kwezindawo ezithile zokusetshenziswa, njengokuhlela amasistimu ezingxoxo. Ukuze kuhlolwe, i-GPU enenani elikhulu lememori iyadingeka (ngempela ukuthi hlobo luni lwenkumbulo olungacacisiwe). Ukulingisa okumile kwemodeli kutholakala esidlangalaleni, kuyilapho esinye sezici ze-Grok chatbot ethuthukiswayo ye-Twitter ukuzivumelanisa nezimo nokuqukethwe okusha okuvelayo (ukuhlanganiswa nenkundla ye-X/Twitter kusetshenziswa ukufinyelela ulwazi olusha).
Yakhelwe ku-Grok, i-chatbot idlula i-GPT-3.5 ezivivinyweni zokuxazulula izinkinga zezibalo zesikole samabanga aphezulu (GSM8k), ikhiqiza izimpendulo zemibuzo ehlukahlukene (MMLU), ukuqedela ikhodi ye-Python (HumanEval), nokuxazulula izinkinga zezibalo zasenyuvesi ezichazwe ngefomethi ye-LaTeX (IZIMBALI) .
Source: opennet.ru
