Developers at Mozilla have unveiled the first release of the Llamafile utility, which allows the creation of universal executable files for running large machine learning language models (LLM). With Llamafile, you can take a file containing the parameters of a machine learning model in GGUF format and convert it into an executable file that can run on six different operating systems on hardware with AMD64 and ARM64 processors. The toolkit is written in C/C++ and is distributed under the Apache 2.0 license.
The models are executed using the llama.cpp framework, which implements the LLaMA model in C/C++, compiling to a universal executable file in APE (Actually Portable Executable) format that is bundled with the model file. The compiled code is linked to the standard C library Cosmopolitan, allowing for the creation of application builds that can run on Linux, FreeBSD, macOS, OpenBSD, NetBSD, and Windows. The code is available for AMD64 and ARM64 architectures, and also includes components for offloading computations to NVIDIA GPUs (using CUDA).
Source: opennet.ru
