On January 24, Intel engineers released a major update to the leading open-source AI toolkit, OpenVINO 2023.3. It provides full support for the new Emerald Rapids and Meteor Lake processors, along with other hardware enhancements from Intel for generative artificial intelligence (GenAI) and large language models (LLM).
OpenVINO 2023.3 introduces the OpenVINO Gen AI repository to showcase examples of C/C++ pipeline for large language models and validate additional models, including Mistral, Zephyr, ChatGLM3, and others. Torch.compile is now fully integrated with OpenVINO.
To extend support for large language models, the INT4 model compression format is supported on Intel Xeon processors, as well as Intel Core and Intel iGPU. The performance of transformer-based LLMs has been improved on both CPUs and GPUs, streamlining optimization for Hugging Face models and much more.
For reference: OpenVINO is a free open-source toolkit that helps developers and data analysts accelerate the development of high-performance solutions for various vision systems. This comprehensive toolkit supports a wide range of computer vision solutions, optimizes deep learning deployment, and ensures easy execution on various Intel platforms. OpenVINO also addresses a variety of tasks, including face detection, automatic object recognition, text and speech recognition, image processing, and much more.
Source: linux.org.ru
