Intel Releases OpenVINO 2024.0 Open Source Toolkit for Optimizing and Deploying AI Systems

Intel Releases OpenVINO 2024.0 Open Source Toolkit for Optimizing and Deploying AI Systems

On March 6, 2024, Intel released the OpenVINO 2024.0 toolkit for neural networks with built-in support for performance analysis (benchmarks), throughput and latency analysis for different models, which allows optimization and deployment of AI systems on different hardware. The source code of the project is posted on GitHub under the Apache License 2.0.

The OpenVINO project is designed to test AI performance not only on x86_64 processors, but also on ARM processors and other architectures, integrated and discrete Intel graphics, and many other hardware. The project supports, with the help of a plugin, the possibility of using neural processing of the NPU unit on the new Intel Core Ultra Meteor Lake processors.

OpenVINO 2024.0 adds new features for working with generative AI (GenAI) including working out of the box with TensorFlow proposal encoder models, Mixture of Experts (MoE) support, and proven Mistral models. The project received JavaScript support for seamless access to the OpenVINO API.

OpenVINO 2024.0 also provides improved INT4 weight compression quality for LLMs (LLM – Large Language Models), improved LLM performance on Intel processors, simplified Hugging Face model optimization and conversion, and other Hugging Face integration improvements.

The developers explained that OpenVINO 2024.0 also provides better performance on ARM processors and received various improvements in its platform code.

In OpenVINO 2024.0, support for the previous Gaussian and Neural Accelerator (Intel GNA) solution from Intel has been discontinued. The project now focuses on NPUs in Meteor Lake processors and newer. p align=”justify”> The Intel NPU plugin for OpenVINO is now distributed as part of the core OpenVINO package on PyPi.

Related posts