PyTorch

• [New] Your PyTorch solutions are even further enhanced with OpenVINO. You can:​
  • Automatically import and convert PyTorch models for use with native OpenVINO APIs.
  • Continue to make the most of OpenVINO tools for advanced model compression and deployment advantages, ensuring flexibility and a range of options.​
• Developers can now use OpenVINO as a back end through PyTorch torch.compile. It empowers developers to use OpenVINO toolkit through PyTorch APIs. This feature is also integrated into the Automatic1111 for Stable Diffusion* web UI, helping developers achieve accelerated performance for Stable Diffusion 1.5 and 2.1 on Intel CPUs and GPUs in built-in Linux* and Windows* platforms. ​

Optimum for Intel

• Improvements around Optimum for Intel. Hugging Face* and Intel have continued to enhance top generative AI models by optimizing execution, making your models run faster and more efficiently on CPUs and GPUs. OpenVINO serves as a runtime for inferencing execution.
• [New] PyTorch automatic import and conversion capabilities are also enabled with added support for weights compression to further performance gains. ​

Generative AI Support

• OpenVINO has enhanced performance and accessibility for generative AI. Significant progress was made in enhancing runtime performance and optimizing memory use, particularly for LLMs.
• Models used for chatbots are enabled for following instructions, generating code, and more. The models include prominent models like BLOOM, Dolly, Llama 2, GPT-J, GPT-NeoX, ChatGLM, Automatic1111, Open-LLaMA, and LoRA.​

LLMs on GPUs

• LLMs on GPUs have been greatly improved. Model coverage is expanded for dynamic shapes support, further helping the performance of generative AI workloads on integrated and discrete GPUs.
• Memory reuse and weight memory consumption for dynamic shapes is improved.

Neural Network Compression Framework (NNCF)

• NNCF now includes an 8-bit weight compression method making it easier to compress and optimize LLM models.
• SmoothQuant is added for more accurate and efficient posttraining quantization for LLMs​.

Integration with MediaPipe*

• Get direct access to this framework for building multipurpose AI pipelines.
• Facilitates integration with OpenVINO Runtime and OpenVINO model server to enhance performance for faster AI model execution.
• Seamless model management and version control.
• Custom logic integration with additional calculators and graphs for tailored AI solutions.
• Scale faster with the ability to delegate deployment to remote hosts via gRPC and REST interfaces for distributed processing.​

Intel® Core™ Ultra (Formerly Code Named Meteor Lake)

• [New] Full support this processor. This new generation of Intel CPUs is tailored to excel in AI workloads with a built-in inference accelerator: Intel® AI Boost (neural processing unit [NPU] or NPU plug-in), but is not limited to just one AI accelerator. You have the option of running AI workloads on iGPUs and CPUs.