OpenVINO™ toolkit allows you to efficiently build, optimize, and deploy AI apps on AI PCs across diverse and heterogeneous engines like CPUs, Intel integrated and discrete GPUs, and NPUs. This session shows the benefits of optimizing PyTorch* models with the OpenVINO toolkit and deploying such models on AI PCs.

To use OpenVINO toolkit as a stand-alone AI inference runtime, developers can:

• Convert PyTorch models to an intermediate representation (IR) format for the toolkit and subsequently load them into the OpenVINO™ Runtime for optimized inference.
• Use the front end for OpenVINO toolkit and PyTorch that directly loads PyTorch models into the toolkit.

Additionally, with the advent of PyTorch 2.0, OpenVINO toolkit is incorporated into PyTorch as a TorchDynamo back end with torch.compile to make inference possible with PyTorch APIs.

The presentation includes a demonstration of the practical implementation of each of these techniques. It provides examples for using relevant APIs and explains the difference between each approach in the context of AI app development for AI PCs.