Why Is the NPU Compile Time So Much Longer than GPU?

• Ran a test to measure the performance of GPU and NPU on OpenVINO.
• Observed NPU compile time is longer compared to GPU.

NPU compilation in OpenVINO takes longer than GPU because it uses Ahead-of-Time (AOT) compilation, applies more graph optimizations, and generates hardware-specific kernels. Unlike GPUs, which rely on precompiled OpenCL kernels, NPUs require additional processing to optimize execution.

To avoid recompilation, OpenVINO caches compiled models in:

• Windows*: C:\Users\<your_username>\.cache\blob_cache
• Linux*/macOS*: ~/.cache/blob_cache

Run the command below to check the cache location in OpenVINO:

from openvino.runtime import Core

ie = Core() cache_path = ie.get_property("GPU", "CACHE_DIR") # Change "GPU" to "NPU" if needed

print(f"OpenVINO Model Cache Directory: {cache_path}"