Support Knowledge Base

Unable to Infer OpenVINO™ Optimized TinyLlama Model on NPU

Content Type: Troubleshooting | Article ID: 000101250 | Last Reviewed: 05/28/2025

Description Resolution

OpenVINO version 2025.0 and lower

Description

Inferred OpenVINO™ Optimized TinyLlama Model on NPU.
Encountered error:
get_shape was called on a descriptor::Tensor with dynamic shape

Upgrade OpenVINO™ version to 2025.1 and install required dependencies.
pip install openvino==2025.1 openvino-tokenizers==2025.1 openvino-genai==2025.1 nncf==2.14.1 onnx==1.17.0 optimum-intel==1.22.0
Infer OpenVINO™ Optimized TinyLlama Model on NPU using OpenVINO™ GenAI.
import openvino_genai as ov_genai
model_path = "TinyLlama"
pipe = ov_genai.LLMPipeline(model_path, "NPU")

print(pipe.generate("What is OpenVINO?", max_length=200))

This article applies to 1 products.

OpenVINO™ toolkit

Contact support