Skip To Main Content
Support Knowledge Base

Unable to Run Benchmark_app for ONNX Model on GPU

Content Type: Troubleshooting   |   Article ID: 000095644   |   Last Reviewed: 07/24/2023

Environment

OpenVINO 2023.0

Description

  • Inferred ONNX model with OpenVINO™ 2023.0 Benchmark_app on CPU and GPU.
  • Inference success on CPU but failed on GPU.

Resolution

Currently, dynamic shape support for GPU is a preview feature and has the following limitations:

  • It mainly supports Natural Language Processing (NLP) models. Not all operations and optimization pass and support dynamic shapes. As a result, a given model may crash or experience significant performance drops.
  • Due to the dominant runtime overhead on the host device, dynamic shapes may perform worse than static shapes on a discrete GPU.
  • Dynamic rank is not supported.

Additional information

Refer to GPU Device for more information.

Related Products

This article applies to 1 products.