Skip To Main Content
Support Knowledge Base

Error: "Bad Cast From __Int64 to Unsigned Int" When Inferencing Quantized Llama 3.1 8b Model With NPU Plugin

Content Type: Error Messages   |   Article ID: 000100229   |   Last Reviewed: 05/27/2025

Description

  1. Installed the OpenVINO™ GenAI 2024.4.
  2. Quantized Llama 3.1 8B model with the following command:
    optimum-cli export openvino -m meta-llama/Meta-Llama-3.1-8B --weight-format int4 --sym --group-size 128 --ratio 1.0 INT4-llama-3.1-8B
  3. Encountered error when inferencing quantized Llama 3.1 8B model with NPU plugin:
    Runtime error: Bad cast from __int64 to unsigned int

Resolution

Update OpenVINO™ GenAI version to 2024.5 or above.

Related Products

This article applies to 1 products.