AN 993: Using Custom Models with Intel® FPGA AI Suite

Download PDF
ID 777190
Date 5/01/2023
Public
Document Table of Contents
Document Table of Contents x
1. Introduction 2. Prerequisites 3. Intel® FPGA AI Suite Model Development Overview 4. Custom Model Examples 5. Common Errors When Using an Custom Model 6. AN 993 Using Custom Models with Intel® FPGA AI Suite Revision History
4. Custom Model Examples x
4.1. Example 1: Customized ResNet-18 Model 4.2. Example 2: Customized Multilayer Perceptron (MLP) Model
1. Introduction
2. Prerequisites
3. Intel® FPGA AI Suite Model Development Overview
4. Custom Model Examples
5. Common Errors When Using an Custom Model
6. AN 993 Using Custom Models with Intel® FPGA AI Suite Revision History

5. Common Errors When Using an Custom Model

When you use a custom model with Intel® FPGA AI Suite, you might get one of these errors when generating the OpenVINO™ IR for the model.

Shape Not Fully Defined Error

A "shape not fully defined" error looks like the following message: 

[ ERROR ] Shape [ -1 299 299 3] is not fully defined for output 0 of "serving_default_input_1".
Use --input_shape with positive integers to override model input shapes.

For instructions on how to fix this error, review the documentation at the following URL:

https://docs.openvino.ai/latest/openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model.html

The follow Model Optimizer command is an example to fix this error: 
mo --input_model <model path>.onnx --input_shape [1,299,299,3] --input <input layer>

Unsupported Layer Type Error

An "unsupported layer type" error looks like the following message: 

[ ERROR ] Failed to compile layer "<number of layer>": unsupported layer type "<layer name>"
To correct this error, implement one of the following fixes:
  • Use a model with supported layers.
  • Modify the model to remove the unsupported layer or replace it.
  • Run the layer in parallel on a supported device

Running the layer in parallel on a supported device requires you to enable heterogeneous execution in OpenVINO™ . For details, refer to the following URL:

https://docs.openvino.ai/latest/openvino_docs_OV_UG_Hetero_execution.html

The following table shows examples of enabling heterogeneous execution in OpenVINO™ :
Language Code Example
Python 
compiled_model = core.compile_model(model, device_name="HETERO:GPU,CPU")
# device priorities via configuration property
compiled_model = core.compile_model(model, device_name="HETERO", \
                 config={"MULTI_DEVICE_PRIORITIES": "GPU<CPU"})
C++ 
auto compiled_model = core.compile_model(model, "HETERO:GPU,CPU");
// or with ov::device::priorities with multiple args
compiled_model = core.compile_model(model,, "HETERO", ov::device::priorities("GPU","CPU"));
// or with ov::device::priorities with a single argument
compiled_model = core.compile_model(model, "HETERO", ov::device::priorities("GPU,CPU"));
Level Two Title