Support Knowledge Base

Is It Possible to Implement OpenVINO™ Runtime Inference Pipeline with Intermediate Representation (IR)?

Content Type: Product Information & Documentation | Article ID: 000092935 | Last Reviewed: 02/28/2023

Description Resolution

Description

Converted TensorFlow* model into IR.
Unable to determine steps to implement OpenVINO™ Runtime inference pipeline with IR.

Create OpenVINO™ Runtime Core
import openvino.runtime as ov
core = ov.Core()
Compile the Model
compiled_model = core.compile_model("model.xml", "AUTO")
Create an Infer Request
infer_request = compiled_model.create_infer_request()
Set Inputs
# Create tensor from external memory
input_tensor = ov.Tensor(array=memory, shared_memory=True)
# Set input tensor for model with one input
infer_request.set_input_tensor(input_tensor)
Start Inference
infer_request.start_async()
infer_request.wait()
Process the Inference Results
# Get output tensor for model with one output
output = infer_request.get_output_tensor()
output_buffer = output.data
# output_buffer[] - accessing output tensor data

This article applies to 3 products.

Intel® Xeon Phi™ Processor Software OpenVINO™ toolkit Performance Libraries

Contact support