Intel® FPGA AI Suite: Getting Started Guide

ID 768970
Date 2/12/2024
Public
Document Table of Contents

6.11. Performing Inference on the Inflated 3D (I3D) Graph

Before you try the instructions in this section, ensure that you have completed the following tasks:
Remember: A model must be converted from a framework (such as Tensorflow, Caffe, or PyTorch) into a .bin/.xml file pair before the Intel® FPGA AI Suite compiler (dla_compiler command) can ingest the model.

Preparing the i3d-rgb-tf Model

The following commands download the i3d-rgb-tf TensorFlow model and run Model Optimizer:
source ~/build-openvino-dev/openvino_env/bin/activate
omz_downloader --name i3d-rgb-tf \
   --output_dir $COREDLA_WORK/demo/models/
omz_converter --name i3d-rgb-tf \
   --download_dir $COREDLA_WORK/demo/models/ \
   --output_dir $COREDLA_WORK/demo/models/
After running these commands, you can find the model in the following directory:
demo/models/public/i3d-rgb-tf/FP32/

Inference on the i3d-rgb-tf Model and Accuracy Calculations

This section details how to run inference on i3d-rgb-tf with the sample videos and ground truth files provided. For 3D input, the batch size determines how many "clips" a video should be broken down into.

To run inference on the i3d-rgb-tf model with 3 clips per video, run the following commands:
cd $COREDLA_WORK/runtime/build_Release/dla_benchmark
gnd=$COREDLA_WORK/demo/sample_videos/groundtruth_batch_size_3.txt
./dla_benchmark \
   -b=3 \
   -niter=4 \
   -m $COREDLA_WORK/demo/models/public/i3d-rgb-tf/FP32/i3d-rgb-tf.xml \
   -d=HETERO:FPGA,CPU \
   -i=$COREDLA_WORK/demo/sample_videos/ \
   -plugins_xml_file=../plugins.xml \
   -arch_file=$curarch \
   -api=async \
   -nireq=4 \
   -perf_est \
   -groundtruth_loc=$gnd \
   -folding_option=1
To run inference on the i3d-rgb-tf model with 1 clip per video, change the ground truth file (-groundtruth_loc) and the -b flag as follows:
cd $COREDLA_WORK/runtime/build_Release/dla_benchmark
gnd=$COREDLA_WORK/demo/sample_videos/groundtruth_batch_size_1.txt
./dla_benchmark \
   -b=1 \
   -niter=4 \
   -m $COREDLA_WORK/demo/models/public/i3d-rgb-tf/FP32/i3d-rgb-tf.xml \
   -d=HETERO:FPGA,CPU \
   -i=$COREDLA_WORK/demo/sample_videos/ \
   -plugins_xml_file=../plugins.xml \
   -arch_file=$curarch \
   -api=async \
   -nireq=4 \
   -perf_est \
   -groundtruth_loc=$gnd \
   -folding_option=1
Important: Loading the video data with the dla_benchmark command places a high memory demand on the host. Using values for the -niter option that are less than 100 results in manageable memory requirements for most hosts. If the memory usage is too high, the operating system might abruptly end the running dla_benchmark command.
Ground truth files are included for both 3 clips per video, and 1 clip per video. To use a different number of clips per video:
  1. Create a new ground truth text file that contains the correct action class for each video being repeated the same amount of time as the amount of clips per video

    See the groundtruth_batch_size_3.txt file for reference. The ground truths are repeated 3 times each in that file.

  2. Change the -groundtruth_loc option to point to the new ground truth text file that you created.

To run the dla_benchmark command on the CPU, change the ‑d=HETERO:FPGA,CPU option to ‑d=HETERO:CPU.