7. Intel® FPGA AI Suite Soc Design Example Software Components
The Intel® FPGA AI Suite SoC design example contains a software environment for the runtime flow.
- Yocto build and runtime Linux environment
- Intel® Distribution of OpenVINO™ Toolkit Version 2022.3.1 LTS (Inference Engine, Heterogeneous plugin)
- OpenVINO™ Arm* CPU plugin
- Intel® FPGA AI Suite runtime plugin
- MMD hardware library
The Intel® FPGA AI Suite SoC design example contains the source files, Makefiles, and scripts to cross compile all the software for the Intel® Arria® 10 SX SoC FPGA Development Kit. The Yocto SDK provides the cross compiler, and is the first component that must be built.
The machine learning network graph is compiled separately using the OpenVINO™ Model Optimizer and the Intel® FPGA AI Suite compiler (dla_compiler) command.. When you compile the graph for the Intel® FPGA AI Suite SoC design example, ensure that you specify the --foutput-format=open_vino_hetero and -o <path_to_file>/CompiledNetwork.bin options.
The AOT file from the Intel® FPGA AI Suite compiler contains the compiled network partitions for FPGA and CPU devices along with the network weights. The network is compiled for a specific Intel® FPGA AI Suite architecture and batch size.
The SoC flow does not support the Just-In-Time (JIT) flow because Arm* libraries are not available for the Intel® FPGA AI Suite compiler.
An architecture file (.arch) describes the Intel® FPGA AI Suite IP architecture to the compiler. You must specify the same architecture file to the Intel® FPGA AI Suite compiler and to the Intel® FPGA AI Suite design example build script (dla_build_example_design.py).
The runtime stack cannot program the FPGA with a bitstream. The bitstream must be built into the SD card (.wic) image that is used to program the flash card, as described in (Optional) Create an SD Card Image (.wic) and Writing the SD Card Image (.wic) to an SD Card.
The runtime inference on the Intel® Arria® 10 SX SoC uses the OpenVINO™ Arm* CPU plugin. To enable fallback to the OpenVINO™ Arm* CPU plugin for graph layers that are not supported on the FPGA, the device flag must be set to HETERO:FPGA,CPU during the AOT compile step and when you run the dla_benchmark command.
In some cases, a layer might be supported by the FPGA even though the OpenVINO™ Arm* CPU plugin does not support the layer. This support is handled by the HETERO plugin and the layer is executed on the FPGA as expected. As an example, 3D convolution layers are not supported by the OpenVINO™ Arm* CPU plugin but still work properly provided that the .arch file used for the Intel® FPGA AI Suite IP configuration has enabled support for 3D convolutions.