6.1. Intel® FPGA AI Suite SoC Design Example Inference Sequence Overview
The Intel® FPGA AI Suite IP works with system memory. To communicate with the system memory, the Intel® FPGA AI Suite has its own multichannel DMA engine.
This DMA engine pulls input commands and data from the system memory. It then writes the output data back to this memory for a Host CPU to collect.
When running inferences, the Intel® FPGA AI Suite continually reads and writes intermediate buffers to and from the system memory. The allocation of all the buffer addresses is done by the Intel® FPGA AI Suite runtime software library.
Running an inference requires minimal interaction between the host CPU and the IP Registers.
The system memory must be primed with all necessary buffers before starting a machine learning inference operation. These buffers are setup by the Intel® FPGA AI Suite runtime library and application that runs on the host CPU.
After the setup is complete, the host CPU pushes a job into the IP registers.
The Intel® FPGA AI Suite IP now performs a single inference. The job-queue registers in the IP are FIFO based, and the host application can store multiple jobs in the system memory and then prime multiple jobs inside the IP. Each job stores results in system memory and results in a CPU interrupt request.
For each inference operation in the M2M model, the host CPU (HPS) must perform an extensive data transfer from host (HPS) DDR memory to the external DDR memory that is allocated to the Intel® FPGA AI Suite IP. As this task has not been FPGA-accelerated in the design, the host operating system and Intel® FPGA AI Suite runtime library must manually transfer the data. This step consumes significant CPU resources. The M2M design uses a DMA engine to help with the data transfer from HPS DDR to the allocated DDR memory.
The Intel® FPGA AI Suite inference application and library software are responsible for keeping the Intel® FPGA AI Suite IP primed with new input data and responsible for consuming the results.
For a detailed overview of the Intel® FPGA AI Suite IP inference sequence, refer to the Intel® FPGA AI Suite IP Reference Manual .