FPGA AI Suite Handbook

Download PDF
ID 863373
Date 11/21/2025
Public
Document Table of Contents
Document Table of Contents x
1. FPGA AI Suite Handbook 2. What is the FPGA AI Suite? 3. Design Considerations Before You Begin 4. Installing FPGA AI Suite 5. FPGA AI Suite Tutorial and Design Example Demonstration Applications 6. Creating an Architecture File for the FPGA AI Suite IP 7. Compiling Your Model with the FPGA AI Suite Compiler 8. Generating the FPGA AI Suite IP for Integration into an FPGA Design 9. Optimizing Your FPGA AI Suite IP 10. Integrating FPGA AI Suite IP into an FPGA Design 11. Using FPGA AI Suite as a PCIe* -Attach Platform 12. Using FPGA AI Suite in Hostless DDR-Free Mode 13. Using the FPGA AI Suite in Hostless JTAG Mode 14. Using FPGA AI Suite as an Embedded Platform 15. Using FPGA AI Suite in Video Applications 16. Using the FPGA AI Suite IP with High Bandwidth Memory on Stratix® 10 MX and Agilex™ 7 M-Series Devices 17. Developing Software Applications with the FPGA AI Suite A. FPGA AI Suite Handbook Archives B. FPGA AI Suite Handbook Document Revision History
2. What is the FPGA AI Suite? x
2.1. FPGA AI Suite Components 2.2. The FPGA AI Suite Tool Flow 2.3. The FPGA AI Suite User Flow 2.4. The FPGA AI Suite IP 2.5. The FPGA AI Suite Compiler 2.6. The FPGA AI Suite Design Examples
2.4. The FPGA AI Suite IP x
2.4.1. The FPGA AI Suite IP Overlay Architecture 2.4.2. Supported Models 2.4.3. Model Performance
2.4.1. The FPGA AI Suite IP Overlay Architecture x
2.4.1.1. How Does the FPGA AI Suite Overlay Architecture Work? 2.4.1.2. Parallelism in the FPGA AI Suite IP 2.4.1.3. FPGA AI Suite IP Datapath Component Organization
2.4.1.2. Parallelism in the FPGA AI Suite IP x
2.4.1.2.1. Comparing ML Architectures using the Compiler Report
2.4.2. Supported Models x
2.4.2.1. MobileNet V2 differences between Caffe and TensorFlow models
2.4.3. Model Performance x
2.4.3.1. Throughput on the MobileNetV1 model (and other very fast models) 2.4.3.2. DDR-Free Streaming Performance
2.6. The FPGA AI Suite Design Examples x
2.6.1. PCIe* -Attach Design Example 2.6.2. Open FPGA Stack (OFS) for PCIe* -Attach Design Examples 2.6.3. Hostless DDR-Free Design Example 2.6.4. Hostless JTAG Design Example 2.6.5. SoC Design Example 2.6.6. Design Example Components
2.6.6. Design Example Components x
2.6.6.1. FPGA AI Suite Design Example Utility 2.6.6.2. Example Architecture Bitstream Files 2.6.6.3. Design Example Software Components
2.6.6.1. FPGA AI Suite Design Example Utility x
2.6.6.1.1. The dla_build_example_design Command 2.6.6.1.2. Listing Available FPGA AI Suite Design Examples 2.6.6.1.3. Building FPGA AI Suite Design Examples
2.6.6.1.3. Building FPGA AI Suite Design Examples x
2.6.6.1.3.1. Staging FPGA AI Suite Design Example Builds
2.6.6.3. Design Example Software Components x
2.6.6.3.1. OpenVINO™ FPGA Runtime Overview 2.6.6.3.2. OpenVINO™ FPGA Runtime Plugin 2.6.6.3.3. FPGA AI Suite Runtime 2.6.6.3.4. FPGA AI Suite Custom Platform 2.6.6.3.5. Memory-Mapped Device (MMD) Driver 2.6.6.3.6. FPGA AI Suite Runtime MMD API 2.6.6.3.7. Board Support Package (BSP) Overview 2.6.6.3.8. Additional FPGA AI Suite SoC Design Example Software Components
2.6.6.3.7. Board Support Package (BSP) Overview x
2.6.6.3.7.1. Terasic* DE10-Agilex Development Board BSP Example 2.6.6.3.7.2. Agilex™ 7 PCIe-Attach OFS-based BSP Example
3. Design Considerations Before You Begin x
3.1. Consider an FPGA AI Suite Design Example as Starting Point 3.2. Determine Your Performance Bottlenecks 3.3. Select Your FPGA Device
4. Installing FPGA AI Suite x
4.1. Using the FPGA AI Suite Docker* Image 4.2. Installing the FPGA AI Suite Compiler and IP Generation Tools 4.3. Installing FPGA AI Suite Design Example Prerequisites
4.1. Using the FPGA AI Suite Docker* Image x
4.1.1. FPGA AI Suite Docker* Image Requirements 4.1.2. Running the FPGA AI Suite Docker* Container 4.1.3. Containerized FPGA AI Suite Docker* Image Quick-Start Tutorial
4.2. Installing the FPGA AI Suite Compiler and IP Generation Tools x
4.2.1. Supported FPGA Families 4.2.2. FPGA AI Suite Operating System Prerequisites 4.2.3. Installing FPGA AI Suite 4.2.4. Installing OpenVINO™ Toolkit 4.2.5. Installing Quartus® Prime Pro Edition Software 4.2.6. Setting Required Environment Variables 4.2.7. Finalizing Your FPGA AI Suite Installation 4.2.8. Downloading Precompiled Bitstreams and SD Card Images
4.2.2. FPGA AI Suite Operating System Prerequisites x
4.2.2.1. Red Hat* Enterprise Linux* Operating System Requirements 4.2.2.2. Ubuntu* Operating System Requirements 4.2.2.3. Windows* 11 Operating System Requirements
4.2.3. Installing FPGA AI Suite x
4.2.3.1. FPGA AI Suite Directory Structure 4.2.3.2. Installing the FPGA AI Suite Using the FPGA AI Suite Docker* Image 4.2.3.3. Installing the FPGA AI Suite Using the Quartus® Prime Installer 4.2.3.4. Installing the FPGA AI Suite with System Package Management Tools
4.2.3.4. Installing the FPGA AI Suite with System Package Management Tools x
4.2.3.4.1. Installing the FPGA AI Suite Into an Alternative Location
4.3. Installing FPGA AI Suite Design Example Prerequisites x
4.3.1. Creating a Working Directory 4.3.2. PCIe Design Example Prerequisites 4.3.3. OFS for PCIe* -Attach Design Example Prerequisites 4.3.4. Hostless DDR-Free Design Example Prerequisites 4.3.5. Hostless JTAG Design Example Prerequisites 4.3.6. SoC Design Example Prerequisites
4.3.2. PCIe Design Example Prerequisites x
4.3.2.1. PCIe Design Example Hardware Prerequisites 4.3.2.2. PCIe Design Example Operating System Prerequisites 4.3.2.3. PCIe Design Example Software Prerequisites 4.3.2.4. Building the FPGA AI Suite PCIe Design Example Runtime
4.3.2.3. PCIe Design Example Software Prerequisites x
4.3.2.3.1. Additional Software Prerequisites for the FPGA AI Suite PCIe Design Example for Agilex™ 7 Devices
4.3.2.4. Building the FPGA AI Suite PCIe Design Example Runtime x
4.3.2.4.1. FPGA AI Suite PCIe Design Example CMake Targets 4.3.2.4.2. FPGA AI Suite PCIe Design Example Build Options
4.3.3. OFS for PCIe* -Attach Design Example Prerequisites x
4.3.3.1. Building the FPGA AI Suite OFS for PCIe* Attach Design Example Runtime
4.3.3.1. Building the FPGA AI Suite OFS for PCIe* Attach Design Example Runtime x
4.3.3.1.1. OFS for PCIe* Attach Design Example CMake Targets 4.3.3.1.2. OFS for PCIe* Attach Design Example Build Options
4.3.4. Hostless DDR-Free Design Example Prerequisites x
4.3.4.1. Hostless DDR-Free Design Example Hardware Requirements 4.3.4.2. Hostless DDR-Free Design Example Software Requirements
4.3.5. Hostless JTAG Design Example Prerequisites x
4.3.5.1. Hostless JTAG Design Example Hardware Requirements 4.3.5.2. Hostless JTAG Design Example Software Requirements 4.3.5.3. Building the FPGA AI Suite Hostless JTAG Design Example Runtime
4.3.6. SoC Design Example Prerequisites x
4.3.6.1. SoC Design Example Development Kit Prerequisites 4.3.6.2. Building the Bitstream and Configuring Your Board for the FPGA AI Suite SoC Design Example
4.3.6.1. SoC Design Example Development Kit Prerequisites x
4.3.6.1.1. Agilex™ 7 FPGA I-Series Transceiver-SoC Development Kit SoC Design Example Hardware Requirements
4.3.6.2. Building the Bitstream and Configuring Your Board for the FPGA AI Suite SoC Design Example x
4.3.6.2.1. Initial Setup for the SoC Design Example 4.3.6.2.2. (Optional) Creating an SD Card Image (.wic) 4.3.6.2.3. Writing the SoC Design Example SD Card Image (.wic) to an SD Card 4.3.6.2.4. Preparing SoC FPGA Development Kits for the FPGA AI Suite SoC Design Example 4.3.6.2.5. Adding Compiled Graphs (AOT files) to the SD Card 4.3.6.2.6. Verifying FPGA Device Drivers
4.3.6.2.2. (Optional) Creating an SD Card Image (.wic) x
4.3.6.2.2.1. Installing Prerequisite Software for Building an SD Card Image 4.3.6.2.2.2. Building the FPGA Bitstreams for the SoC Design Example 4.3.6.2.2.3. Installing HPS Disk Image Build Prerequisites 4.3.6.2.2.4. (Optional) Downloading the ImageNet Categories 4.3.6.2.2.5. Building the SD Card Image for the SoC Design Example
4.3.6.2.4. Preparing SoC FPGA Development Kits for the FPGA AI Suite SoC Design Example x
4.3.6.2.4.1. Preparing the Agilex™ 5 FPGA E-Series 065B Modular Development Kit for the SoC Design Example 4.3.6.2.4.2. Preparing the Agilex™ 7 FPGA I-Series Transceiver-SoC Development Kit for the SoC Design Example 4.3.6.2.4.3. Preparing the Arria® 10 SX SoC FPGA Development Kit for the SoC Design Example 4.3.6.2.4.4. Configuring the SoC FPGA Development Kit UART Connection 4.3.6.2.4.5. Determining the SoC FPGA Development Kit IP Address
4.3.6.2.5. Adding Compiled Graphs (AOT files) to the SD Card x
4.3.6.2.5.1. Preparing OpenVINO™ Model Zoo for the SoC Design Example 4.3.6.2.5.2. Preparing a Model for the SoC Design Example 4.3.6.2.5.3. Compiling the Graphs for the SoC Design Example 4.3.6.2.5.4. Copying the Compiled Graphs to the SD card
5. FPGA AI Suite Tutorial and Design Example Demonstration Applications x
5.1. FPGA AI Suite Quick Start Tutorial 5.2. FPGA AI Suite Design Example Demonstration Applications
5.1. FPGA AI Suite Quick Start Tutorial x
5.1.1. Preparing OpenVINO™ Model Zoo for the PCIe Design Example 5.1.2. Preparing a Model for the PCIe Design Example 5.1.3. Running the Graph Compiler 5.1.4. Preparing an Image Set 5.1.5. Programming the FPGA Device 5.1.6. Performing Inference on the PCIe Design Example 5.1.7. Building an FPGA Bitstream for the PCIe Design Example 5.1.8. Building the Example FPGA Bitstreams 5.1.9. Preparing a ResNet50 v1 Model 5.1.10. Performing Inference on the Inflated 3D (I3D) Graph 5.1.11. Performing Inference on YOLOv3 and Calculating Accuracy Metrics 5.1.12. Performing Inference Without an FPGA Board
5.1.11. Performing Inference on YOLOv3 and Calculating Accuracy Metrics x
5.1.11.1. Preparing a YOLOv3 Model 5.1.11.2. Preparing a COCO Validation Dataset and Annotations 5.1.11.3. Inference on YOLOv3 and Calculating Accuracy Scores
5.2. FPGA AI Suite Design Example Demonstration Applications x
5.2.1. Running the PCIe* -Attach Design Example Demonstration Applications 5.2.2. Running the Hostless DDR-Free Design Example 5.2.3. Running the JTAG Design Example Demonstration Application 5.2.4. Running the SoC Design Example Demonstration Applications
5.2.1. Running the PCIe* -Attach Design Example Demonstration Applications x
5.2.1.1. Setup the OFS Environment for the FPGA Device in the OFS for PCIe* -Attach Design Example 5.2.1.2. Exporting Trained Graphs from Source Frameworks in the PCIe* Design Example 5.2.1.3. Compiling Exported Graphs Through the FPGA AI Suite in the PCIe* Design Example 5.2.1.4. Compiling the PCIe* Design Example 5.2.1.5. Programming the FPGA Device in the PCIe* Design Example 5.2.1.6. Performing Accelerated Inference with the dla_benchmark Application in the PCIe* Design Example 5.2.1.7. Running the Ported OpenVINO™ Demonstration Applications in the PCIe* Design Example
5.2.1.6. Performing Accelerated Inference with the dla_benchmark Application in the PCIe* Design Example x
5.2.1.6.1. Inference on Image Classification Graphs in the PCIe* Design Example 5.2.1.6.2. Inference on Object Detection Graphs in the PCIe* Design Example 5.2.1.6.3. Additional dla_benchmark Options in the PCIe* Design Example 5.2.1.6.4. The dla_benchmark Performance Metrics in the PCIe* Design Example
5.2.1.6.2. Inference on Object Detection Graphs in the PCIe* Design Example x
5.2.1.6.2.1. The mAP and COCO AP Metrics in the PCIe* Design Example 5.2.1.6.2.2. Specifying Ground Truth in the PCIe* Design Example 5.2.1.6.2.3. Example of Inference on Object Detection Graphs in the PCIe* Design Example
5.2.1.6.4. The dla_benchmark Performance Metrics in the PCIe* Design Example x
5.2.1.6.4.1. Interpreting System Throughput and Latency Metrics in the PCIe* Design Example
5.2.1.7. Running the Ported OpenVINO™ Demonstration Applications in the PCIe* Design Example x
5.2.1.7.1. Example Running the Object Detection Demonstration Application in the PCIe* Design Example
5.2.3. Running the JTAG Design Example Demonstration Application x
5.2.3.1. Building an FPGA Bitstream for the JTAG Design Examples 5.2.3.2. Programming the FPGA Device with JTAG Design Example Bitstream 5.2.3.3. Preparing JTAG Design Example Graphs for Inference with FPGA AI Suite 5.2.3.4. Performing Inference with the JTAG Design Example 5.2.3.5. JTAG Design Example Inference Performance Measurement 5.2.3.6. JTAG Design Example Known Issues and Limitations
5.2.4. Running the SoC Design Example Demonstration Applications x
5.2.4.1. Running the M2M Mode Demonstration Application in the SoC Design Example 5.2.4.2. Running the S2M Mode Demonstration Application in the SoC Design Example 5.2.4.3. Troubleshooting the Demonstration Applications in the SoC Design Example
6. Creating an Architecture File for the FPGA AI Suite IP x
6.1. Predefined FPGA AI Suite Architecture Files 6.2. FPGA AI Suite Architecture File Breakdown 6.3. FPGA AI Suite IP Parameterization
6.1. Predefined FPGA AI Suite Architecture Files x
6.1.1. Architecture Description File Format for Instance Parameterization
6.2. FPGA AI Suite Architecture File Breakdown x
6.2.1. FPGA AI Suite IP Supported Layers and Hyperparameter Ranges 6.2.2. Architecture Description File Parameters
6.2.2. Architecture Description File Parameters x
6.2.2.1. Parameter Group: Global Parameters 6.2.2.2. Parameter Group: activation 6.2.2.3. Parameter Group: pe_array 6.2.2.4. Parameter Group: pool 6.2.2.5. Parameter Group: depthwise 6.2.2.6. Module: softmax 6.2.2.7. Parameter Group: dma 6.2.2.8. Parameter Group: xbar 6.2.2.9. Parameter Group: filter_scratchpad 6.2.2.10. Parameter Group: input_stream_interface 6.2.2.11. Parameter Group: output_stream_interface 6.2.2.12. Parameter Group: config_network 6.2.2.13. Parameter Group: layout_transform_params 6.2.2.14. Parameter Group: lightweight_layout_transform_params
7. Compiling Your Model with the FPGA AI Suite Compiler x
7.1. Compiling a Graph 7.2. Estimating Graph Performance 7.3. Estimating the Area and Power of an Architecture 7.4. The FPGA AI Suite Compiler Report 7.5. FPGA AI Suite Compiler Command Line Options 7.6. Compiler Inputs and Outputs
7.4. The FPGA AI Suite Compiler Report x
7.4.1. Partitioning Table Report 7.4.2. Model Analyzer Reports 7.4.3. Visualizing How FPGA AI Suite Implements Your Graph
7.4.3. Visualizing How FPGA AI Suite Implements Your Graph x
7.4.3.1. Model Analyzer GUI Description 7.4.3.2. Fixing Unsupported Layers Assigned To CPU
7.5. FPGA AI Suite Compiler Command Line Options x
7.5.1. Inputs (dla_compiler Command Options) 7.5.2. Outputs (dla_compiler Command Options) 7.5.3. Reporting (dla_compiler Command Options) 7.5.4. Compilation Options (dla_compiler Command Options) 7.5.5. Architecture Options (dla_compiler Command Options) 7.5.6. Architecture Optimizer Options (dla_compiler Command Options) 7.5.7. Analyzer Tool Options (dla_compiler Command Options) 7.5.8. Miscellaneous Options (dla_compiler Command Options)
7.6. Compiler Inputs and Outputs x
7.6.1. FPGA AI Suite Input File Formats 7.6.2. FPGA AI Suite Compiler Graph Export Formats
8. Generating the FPGA AI Suite IP for Integration into an FPGA Design x
8.1. IP Generation Utility Execution Flows 8.2. IP Generation Utility Inputs 8.3. IP Generation Utility Outputs 8.4. IP Generation Utility Command Line Options
8.4. IP Generation Utility Command Line Options x
8.4.1. The --flow create_ip Flow 8.4.2. The --flow add_arch Flow 8.4.3. The --flow list Flow 8.4.4. The --flow remove_arch Flow
9. Optimizing Your FPGA AI Suite IP x
9.1. Folding Input 9.2. Parallelizing Inference Using FPGA AI Suite with Multiple Lanes and Multiple Instances 9.3. Transforming Input Data Layout 9.4. Make Precision vs. Performance Trade-offs for Your FPGA AI Suite IP 9.5. FPGA AI Suite IP Supported Layers and Hyperparameter Ranges 9.6. FPGA AI Suite IP Parameterization 9.7. Generating an Optimized Architecture
9.3. Transforming Input Data Layout x
9.3.1. Full Input Layout Transform 9.3.2. Lightweight Layout Transform
9.3.1. Full Input Layout Transform x
9.3.1.1. Input Feature Tensor In-Memory Format 9.3.1.2. Output Tensor In-Memory Format
9.3.1.1. Input Feature Tensor In-Memory Format x
9.3.1.1.1. Multiple Input Graphs 9.3.1.1.2. Input Scale and Shift 9.3.1.1.3. Input Transform Mapping
9.3.2. Lightweight Layout Transform x
9.3.2.1. How the Lightweight Layout Transform Works 9.3.2.2. Enabling the Lightweight Layout Transform
9.4. Make Precision vs. Performance Trade-offs for Your FPGA AI Suite IP x
9.4.1. Block Floating Point (BFP) 9.4.2. Improving Layer Accuracy by using Mixed Precision 9.4.3. Using the Mixed Precision Feature
9.4.3. Using the Mixed Precision Feature x
9.4.3.1. Performance Impact of Mixed Precision
9.7. Generating an Optimized Architecture x
9.7.1. Generating an Architecture for Highest Performance 9.7.2. Generating an Architecture Optimized for a Frame Rate Target Value
10. Integrating FPGA AI Suite IP into an FPGA Design x
10.1. FPGA AI Suite IP Directory Structure 10.2. Interfacing FPGA AI Suite IP an FPGA Design for a Typical System 10.3. FPGA AI Suite IP Interface 10.4. Instantiating the FPGA AI Suite IP in Platform Designer
10.3. FPGA AI Suite IP Interface x
10.3.1. Clock and Reset 10.3.2. AXI Interfaces 10.3.3. AXI Streaming Interface 10.3.4. CSR Map and Descriptor Queue Register and Bit Attribute Definitions Discovery ROM Interrupt Control DMA Descriptor Queue DMA Control Registers Performance Registers Debug Network Registers DMA License Register DMA Transaction Counters Model Update Registers 10.3.5. Interfacing the FPGA AI Suite IP to Avalon® Memory Map (AVMM)
10.3.3. AXI Streaming Interface x
10.3.3.1. Input Streaming 10.3.3.2. Output Streaming
10.4. Instantiating the FPGA AI Suite IP in Platform Designer x
10.4.1. Resource Utilization of FPGA AI Suite IP
11. Using FPGA AI Suite as a PCIe* -Attach Platform x
11.1. PCIe* -Attach Design Example System Architecture 11.2. OFS PCIe* -Attach Design Example Components
11.1. PCIe* -Attach Design Example System Architecture x
11.1.1. PCIe* -Attach System Overview 11.1.2. PCIe* -Attach Design Example Hardware
11.1.2. PCIe* -Attach Design Example Hardware x
11.1.2.1. PCIe* -Attach Design Example PLL Adjustment
11.2. OFS PCIe* -Attach Design Example Components x
11.2.1. OFS PCIe* -Attach Hardware Components 11.2.2. OFS PCIe* -Attach Software Components
12. Using FPGA AI Suite in Hostless DDR-Free Mode x
12.1. Generating Artifacts for Hostless DDR-Free Operation 12.2. Hostless DDR-Free Design Example System Architecture 12.3. Hostless DDR-Free Design Example Quartus® Prime System Console 12.4. Hostless DDR-Free Design Example JTAG to Avalon MM Host Register Map 12.5. Changing the ML Graph in a Hostless DDR-Free Architecture
12.2. Hostless DDR-Free Design Example System Architecture x
12.2.1. Hostless DDR-Free Design Example System Overview 12.2.2. Hostless DDR-Free Design Example Hardware
12.2.2. Hostless DDR-Free Design Example Hardware x
12.2.2.1. The Modular Scatter-Gather DMA (mSGDMA) Engines in the Hostless DDR-Free Design Example 12.2.2.2. On-Chip Memory Modules in the Hostless DDR-Free Design Example 12.2.2.3. Platform Designer System in the Hostless DDR-Free Design Example 12.2.2.4. PLL Adjustment in the Hostless DDR-Free Design Example
12.3. Hostless DDR-Free Design Example Quartus® Prime System Console x
12.3.1. Hostless DDR-Free Design Example Quartus® Prime System Console Script Options 12.3.2. Hostless DDR-Free Design Example Inference Functionality 12.3.3. Hostless DDR-Free Design Example System Reset 12.3.4. Hostless DDR-Free Design Example Input Data Conversion 12.3.5. Measuring Performance in the Hostless DDR-Free Design Example
12.5. Changing the ML Graph in a Hostless DDR-Free Architecture x
12.5.1. Updating Hostless DDR-Free MIF Files Through the CSR
13. Using the FPGA AI Suite in Hostless JTAG Mode x
13.1. Hostless JTAG Design Example Components
13.1. Hostless JTAG Design Example Components x
13.1.1. Hostless JTAG Hardware Components 13.1.2. Hostless JTAG Software Components
14. Using FPGA AI Suite as an Embedded Platform x
14.1. FPGA AI Suite SoC Design Example Inference Sequence Overview 14.2. Memory-to-Memory (M2M) Variant Design 14.3. Streaming-to-Memory (S2M) Variant Design 14.4. Top Level 14.5. The SoC Design Example Platform Designer System 14.6. Fabric EMIF Design Component 14.7. PLL Configuration 14.8. Yocto Build and Runtime Linux Environment 14.9. SoC Design Example MMD Layer Hardware Interaction Library 14.10. FPGA AI Suite SoC Design Example Run Process 14.11. FPGA AI Suite SoC Design Example Build Process
14.2. Memory-to-Memory (M2M) Variant Design x
14.2.1. The mSGDMA FPGA IP 14.2.2. RAM considerations
14.3. Streaming-to-Memory (S2M) Variant Design x
14.3.1. Streaming Enablement for FPGA AI Suite 14.3.2. Nios® V Subsystem 14.3.3. Streaming System Operation 14.3.4. Resolving Input Rate Mismatches Between the FPGA AI Suite IP and the Streaming Input 14.3.5. The Layout Transform IP as an Application-Specific Block in the SoC Design Example
14.3.3. Streaming System Operation x
14.3.3.1. Streaming System Buffer Management 14.3.3.2. Streaming System Inference Job Management
14.3.5. The Layout Transform IP as an Application-Specific Block in the SoC Design Example x
14.3.5.1. Layout Transform Considerations 14.3.5.2. Layout Transform IP Register Map 14.3.5.3. Layout Transform Configuration Options
14.4. Top Level x
14.4.1. Clock Domains
14.5. The SoC Design Example Platform Designer System x
14.5.1. The dla_0 Platform Designer Layer (dla.qsys) 14.5.2. The hps_0 Platform Designer Layer (hps.qys)
14.8. Yocto Build and Runtime Linux Environment x
14.8.1. Yocto Recipe: recipes-core/images/coredla-image.bb 14.8.2. Yocto Recipe: recipes-bsp/u-boot/u-boot-socfpga_%.bbappend 14.8.3. Yocto Recipe: recipes-drivers/msgdma-userio/msgdma-userio.bb 14.8.4. Yocto Recipe: recipes-drivers/uio-devices/uio-devices.bb 14.8.5. Yocto Recipe: recipes-kernel/linux/linux-socfpga-lts_%.bbappend 14.8.6. Yocto Recipe: recipes-support/devmem2/devmem2_2.0.bb 14.8.7. Yocto Recipe: wic
14.9. SoC Design Example MMD Layer Hardware Interaction Library x
14.9.1. MMD Layer Hardware Interaction Library Class mmd_device 14.9.2. MMD Layer Hardware Interaction Library Class uio_device 14.9.3. MMD Layer Hardware Interaction Library Class dma_device
14.10. FPGA AI Suite SoC Design Example Run Process x
14.10.1. Exporting Trained Graphs from Source Frameworks 14.10.2. Compiling Exported Graphs Through the FPGA AI Suite
14.11. FPGA AI Suite SoC Design Example Build Process x
14.11.1. Building the Quartus® Prime Project 14.11.2. Building the Bootable SD Card Image (.wic)
14.11.1. Building the Quartus® Prime Project x
14.11.1.1. Quartus® Prime Build Flow 14.11.1.2. Build Script Options 14.11.1.3. Build Directory
14.11.1.1. Quartus® Prime Build Flow x
14.11.1.1.1. Build Synchronization of FPGA with Software
14.11.1.3. Build Directory x
14.11.1.3.1. The build_stream_controller.sh Script
15. Using FPGA AI Suite in Video Applications x
15.1. Nios® Subsystem 15.2. Building the Stream Controller Module 15.3. Building the Streaming Demonstration Applications 15.4. Running the Streaming Demonstration
15.1. Nios® Subsystem x
15.1.1. Stream Controller Communication Protocol 15.1.2. Buffer Flow in Streaming Mode using Nios® V Software Scheduler
15.1.2. Buffer Flow in Streaming Mode using Nios® V Software Scheduler x
15.1.2.1. Review of M2M mode 15.1.2.2. External Streaming Mode Buffer Flow 15.1.2.3. Nios® V Stream Controller State Machine Buffer Flow
15.4. Running the Streaming Demonstration x
15.4.1. The streaming_inference_app Application 15.4.2. The image_streaming_app Application
16. Using the FPGA AI Suite IP with High Bandwidth Memory on Stratix® 10 MX and Agilex™ 7 M-Series Devices x
16.1. Current FPGA AI Suite IP Implementation With DDR 16.2. HBM Connection Considerations
16.2. HBM Connection Considerations x
16.2.1. Memory Size and Bandwidth Considerations 16.2.2. Memory Data Width Considerations 16.2.3. Conclusion
17. Developing Software Applications with the FPGA AI Suite x
17.1. Understanding the FPGA AI Suite Runtime Software Stack 17.2. Using the FPGA AI Suite Software Emulation 17.3. Running Inference on the FPGA AI Suite IP Without the OpenVINO™ Runtime
17.3. Running Inference on the FPGA AI Suite IP Without the OpenVINO™ Runtime x
17.3.1. Files Generated by the FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 17.3.2. Building the FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 17.3.3. Running the FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 17.3.4. FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility Example Application
B. FPGA AI Suite Handbook Document Revision History x
B.1. FPGA AI Suite Getting Started Guide Document Revision History B.2. FPGA AI Suite Compiler Reference Manual Document Revision History B.3. FPGA AI Suite IP Reference Manual Document Revision History B.4. FPGA AI Suite Example Designs User Guide Revision History B.5. AN 1008: Using the FPGA AI Suite Docker* Image Document Revision History B.6. AN 1020: Using the FPGA AI Suite IP with High Bandwidth Memory on Stratix® 10 MX and Agilex™ 7 M-Series Devices Document Revision History
B.4. FPGA AI Suite Example Designs User Guide Revision History x
B.4.1. FPGA AI Suite PCIe-based Design Example User Guide Document Revision History B.4.2. FPGA AI Suite SoC Design Example User Guide Document Revision History
1. FPGA AI Suite Handbook
2. What is the FPGA AI Suite?
3. Design Considerations Before You Begin
4. Installing FPGA AI Suite
5. FPGA AI Suite Tutorial and Design Example Demonstration Applications
6. Creating an Architecture File for the FPGA AI Suite IP
7. Compiling Your Model with the FPGA AI Suite Compiler
8. Generating the FPGA AI Suite IP for Integration into an FPGA Design
9. Optimizing Your FPGA AI Suite IP
10. Integrating FPGA AI Suite IP into an FPGA Design
11. Using FPGA AI Suite as a PCIe* -Attach Platform
12. Using FPGA AI Suite in Hostless DDR-Free Mode
13. Using the FPGA AI Suite in Hostless JTAG Mode
14. Using FPGA AI Suite as an Embedded Platform
15. Using FPGA AI Suite in Video Applications
16. Using the FPGA AI Suite IP with High Bandwidth Memory on Stratix® 10 MX and Agilex™ 7 M-Series Devices
17. Developing Software Applications with the FPGA AI Suite
A. FPGA AI Suite Handbook Archives
B. FPGA AI Suite Handbook Document Revision History

10.3.4. CSR Map and Descriptor Queue

The CSR interface uses a 32-bit data path in which all accesses are aligned to 32 bits; however the address is a byte address. The size of the CSR address space is 2048 bytes (11 bit addressable). The regions within the CSR address space are listed in the table that follows.

Table 42.  CSR Address Space Regions

Feature

Base Address

Discovery ROM (512 word)

0x000

Interrupt Control

0x200

DMA Descriptor Queue

0x210

DMA Control Registers

0x220

Performance Registers

0x240

Debug Network Registers

0x250
DMA License Register 0x260
DMA Transaction Counters 0x264
Model Update Registers 0x300

Register and Bit Attribute Definitions

The following notation describes the CSR registers.

Table 43.  Register and Bit Attribute Definitions

Attribute

Expansion

Description

RW

Read/Write

This bit can be read or written by software.

RO

Read Only

The bit is set by hardware only. Software can only read this bit. Writes have no effect.

RW1C

Read/Write 1to Clear

Software can read or clear this bit. Software must write 1 to clear this bit. Writing zero to an RW1C bit has no effect.

A multibit RW1C field can exist. In that case, all bits in the field are cleared if a 1 is written to any of the bits.

RsvdZ

Reserved and zero

Reserved for future RW1C implementations.

When you write to a register with RsvdZ bits, only write zeros to these bits.

Discovery ROM

The discovery ROM stores metadata. The metadata includes a hash for the architecture that the IP corresponds to and the FPGA AI Suite version that was used to create the IP.

The host runtime can use this information to determine whether the incoming inference job can be run on the IP instances. For example, if the architectures do not match each other, then inference is not possible.

The layout of the discovery ROM is as follows:

Table 44.  Discovery ROM Layout
Base Byte Address Length (in bytes) Feature

0x000

16

Hash of the Architecture Description File (.arch)

0x010

32

Human-readable FPGA AI Suite version string

Interrupt Control

The interrupt control feature registers are as follows:

Table 45.  Interrupt Control Feature Registers

Register

Offset

Attribute

Description

ICR

0x000

RW1C

DMA Interrupt control register

IMR

0x004

RW

DMA Interrupt mask register

The DMA optionally generates level sensitive interrupt signals in response to various events.

The hardware sets the corresponding bit within the ICR register whenever such an event occurs.

An interrupt is generated upon a 0-to-1 transition of a bit within ICR only if the corresponding bit in the IMR is set to one. A 0-to-1 transition of a bit within the IMR also generates an interrupt if the corresponding bit within the ICR is set to 1.

Table 46.  Interrupt Control Register (ICR) Fields

Field

Bit

Description

Reserved

31:2

RsvdZ (Reserved; software must write 0)

Inference_complete

1

Indicates that an inference request has completed

Error

0

Indicates that an error condition has been triggered
Table 47.  Interrupt Mask Register (IMR) Fields

Field

Bit

Description

Reserved

31:2

RsvdZ (Reserved; software must write 0)

Inference_complete_mask

1

Set to one to enable interrupt generation on inference completion

Error_mask

0

Set to one to enable interrupt generation on error condition

DMA Descriptor Queue

The DMA contains a single descriptor FIFO for enqueuing inference requests. Descriptors potentially require multiple register writes and are added to the queue upon writing to the desc_input_output_base_addr register.

The desc_cfg_filter_base_addr and desc_cfg_num_words are registers that hold their value.

If you already enqueued a DMA descriptor and want to enqueue another descriptor with the same values for the desc_cfg_filter_base_addr and desc_cfg_num_words registers, then write to the desc_input_output_base_addr register.

If you want to change the desc_cfg_filter_base_addr and desc_cfg_num_words registers for the next descriptor, then you must set new values before writing to the desc_input_output_base_addr register.

Table 48.  DMA Descriptor Queue Registers

Register

Offset

Attribute

Description

desc_cfg_filter_base_addr

0x000

RW

Base address pointer for the configuration buffer and for the filter buffer.

The filters are located at desc_cfg_filter_base_addr + desc_cfg_num_words, which is encoded in the address provided to the filter reader as configuration data.

Must be aligned to a multiple of the DDR word size.

desc_cfg_num_words - 2

0x004

RW

Length of the configuration buffer - 2, in config words (64 bits – 32 for instruction, 32 for data)

desc_input_output_base_addr

0x008

RW

Base address pointer for the input feature data and output inference results (written to an offset from the base address).

Must be aligned to a multiple of the DDR word size.

Writing to this register enqueues a descriptor into the internal DMA descriptor queue.
desc_diagnostics

0x00C

RO

This register is useful for debugging. Production software should not need to read from this.

Bit 0: Asserts if the descriptor queue overflows; this is a sticky bit which only clears after reset.

Bit 1: Descriptor queue is full or almost full.

Bit 2: Asserts if the inference limit for an unlicensed IP is reached. When asserted, inference requests are rejected.

All other bits are reserved.

DMA Control Registers

Table 49.  DMA Control Registers

Register

Offset

Attribute

Description

Intermediate_ddr_base_address

0x000

RW

Base address for the DDR intermediate data. This is a shared address across all graphs. Only required to be set once upon startup. Must be aligned to a multiple of the DDR word size.

Inference_completion_count

0x004

RO

Number of inference request completions by the FPGA AI Suite IP.
IP_reset 0x008 RW Write any non-zero value to this address to trigger a reset of the FPGA AI Suite IP.

The value is automatically cleared upon reset.

Reading from this register always returns 0.
Activate_streaming 0x00C RW When streaming is enabled in the architecture, writing "1" to this register makes the FPGA AI Suite IP begin queuing descriptors and start listening for streaming inputs.

Writing "0" stops queuing descriptors and turns off the input streaming interface.

Performance Registers

Hardware counters are provided to measure how many clock cycles that the IP is active. A job is considered active after the first word of its descriptor is read from the descriptor queue. A job is considered finished just before the done interrupt is raised and the completion count is updated.

The IP and supporting host form an elastic pipeline in which multiple jobs can be in flight. The IP tracks both the overall latency (for example, the length of time required to process 100 jobs) as well as the average latency for each of those jobs. The hardware tracks the total latency of every job but knowing the total number of jobs software can compute the average.

64-bit counters mitigate against overflow. There is no synchronization between reading the lower or upper 32 bits of a counter, therefore the software should not read the counters while the IP is active.

Table 50.  Performance Registers

Register

Offset

Attribute

Description

Total clocks active

(lower 32 bits)

0x000

RO

On each clock cycle, if any IP job is active, increment the counter by 1.

Total clocks active

(upper 32 bits)

0x004

RO

Same as above.

Total clocks for all jobs

(lower 32 bits)

0x008

RO

On each clock cycle, if there are N IP jobs active, increment the counter by N.

Total clocks for all jobs

(upper 32 bits)

0x00C

RO

Same as above.

Debug Network Registers

The debug network has the following registers available from the CSR:

Table 51.  Debug Network Registers

Register

Offset

Attribute

Description
DLA_DMA_CSR_OFFSET_DEBUG_NETWORK_ADDR

0x000

RO

Address that the debug network uses to issue a read request.
DLA_DMA_CSR_OFFSET_DEBUG_NETWORK_VALID

0x004

RO

Indicates that a read response has been received from the debug network.
DLA_DMA_CSR_OFFSET_DEBUG_NETWORK_DATA

0x008

RO

Data from debug network.

DMA License Register

Table 52.  DMA License Register

Register

Offset

Attribute

Description

license_flag

0x000

RO
Indicates whether the IP is licensed:
  • 0: unlicensed
  • 1: licensed

DMA Transaction Counters

Hardware counters are provided to measure the number of data words accessed by the DMA from the external DDR memory.

The counter values are separated into input feature reads, input weights and biases reads, and output feature writes. The width of each memory word in bytes matches the dma/ddr_data_bytes value in the architecture description file.

Table 53.  DMA Transaction Counter Registers

Register

Offset

Attribute

Description
Total number of input feature words read by the FPGA AI Suite IP

(lower 32 bits)

0x000

RO

This counter is incremented by 1 for every input feature word transferred from the external memory to the IP DMA on the AXI4 read bus.
Total number of input feature words read by the FPGA AI Suite IP

(upper 32 bits)

0x004

RO

Same as above.
Total number of input filter and biases words read by the FPGA AI Suite IP

(lower 32 bits)

0x008

RO

 This counter is incremented by 1 for every filter-bias word transferred from the external memory to the IP DMA on the AXI4 read bus.
Total number of input filter and biases words read by the FPGA AI Suite IP

(upper 32 bits)

0x00C

RO

Same as above.
Total number of output feature words written by the FPGA AI Suite IP

(lower 32 bits)

 0x010 RO This counter is incremented by 1 for every feature word written to the external memory by the IP DMA on the AXI4 write bus.
Total number of output feature words written by the FPGA AI Suite IP

(upper 32 bits)

 0x00C RO Same as above.

Model Update Registers

When the FPGA AI Suite IP is configured as DDR-Free, its model can be updated via writes to the following registers:
Register Offset Attribute Description
MODEL_UPDATE_WORD_0 0x000 W 32-bit chunk of a scratchpad or configuration word, index 0 (least significant)
MODEL_UPDATE_WORD_1 0x004 W Same as above, index 1
MODEL_UPDATE_WORD_2 0x008 W Same as above, index 2
MODEL_UPDATE_WORD_3 0x00C W Same as above, index 3
MODEL_UPDATE_WORD_4 0x010 W Same as above, index 4
MODEL_UPDATE_WORD_5 0x014 W Same as above, index 5
MODEL_UPDATE_WORD_6 0x018 W Same as above, index 6
MODEL_UPDATE_WORD_7 0x01C W Same as above, index 7
MODEL_UPDATE_WORD_8 0x020 W Same as above, index 8
MODEL_UPDATE_WORD_9 0x024 W Same as above, index 9
MODEL_UPDATE_WORD_10 0x028 W Same as above, index 10
MODEL_UPDATE_WORD_11 0x02C W Same as above, index 11
MODEL_UPDATE_WORD_12 0x030 W Same as above, index 12
MODEL_UPDATE_WORD_13 0x034 W Same as above, index 13
MODEL_UPDATE_WORD_14 0x038 W Same as above, index 14
MODEL_UPDATE_WORD_15 0x03C W Same as above, index 15
MODEL_UPDATE_WORD_16 0x040 W Same as above, index 16
MODEL_UPDATE_WORD_17 0x044 W Same as above, index 17
MODEL_UPDATE_WORD_18 0x048 W Same as above, index 18
MODEL_UPDATE_WORD_19 0x04C W Same as above, index 19
MODEL_UPDATE_WORD_20 0x050 W Same as above, index 20
MODEL_UPDATE_WORD_21 0x054 W Same as above, index 21
MODEL_UPDATE_WORD_22 0x058 W Same as above, index 22
MODEL_UPDATE_WORD_23 0x05C W Same as above, index 23
MODEL_UPDATE_WORD_24 0x060 W Same as above, index 24
MODEL_UPDATE_WORD_25 0x064 W Same as above, index 25
MODEL_UPDATE_WORD_26 0x068 W Same as above, index 26
MODEL_UPDATE_WORD_27 0x06C W Same as above, index 27
MODEL_UPDATE_WORD_28 0x070 W Same as above, index 28
MODEL_UPDATE_WORD_29 0x074 W Same as above, index 29
MODEL_UPDATE_WORD_30 0x078 W Same as above, index 30
MODEL_UPDATE_WORD_31 0x07C W Same as above, index 31
MODEL_UPDATE_CONTROL 0x080 W Type of word and target address of the update

To see how to use these registers to update the DDR-free model on the FPGA device, refer to Updating Hostless DDR-Free MIF Files Through the CSR

Level Two Title