FPGA AI Suite: Design Examples User Guide

Download PDF
ID 848957
Date 4/30/2025
Public
Document Table of Contents
Document Table of Contents x
1. FPGA AI Suite Design Examples User Guide 2. FPGA AI Suite Design Examples 3. Design Example Components 4. [PCIE] Getting Started with the FPGA AI Suite PCIe* -based Design Example 5. [PCIE] Building the FPGA AI Suite Runtime 6. [PCIE] Running the Design Example Demonstration Applications 7. [PCIE] Design Example System Architecture for the Agilex™ 7 FPGA 8. [OFS-PCIE] Getting Started with Open FPGA Stack (OFS) for PCIe* -Attach Design Examples 9. [OFS-PCIE] Design Example Components 10. [HL-NO-DDR] Getting Started with the FPGA AI Suite DDR-Free Design Example 11. [HL-NO-DDR] Running the Hostless DDR-Free Design Example 12. [HL-NO-DDR] Design Example System Architecture 13. [HL-NO-DDR] Quartus® Prime System Console 14. [HL-NO-DDR] JTAG to Avalon MM Host Register Map 15. [HL-NO-DDR] Updating MIF Files 16. [HL-JTAG] Getting Started 17. [HL-JTAG] Design Example Components 18. [SOC] FPGA AI Suite SoC Design Example Prerequisites 19. [SOC] FPGA AI Suite SoC Design Example Quick Start Tutorial 20. [SOC] FPGA AI Suite SoC Design Example Run Process 21. [SOC] FPGA AI Suite SoC Design Example Build Process 22. [SOC] FPGA AI Suite SoC Design Example Quartus® Prime System Architecture 23. [SOC] FPGA AI Suite SoC Design Example Software Components 24. [SOC] Streaming-to-Memory (S2M) Streaming Demonstration A. FPGA AI Suite Example Designs User Guide Archives B. FPGA AI Suite Example Designs User Guide Revision History
2. FPGA AI Suite Design Examples x
2.1. About the PCIe* -Attach Design Example 2.2. About the Open FPGA Stack (OFS) for PCIe* -Attach Design Examples 2.3. About the Hostless DDR-Free Design Example 2.4. About the Hostless JTAG Design Example 2.5. About the SoC Design Example
3. Design Example Components x
3.1. FPGA AI Suite Design Example Utility 3.2. Example Architecture Bitstream Files 3.3. Design Example Software Components
3.1. FPGA AI Suite Design Example Utility x
3.1.1. The dla_build_example_design.py Command 3.1.2. Listing Available FPGA AI Suite Design Examples 3.1.3. Building FPGA AI Suite Design Examples
3.1.3. Building FPGA AI Suite Design Examples x
3.1.3.1. Staging FPGA AI Suite Design Example Builds 3.1.3.2. WSL 2 FPGA AI Suite Design Example Builds
3.3. Design Example Software Components x
3.3.1. OpenVINO™ FPGA Runtime Overview 3.3.2. OpenVINO™ FPGA Runtime Plugin 3.3.3. FPGA AI Suite Runtime 3.3.4. FPGA AI Suite Custom Platform 3.3.5. Memory-Mapped Device (MMD) Driver 3.3.6. FPGA AI Suite Runtime MMD API 3.3.7. Board Support Package (BSP) Overview
3.3.7. Board Support Package (BSP) Overview x
3.3.7.1. Terasic* DE10-Agilex Development Board BSP Example 3.3.7.2. Agilex™ 7 PCIe-Attach OFS-based BSP Example
5. [PCIE] Building the FPGA AI Suite Runtime x
5.1. [PCIE] CMake Targets 5.2. [PCIE] Build Options
6. [PCIE] Running the Design Example Demonstration Applications x
6.1. [PCIE] Exporting Trained Graphs from Source Frameworks 6.2. [PCIE] Compiling Exported Graphs Through the FPGA AI Suite 6.3. [PCIE] Compiling the PCIe* -based Example Design 6.4. [PCIE] Programming the FPGA Device ( Agilex™ 7) 6.5. [PCIE] Performing Accelerated Inference with the dla_benchmark Application 6.6. [PCIE] Running the Ported OpenVINO™ Demonstration Applications
6.5. [PCIE] Performing Accelerated Inference with the dla_benchmark Application x
6.5.1. [PCIE] Inference on Image Classification Graphs 6.5.2. [PCIE] Inference on Object Detection Graphs 6.5.3. [PCIE] Additional dla_benchmark Options 6.5.4. [PCIE] The dla_benchmark Performance Metrics
6.5.2. [PCIE] Inference on Object Detection Graphs x
6.5.2.1. [PCIE] The mAP and COCO AP Metrics 6.5.2.2. [PCIE] Specifying Ground Truth 6.5.2.3. [PCIE] Example of Inference on Object Detection Graphs
6.5.4. [PCIE] The dla_benchmark Performance Metrics x
6.5.4.1. [PCIE] Interpreting System Throughput and Latency Metrics
6.6. [PCIE] Running the Ported OpenVINO™ Demonstration Applications x
6.6.1. [PCIE] Example Running the Object Detection Demonstration Application
7. [PCIE] Design Example System Architecture for the Agilex™ 7 FPGA x
7.1. [PCIE] System Overview 7.2. [PCIE] Hardware
7.2. [PCIE] Hardware x
7.2.1. [PCIE] PLL Adjustment
8. [OFS-PCIE] Getting Started with Open FPGA Stack (OFS) for PCIe* -Attach Design Examples x
8.1. [OFS-PCIE] Building the FPGA AI Suite Runtime 8.2. [OFS-PCIE] Running the Design Example Demonstration Applications
8.1. [OFS-PCIE] Building the FPGA AI Suite Runtime x
8.1.1. [OFS-PCIE] CMake Targets 8.1.2. [OFS-PCIE] Build Options
8.2. [OFS-PCIE] Running the Design Example Demonstration Applications x
8.2.1. [OFS-PCIE] Setup the OFS Environment for the FPGA Device 8.2.2. [OFS-PCIE] Exporting Trained Graphs from Source Frameworks. 8.2.3. [OFS-PCIE] Compiling Exported Graphs Through the FPGA AI Suite 8.2.4. [OFS-PCIE] Compiling the OFS for PCIe* Attach Design Example 8.2.5. [OFS-PCIE] Programming the FPGA Green Bitstream 8.2.6. [OFS-PCIE] Performing Accelerated Inference with the dla_benchmark application
8.2.6. [OFS-PCIE] Performing Accelerated Inference with the dla_benchmark application x
8.2.6.1. [OFS-PCIE] Inference on Image Classification Graphs 8.2.6.2. [OFS-PCIE] Inference on Object Detection Graphs 8.2.6.3. [OFS-PCIE] Additional dla_benchmark Options 8.2.6.4. [OFS-PCIE] The dla_benchmark Performance Metrics
8.2.6.2. [OFS-PCIE] Inference on Object Detection Graphs x
8.2.6.2.1. [OFS-PCIE] The mAP and COCO AP Metrics 8.2.6.2.2. [OFS-PCIE] Specifying Ground Truth 8.2.6.2.3. [OFS-PCIE] Example of Inference on Object Detection Graphs
8.2.6.4. [OFS-PCIE] The dla_benchmark Performance Metrics x
8.2.6.4.1. [OFS-PCIE] Interpreting System Throughput and Latency Metrics
9. [OFS-PCIE] Design Example Components x
9.1. [OFS-PCIE] Hardware Components 9.2. [OFS-PCIE] Software Components
10. [HL-NO-DDR] Getting Started with the FPGA AI Suite DDR-Free Design Example x
10.1. [HL-NO-DDR] Hardware Requirements 10.2. [HL-NO-DDR] Software Requirements
12. [HL-NO-DDR] Design Example System Architecture x
12.1. [HL-NO-DDR] System Overview 12.2. [HL-NO-DDR] Hardware
12.2. [HL-NO-DDR] Hardware x
12.2.1. [HL-NO-DDR] The Modular Scatter-Gather DMA (mSGDMA) Engines 12.2.2. [HL-NO-DDR] On-Chip Memory Modules 12.2.3. [HL-NO-DDR] Platform Designer System 12.2.4. [HL-NO-DDR] PLL Adjustment
13. [HL-NO-DDR] Quartus® Prime System Console x
13.1. [HL-NO-DDR] Quartus® Prime System Console Script Options 13.2. [HL-NO-DDR] Functionality 13.3. [HL-NO-DDR] System Reset 13.4. [HL-NO-DDR] Input Data Conversion 13.5. [HL-NO-DDR] Measuring Performance
16. [HL-JTAG] Getting Started x
16.1. [HL-JTAG] Prerequisites 16.2. [HL-JTAG] Building the FPGA AI Suite Runtime 16.3. [HL-JTAG] Building an FPGA Bitstream for the JTAG Design Examples 16.4. [HL-JTAG] Programming the FPGA Device 16.5. [HL-JTAG] Preparing Graphs for Inference with FPGA AI Suite 16.6. [HL-JTAG] Performing Inference on the Agilex™ 5 FPGA E-Series 065B Modular Development Kit 16.7. [HL-JTAG] Inference Performance Measurement 16.8. [HL-JTAG] Known Issues and Limitations
16.1. [HL-JTAG] Prerequisites x
16.1.1. [HL-JTAG] Software Requirements 16.1.2. [HL-JTAG] Hardware Requirements
17. [HL-JTAG] Design Example Components x
17.1. [HL-JTAG] Hardware Components 17.2. [HL-JTAG] Software Components
18. [SOC] FPGA AI Suite SoC Design Example Prerequisites x
18.1. [SOC] Agilex™ 7 FPGA I-Series Transceiver-SoC Development Kit Hardware Requirements
19. [SOC] FPGA AI Suite SoC Design Example Quick Start Tutorial x
19.1. [SOC] Initial Setup 19.2. [SOC] Initializing a Work Directory 19.3. [SOC] (Optional) Create an SD Card Image (.wic) 19.4. [SOC] Writing the SD Card Image (.wic) to an SD Card 19.5. [SOC] Preparing SoC FPGA Development Kits for the FPGA AI Suite SoC Design Example 19.6. [SOC] Adding Compiled Graphs (AOT files) to the SD Card 19.7. [SOC] Verifying FPGA Device Drivers 19.8. [SOC] Running the Demonstration Applications
19.3. [SOC] (Optional) Create an SD Card Image (.wic) x
19.3.1. [SOC] Installing Prerequisite Software for Building an SD Card Image 19.3.2. [SOC] Building the FPGA Bitstreams 19.3.3. [SOC] Installing HPS Disk Image Build Prerequisites 19.3.4. [SOC] (Optional) Downloading the ImageNet Categories 19.3.5. [SOC] Building the SD Card Image
19.5. [SOC] Preparing SoC FPGA Development Kits for the FPGA AI Suite SoC Design Example x
19.5.1. [SOC] Preparing the Agilex™ 5 FPGA E-Series 065B Modular Development Kit 19.5.2. [SOC] Preparing the Agilex™ 7 FPGA I-Series Transceiver-SoC Development Kit 19.5.3. [SOC] Preparing the Arria® 10 SX SoC FPGA Development Kit 19.5.4. [SOC] Configuring the SoC FPGA Development Kit UART Connection 19.5.5. [SOC] Determining the SoC FPGA Development Kit IP Address
19.5.1. [SOC] Preparing the Agilex™ 5 FPGA E-Series 065B Modular Development Kit x
19.5.1.1. [SOC] Confirming the Agilex™ 5 FPGA E-Series 065B Modular Development Kit Board Setup 19.5.1.2. [SOC] Programming the Agilex™ 5 FPGA Device with the JTAG Indirect Configuration (.jic) File 19.5.1.3. [SOC] Programming the Agilex™ 5 FPGA Device with the SRAM Object File (.sof) 19.5.1.4. [SOC] Connecting the Agilex™ 5 FPGA E-Series 065B Modular Development Kit to the Host Development System
19.5.2. [SOC] Preparing the Agilex™ 7 FPGA I-Series Transceiver-SoC Development Kit x
19.5.2.1. [SOC] Confirming Agilex™ 7 FPGA I-Series Transceiver-SoC Development Kit Board Set Up 19.5.2.2. [SOC] Programming the Agilex™ 7 FPGA Device with the JTAG Indirect Configuration (.jic) File 19.5.2.3. [SOC] Programming the Agilex™ 7 FPGA Device with the SRAM Object File (.sof) 19.5.2.4. [SOC] Connecting the Agilex™ 7 FPGA I-Series Transceiver-SoC Development Kit to the Host Development System
19.5.3. [SOC] Preparing the Arria® 10 SX SoC FPGA Development Kit x
19.5.3.1. [SOC] Confirming Arria® 10 SX SoC FPGA Development Kit Board Settings 19.5.3.2. [SOC] Connecting the Arria® 10 SX SoC FPGA Development Kit to the Host Development System
19.6. [SOC] Adding Compiled Graphs (AOT files) to the SD Card x
19.6.1. [SOC] Preparing OpenVINO™ Model Zoo 19.6.2. [SOC] Preparing a Model 19.6.3. [SOC] Compiling the Graphs 19.6.4. [SOC] Copying the Compiled Graphs to the SD card
19.8. [SOC] Running the Demonstration Applications x
19.8.1. [SOC] Running the M2M Mode Demonstration Application 19.8.2. [SOC] Running the S2M Mode Demonstration Application 19.8.3. [SOC] Troubleshooting the Demonstration Applications
20. [SOC] FPGA AI Suite SoC Design Example Run Process x
20.1. [SOC] Exporting Trained Graphs from Source Frameworks 20.2. [SOC] Compiling Exported Graphs Through the FPGA AI Suite
21. [SOC] FPGA AI Suite SoC Design Example Build Process x
21.1. [SOC] Building the Quartus® Prime Project 21.2. [SOC] Building the Bootable SD Card Image (.wic)
21.1. [SOC] Building the Quartus® Prime Project x
21.1.1. [SOC] Quartus® Prime Build Flow 21.1.2. [SOC] Build Script Options 21.1.3. [SOC] Build Directory
21.1.1. [SOC] Quartus® Prime Build Flow x
21.1.1.1. [SOC] Build Synchronization of FPGA with Software
21.1.3. [SOC] Build Directory x
21.1.3.1. [SOC] The build_stream_controller.sh Script
22. [SOC] FPGA AI Suite SoC Design Example Quartus® Prime System Architecture x
22.1. [SOC] FPGA AI Suite SoC Design Example Inference Sequence Overview 22.2. [SOC] Memory-to-Memory (M2M) Variant Design 22.3. [SOC] Streaming-to-Memory (S2M) Variant Design 22.4. [SOC] Top Level 22.5. [SOC] The SoC Design Example Platform Designer System 22.6. [SOC] Fabric EMIF Design Component 22.7. [SOC] PLL Configuration
22.2. [SOC] Memory-to-Memory (M2M) Variant Design x
22.2.1. [SOC] The mSGDMA Intel FPGA IP 22.2.2. [SOC] RAM considerations
22.3. [SOC] Streaming-to-Memory (S2M) Variant Design x
22.3.1. [SOC] Streaming Enablement for FPGA AI Suite 22.3.2. [SOC] Nios® V Subsystem 22.3.3. [SOC] Streaming System Operation 22.3.4. [SOC] Resolving Input Rate Mismatches Between the FPGA AI Suite IP and the Streaming Input 22.3.5. [SOC] The Layout Transform IP as an Application-Specific Block
22.3.3. [SOC] Streaming System Operation x
22.3.3.1. [SOC] Streaming System Buffer Management 22.3.3.2. [SOC] Streaming System Inference Job Management
22.3.5. [SOC] The Layout Transform IP as an Application-Specific Block x
22.3.5.1. [SOC] Layout Transform Considerations 22.3.5.2. [SOC] Layout Transform IP Register Map 22.3.5.3. [SOC] Layout Transform Configuration Options
22.4. [SOC] Top Level x
22.4.1. [SOC] Clock Domains
22.5. [SOC] The SoC Design Example Platform Designer System x
22.5.1. [SOC] The dla_0 Platform Designer Layer (dla.qsys) 22.5.2. [SOC] The hps_0 Platform Designer Layer (hps.qys)
23. [SOC] FPGA AI Suite SoC Design Example Software Components x
23.1. [SOC] Yocto Build and Runtime Linux Environment 23.2. [SOC] FPGA AI Suite Runtime Plugin 23.3. [SOC] Runtime Interaction with the MMD Layer 23.4. [SOC] MMD Layer Hardware Interaction Library
23.1. [SOC] Yocto Build and Runtime Linux Environment x
23.1.1. [SOC] Yocto Recipe: recipes-core/images/coredla-image.bb 23.1.2. [SOC] Yocto Recipe: recipes-bsp/u-boot/u-boot-socfpga_%.bbappend 23.1.3. [SOC] Yocto Recipe: recipes-drivers/msgdma-userio/msgdma-userio.bb 23.1.4. [SOC] Yocto Recipe: recipes-drivers/uio-devices/uio-devices.bb 23.1.5. [SOC] Yocto Recipe: recipes-kernel/linux/linux-socfpga-lts_%.bbappend 23.1.6. [SOC] Yocto Recipe: recipes-support/devmem2/devmem2_2.0.bb 23.1.7. [SOC] Yocto Recipe: wic
23.4. [SOC] MMD Layer Hardware Interaction Library x
23.4.1. [SOC] MMD Layer Hardware Interaction Library Class mmd_device 23.4.2. [SOC] MMD Layer Hardware Interaction Library Class uio_device 23.4.3. [SOC] MMD Layer Hardware Interaction Library Class dma_device
24. [SOC] Streaming-to-Memory (S2M) Streaming Demonstration x
24.1. [SOC] Nios® Subsystem 24.2. [SOC] Building the Stream Controller Module 24.3. [SOC] Building the Streaming Demonstration Applications 24.4. [SOC] Running the Streaming Demonstration
24.1. [SOC] Nios® Subsystem x
24.1.1. [SOC] Stream Controller Communication Protocol 24.1.2. [SOC] Buffer Flow in Streaming Mode using Nios® V Software Scheduler
24.1.2. [SOC] Buffer Flow in Streaming Mode using Nios® V Software Scheduler x
24.1.2.1. [SOC] Review of M2M mode 24.1.2.2. [SOC] External Streaming Mode Buffer Flow 24.1.2.3. [SOC] Nios® V Stream Controller State Machine Buffer Flow
24.4. [SOC] Running the Streaming Demonstration x
24.4.1. [SOC] The streaming_inference_app Application 24.4.2. [SOC] The image_streaming_app Application
B. FPGA AI Suite Example Designs User Guide Revision History x
B.1. FPGA AI Suite PCIe-based Design Example User Guide Document Revision History B.2. FPGA AI Suite SoC Design Example User Guide Document Revision History
1. FPGA AI Suite Design Examples User Guide
2. FPGA AI Suite Design Examples
3. Design Example Components
4. [PCIE] Getting Started with the FPGA AI Suite PCIe* -based Design Example
5. [PCIE] Building the FPGA AI Suite Runtime
6. [PCIE] Running the Design Example Demonstration Applications
7. [PCIE] Design Example System Architecture for the Agilex™ 7 FPGA
8. [OFS-PCIE] Getting Started with Open FPGA Stack (OFS) for PCIe* -Attach Design Examples
9. [OFS-PCIE] Design Example Components
10. [HL-NO-DDR] Getting Started with the FPGA AI Suite DDR-Free Design Example
11. [HL-NO-DDR] Running the Hostless DDR-Free Design Example
12. [HL-NO-DDR] Design Example System Architecture
13. [HL-NO-DDR] Quartus® Prime System Console
14. [HL-NO-DDR] JTAG to Avalon MM Host Register Map
15. [HL-NO-DDR] Updating MIF Files
16. [HL-JTAG] Getting Started
17. [HL-JTAG] Design Example Components
18. [SOC] FPGA AI Suite SoC Design Example Prerequisites
19. [SOC] FPGA AI Suite SoC Design Example Quick Start Tutorial
20. [SOC] FPGA AI Suite SoC Design Example Run Process
21. [SOC] FPGA AI Suite SoC Design Example Build Process
22. [SOC] FPGA AI Suite SoC Design Example Quartus® Prime System Architecture
23. [SOC] FPGA AI Suite SoC Design Example Software Components
24. [SOC] Streaming-to-Memory (S2M) Streaming Demonstration
A. FPGA AI Suite Example Designs User Guide Archives
B. FPGA AI Suite Example Designs User Guide Revision History

11. [HL-NO-DDR] Running the Hostless DDR-Free Design Example

Procedure

To run the hostless DDR-free design example with a ResNet-18 PyTorch Model:
  1. Download and prepare the ResNet-18 PyTorch Model with the OpenVINO™ Open Model Zoo tools with the following commands:
    source ~/build-openvino-dev/openvino_env/bin/activate

omz_downloader --name resnet-18-pytorch \
    --output_dir $COREDLA_WORK/demo/models/

omz_converter --name resnet-18-pytorch \
    --download_dir $COREDLA_WORK/demo/models/ \
    --output_dir $COREDLA_WORK/demo/models/
    Important: The OpenVINO™ Open Model Zoo (OMZ) PyTorch models do not include a softmax operation at the end of the model.
  2. Generate the parameter ROMs as .mif files by running the FPGA AI Suite compiler with the following command:
    dla_compiler \
  --batch-size=1 \
  --network-file <path/to/graph> \
  --march $COREDLA_ROOT/example_architectures/AGX7_Streaming_Ddrfree_Resnet18.arch \
  --foutput-format=open_vino_hetero \
  --o <compiler output .bin file name>  \
  --fplugin HETERO:FPGA \
  --dumpdir $COREDLA_WORK/resnet-18-dlac-out/

    The .mif files are created in a subdirectory of the directory specified by the --dumpdir option. This subdirectory is called parameter_rom.

    For details about creating the .mif files required for DDR-free operation, refer to "Generating Artifacts for DDR-Free Operation" in the FPGA AI Suite Compiler Reference Manual .

  3. Build the example design with the following command:
    dla_build_example_design.py build \
  --output-dir <path/to/build/dir> \
  --num-instances 1 \
  --seed 1 \
  --parameter-rom-dir $COREDLA_WORK/resnet-18-dlac-out/parameter_rom/ \
  agx7_iseries_ddrfree \
  $COREDLA_ROOT/example_architectures/AGX7_Streaming_Ddrfree_Resnet18.arch

    Building the example design creates the bitstream needed to program the FPGA device.

    For more information about the dla_build_example_design.py command, refer to The dla_build_example_design.py Command.

  4. Program the FPGA device with the Quartus® Prime Programmer.

    The bitstream used to program the device is <path/to/build/dir>/hw/output_files/top.sof.

    Program the FPGA device with the following command: 
    quartus_pgm -c 1 -m jtag -o "p;top.sof@1"

    For more information about the Quartus® Prime Programmer, refer to Quartus® Prime Pro Edition User Guide: Programmer .

  5. Use the Quartus® Prime System Console to run inference on the example design.

    Because this example design is hostless, operations that typically come from the host are performed through Quartus® Prime System Console instead. For more information about the Quartus® Prime System Console, refer to "Analyzing and Debugging Designs with System Console" in Quartus® Prime Pro Edition User Guide: Debug Tools .

    Use the System Console to complete the following steps:
    1. Store input features in the FPGA on-chip memory.
    2. Prime the FPGA AI Suite IP registers for inference.
    3. Configure an ingress Modular Scatter-Gather DMA (mSGDMA) core to read the input features from on-chip memory and stream data into the FPGA AI Suite IP.
    4. Configure an egress mSGDMA core to stream data from the FPGA AI Suite IP into on-chip memory.
    5. Read the inference results from on-chip memory.

    The design example provides a System Console script to automate these operations for you. You can find the script in the $CORDLA_ROOT/runtime/streaming/ed0_streaming_example folder.

    To use the design example System Console script:
    1. Run the following command:
      system-console --script=system_console_script.tcl \
               --input <path-to-img.bin> \
               --num_inferences <#-of-inferences> \
               --output_shape <[C H W]> \
               --functional

      The design example Quartus® Prime System Console script generates a file called output.bin that contains the raw inference results.

    2. (Optional) To measure the performance of the design example, run the following command:
      system-console --script=system_console_script.tcl \
               --input <path-to-img.bin> \
               --output_shape <[C H W]> \
               --core_ip_performance
    For more information about the System Console script, refer to [HL-NO-DDR] Quartus Prime System Console.
  6. Postprocess the raw inference output for readability with the following command:
    python3 $COREDLA_ROOT/bin/streaming_post_processing.py <path-to-output.bin>

    This script cleans the raw output binary file by script some invalid bytes and storing an FP16 formatted result_hw.txt file for readability.

Level Two Title