Multi Channel DMA Intel® FPGA IP for PCI Express* Design Example User Guide

Download PDF
ID 683517
Date 1/19/2024
Public
Document Table of Contents
Document Table of Contents x
1. Introduction 2. Design Example Detailed Description 3. Design Example Quick Start Guide 4. Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide Archives 5. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide
1. Introduction x
1.1. Terms and Acronyms 1.2. MCDMA IP Modes
2. Design Example Detailed Description x
2.1. Design Example Overview 2.2. Hardware and Software Requirements 2.3. PIO using MCDMA Bypass Mode 2.4. Avalon-ST Packet Generate/Check 2.5. Avalon-ST Device-side Packet Loopback 2.6. Avalon-MM DMA 2.7. BAM_BAS Traffic Generator and Checker 2.8. External Descriptor Controller
2.1. Design Example Overview x
2.1.1. H-Tile MCDMA IP - Design Examples for Endpoint 2.1.2. P-Tile MCDMA IP - Design Examples for Endpoint 2.1.3. F-Tile MCDMA IP - Design Examples for Endpoint 2.1.4. R-Tile MCDMA IP - Design Examples for Endpoint
2.3. PIO using MCDMA Bypass Mode x
2.3.1. Avalon-ST PIO using MCDMA Bypass Mode 2.3.2. Avalon-MM PIO using MCDMA Bypass Mode
2.3.1. Avalon-ST PIO using MCDMA Bypass Mode x
2.3.1.1. Single-Port Avalon-ST PIO using MCDMA Bypass Mode
2.3.2. Avalon-MM PIO using MCDMA Bypass Mode x
2.3.2.1. Simulation Results 2.3.2.2. Hardware Test Results
2.4. Avalon-ST Packet Generate/Check x
2.4.1. Single-Port Avalon-ST Packet Generate/Check
2.4.1. Single-Port Avalon-ST Packet Generate/Check x
2.4.1.1. Simulation Waveforms 2.4.1.2. Simulation Log 2.4.1.3. Hardware Test Result
2.5. Avalon-ST Device-side Packet Loopback x
2.5.1. Simulation Results 2.5.2. Hardware Test Results
2.6. Avalon-MM DMA x
2.6.1. Simulation Results 2.6.2. Hardware Test Results
2.7. BAM_BAS Traffic Generator and Checker x
2.7.1. BAM_BAS Traffic Generator and Checker Example Design Register Map
2.8. External Descriptor Controller x
2.8.1. Registers 2.8.2. Hardware Test Results
2.8.1. Registers x
2.8.1.1. GCSR Registers (Base address 64’h00000) 2.8.1.2. QCSR Registers(Base address 64’h10000)
3. Design Example Quick Start Guide x
3.1. Design Example Directory Structure 3.2. Generating the Example Design using Intel® Quartus® Prime 3.3. Simulating the Design Example 3.4. Compiling the Example Design in Intel® Quartus® Prime 3.5. Running the Design Example Application on a Hardware Setup
3.2. Generating the Example Design using Intel® Quartus® Prime x
3.2.1. Procedure
3.3. Simulating the Design Example x
3.3.1. Testbench Overview 3.3.2. Supported Simulators 3.3.3. Example Testbench Flow for DMA Test with Avalon-ST Packet Generate/Check Design Example 3.3.4. Run the Simulation Script 3.3.5. Steps to Run the Simulation 3.3.6. View the Results
3.3.5. Steps to Run the Simulation x
3.3.5.1. Steps to Run the Simulation : QuestaSim* / Questa* Intel® FPGA Edition 3.3.5.2. Steps to Run the Simulation : VCS* / VCS* MX 3.3.5.3. Steps to Run the Simulation : Xcelium*
3.5. Running the Design Example Application on a Hardware Setup x
3.5.1. Program the FPGA 3.5.2. Quick Start Guide
3.5.2. Quick Start Guide x
3.5.2.1. Software Test Setup 3.5.2.2. Driver Support 3.5.2.3. MCDMA Custom Driver 3.5.2.4. MCDMA DPDK Poll Mode Driver 3.5.2.5. MCDMA Kernel Mode Network Device Driver
3.5.2.3. MCDMA Custom Driver x
3.5.2.3.1. Prerequisites 3.5.2.3.2. Software Setup 3.5.2.3.3. Run the Reference Example Application 3.5.2.3.4. Testing Bitstream Configuration beyond 256 Channels (for MCDMA Custom Driver) 3.5.2.3.5. External Descriptor Mode Verification
3.5.2.3.1. Prerequisites x
3.5.2.3.1.1. Configuration Changes from BIOS 3.5.2.3.1.2. External Packages 3.5.2.3.1.3. Set the Boot Parameters
3.5.2.3.2. Software Setup x
3.5.2.3.2.1. Installing the Linux Kernel Driver 3.5.2.3.2.2. Build and Install User Space Library 3.5.2.3.2.3. Enabling VFs and Create Guest VM by Using QEMU 3.5.2.3.2.4. Establish Communication Between Host and QEMU
3.5.2.3.3. Run the Reference Example Application x
3.5.2.3.3.1. Meta Data Test 3.5.2.3.3.2. Custom PIO Read Write Test 3.5.2.3.3.3. BAM Test 3.5.2.3.3.4. BAS Test 3.5.2.3.3.5. Packet Gen Test
3.5.2.4. MCDMA DPDK Poll Mode Driver x
3.5.2.4.1. Prerequisites 3.5.2.4.2. Install PMD and Test Application (for CentOS) 3.5.2.4.3. Install PMD and Test Application (for Ubuntu) 3.5.2.4.4. Create VM Environment 3.5.2.4.5. Run the Reference Example Application
3.5.2.4.1. Prerequisites x
3.5.2.4.1.1. Set the Boot Parameters 3.5.2.4.1.2. Configuration Changes from BIOS 3.5.2.4.1.3. Install and Build Testpmd
3.5.2.4.4. Create VM Environment x
3.5.2.4.4.1. Enable VFs 3.5.2.4.4.2. Create Guest VM by using QEMU 3.5.2.4.4.3. Establish Communication Between Host and VM
3.5.2.4.5. Run the Reference Example Application x
3.5.2.4.5.1. Channel ID VF PF Verification 3.5.2.4.5.2. Example of Verifying on an AVMM Design 3.5.2.4.5.3. Testing Bitstream Configuration beyond 256 Channels (for DPDK Poll Mode Driver) 3.5.2.4.5.4. BAM Test 3.5.2.4.5.5. BAS Test
3.5.2.5. MCDMA Kernel Mode Network Device Driver x
3.5.2.5.1. Build and Install Netdev Driver 3.5.2.5.2. Enable VFs if SRIOV is Supported 3.5.2.5.3. Configure the Number of Channels Supported on the Device 3.5.2.5.4. Configure the MTU Value 3.5.2.5.5. Configure the Device Communication 3.5.2.5.6. Configure Transmit Queue Selection Mechanism 3.5.2.5.7. Test Procedure by Using Name Space Environment 3.5.2.5.8. PIO Test
3.5.2.5.6. Configure Transmit Queue Selection Mechanism x
3.5.2.5.6.1. MCDMA Queue Selection 3.5.2.5.6.2. Transmit Packet Steering (XPS) 3.5.2.5.6.3. Default
1. Introduction
2. Design Example Detailed Description
3. Design Example Quick Start Guide
4. Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide Archives
5. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide

3.5.2.3.3. Run the Reference Example Application

  1. Build the reference application.
    $ cd software/user/cli/perfq_app/
$ make clean && make all
$ ./perfq_app -h
  2. This command displays the available options in the application, as shown in the image below:

    Refer to the README file located in the software/user/cli/perfq_app directory for more information.

  3. Perform a PIO test to check if the setup is correct. If successful, the application displays a Pass status as shown in the image below:

    Here the -b option should be provided with the correct BDF in the system.

  4. Perform IP reset.

    This step will perform an IP Reset. You can perform this step before every run.

    Build devmem utility: 
    $ cd software/user/cli/devmem
$ make clean all
$ ./devmem 0000:01:00.0 0 0x00200120 0x1

  5. For Channel ID VF PF verification: Channel ID , VF and PF information is inserted in AVST data pattern. To test dma in proper channel and VF, it can be tested as described below:

    For AVST LB : In the file software/user/common/mk/common.mk, enable CID _PAT.

    For Example: __ 
    cflags += -DCID_PAT

    Data validation mode supported. Performance mode is disabled. 

    __cflags += -DPERFQ_LOAD_DATA \ 
__cflags += -UPERFQ_PERF

    Supported direction: Applicable Bi-directional (-i) only.

    For AVST PKT GEN: The following configuration should be made in software/user/common/mk/common.mk

    Enable both CID_PAT and IFC_PROG_DATA_EN 

    __cflags += -DCID_PAT
__cflags += -DIFC_PROG_DATA_EN
DIFC_PROG_DATA_EN

    In validation mode, enable DPERFQ_LOAD_DATA flag and disable PERFQ_PERF flag. 

    __cflags += -DPERFQ_LOAD_DATA
__cflags += -UPERFQ_PERF

    Supported direction: Bi-directional (-Z), Tx (-t) and Rx (-r) as well.

  6. Example of testing a loopback design with the following configuration:
    Command: 
    $ ./perfq_app -b 0000:01:00.0 -p 32768 -l 5 -i -c 2 -d 2 -a 4

    Configuration:

    1. bdf (-b 0000:01:00.0)
    2. 2 channels (-c 2)
    3. Loopback (-i)
    4. Payload length of 32,768 bytes in each descriptor (-p 32768)
    5. Time Limit (-l 5)
    6. Debug log enabled (-d 2)
    7. One thread per queue (-a 4)
    Note: This hardware test was run with the Intel® Stratix® 10 GX H-tile PCIe Gen3 x16 configuration.
    Note: This hardware test was run with the Intel Agilex® 7 P-Tile PCIe Gen4x16 configuration.
    Figure 31. Custom AVST DMA Gen4 x16 : P-Tile Hardware Test Result
    Note: For Rx/Tx test:
    1. AVMM should use -u instead of -i
    2. -i should be used for AVST Loopback case
    3. -z should be used in AVST PKT case
  7. Example of verifying on AVMM DMA:
    Modify the below micro in the file: user/common/include/ifc_libmqdma.h 
    #define PCIe_SLOT 0 /* 0 - x16, 1 - x8 */
    Command: 
    $ ./perfq_app -b 0000:01:00.0 -p 32768 -l 5 -u -c 2 -d 2 -a 4

    Configuration:

    1. bdf (-b 0000:01:00.0)
    2. 2 channels (-c 2)
    3. bi-directional DMA transfer (-u)
    4. Payload length of 32768 bytes in each descriptor (-p 32768)
    5. Time Limit set to 5 (-l 5)
    6. Debug log enabled (-d 2)
    7. One thread per queue (-a 4)
    Note: To test the data validity, you need to perform H2D then D2H operations.
    Figure 32. Custom AVMM DMA Gen4 x16 : P-Tile Hardware Test Result
Note: Execute the following command for MSIX: 
$ulimit -n 8192
To enable data validation using -v option, set the software flags in user/common/mk/common.mk as follows: 
cflags += -UPERFQ_PERF
cflags += -DPERFQ_LOAD_DATA

Section Content
Meta Data Test
Custom PIO Read Write Test
BAM Test
BAS Test
Packet Gen Test

Level Two Title