Multi Channel DMA Intel® FPGA IP for PCI Express* Design Example User Guide

Download PDF
ID 683517
Date 6/09/2025
Public
Document Table of Contents
Document Table of Contents x
1. Introduction 2. Design Example Detailed Description 3. Design Example Quick Start Guide 4. Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide Archives 5. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide
1. Introduction x
1.1. Terms and Acronyms 1.2. MCDMA IP Modes
2. Design Example Detailed Description x
2.1. Design Example Overview 2.2. PIO using MCDMA Bypass Mode 2.3. Avalon-ST Packet Generate/Check 2.4. Avalon-ST Device-side Packet Loopback 2.5. Avalon-MM DMA 2.6. BAM_BAS Traffic Generator and Checker 2.7. External Descriptor Controller
2.1. Design Example Overview x
2.1.1. H-Tile MCDMA IP - Design Examples for Endpoint 2.1.2. P-Tile MCDMA IP - Design Examples for Endpoint 2.1.3. F-Tile MCDMA IP - Design Examples for Endpoint 2.1.4. R-Tile MCDMA IP - Design Examples for Endpoint 2.1.5. Design Example BAR Mappings 2.1.6. Design Example Driver Support
2.2. PIO using MCDMA Bypass Mode x
2.2.1. Avalon-ST PIO using MCDMA Bypass Mode 2.2.2. Avalon-MM PIO using MCDMA Bypass Mode
2.2.1. Avalon-ST PIO using MCDMA Bypass Mode x
2.2.1.1. Single-Port Avalon-ST PIO using MCDMA Bypass Mode
2.2.2. Avalon-MM PIO using MCDMA Bypass Mode x
2.2.2.1. Simulation Results 2.2.2.2. Hardware Test Results
2.3. Avalon-ST Packet Generate/Check x
2.3.1. Single-Port Avalon-ST Packet Generate/Check
2.3.1. Single-Port Avalon-ST Packet Generate/Check x
2.3.1.1. Simulation Waveforms 2.3.1.2. Simulation Log 2.3.1.3. Hardware Test Result
2.4. Avalon-ST Device-side Packet Loopback x
2.4.1. Simulation Results 2.4.2. Hardware Test Results
2.5. Avalon-MM DMA x
2.5.1. Simulation Results 2.5.2. Hardware Test Results
2.6. BAM_BAS Traffic Generator and Checker x
2.6.1. BAM_BAS Traffic Generator and Checker Example Design Register Map
2.7. External Descriptor Controller x
2.7.1. Registers 2.7.2. Hardware Test Results
2.7.1. Registers x
2.7.1.1. GCSR Registers (Base address 64’h00000) 2.7.1.2. QCSR Registers(Base address 64’h10000)
3. Design Example Quick Start Guide x
3.1. Design Example Directory Structure 3.2. Generating the Example Design using Quartus® Prime 3.3. Simulating the Design Example 3.4. Compiling the Example Design in Quartus® Prime 3.5. Running the Design Example Application on a Hardware Setup
3.2. Generating the Example Design using Quartus® Prime x
3.2.1. Procedure
3.3. Simulating the Design Example x
3.3.1. Testbench Overview 3.3.2. Supported Simulators 3.3.3. Example Testbench Flow for DMA Test with Avalon-ST Packet Generate/Check Design Example 3.3.4. Run the Simulation Script 3.3.5. Steps to Run the Simulation 3.3.6. View the Results
3.3.5. Steps to Run the Simulation x
3.3.5.1. Steps to Run the Simulation : QuestaSim* / Questa* Intel® FPGA Edition 3.3.5.2. Steps to Run the Simulation : VCS* / VCS* MX 3.3.5.3. Steps to Run the Simulation : Xcelium*
3.5. Running the Design Example Application on a Hardware Setup x
3.5.1. Hardware Requirements 3.5.2. Software Requirements 3.5.3. Set Up the Hardware and Program the FPGA 3.5.4. Configuration Changes from BIOS 3.5.5. Host Operating System Check for Ubuntu v22.04 3.5.6. Installing the Required Kernel Version for Ubuntu v24.04 3.5.7. Set the Boot Parameters 3.5.8. MCDMA Custom Driver 3.5.9. MCDMA DPDK Poll Mode Driver 3.5.10. MCDMA Kernel Mode Network Device Driver
3.5.8. MCDMA Custom Driver x
3.5.8.1. Prerequisites 3.5.8.2. PIO Test 3.5.8.3. DMA Test 3.5.8.4. BAM Test 3.5.8.5. BAS Test
3.5.8.1. Prerequisites x
3.5.8.1.1. Install the Linux Kernel Driver 3.5.8.1.2. Enable Virtual Function 3.5.8.1.3. Build and Install the User Space Library 3.5.8.1.4. Build the Reference Application
3.5.8.2. PIO Test x
3.5.8.2.1. Custom PIO Read Write Test
3.5.8.3. DMA Test x
3.5.8.3.1. Avalon-MM DMA (AVMM DMA) 3.5.8.3.2. Avalon-ST Device-side Packet Loopback (Device-side Packet Loopback) 3.5.8.3.3. Avalon-ST Packet Generate/Check (Packet Generate/Check) 3.5.8.3.4. External Descriptor Controller
3.5.8.3.2. Avalon-ST Device-side Packet Loopback (Device-side Packet Loopback) x
3.5.8.3.2.1. DMA Test Beyond 256 Channels
3.5.8.3.3. Avalon-ST Packet Generate/Check (Packet Generate/Check) x
3.5.8.3.3.1. DMA Test Beyond 256 Channels
3.5.9. MCDMA DPDK Poll Mode Driver x
3.5.9.1. Prerequisites 3.5.9.2. PIO Test 3.5.9.3. DMA Test 3.5.9.4. BAM Test 3.5.9.5. BAS Test
3.5.9.1. Prerequisites x
3.5.9.1.1. Install the PMD Driver 3.5.9.1.2. Bind the Device 3.5.9.1.3. Enable Virtual Function
3.5.9.3. DMA Test x
3.5.9.3.1. Avalon-MM DMA (AVMM DMA) 3.5.9.3.2. Avalon-ST Packet Generate/Check (Packet Generate/Check) 3.5.9.3.3. Avalon-ST Device-side Packet Loopback (Device-side Packet Loopback)
3.5.9.3.2. Avalon-ST Packet Generate/Check (Packet Generate/Check) x
3.5.9.3.2.1. DMA Test Beyond 256 Channels
3.5.9.3.3. Avalon-ST Device-side Packet Loopback (Device-side Packet Loopback) x
3.5.9.3.3.1. DMA Test Beyond 256 Channels
3.5.10. MCDMA Kernel Mode Network Device Driver x
3.5.10.1. Prerequisites 3.5.10.2. PIO Test 3.5.10.3. Ping Test
3.5.10.1. Prerequisites x
3.5.10.1.1. Build and Install Netdev Driver 3.5.10.1.2. Enable Virtual Function
1. Introduction
2. Design Example Detailed Description
3. Design Example Quick Start Guide
4. Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide Archives
5. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide

3.5.8.1.1. Install the Linux Kernel Driver

Both UIO and VFIO drivers are supported. Either one can be used.

User Space I/O (UIO): A kernel base module that the PCIe device uses to expose its resources to the user space.

Virtual Function I/O (VFIO) driver: An IOMMU/device agnostic framework for exposing direct device access to the user space in a secure, IOMMU-protected environment.

Steps for UIO driver installation:

  1. In the host system, install the UIO driver by running the following command if you are proceeding with UIO support. If you are proceeding with VFIO, this step is not required:

    $ sudo modprobe uio

  2. Install "make".
    1. Check if "make" is installed using the following command:

      $ make --version

    2. Install "make" using the following command:

      $ sudo apt install make

  3. Build the mcdma kernel driver and load:
    1. $ make clean all -C p0_software/kernel/driver/kmod/mcdma-custom-driver
    2. $ sudo insmod p0_software/kernel/driver/kmod/mcdma-custom-driver/ifc_uio.ko
  4. Verify whether the driver is loaded or not:

    $ lspci -d 1172:000 -v | grep ifc_uio

    (Kernel driver in use: ifc_uio)

Steps for VFIO driver installation:

This section can be bypassed if you are proceeding with UIO. Unlike UIO, the design example with and without SR-IOV enabled needs to be handled differently when using the VFIO Linux Kernel driver. PFs and their VFs should not be bound to the VFIO driver at the same time. PFs should bind to the ifc_uio driver, and VFs should bind to the VFIO driver.

  1. To use the VFIO Linux Kernel driver for the design example with SR-IOV enabled, create two copies of the software driver, p0_software, which is generated along with the design example in the host system, one for PFs and the other for VFs. If you are testing a design example without SR-IOV, skip to step 4.
  2. Follow the steps outlined in Steps for UIO driver installation to install the UIO Linux kernel driver for PFs, then follow the instructions in Enable Virtual Function to enable the VFs.
  3. Unbind the VFs from ifc_uio by using the following command:

    $ echo "<bdf>" | sudo tee /sys/bus/pci/devices/<bdf>/driver/unbind

    E.g: echo "0000:01:00.0" | sudo tee /sys/bus/pci/devices/0000\:01\:00.0/driver/unbind

  4. Before installing the VFIO driver, undefine UIO_SUPPORT in

    p0_software/user/common/mk/common.mk as shown below:

    __cflags += -UUIO_SUPPORT

    Note: For design examples with SR-IOV enabled or PFs which are bound to the ifc_uio driver, use a separate copy of the p0_software folder to undefine UIO_SUPPORT for the VFIO Linux Kernel driver installation.
  5. Install the vfio-pci module by running the following command:

    $ sudo modprobe vfio-pci

  6. Bind the device to vfio-pci using the following command:

    $ echo <PCI Vendor ID> <PCI Device ID> | sudo tee /sys/bus/pci/drivers/vfio-pci/new_id

    Example: echo 1172 0000 | sudo tee /sys/bus/pci/drivers/vfio-pci/new_id

Level Two Title