Multi Channel DMA Intel® FPGA IP for PCI Express* Design Example User Guide

Download PDF
ID 683517
Date 6/09/2025
Public

Visible to Intel only — GUID: egy1745601971678

Ixiasoft

View Details

Document Table of Contents
Document Table of Contents x
1. Introduction 2. Design Example Detailed Description 3. Design Example Quick Start Guide 4. Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide Archives 5. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide
1. Introduction x
1.1. Terms and Acronyms 1.2. MCDMA IP Modes
2. Design Example Detailed Description x
2.1. Design Example Overview 2.2. PIO using MCDMA Bypass Mode 2.3. Avalon-ST Packet Generate/Check 2.4. Avalon-ST Device-side Packet Loopback 2.5. Avalon-MM DMA 2.6. BAM_BAS Traffic Generator and Checker 2.7. External Descriptor Controller
2.1. Design Example Overview x
2.1.1. H-Tile MCDMA IP - Design Examples for Endpoint 2.1.2. P-Tile MCDMA IP - Design Examples for Endpoint 2.1.3. F-Tile MCDMA IP - Design Examples for Endpoint 2.1.4. R-Tile MCDMA IP - Design Examples for Endpoint 2.1.5. Design Example BAR Mappings 2.1.6. Design Example Driver Support
2.2. PIO using MCDMA Bypass Mode x
2.2.1. Avalon-ST PIO using MCDMA Bypass Mode 2.2.2. Avalon-MM PIO using MCDMA Bypass Mode
2.2.1. Avalon-ST PIO using MCDMA Bypass Mode x
2.2.1.1. Single-Port Avalon-ST PIO using MCDMA Bypass Mode
2.2.2. Avalon-MM PIO using MCDMA Bypass Mode x
2.2.2.1. Simulation Results 2.2.2.2. Hardware Test Results
2.3. Avalon-ST Packet Generate/Check x
2.3.1. Single-Port Avalon-ST Packet Generate/Check
2.3.1. Single-Port Avalon-ST Packet Generate/Check x
2.3.1.1. Simulation Waveforms 2.3.1.2. Simulation Log 2.3.1.3. Hardware Test Result
2.4. Avalon-ST Device-side Packet Loopback x
2.4.1. Simulation Results 2.4.2. Hardware Test Results
2.5. Avalon-MM DMA x
2.5.1. Simulation Results 2.5.2. Hardware Test Results
2.6. BAM_BAS Traffic Generator and Checker x
2.6.1. BAM_BAS Traffic Generator and Checker Example Design Register Map
2.7. External Descriptor Controller x
2.7.1. Registers 2.7.2. Hardware Test Results
2.7.1. Registers x
2.7.1.1. GCSR Registers (Base address 64’h00000) 2.7.1.2. QCSR Registers(Base address 64’h10000)
3. Design Example Quick Start Guide x
3.1. Design Example Directory Structure 3.2. Generating the Example Design using Quartus® Prime 3.3. Simulating the Design Example 3.4. Compiling the Example Design in Quartus® Prime 3.5. Running the Design Example Application on a Hardware Setup
3.2. Generating the Example Design using Quartus® Prime x
3.2.1. Procedure
3.3. Simulating the Design Example x
3.3.1. Testbench Overview 3.3.2. Supported Simulators 3.3.3. Example Testbench Flow for DMA Test with Avalon-ST Packet Generate/Check Design Example 3.3.4. Run the Simulation Script 3.3.5. Steps to Run the Simulation 3.3.6. View the Results
3.3.5. Steps to Run the Simulation x
3.3.5.1. Steps to Run the Simulation : QuestaSim* / Questa* Intel® FPGA Edition 3.3.5.2. Steps to Run the Simulation : VCS* / VCS* MX 3.3.5.3. Steps to Run the Simulation : Xcelium*
3.5. Running the Design Example Application on a Hardware Setup x
3.5.1. Hardware Requirements 3.5.2. Software Requirements 3.5.3. Set Up the Hardware and Program the FPGA 3.5.4. Configuration Changes from BIOS 3.5.5. Host Operating System Check for Ubuntu v22.04 3.5.6. Installing the Required Kernel Version for Ubuntu v24.04 3.5.7. Set the Boot Parameters 3.5.8. MCDMA Custom Driver 3.5.9. MCDMA DPDK Poll Mode Driver 3.5.10. MCDMA Kernel Mode Network Device Driver
3.5.8. MCDMA Custom Driver x
3.5.8.1. Prerequisites 3.5.8.2. PIO Test 3.5.8.3. DMA Test 3.5.8.4. BAM Test 3.5.8.5. BAS Test
3.5.8.1. Prerequisites x
3.5.8.1.1. Install the Linux Kernel Driver 3.5.8.1.2. Enable Virtual Function 3.5.8.1.3. Build and Install the User Space Library 3.5.8.1.4. Build the Reference Application
3.5.8.2. PIO Test x
3.5.8.2.1. Custom PIO Read Write Test
3.5.8.3. DMA Test x
3.5.8.3.1. Avalon-MM DMA (AVMM DMA) 3.5.8.3.2. Avalon-ST Device-side Packet Loopback (Device-side Packet Loopback) 3.5.8.3.3. Avalon-ST Packet Generate/Check (Packet Generate/Check) 3.5.8.3.4. External Descriptor Controller
3.5.8.3.2. Avalon-ST Device-side Packet Loopback (Device-side Packet Loopback) x
3.5.8.3.2.1. DMA Test Beyond 256 Channels
3.5.8.3.3. Avalon-ST Packet Generate/Check (Packet Generate/Check) x
3.5.8.3.3.1. DMA Test Beyond 256 Channels
3.5.9. MCDMA DPDK Poll Mode Driver x
3.5.9.1. Prerequisites 3.5.9.2. PIO Test 3.5.9.3. DMA Test 3.5.9.4. BAM Test 3.5.9.5. BAS Test
3.5.9.1. Prerequisites x
3.5.9.1.1. Install the PMD Driver 3.5.9.1.2. Bind the Device 3.5.9.1.3. Enable Virtual Function
3.5.9.3. DMA Test x
3.5.9.3.1. Avalon-MM DMA (AVMM DMA) 3.5.9.3.2. Avalon-ST Packet Generate/Check (Packet Generate/Check) 3.5.9.3.3. Avalon-ST Device-side Packet Loopback (Device-side Packet Loopback)
3.5.9.3.2. Avalon-ST Packet Generate/Check (Packet Generate/Check) x
3.5.9.3.2.1. DMA Test Beyond 256 Channels
3.5.9.3.3. Avalon-ST Device-side Packet Loopback (Device-side Packet Loopback) x
3.5.9.3.3.1. DMA Test Beyond 256 Channels
3.5.10. MCDMA Kernel Mode Network Device Driver x
3.5.10.1. Prerequisites 3.5.10.2. PIO Test 3.5.10.3. Ping Test
3.5.10.1. Prerequisites x
3.5.10.1.1. Build and Install Netdev Driver 3.5.10.1.2. Enable Virtual Function
1. Introduction
2. Design Example Detailed Description
3. Design Example Quick Start Guide
4. Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide Archives
5. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide

3.5.10.3. Ping Test

  1. Configure the number of channels supported on the device. It must be consistent with the number set in the IP Parameter Editor. However, you have an option to use less than the maximum available channels.

    To check the number of channels supported, run the following command:

    $ ethtool -l ifc_mcdma0

    To set the number of channels to be used, run the following command:

    $ sudo ethtool -L <device> rx <chnls> tx <chnls>

    In a design with two physical functions, each allocated 16 channels, the following command set is used to configure a single channel: 

    $ sudo ethtool -L ifc_mcdma0 rx 1 tx 1
$ sudo ethtool -L ifc_mcdma1 rx 1 tx 1
  2. Configure the MTU value. Select the MTU value so that the sum of the MTU value and the Ethernet header length is aligned to 64.

    Use the following command to set the MTU value:

    $ ifconfig ifc_mcdma0 mtu <mtu value>

    For example, the following command sets the MTU value as 1500 (standard MTU size).

     $ sudo ifconfig ifc_mcdma0 mtu 1500
 $ sudo ifconfig ifc_mcdma1 mtu 1500
  3. Configure the device communication. Configure for PF-PF communication by using the debugfs interface. The Avalon-ST device-side packet loopback design example has the provision to switch the packets from one channel to another.

    Configure the communication from PF0 to PF1 and PF1 to PF0:

    $ echo src_chnl_no | sudo tee /sys/kernel/debug/ifc_mcdma_config/src_chnl
$ echo dst_chnl_no | sudo tee /sys/kernel/debug/ifc_mcdma_config/dst_chnl
$ echo 1 | sudo tee /sys/kernel/debug/ifc_mcdma_config/update

    For example, to configure a loopback between PF0 and PF1:

    Note: The physical channel number should be considered based on the design example configuration on channels per PF/VF function. In the example below, PF0 has channels assigned from 0-15 and PF1 has channels assigned from 16-31.
    • Configure the communication from PF0 (H2D queue) to PF1 (D2H queue). 
      	$ echo 0 | sudo tee /sys/kernel/debug/ifc_mcdma_config/src_chnl
	$ echo 16 | sudo tee /sys/kernel/debug/ifc_mcdma_config/dst_chnl
	$ echo 1 | sudo tee /sys/kernel/debug/ifc_mcdma_config/update
    • Configure the communication from PF1 (H2D queue) to PF0 (D2H queue). 
      $ echo 16 | sudo tee /sys/kernel/debug/ifc_mcdma_config/src_chnl
$ echo 0 | sudo tee /sys/kernel/debug/ifc_mcdma_config/dst_chnl
$ echo 1 | sudo tee /sys/kernel/debug/ifc_mcdma_config/update

    Display the configuration:

    $ sudo cat /sys/kernel/debug/ifc_mcdma_config/show

    Expected output:

    0 → 16

    16 → 0

  4. Ping test using the name space environment: use the following procedure to create name spaces and execute the commands.
    1. Stop the network manager by using the following command:

      $ sudo systemctl stop NetworkManager.service

    2. Create the name spaces:

      $ ip netns add <VM name>

      For example:

      $ sudo ip netns add vm0
$ sudo ip netns add vm1
    3. Assign the interface to name spaces using the following command:

      $ sudo ip link set <interface> netns <namespace>

      For example:

      $ sudo ip link set ifc_mcdma0 netns vm0
$ sudo ip link set ifc_mcdma1 netns vm1
    4. Bring up the interface:

      $ ip netns exec vm0 ifconfig <interface> up

      For example:

      $ sudo ip netns exec vm0 ifconfig ifc_mcdma0 up
$ sudo ip netns exec vm1 ifconfig ifc_mcdma1 up
    5. Assign the IP address by running the following commands:

      $ ip netns exec <namespace> ip addr add <ipaddr> dev <interfacename>

      For example:

      $ sudo ip netns exec vm0 ip addr add 1.1.1.10/24 dev ifc_mcdma0
$ sudo ip netns exec vm1 ip addr add 1.1.1.11/24 dev ifc_mcdma1
  5. Perform a ping from one interface to another interface:

    ip netns exec <namespace> ping <ip address>

    Run the following command to perform a ping with five packets:

    $ sudo ip netns exec vm0 ping -c 5 1.1.1.11

    Figure 54. Ping Test Results
Note: Observing the pkt-loss currently with the TCP and UDP traffic in overtime after starting the traffic. This leads to the iperf or netperf connections hanging and closing.
Level Two Title