GTS AXI Multichannel DMA IP for PCI Express* User Guide

Download PDF
ID 847470
Date 8/25/2025
Public
Document Table of Contents
Document Table of Contents x
1. Overview 2. Quick Start Guide 3. Configuring and Generating the GTS AXI Multichannel DMA IP for PCI Express 4. Integrating the IP With Your Application 5. Simulating the IP 6. Validating the IP 7. Known Issues A. Functional Description B. Software Programming Model C. Registers D. Document Revision History for the GTS AXI Multichannel DMA IP for PCI Express*
1. Overview x
1.1. About the GTS AXI Multichannel DMA IP for PCI Express* 1.2. About the Document 1.3. Applications 1.4. Key Concepts 1.5. IP Design Flow 1.6. Design Guidelines 1.7. Design Flow Requirements
1.1. About the GTS AXI Multichannel DMA IP for PCI Express* x
1.1.1. IP Release Information 1.1.2. IP Features 1.1.3. IP High-level Block Diagram 1.1.4. Licensing the IP 1.1.5. Device Family Support 1.1.6. Device Speed Grade Support 1.1.7. Resource Utilization 1.1.8. IP and Design Example Support
1.1.2. IP Features x
1.1.2.1. Root Port Mode Features 1.1.2.2. Endpoint Mode Features
1.2. About the Document x
1.2.1. Target Audience 1.2.2. Conventions Used in the Document 1.2.3. Terminology 1.2.4. Acronyms 1.2.5. Reference Documents and Training
1.4. Key Concepts x
1.4.1. PCI Express and DMA 1.4.2. Altera FPGA IPs for PCI Express and DMA Controller 1.4.3. Example Use Case
1.6. Design Guidelines x
1.6.1. Supporting IPs 1.6.2. Aligning IP Settings with the GTS AXI Streaming IP for PCI Express
1.7. Design Flow Requirements x
1.7.1. Software Requirements 1.7.2. Hardware Requirements 1.7.3. Supported EDA Tools
2. Quick Start Guide x
2.1. Step-by-step IP Bring-up 2.2. Reset Sequence
2.1. Step-by-step IP Bring-up x
2.1.1. Downloading and Installing Quartus® Prime Software 2.1.2. Configuring and Generating the GTS AXI Multichannel DMA IP for PCI Express 2.1.3. Configuring and Generating the GTS AXI Streaming IP for PCI Express 2.1.4. Configuring and Generating the GTS System PLL Clocks IP 2.1.5. Configuring and Generating the GTS Reset Sequencer IP 2.1.6. Configuring and Generating the Reset Release IP 2.1.7. Instantiating and Connecting the IP Interfaces 2.1.8. Simulate, Compile and Validate the Design on Hardware
2.2. Reset Sequence x
2.2.1. Cold Reset Sequence Triggered by PERST# Signal 2.2.2. Warm Reset Sequence Triggered by Hot Reset 2.2.3. Cold Reset Triggered by User 2.2.4. Warm Reset Triggered by User
3. Configuring and Generating the GTS AXI Multichannel DMA IP for PCI Express x
3.1. Creating the Quartus® Prime Project 3.2. Configuring the GTS AXI Multichannel DMA IP for PCI Express 3.3. Generating HDL for Simulation and Synthesis 3.4. Generating the Design Example 3.5. Compiling the Design Example
3.2. Configuring the GTS AXI Multichannel DMA IP for PCI Express x
3.2.1. IP Settings 3.2.2. PCIe Settings
3.2.2. PCIe Settings x
3.2.2.1. MCDMA Settings 3.2.2.2. PCI Express / PCI Capabilities 3.2.2.3. Base Address Registers
3.2.2.1. MCDMA Settings x
3.2.2.1.1. Multichannel DMA Mode 3.2.2.1.2. Bursting Master Mode 3.2.2.1.3. Bursting Slave Mode 3.2.2.1.4. BAM + BAS Mode 3.2.2.1.5. BAM + MCDMA Mode 3.2.2.1.6. BAM + BAS + MCDMA Mode
3.2.2.2. PCI Express / PCI Capabilities x
3.2.2.2.1. PCIe Device 3.2.2.2.2. PCIe MSI-X
3.2.2.2.2. PCIe MSI-X x
3.2.2.2.2.1. PCIe PF MSI-X 3.2.2.2.2.2. PCIe VF MSI-X
3.3. Generating HDL for Simulation and Synthesis x
3.3.1. IP Core Generation Output
4. Integrating the IP With Your Application x
4.1. Overview 4.2. Implementing Required Clocking 4.3. Implementing Required Resets 4.4. Connecting the GTS AXI Streaming IP-Facing Interfaces 4.5. Connect User Logic-Facing Interfaces
4.4. Connecting the GTS AXI Streaming IP-Facing Interfaces x
4.4.1. PCIe AXI-Stream TX Interface (ss_tx_st) 4.4.2. PCIe AXI-Stream RX Interface (ss_rx_st) 4.4.3. Control and Status Register Interface (ss_csr_lite) 4.4.4. Transmit Flow Control Credit Interface (ss_txcrdt) 4.4.5. Configuration Intercept Interface (CII) 4.4.6. Completion Timeout Interface (ss_cplto) 4.4.7. Function Level Reset (FLR) Interface 4.4.8. Control Shadow Interface (ss_ctrlshadow) 4.4.9. Error Interface
4.4.5. Configuration Intercept Interface (CII) x
4.4.5.1. Configuration Intercept Request Interface (ss_cii_req) 4.4.5.2. Configuration Intercept Response Interface (ss_cii_resp)
4.4.7. Function Level Reset (FLR) Interface x
4.4.7.1. FLR Received Interface (ss_flrrcvd) 4.4.7.2. FLR Completion Interface (ss_flrcmpl)
4.5. Connect User Logic-Facing Interfaces x
4.5.1. H2D AXI-Stream Manager (h2d_st_initatr) 4.5.2. D2H AXI-Stream Subordinate (d2h_st_respndr) 4.5.3. H2D/D2H AXI-MM Manager (dma_mm_initatr) 4.5.4. BAM AXI-MM Manager (bam_mm_initatr) 4.5.5. BAS AXI-MM Subordinate (bas_mm_respndr) 4.5.6. PIO AXI-Lite Manager (pio_lite_initiatr) 4.5.7. HIP Reconfiguration AXI-Lite Subordinate (user_csr_lite) 4.5.8. User Event MSI-X (user_msix) 4.5.9. User Event MSI (user_msi) 4.5.10. User Function Level Reset (user_flr) 4.5.11. User Configuration Intercept Interface 4.5.12. Configuration Slave (cs_lite_respndr)
4.5.11. User Configuration Intercept Interface x
4.5.11.1. User Configuration Intercept Request Interface (user_cii_req) 4.5.11.2. User Configuration Intercept Response Interface (user_cii_resp)
5. Simulating the IP x
5.1. Design Example Overview 5.2. Simulating the Design Example
5.1. Design Example Overview x
5.1.1. AXI-S Device-side Packet Loopback Design Example 5.1.2. AXI-S Packet Generate/Check Design Example 5.1.3. AXI-MM Traffic Generator/Checker Design Example 5.1.4. AXI-MM BAM EP Memory Design Example
5.1.3. AXI-MM Traffic Generator/Checker Design Example x
5.1.3.1. AXI-MM Traffic Generator/Checker Design Example Register Map
6. Validating the IP x
6.1. Driver Support for MCDMA Design Examples 6.2. Hardware and Software Requirements 6.3. Running the Design Example Application on a Hardware Setup
6.3. Running the Design Example Application on a Hardware Setup x
6.3.1. Program the FPGA 6.3.2. Configuration Changes from BIOS 6.3.3. Installing the Required Kernel Version (if Working with Ubuntu 24.04) 6.3.4. Set the Boot Parameters 6.3.5. Custom Driver 6.3.6. DPDK Poll Mode Driver
6.3.5. Custom Driver x
6.3.5.1. Prerequisites 6.3.5.2. PIO Test 6.3.5.3. Custom PIO Read Write Test 6.3.5.4. DMA Test 6.3.5.5. BAM Test 6.3.5.6. BAS Test
6.3.5.1. Prerequisites x
6.3.5.1.1. Install the Linux Kernel Driver 6.3.5.1.2. Enable Virtual Function 6.3.5.1.3. Build and Install User Space Library 6.3.5.1.4. Build the Reference Application
6.3.5.4. DMA Test x
6.3.5.4.1. AXI-S Device-side Packet Loopback (Device-side Packet Loopback) 6.3.5.4.2. AXI-S Packet Generate/Check
6.3.6. DPDK Poll Mode Driver x
6.3.6.1. Prerequisites 6.3.6.2. PIO Test 6.3.6.3. DMA Test 6.3.6.4. BAM Test 6.3.6.5. BAS Test
6.3.6.1. Prerequisites x
6.3.6.1.1. Install PMD Driver 6.3.6.1.2. Bind the Device 6.3.6.1.3. Enable Virtual Function
6.3.6.3. DMA Test x
6.3.6.3.1. Device-side Packet Loopback 6.3.6.3.2. AXI-MM DMA 6.3.6.3.3. AXI-S Packet Generate/Check
A. Functional Description x
A.1. High-level Functional Overview
A.1. High-level Functional Overview x
A.1.1. Multichannel DMA A.1.2. Bursting Master (BAM) A.1.3. Bursting Slave (BAS) A.1.4. MSI Interrupt A.1.5. Configuration Slave (CS) A.1.6. Root Port Address Translation Table Enablement A.1.7. Control and Status Register Interface A.1.8. Configuration Intercept Interface
A.1.1. Multichannel DMA x
A.1.1.1. H2D Data Mover A.1.1.2. D2H Data Mover A.1.1.3. Descriptors A.1.1.4. AXI4-Lite PIO Manager A.1.1.5. AXI-MM Write (H2D) and Read (D2H) Manager A.1.1.6. AXI-Stream Manager (H2D) and Subordinate (D2H) A.1.1.7. User MSI-X A.1.1.8. User Function Level Reset (FLR) A.1.1.9. Control and Status Registers
A.1.1.2. D2H Data Mover x
A.1.1.2.1. D2H Descriptor Fetch
A.1.1.3. Descriptors x
A.1.1.3.1. Support for Unaligned (or Byte-Aligned) Data Transfer A.1.1.3.2. Metadata Support A.1.1.3.3. MSI-X/Writeback
A.1.4. MSI Interrupt x
A.1.4.1. Endpoint MSI Support Through BAS A.1.4.2. MSI Interrupt Controller
A.1.5. Configuration Slave (CS) x
A.1.5.1. 14-bit AXI-MM Address Format A.1.5.2. Configuration Access Mechanism
B. Software Programming Model x
B.1. GTS AXI MCDMA IP Custom Driver B.2. GTS AXI MCDMA IP DPDK Poll Mode-Based Driver
B.1. GTS AXI MCDMA IP Custom Driver x
B.1.1. Architecture B.1.2. libmqdma library details B.1.3. Application B.1.4. Software Flow B.1.5. API Flow B.1.6. libmqdma Library API List B.1.7. Request Structures
B.1.2. libmqdma library details x
B.1.2.1. Channel Initialization B.1.2.2. Descriptor Memory Management B.1.2.3. Descriptor Completion Status Update
B.1.5. API Flow x
B.1.5.1. Single Descriptor Load and Submit B.1.5.2. Multiple-Descriptor Load and Submit
B.1.6. libmqdma Library API List x
B.1.6.1. ifc_api_start B.1.6.2. ifc_mcdma_port_by_name B.1.6.3. ifc_qdma_device_get B.1.6.4. ifc_num_channels_get B.1.6.5. ifc_qdma_channel_get B.1.6.6. ifc_qdma_acquire_channels B.1.6.7. ifc_qdma_release_all_channels B.1.6.8. ifc_qdma_device_put B.1.6.9. ifc_qdma_channel_put B.1.6.10. ifc_qdma_completion_poll B.1.6.11. ifc_qdma_request_start B.1.6.12. ifc_qdma_request_prepare B.1.6.13. ifc_qdma_descq_queue_batch_load B.1.6.14. ifc_qdma_request_submit B.1.6.15. ifc_qdma_pio_read32 B.1.6.16. ifc_qdma_pio_write32 B.1.6.17. ifc_qdma_pio_read64 B.1.6.18. ifc_qdma_pio_write64 B.1.6.19. ifc_qdma_pio_read128 B.1.6.20. ifc_qdma_pio_write128 B.1.6.21. ifc_qdma_pio_read256 B.1.6.22. ifc_qdma_pio_write256 B.1.6.23. ifc_request_malloc B.1.6.24. ifc_request_free B.1.6.25. ifc_app_stop B.1.6.26. ifc_qdma_poll_init B.1.6.27. ifc_qdma_poll_add B.1.6.28. ifc_qdma_poll_wait B.1.6.29. ifc_mcdma_port_by_name
B.2. GTS AXI MCDMA IP DPDK Poll Mode-Based Driver x
B.2.1. Architecture B.2.2. GTS AXI MCDMA Poll Mode Driver B.2.3. DPDK based application B.2.4. Request Structures B.2.5. Software Flow B.2.6. API Flow B.2.7. API List
B.2.2. GTS AXI MCDMA Poll Mode Driver x
B.2.2.1. Completion Status Update B.2.2.2. Metadata Support B.2.2.3. User MSI-X
C. Registers x
C.1. Queue Control C.2. MSI-X Memory Space C.3. Control Register (GCSR)
1. Overview
2. Quick Start Guide
3. Configuring and Generating the GTS AXI Multichannel DMA IP for PCI Express
4. Integrating the IP With Your Application
5. Simulating the IP
6. Validating the IP
7. Known Issues
A. Functional Description
B. Software Programming Model
C. Registers
D. Document Revision History for the GTS AXI Multichannel DMA IP for PCI Express*

6.3.5.6. BAS Test

Note: For the Traffic Generator/Checker design example, you must disable the MSI-X parameter, IFC_QDMA_MSIX_ENABLE, in the Custom Driver's p0_software/kernel/common/include/mcdma_ip_params.h file if MSI-X is not enabled in the IP Parameter Editor. By default, the Custom Driver software parameter is enabled and MSI-X is disabled in the IP. This mismatch prevents the ifc_uio kernel module from being loaded.

For BAS x4:

BAS x4 supports a burst length of 32 by default. In the file perfq_app.h (p0_software/user/cli/perfq_app/perfq_app.h):

#define IFC_MCDMA_BAS_X4_BURST_LENGTH 32

To enable BAS, set the following software flag in p0_software/user/common/mk/common.mk):

__cflags += -DIFC_MCDMA_BAS_EN

Note: Refer to Design Example Variants and BAR Mappings for BAR mappings information.
Note: A BAS test failure may be observed when the request size exceeds 2 kB. A fix is planned in a future release.

Commands:

To verify the write operation:

$ sudo ./cli/perfq_app/perfq_app -b 0000:08:00.0 -s 512 -e -t --bar=<bar number>

Figure 38. Write Operation Results

To verify the read operation:

$ sudo ./cli/perfq_app/perfq_app -b 0000:01:00.0 -s 512 -e -r --bar=<bar number>

Figure 39. Read Operation Results

To verify the write and read operations:

$ sudo ./cli/perfq_app/perfq_app -b 0000:01:00.0 -s 512 -e -z --bar=<bar number>

Figure 40. Write and Read Operation Results

Performance test:

$ sudo ./cli/perfq_app/perfq_app -b 0000:08:00.0 -s 16384 --bas_perf -z --bar=<bar number>

Figure 41. Performance Test Results
Note: You may not be able to proceed with the -z option. Add the flag #define IFC_QDMA_INTF_ST in user/common/include/mcdma_ip_params.h as a workaround to make it work.
Note: In the case of VFIO, to run BAM+BAS+MCDMA, you need to create at least three VFs and run on each VF respectively. If you try to use one VF to run BAM+BAS+MCDMA simultaneously in the case of VFIO, it gives a "resource busy" prompt.
Level Two Title