GTS AXI Multichannel DMA IP for PCI Express User Guide

Download PDF
ID 847470
Date 5/06/2025
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Overview 2. Quick Start Guide 3. Configuring and Generating the GTS AXI Multichannel DMA IP for PCI Express 4. Integrating the IP With Your Application 5. Simulating the IP 6. Validating the IP A. Appendix A: Functional Description B. Appendix B: Registers C. Document Revision History for the GTS AXI Multichannel DMA IP for PCI Express*
1. Overview x
1.1. About the GTS AXI Multichannel DMA IP for PCI Express* 1.2. About the Document 1.3. Applications 1.4. Key Concepts 1.5. IP Design Flow 1.6. Design Guidelines 1.7. Design Flow Requirements
1.1. About the GTS AXI Multichannel DMA IP for PCI Express* x
1.1.1. IP Release Information 1.1.2. IP Features 1.1.3. IP High-level Block Diagram 1.1.4. Licensing the IP 1.1.5. Device Family Support 1.1.6. Device Speed Grade Support 1.1.7. Resource Utilization 1.1.8. IP and Design Example Support
1.1.2. IP Features x
1.1.2.1. Root Port Mode Features 1.1.2.2. Endpoint Mode Features
1.2. About the Document x
1.2.1. Target Audience 1.2.2. Conventions Used in the Document 1.2.3. Terminology 1.2.4. Acronyms 1.2.5. Reference Documents and Training
1.4. Key Concepts x
1.4.1. PCI Express and DMA 1.4.2. Altera FPGA IPs for PCI Express and DMA Controller 1.4.3. Example Use Case
1.6. Design Guidelines x
1.6.1. Supporting IPs 1.6.2. Aligning IP Settings with the GTS AXI Streaming IP for PCI Express
1.7. Design Flow Requirements x
1.7.1. Software Requirements 1.7.2. Hardware Requirements 1.7.3. Supported EDA Tools
2. Quick Start Guide x
2.1. Step-by-step IP Bring-up 2.2. Reset Sequence
2.1. Step-by-step IP Bring-up x
2.1.1. Downloading and Installing Quartus® Prime Software 2.1.2. Configuring and Generating the GTS AXI Multichannel DMA IP for PCI Express 2.1.3. Configuring and Generating the GTS AXI Streaming Intel FPGA IP for PCI Express 2.1.4. Configuring and Generating the GTS System PLL Clocks Intel FPGA IP 2.1.5. Configuring and Generating the GTS Reset Sequencer Intel FPGA IP 2.1.6. Configuring and Generating the Reset Release IP 2.1.7. Instantiating and Connecting the IP Interfaces 2.1.8. Simulate, Compile and Validate the Design on Hardware
2.2. Reset Sequence x
2.2.1. Cold Reset Sequence Triggered by PERST# Signal 2.2.2. Warm Reset Sequence Triggered by Hot Reset 2.2.3. Cold Reset Triggered by User 2.2.4. Warm Reset Triggered by User
3. Configuring and Generating the GTS AXI Multichannel DMA IP for PCI Express x
3.1. Creating the Quartus® Prime Project 3.2. Configuring the GTS AXI Multichannel DMA IP for PCI Express 3.3. Generating HDL for Simulation and Synthesis 3.4. Generating the Design Example 3.5. Compiling the Design Example
3.2. Configuring the GTS AXI Multichannel DMA IP for PCI Express x
3.2.1. IP Settings 3.2.2. PCIe Settings
3.2.2. PCIe Settings x
3.2.2.1. MCDMA Settings 3.2.2.2. PCI Express / PCI Capabilities 3.2.2.3. Base Address Registers
3.2.2.1. MCDMA Settings x
3.2.2.1.1. Multichannel DMA Mode 3.2.2.1.2. Bursting Master Mode 3.2.2.1.3. Bursting Slave Mode 3.2.2.1.4. BAM + BAS Mode 3.2.2.1.5. BAM + MCDMA Mode 3.2.2.1.6. BAM + BAS + MCDMA Mode
3.2.2.2. PCI Express / PCI Capabilities x
3.2.2.2.1. PCIe Device 3.2.2.2.2. PCIe MSI-X
3.2.2.2.2. PCIe MSI-X x
3.2.2.2.2.1. PCIe PF MSI-X 3.2.2.2.2.2. PCIe VF MSI-X
3.3. Generating HDL for Simulation and Synthesis x
3.3.1. IP Core Generation Output
4. Integrating the IP With Your Application x
4.1. Overview 4.2. Implementing Required Clocking 4.3. Implementing Required Resets 4.4. Connecting the GTS AXI Streaming IP-Facing Interfaces 4.5. Connect User Logic-Facing Interfaces
4.4. Connecting the GTS AXI Streaming IP-Facing Interfaces x
4.4.1. PCIe AXI-Stream TX Interface (ss_tx_st) 4.4.2. PCIe AXI-Stream RX Interface (ss_rx_st) 4.4.3. Control and Status Register Interface (ss_csr_lite) 4.4.4. Transmit Flow Control Credit Interface (ss_txcrdt) 4.4.5. Configuration Intercept Interface (CII) 4.4.6. Completion Timeout Interface (ss_cplto) 4.4.7. Function Level Reset (FLR) Interface 4.4.8. Control Shadow Interface (ss_ctrlshadow) 4.4.9. Error Interface
4.4.5. Configuration Intercept Interface (CII) x
4.4.5.1. Configuration Intercept Request Interface (ss_cii_req) 4.4.5.2. Configuration Intercept Response Interface (ss_cii_resp)
4.4.7. Function Level Reset (FLR) Interface x
4.4.7.1. FLR Received Interface (ss_flrrcvd) 4.4.7.2. FLR Completion Interface (ss_flrcmpl)
4.5. Connect User Logic-Facing Interfaces x
4.5.1. H2D AXI-Stream Manager (h2d_st_initatr) 4.5.2. D2H AXI-Stream Subordinate (d2h_st_respndr) 4.5.3. H2D/D2H AXI-MM Manager (dma_mm_initatr) 4.5.4. BAM AXI-MM Manager (bam_mm_initatr) 4.5.5. BAS AXI-MM Subordinate (bas_mm_respndr) 4.5.6. PIO AXI-Lite Manager (pio_lite_initiatr) 4.5.7. HIP Reconfiguration AXI-Lite Subordinate (user_csr_lite) 4.5.8. User Event MSI-X (user_msix) 4.5.9. User Event MSI (user_msi) 4.5.10. User Function Level Reset (user_flr) 4.5.11. User Configuration Intercept Interface 4.5.12. Configuration Slave (cs_lite_respndr)
4.5.11. User Configuration Intercept Interface x
4.5.11.1. User Configuration Intercept Request Interface (user_cii_req) 4.5.11.2. User Configuration Intercept Response Interface (user_cii_resp)
5. Simulating the IP x
5.1. Design Example Overview 5.2. Design Example Components 5.3. Simulating the Design Example
5.1. Design Example Overview x
5.1.1. AXI-S Device-side Packet Loopback Design Example
6. Validating the IP x
6.1. Hardware and Software Requirements 6.2. Running the Design Example Application on a Hardware Setup
6.2. Running the Design Example Application on a Hardware Setup x
6.2.1. Program the FPGA 6.2.2. Configuration Changes from BIOS 6.2.3. Host Operating System Check (if Working with Ubuntu 22.04) 6.2.4. Installing the Required Kernel Version (if Working with Ubuntu 24.04) 6.2.5. Set the Boot Parameters 6.2.6. Custom Driver 6.2.7. DPDK Poll Mode Driver
6.2.6. Custom Driver x
6.2.6.1. Prerequisites 6.2.6.2. PIO Test 6.2.6.3. Custom PIO Read Write Test 6.2.6.4. DMA Test
6.2.6.1. Prerequisites x
6.2.6.1.1. Steps for UIO Driver Installation 6.2.6.1.2. Enable Virtual Function 6.2.6.1.3. Build and Install User Space Library 6.2.6.1.4. Build the Reference Application
6.2.6.4. DMA Test x
6.2.6.4.1. AXI-S Device-side Packet Loopback (Device-side Packet Loopback)
6.2.7. DPDK Poll Mode Driver x
6.2.7.1. Prerequisites 6.2.7.2. PIO Test 6.2.7.3. DMA Test
6.2.7.1. Prerequisites x
6.2.7.1.1. Install PMD Driver 6.2.7.1.2. Bind the Device 6.2.7.1.3. Enable Virtual Function
6.2.7.3. DMA Test x
6.2.7.3.1. Device-side Packet Loopback
A. Appendix A: Functional Description x
A.1. High-level Functional Overview
A.1. High-level Functional Overview x
A.1.1. Multichannel DMA A.1.2. Bursting Master (BAM) A.1.3. Bursting Slave (BAS) A.1.4. MSI Interrupt A.1.5. Configuration Slave (CS) A.1.6. Root Port Address Translation Table Enablement A.1.7. Control and Status Register Interface A.1.8. Configuration Intercept Interface
A.1.1. Multichannel DMA x
A.1.1.1. H2D Data Mover A.1.1.2. D2H Data Mover A.1.1.3. Descriptors A.1.1.4. AXI4-Lite PIO Manager A.1.1.5. AXI-MM Write (H2D) and Read (D2H) Manager A.1.1.6. AXI-Stream Manager (H2D) and Subordinate (D2H) A.1.1.7. User MSI-X A.1.1.8. User Function Level Reset (FLR) A.1.1.9. Control and Status Registers
A.1.1.2. D2H Data Mover x
A.1.1.2.1. D2H Descriptor Fetch
A.1.1.3. Descriptors x
A.1.1.3.1. Support for Unaligned (or Byte-Aligned) Data Transfer A.1.1.3.2. Metadata Support A.1.1.3.3. MSI-X/Writeback
A.1.4. MSI Interrupt x
A.1.4.1. Endpoint MSI Support Through BAS A.1.4.2. MSI Interrupt Controller
A.1.5. Configuration Slave (CS) x
A.1.5.1. 14-bit AXI-MM Address Format A.1.5.2. Configuration Access Mechanism
B. Appendix B: Registers x
B.1. Queue Control B.2. MSI-X Memory Space B.3. Control Register (GCSR)
1. Overview
2. Quick Start Guide
3. Configuring and Generating the GTS AXI Multichannel DMA IP for PCI Express
4. Integrating the IP With Your Application
5. Simulating the IP
6. Validating the IP
A. Appendix A: Functional Description
B. Appendix B: Registers
C. Document Revision History for the GTS AXI Multichannel DMA IP for PCI Express*

A.1.1.2. D2H Data Mover

The D2H Data Mover (D2HDM) transfers data from the device memory to host memory. It receives the data from the user logic through the AXI-MM Subordinate/AXI-Stream Subordinate interface and generates MemWr TLPs to move the data to the host based on descriptor information such as the PCIe address (destination), data size, and MPS value to transmit data to the receive buffer in the host memory.

In AXI-MM mode, the D2HDM sends a series of AXI-MM reads based on PCIe address, MPS, and DMA transfer size. The AXI-MM read is generated as follows:

  • First AXI-MM read to the 64-byte address boundary. Multiple bursts are read on first AXI-MM read if:
    • AXI-MM address is 64-byte aligned.
    • Total payload count of the descriptor is 64-byte aligned and less than the maximum supported MPS.
  • Following with AXI-MM reads with the maximum supported MPS size.
  • Last AXI-MM Read of the remaining size.

In AXI-S mode, D2HDM deasserts ready when descriptors are not available.

  • Host sets up software descriptors for a port. Maximum payload count can be up to 1MB. SOF/EOF fields in the descriptor may not be set by the Host.
    • D2HDM uses a descriptor update sequence to update SOF, EOF, Rx payload count fields in the software descriptor at a Host location through a Memory Write request.
  • Once there is an available descriptor, incoming data from the user logic triggers a descriptor update sequence by D2HDM to mark the start of the AXI-Stream frame.
    • D2HDM issues a MWr to set the SOF field in the descriptor.
    • WB/MSI-X, if set in the descriptor, is issued.
  • AXI-Stream d2h_axi_st_tlast signal assertion triggers a descriptor update sequence by D2HDM to mark the end of the AXI-Stream frame. The descriptor update sequence is as follows:
    • D2HDM terminates the descriptor and initiates a descriptor update sequence.
    • During the descriptor update sequence, a MWr is issued to set the EOF field in the descriptor and update the Rx payload count field with the total bytes transferred.
    • WB/MSI-X, if set in the descriptor, is issued.
  • The descriptor immediately after the EOF sequence is considered as the start of the next AVST data frame and initiates a descriptor update sequence to set the SOF field.

When a descriptor is completed, that is, all DMA data corresponding to the descriptor has been sent to the host, the D2HDM performs housekeeping tasks that include:

  • Schedule MSI-X for a completed queue, if enabled in the descriptor.
  • Schedule Writeback Consumed Head Pointer for a completed queue, if enabled in the descriptor.
  • Update Consume Head Pointer for software polling.
  • MSI-X and Writeback are memory writes to the host via the D2HDM to avoid a race condition due to out-of-order writes.

Based on the updated status, software can proceed with releasing the receive buffer and reuse the descriptor ring entries.


Section Content
D2H Descriptor Fetch

Level Two Title