GTS AXI Multichannel DMA IP for PCI Express User Guide

Download PDF
ID 847470
Date 5/06/2025
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Overview 2. Quick Start Guide 3. Configuring and Generating the GTS AXI Multichannel DMA IP for PCI Express 4. Integrating the IP With Your Application 5. Simulating the IP 6. Validating the IP A. Appendix A: Functional Description B. Appendix B: Registers C. Document Revision History for the GTS AXI Multichannel DMA IP for PCI Express*
1. Overview x
1.1. About the GTS AXI Multichannel DMA IP for PCI Express* 1.2. About the Document 1.3. Applications 1.4. Key Concepts 1.5. IP Design Flow 1.6. Design Guidelines 1.7. Design Flow Requirements
1.1. About the GTS AXI Multichannel DMA IP for PCI Express* x
1.1.1. IP Release Information 1.1.2. IP Features 1.1.3. IP High-level Block Diagram 1.1.4. Licensing the IP 1.1.5. Device Family Support 1.1.6. Device Speed Grade Support 1.1.7. Resource Utilization 1.1.8. IP and Design Example Support
1.1.2. IP Features x
1.1.2.1. Root Port Mode Features 1.1.2.2. Endpoint Mode Features
1.2. About the Document x
1.2.1. Target Audience 1.2.2. Conventions Used in the Document 1.2.3. Terminology 1.2.4. Acronyms 1.2.5. Reference Documents and Training
1.4. Key Concepts x
1.4.1. PCI Express and DMA 1.4.2. Altera FPGA IPs for PCI Express and DMA Controller 1.4.3. Example Use Case
1.6. Design Guidelines x
1.6.1. Supporting IPs 1.6.2. Aligning IP Settings with the GTS AXI Streaming IP for PCI Express
1.7. Design Flow Requirements x
1.7.1. Software Requirements 1.7.2. Hardware Requirements 1.7.3. Supported EDA Tools
2. Quick Start Guide x
2.1. Step-by-step IP Bring-up 2.2. Reset Sequence
2.1. Step-by-step IP Bring-up x
2.1.1. Downloading and Installing Quartus® Prime Software 2.1.2. Configuring and Generating the GTS AXI Multichannel DMA IP for PCI Express 2.1.3. Configuring and Generating the GTS AXI Streaming Intel FPGA IP for PCI Express 2.1.4. Configuring and Generating the GTS System PLL Clocks Intel FPGA IP 2.1.5. Configuring and Generating the GTS Reset Sequencer Intel FPGA IP 2.1.6. Configuring and Generating the Reset Release IP 2.1.7. Instantiating and Connecting the IP Interfaces 2.1.8. Simulate, Compile and Validate the Design on Hardware
2.2. Reset Sequence x
2.2.1. Cold Reset Sequence Triggered by PERST# Signal 2.2.2. Warm Reset Sequence Triggered by Hot Reset 2.2.3. Cold Reset Triggered by User 2.2.4. Warm Reset Triggered by User
3. Configuring and Generating the GTS AXI Multichannel DMA IP for PCI Express x
3.1. Creating the Quartus® Prime Project 3.2. Configuring the GTS AXI Multichannel DMA IP for PCI Express 3.3. Generating HDL for Simulation and Synthesis 3.4. Generating the Design Example 3.5. Compiling the Design Example
3.2. Configuring the GTS AXI Multichannel DMA IP for PCI Express x
3.2.1. IP Settings 3.2.2. PCIe Settings
3.2.2. PCIe Settings x
3.2.2.1. MCDMA Settings 3.2.2.2. PCI Express / PCI Capabilities 3.2.2.3. Base Address Registers
3.2.2.1. MCDMA Settings x
3.2.2.1.1. Multichannel DMA Mode 3.2.2.1.2. Bursting Master Mode 3.2.2.1.3. Bursting Slave Mode 3.2.2.1.4. BAM + BAS Mode 3.2.2.1.5. BAM + MCDMA Mode 3.2.2.1.6. BAM + BAS + MCDMA Mode
3.2.2.2. PCI Express / PCI Capabilities x
3.2.2.2.1. PCIe Device 3.2.2.2.2. PCIe MSI-X
3.2.2.2.2. PCIe MSI-X x
3.2.2.2.2.1. PCIe PF MSI-X 3.2.2.2.2.2. PCIe VF MSI-X
3.3. Generating HDL for Simulation and Synthesis x
3.3.1. IP Core Generation Output
4. Integrating the IP With Your Application x
4.1. Overview 4.2. Implementing Required Clocking 4.3. Implementing Required Resets 4.4. Connecting the GTS AXI Streaming IP-Facing Interfaces 4.5. Connect User Logic-Facing Interfaces
4.4. Connecting the GTS AXI Streaming IP-Facing Interfaces x
4.4.1. PCIe AXI-Stream TX Interface (ss_tx_st) 4.4.2. PCIe AXI-Stream RX Interface (ss_rx_st) 4.4.3. Control and Status Register Interface (ss_csr_lite) 4.4.4. Transmit Flow Control Credit Interface (ss_txcrdt) 4.4.5. Configuration Intercept Interface (CII) 4.4.6. Completion Timeout Interface (ss_cplto) 4.4.7. Function Level Reset (FLR) Interface 4.4.8. Control Shadow Interface (ss_ctrlshadow) 4.4.9. Error Interface
4.4.5. Configuration Intercept Interface (CII) x
4.4.5.1. Configuration Intercept Request Interface (ss_cii_req) 4.4.5.2. Configuration Intercept Response Interface (ss_cii_resp)
4.4.7. Function Level Reset (FLR) Interface x
4.4.7.1. FLR Received Interface (ss_flrrcvd) 4.4.7.2. FLR Completion Interface (ss_flrcmpl)
4.5. Connect User Logic-Facing Interfaces x
4.5.1. H2D AXI-Stream Manager (h2d_st_initatr) 4.5.2. D2H AXI-Stream Subordinate (d2h_st_respndr) 4.5.3. H2D/D2H AXI-MM Manager (dma_mm_initatr) 4.5.4. BAM AXI-MM Manager (bam_mm_initatr) 4.5.5. BAS AXI-MM Subordinate (bas_mm_respndr) 4.5.6. PIO AXI-Lite Manager (pio_lite_initiatr) 4.5.7. HIP Reconfiguration AXI-Lite Subordinate (user_csr_lite) 4.5.8. User Event MSI-X (user_msix) 4.5.9. User Event MSI (user_msi) 4.5.10. User Function Level Reset (user_flr) 4.5.11. User Configuration Intercept Interface 4.5.12. Configuration Slave (cs_lite_respndr)
4.5.11. User Configuration Intercept Interface x
4.5.11.1. User Configuration Intercept Request Interface (user_cii_req) 4.5.11.2. User Configuration Intercept Response Interface (user_cii_resp)
5. Simulating the IP x
5.1. Design Example Overview 5.2. Design Example Components 5.3. Simulating the Design Example
5.1. Design Example Overview x
5.1.1. AXI-S Device-side Packet Loopback Design Example
6. Validating the IP x
6.1. Hardware and Software Requirements 6.2. Running the Design Example Application on a Hardware Setup
6.2. Running the Design Example Application on a Hardware Setup x
6.2.1. Program the FPGA 6.2.2. Configuration Changes from BIOS 6.2.3. Host Operating System Check (if Working with Ubuntu 22.04) 6.2.4. Installing the Required Kernel Version (if Working with Ubuntu 24.04) 6.2.5. Set the Boot Parameters 6.2.6. Custom Driver 6.2.7. DPDK Poll Mode Driver
6.2.6. Custom Driver x
6.2.6.1. Prerequisites 6.2.6.2. PIO Test 6.2.6.3. Custom PIO Read Write Test 6.2.6.4. DMA Test
6.2.6.1. Prerequisites x
6.2.6.1.1. Steps for UIO Driver Installation 6.2.6.1.2. Enable Virtual Function 6.2.6.1.3. Build and Install User Space Library 6.2.6.1.4. Build the Reference Application
6.2.6.4. DMA Test x
6.2.6.4.1. AXI-S Device-side Packet Loopback (Device-side Packet Loopback)
6.2.7. DPDK Poll Mode Driver x
6.2.7.1. Prerequisites 6.2.7.2. PIO Test 6.2.7.3. DMA Test
6.2.7.1. Prerequisites x
6.2.7.1.1. Install PMD Driver 6.2.7.1.2. Bind the Device 6.2.7.1.3. Enable Virtual Function
6.2.7.3. DMA Test x
6.2.7.3.1. Device-side Packet Loopback
A. Appendix A: Functional Description x
A.1. High-level Functional Overview
A.1. High-level Functional Overview x
A.1.1. Multichannel DMA A.1.2. Bursting Master (BAM) A.1.3. Bursting Slave (BAS) A.1.4. MSI Interrupt A.1.5. Configuration Slave (CS) A.1.6. Root Port Address Translation Table Enablement A.1.7. Control and Status Register Interface A.1.8. Configuration Intercept Interface
A.1.1. Multichannel DMA x
A.1.1.1. H2D Data Mover A.1.1.2. D2H Data Mover A.1.1.3. Descriptors A.1.1.4. AXI4-Lite PIO Manager A.1.1.5. AXI-MM Write (H2D) and Read (D2H) Manager A.1.1.6. AXI-Stream Manager (H2D) and Subordinate (D2H) A.1.1.7. User MSI-X A.1.1.8. User Function Level Reset (FLR) A.1.1.9. Control and Status Registers
A.1.1.2. D2H Data Mover x
A.1.1.2.1. D2H Descriptor Fetch
A.1.1.3. Descriptors x
A.1.1.3.1. Support for Unaligned (or Byte-Aligned) Data Transfer A.1.1.3.2. Metadata Support A.1.1.3.3. MSI-X/Writeback
A.1.4. MSI Interrupt x
A.1.4.1. Endpoint MSI Support Through BAS A.1.4.2. MSI Interrupt Controller
A.1.5. Configuration Slave (CS) x
A.1.5.1. 14-bit AXI-MM Address Format A.1.5.2. Configuration Access Mechanism
B. Appendix B: Registers x
B.1. Queue Control B.2. MSI-X Memory Space B.3. Control Register (GCSR)
1. Overview
2. Quick Start Guide
3. Configuring and Generating the GTS AXI Multichannel DMA IP for PCI Express
4. Integrating the IP With Your Application
5. Simulating the IP
6. Validating the IP
A. Appendix A: Functional Description
B. Appendix B: Registers
C. Document Revision History for the GTS AXI Multichannel DMA IP for PCI Express*

1.6.2. Aligning IP Settings with the GTS AXI Streaming IP for PCI Express

As the GTS AXI MCDMA IP needs to interface with the GTS AXI Streaming IP for PCI Express, IP settings and interfaces of both IPs must be aligned to ensure that both IPs can operate smoothly.

The table below covers IP settings that must be aligned between the two IPs. For example, when the Hard IP Mode of the GTS AXI Streaming IP is configured as a Gen3 x4 Interface 128 bits, the PCIe Mode of the GTS AXI MCDMA IP must be aligned and configured to Gen3 1x4.

Table 11.  IP Settings of the GTS AXI MCDMA IP and GTS AXI Streaming IP
GTS AXI MCDMA IP Settings GTS AXI Streaming IP Settings
PCIe Mode Gen3 1x4 Hard IP Mode Gen3 x4 Interface 128 bits
Gen4 1x4 Gen4 x4 Interface 256 bits
Port Mode Native Endpoint Port Mode Native Endpoint
Root Port Root Port
Multichannel DMA Mode
BAR2 Address Width 4 KBytes - 12 bits

...

8 EBytes - 63 bits

 BAR2 Type 64-bit prefetchable memory
BAR2 Size 4 KBytes - 12 bits

...

8 EBytes - 63 bits
Enable Multiple Physical Functions On/Off Enable Multiple Physical Functions On/Off
Total Physical Functions (PFs) 1-4 Total Physical Functions (PFs) 1-4
Enable SR-IOV Support On/Off Enable SR-IOV Support On/Off
Total Virtual Functions of Physical Function 0 - 256 Total Virtual Functions of Physical Function 0 - 256
Enable MSI-X On Enable MSI-X On
BAR0 Type 64-bit prefetchable memory BAR0 Type 64-bit prefetchable memory
BAR0 Size 4 Mbytes – 22 bits BAR0 Size 4 Mbytes – 22 bits
BAR1/3/4/5 Type Disabled BAR1/3/4/5 Type Disabled
Expansion ROM Size

Disabled

…

16 Mbytes – 24 bits

 Expansion ROM Size

Disabled

…

16 Mbytes – 24 bits
Bursting Slave, BAM + BAS User Modes
Enable MSI Capability On/Off PF0 Enable MSI On/Off
Enable ATT (Root Port mode only) On/Off Enable Address Translation Services (ATS) On/Off
Endpoint Bursting Master, Bursting Slave, BAM + BAS User Modes
BAR0/2/4 Type

Disabled

64-bit prefetchable memory

64-bit non-prefetchable memory

 BAR0/2/4 Type

Disabled

64-bit prefetchable memory

64-bit non-prefetchable memory
BAR0/2/4 Size

4 KBytes - 12 bits

...

16 EBytes - 64 bits

 BAR0/2/4 Size

4 KBytes - 12 bits

...

16 EBytes - 64 bits
BAR1/3/5 Type Disabled BAR1/3/5 Type Disabled
Expansion ROM Size

Disabled

…

16 Mbytes – 24 bits

 Expansion ROM Size

Disabled

…

16 Mbytes – 24 bits
Endpoint BAM + MCDMA, BAM + BAS + MCDMA User Modes
BAR0 Type 64-bit prefetchable memory BAR0 Type 64-bit prefetchable memory
BAR0 Size 4 MBytes - 22 bits BAR0 Size 4 MBytes - 22 bits
BAR2/4 Type

Disabled

64-bit prefetchable memory

64-bit non-prefetchable memory

 BAR2/4 Type

Disabled

64-bit prefetchable memory

64-bit non-prefetchable memory
BAR2/4 Size

4 KBytes - 12 bits

...

16 EBytes - 64 bits

 BAR2/4 Size

4 KBytes - 12 bits

...

16 EBytes - 64 bits
BAR3/5 Type Disabled BAR3/5 Type Disabled
Expansion ROM Size

Disabled

…

16 Mbytes – 24 bits

 Expansion ROM Size

Disabled

…

16 Mbytes – 24 bits

For more details on each of the parameters, refer to the IP Settings section.

Some of the GTS AXI MCDMA IP interfaces are enabled by default. This requires the corresponding optional interface of the GTS AXI Streaming IP to be enabled so that signals from both IPs can be connected.

Table 12.  Optional Interface to be Enabled on the GTS AXI Streaming IP
GTS AXI MCDMA IP Interfaces Enabled by Default GTS AXI Streaming IP Settings
Control Shadow Interface (Endpoint mode) Enable PCIe0 Control Shadow Interface On
Completion Timeout Interface Enable PCIe0 Completion Timeout Interface On
Error Interface Enable PCIe0 Error Interface On
Configuration Intercept Interface Enable PCIe0 Configuration Intercept interface On

As the axi_st_clk and axi_mm_clk of the GTS AXI MCDMA IP are expected to be driven by coreclkout_hip_toapp of the GTS AXI Streaming IP, the PLD Clock Frequency IP parameter must be configured to be aligned with the supported frequencies of the GTS AXI MCDMA IP.

Table 13.  PLD Clock Frequency Settings
GTS AXI MCDMA IP Supported Clock Frequencies GTS AXI Streaming IP Settings

Gen4 1x4: 300/250/200 MHz

Gen3 1x4: 250/200 MHz 		PLD Clock Frequency 300/250/200 MHz

The GTS AXI Streaming IP supports both inband and sideband TLP header formats. However, the GTS AXI MCDMA IP only supports the sideband header format. Therefore, the PCIe0 AXI-S Sideband Header parameter must be enabled to ensure the packet streaming format of both IPs are aligned.

Table 14.  AXI-S Sideband Header Settings
GTS AXI MCDMA IP Header Format GTS AXI Streaming IP Settings
Only the sideband TLP header format is supported. PCIe0 AXI-S Sideband Header On
Level Two Title