AXI Streaming Intel® FPGA IP for PCI Express* User Guide

Download PDF
ID 790711
Date 4/12/2024
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Introduction 2. Features 3. Getting Started with the AXI Streaming Intel® FPGA IP for PCI Express* 4. IP Architecture and Functional Description 5. AXI Streaming Intel® FPGA IP for PCI Express* Parameters 6. Interfaces and Signals 7. Register Descriptions 8. Document Revision History for the AXI Streaming Intel® FPGA IP for PCI Express* User Guide A. Specifications B. Simulating the Design Example
1. Introduction x
1.1. Goal of the AXI Streaming Intel® FPGA IP for PCI Express* User Guide 1.2. Intended Audience for the AXI Streaming Intel® FPGA IP for PCI Express* User Guide 1.3. What is PCI Express* ? 1.4. What are the Intel® FPGA IPs for PCI Express* ? 1.5. What is the AXI Streaming Intel® FPGA IP for PCI Express* ? 1.6. Example Use Models 1.7. Design Flow Requirements
1.7. Design Flow Requirements x
1.7.1. Design Software 1.7.2. Hardware
2. Features x
2.1. Supported Features 2.2. Device Family Support 2.3. Performance
3. Getting Started with the AXI Streaming Intel® FPGA IP for PCI Express* x
3.1. Download and Install Quartus Software 3.2. Obtain and Install Intel FPGA IPs and Licenses 3.3. Configure and Generate the AXI Streaming Intel® FPGA IP for PCI Express* 3.4. Instantiate and Connect the AXI Streaming Intel® FPGA IP for PCI Express* Interfaces 3.5. Simulate the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant 3.6. Compile the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant 3.7. Software Drivers for AXI Streaming Intel® FPGA IP for PCI Express* IP Variant 3.8. Build the Application for the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant 3.9. Verification with the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant 3.10. Debugging with the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant
3.4. Instantiate and Connect the AXI Streaming Intel® FPGA IP for PCI Express* Interfaces x
3.4.1. Clocking and Resets 3.4.2. Application Packet Datapath 3.4.3. Enabling Additional Features 3.4.4. Enabling Optional Interfaces
3.5. Simulate the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant x
3.5.1. Simulating the Design Example
3.5.1. Simulating the Design Example x
3.5.1.1. Steps to Run Simulations
3.5.1.1. Steps to Run Simulations x
3.5.1.1.1. VCS* Simulator
3.7. Software Drivers for AXI Streaming Intel® FPGA IP for PCI Express* IP Variant x
3.7.1. Installing the Linux Kernel Driver
3.8. Build the Application for the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant x
3.8.1. Running the Design Example
3.8.1. Running the Design Example x
3.8.1.1. Running the PIO Design Example
3.10. Debugging with the AXI Streaming Intel® FPGA IP for PCI Express* IP Variant x
3.10.1. Debugging Link Training Issues 3.10.2. Debugging Data Transfer and Performance Issues
3.10.1. Debugging Link Training Issues x
3.10.1.1. Operating System Tools and Utilities 3.10.1.2. Signal Tap Logic Analyzer 3.10.1.3. Additional Debug Tools
3.10.2. Debugging Data Transfer and Performance Issues x
3.10.2.1. Advanced Error Reporting (AER) 3.10.2.2. Second-Level Debug Tools
4. IP Architecture and Functional Description x
4.1. Clocks and Resets 4.2. PCIe Hard IP (HIP) 4.3. HIP Interface (IF) Adaptor 4.4. Application Error Reporting 4.5. Debug Toolkit and Hard IP (HIP) Reconfiguration Interface 4.6. Configuration Space Extension 4.7. Control Shadow 4.8. Configuration Intercept Interface 4.9. Power Management 4.10. Legacy Interrupt 4.11. Credit Handling 4.12. Completion Timeout 4.13. Transaction Ordering 4.14. Page Request Service (PRS) Events 4.15. TX Non-Posted Metering Requirement on Application 4.16. MSI Pending 4.17. D-State Status 4.18. Configuration Retry Status Enable 4.19. AXI-Streaming Interface 4.20. Precision Time Measurement (PTM) [F/R-Tiles Only]
5. AXI Streaming Intel® FPGA IP for PCI Express* Parameters x
5.1. Parameter Editor Parameters
6. Interfaces and Signals x
6.1. Overview 6.2. Clocks and Resets 6.3. Application Packet Interface 6.4. Configuration Extension Bus Interface 6.5. Configuration Intercept Interface 6.6. Function Level Reset Interface 6.7. Control Shadow Interface (st_ctrlshadow) 6.8. Completion Timeout Interface (st_cplto) 6.9. Miscellaneous Signals 6.10. Control and Status Register Responder Interface (lite_csr) 6.11. VF Error Flag Interface (vf_err/sent_vfnonfatalmsg) 6.12. VIRTIO PCI* Configuration Access Interface 6.13. Serial Data Signals
6.2. Clocks and Resets x
6.2.1. Interface Clock Signals 6.2.2. Interface Reset Signals
6.3. Application Packet Interface x
6.3.1. Header Format 6.3.2. Prefix Format 6.3.3. Function Number Format 6.3.4. BAR Number Format 6.3.5. Data and Header Packing Schemes 6.3.6. Application Packet Transmit Interface (st_tx) 6.3.7. Application Packet Receive Interface (st_rx) 6.3.8. Flow Control Credit Handling
6.3.5. Data and Header Packing Schemes x
6.3.5.1. HIP Native Mode Packing 6.3.5.2. Compact Packing 6.3.5.3. Simple Packing
6.3.5.1. HIP Native Mode Packing x
6.3.5.1.1. Application Logic Guidelines for the AXI Streaming TX Interface in HIP Native Mode (R-Tile) 6.3.5.1.2. Application Logic Guidelines for the AXI Streaming RX Interface in HIP Native Mode
6.3.5.2. Compact Packing x
6.3.5.2.1. Application Logic Guidelines for the AXI Streaming TX Interface in Compact Mode (P/R-Tiles) 6.3.5.2.2. Application Logic Guidelines for the AXI Streaming RX Interface in Compact Mode (P/R-Tiles)
6.3.5.3. Simple Packing x
6.3.5.3.1. Application Logic Guidelines for the AXI Streaming TX Interface in Simple Mode (P/R-Tiles) 6.3.5.3.2. Application Logic Guidelines for the AXI Streaming RX Interface in Simple Mode (P/R-Tiles)
6.3.8. Flow Control Credit Handling x
6.3.8.1. Application Transmit Flow Control Credit Interface (st_txcrdt) 6.3.8.2. Application Receive Flow Control Credit Interface (st_rxcrdt) (R-Tile Only)
6.4. Configuration Extension Bus Interface x
6.4.1. Configuration Extension Bus Request Interface (st_cebreq) 6.4.2. Configuration Extension Bus Response Interface (st_cebresp)
6.5. Configuration Intercept Interface x
6.5.1. Configuration Intercept Request Interface (st_ciireq) 6.5.2. Configuration Intercept Response Interface (st_ciiresp)
6.6. Function Level Reset Interface x
6.6.1. Function Level Reset Received Interface (st_flrrcvd) 6.6.2. Function Level Reset Completion Interface (st_flrcmpl)
6.12. VIRTIO PCI* Configuration Access Interface x
6.12.1. VIRTIO PCI* Config Access Request Interface (st_virtio_pcicfgreq) 6.12.2. VIRTIO PCI* Config Access Completion Interface (st_virtio_pcicfgcmpl)
7. Register Descriptions x
7.1. Register Address Map 7.2. PCIe Configuration Space 7.3. AXI Streaming Intel® FPGA IP for PCI Express* Soft Register Address Map
7.2. PCIe Configuration Space x
7.2.1. PCIe Configuration Space Registers
7.3. AXI Streaming Intel® FPGA IP for PCI Express* Soft Register Address Map x
7.3.1. AXI Streaming Intel® FPGA IP for PCI Express* Control Registers 7.3.2. IP Debug Registers 7.3.3. IP Performance Monitor Registers
7.3.1. AXI Streaming Intel® FPGA IP for PCI Express* Control Registers x
7.3.1.1. AXI Streaming Intel® FPGA IP for PCI Express* Version 7.3.1.2. AXI Streaming Intel® FPGA IP for PCI Express* Features 7.3.1.3. AXI Streaming Intel® FPGA IP for PCI Express* Interface Attributes 7.3.1.4. ERROR GEN CTRL 7.3.1.5. ERROR GEN ATTR 7.3.1.6. ERROR TLP Header DW0-3 7.3.1.7. ERROR TLP Prefix 7.3.1.8. HOT PLUG GEN CTRL 7.3.1.9. POWER MANAGEMENT CTRL 7.3.1.10. LEGACY INTERRUPT CTRL 7.3.1.11. CFG REG IA CTRL 7.3.1.12. CFG REG IA FN NUM 7.3.1.13. CFG REG IA WRDATA 7.3.1.14. CFG REG IA RDDATA 7.3.1.15. PRS CTRL 7.3.1.16. MSI PENDING CTRL 7.3.1.17. MSI PENDING 7.3.1.18. D-STATE STS 7.3.1.19. CFG RETRY CTRL
7.3.2. IP Debug Registers x
7.3.2.1. HIP Status 7.3.2.2. HIP BP CYCLES 7.3.2.3. HIA BP CYCLES 7.3.2.4. APP BP CYCLES 7.3.2.5. HIA RX BP CYCLES
7.3.3. IP Performance Monitor Registers x
7.3.3.1. PERFMON CTRL 7.3.3.2. TX MRD TLP 7.3.3.3. TX MWR TLP 7.3.3.4. TX MSG TLP 7.3.3.5. TX CFGWR TLP 7.3.3.6. TX CFGRD TLP 7.3.3.7. RX MRD TLP 7.3.3.8. RX MWR TLP 7.3.3.9. RX MSG TLP 7.3.3.10. RX CFGWR TLP 7.3.3.11. RX CFGRD TLP 7.3.3.12. TX MEM DATA 7.3.3.13. TX CPL DATA 7.3.3.14. RX MEM DATA 7.3.3.15. RX CPL DATA
A. Specifications x
A.1. P-Tile Specifications A.2. F-Tile Specifications A.3. R-Tile Specifications
A.1. P-Tile Specifications x
A.1.1. P-Tile Completion Buffer Size A.1.2. Power Management A.1.3. MSI and MSI-X
A.2. F-Tile Specifications x
A.2.1. F-Tile Completion Buffer Size A.2.2. Power Management
A.3. R-Tile Specifications x
A.3.1. R-Tile Completion Buffer Size A.3.2. Power Management A.3.3. MSI and MSI-X
B. Simulating the Design Example x
B.1. Testbench
B.1. Testbench x
B.1.1. Testbench Modules B.1.2. Test Driver Module B.1.3. PIO Design Example Testbench B.1.4. Root Port BFM
1. Introduction
2. Features
3. Getting Started with the AXI Streaming Intel® FPGA IP for PCI Express*
4. IP Architecture and Functional Description
5. AXI Streaming Intel® FPGA IP for PCI Express* Parameters
6. Interfaces and Signals
7. Register Descriptions
8. Document Revision History for the AXI Streaming Intel® FPGA IP for PCI Express* User Guide
A. Specifications
B. Simulating the Design Example

2.1. Supported Features

The AXI Streaming Intel® FPGA IP for PCI Express* provides you control over the PCIe HIP, by providing you with finer control over the PCIe Transaction Layer Packet (TLP), credit handling and various modes directly to the application layer. The IP sends TLPs received from the Link to the user logic side with some additional information like BAR number, function number, etc. The IP supports the following features:

PCIe* Features:

  • Complete protocol stack including the Transaction, Data Link, and Physical Layers implemented as Hard IP.
  • Configurations supported:
    Gen3/Gen4/Gen5 1x16 Gen3/Gen4 1x8 Gen3/Gen4/Gen5 2x8
    Endpoint (EP) Yes Yes (P-Tile and F-Tile only) Yes
    Note:
    1. Currently supported with Agilex™ 7 devices with P-Tile (e.g., AGFB014R24B2E2V), Agilex™ 7 devices with F-Tile (e.g., AGIB027R29A1E2VR2), and Agilex™ 7 devices with R-Tile (e.g., AGIB027R29A1E2VR3).
    2. Gen1/Gen2 configurations are supported via link down-training.
  • Static port bifurcation: two x8s endpoints
  • Clocking architecture:
    • Separate reference clock with independent spread spectrum clocking (SRIS)
    • Separate reference clock with no spread spectrum clocking (SRNS)
    • Common reference clock architecture
  • Single Virtual Channel (VC)
  • Capability Registers:
    • Message Signaled Interrupt (MSI)
    • Message Signal Interrupt Extended (MSI-X)
    • Advanced Error Reporting (AER) (PF only)
    • Power Management (PM – D0 and D3 PCIe* power states) (PF only)
    • Alternative Routing ID (ARI)
    • Address Translation Services (ATS)
    • Page Request Service (PRS)
    • Transaction Processing Hints (TPH) ("No Steering Tag (ST)" mode only)
    • Access Control Services (ACS) (For ACS, only ports 0 and 1 are supported)
    • Latency Tolerance Reporting (LTR)
    • Process Address Space ID (PASID)
    • Vendor Specific Capability
  • PCI Express* Advanced Error Reporting (AER) (PF only)
  • Supports up to 512-byte maximum payload size (MPS)
  • Supports up to 4096-byte (4 KB) maximum read request size (MRRS)
  • 32/64-bit BAR support (Prefetchable/Non-Prefetchable)
  • Expansion ROM BAR support
  • Number of tags – 32, 64, 128, 256, 512, 768 (Gen5 x16 only)
  • Application error handling (UR/CA/Completion Timeout/Poison)
  • MSI support - Supports multiple MSI, per-vector masking

Multifunction and Virtualization Features (Optional):

  • SR-IOV support (Maximum 8PFs, 2048 VFs across all Endpoints in a design)
  • Supports single TLP prefix per TLP (1DW)
  • Supports VIRTIO PCI* Configuration Registers
  • Function Level Reset (FLR) – communicated to application through separate interface

User Interface Features:

  • AXI4 (Streaming, Lite) user interface for data and control signals.
  • AXI Streaming Interface: There are separate interfaces for header and data in both the Transmit and Receive directions.
    • The AXI Streaming Source Interface comprises the master signals, and provides the start of the transaction.
    • The AXI Streaming Source Interface is a single stream interface.
    • The AXI Streaming Sink Interface comprises the slave signals, and provides the response to the transaction from the source.
    • The AXI Streaming Sink Interface provides support for basic bare metal mode (e.g., Single physical function, AER, etc.) and virtualization mode (e.g., Multiple physical functions, function level reset, etc).
    • Selects the application’s AXI streaming data bus width. The interface width is defined in terms of number of Bytes. This interface supports a scalable data bus width (32, 64, 128-byte widths) and frequency.
    • Supports the following data packing schemes:
      • HIP Native: Interface width, segment size of the AXI-ST interface are the same as those of the Native Hard IP.
      • Non-HIP Native/Compact: Interface width, segment size of the AXI-ST interface are different than those of the Native Hard IP.
    • Supports operating frequency selection options of 250, 350, 400, 470, 500 MHz. Refer to Device Family Support for the valid combinations of data bus width and frequencies.
  • AXI Lite Responder Interface
    • This is the Control and Status Register Interface to access registers implemented in the IP modules, including PCI* / PCIe* Configuration Registers of all Functions.
    • 32-bit or 64-bit at 100-250 MHz.
  • Configuration Extension Bus (CEB) Interface provided to extend the configuration capabilities beyond the PCI* / PCIe* capabilities and implement Customer Specific Capabilities.
  • Configuration Intercept Interface (CII) allows the application logic to detect the occurrence of a Configuration (CFG) request on the link and to modify its behavior.
    Note: This interface is mutually exclusive with the Configuration Extension Bus (CEB) interface.
    Note: This interface is provided so that the IP is backward compatible with any legacy application logic that relies on CII for their functionality. Newly defined application logic should avoid using the CII interface and move to the CEB interface.
  • Error reporting by the application logic: The IP implements Error Reporting registers. These registers allow user to indicate various errors. The IP then forwards this error information to the HIP block (UR/CA/Completion Timeout/Poison).
  • Supports Link Partner Credits (exposed via credit interface) - The IP exposes link partner credit to user in the Transmit and Receive directions depending on the tile used. The credits are advertised as a limit value specified in the PCIe* spec. You must check the availability of credits for transmitting and receiving the TLP.
  • Transaction ordering, deadlock avoidance
    • You must implement transaction ordering in the user application logic.
  • Control Shadow Interface provided to shadow the control information from control / command registers (Optional).
  • Completion timeout interface (Optional) - The PCIe* IP can optionally track outgoing non-posted packets to report completion timeout information to the application.
  • Supports Autonomous Hard IP mode - This mode allows the PCIe* Hard IP to communicate with the Host before the FPGA configuration and entry into User mode are complete.
    Note: Unless Readiness Notifications mechanisms are used, the Root Complex or system software must allow at least 1.0 s after a Conventional Reset of a device before it may determine that a device that fails to return a Successful Completion status for a valid Configuration Request is a broken device. This period is independent of how quickly Link training completes.
  • FPGA core configuration via Protocol (CvP Init and CvP Update) (Optional)
    Note: For Gen3, Gen4 and Gen5 x16 variants, Port 0 (corresponding to lanes 0 - 15) supports the CvP features. For Gen3, Gen4 and Gen5 x8 variants, only Port 0 (corresponding to lanes 0 - 7) supports the CvP features. Port 1 (corresponding to lanes 8 - 15) does not support CvP.
  • Debug Toolkit for register accesses and debug (Optional).
    Note: This feature is not supported in the current Quartus® Prime release.
  • Design example generation: Currently available when using P-Tile (Gen4 x16) and R-Tile (Gen5 x16).
  • Software Driver support.
    • Available along with the Design Examples and Intel Open FPGA Stack reference design.
      Note: This feature is not supported in the current Quartus® Prime release.
Related Information
Level Two Title