F-Tile Avalon® Streaming Intel® FPGA IP for PCI Express* User Guide

Download PDF
ID 683140
Date 4/27/2023
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Acronyms 2. Introduction 3. IP Architecture and Functional Description 4. Advanced Features 5. Interfaces 6. Parameters 7. Testbench 8. Troubleshooting/Debugging 9. F-Tile Avalon Streaming Intel FPGA IP for PCI Express User Guide Archives 10. Revision History for the F-Tile Avalon Streaming Intel FPGA IP for PCI Express User Guide A. Configuration Space Registers B. Implementation of Address Translation Services (ATS) in Endpoint Mode C. Packets Forwarded to the User Application in TLP Bypass Mode D. Root Port Enumeration E. Bifurcated Endpoint Support for Independent Resets
2. Introduction x
2.1. Overview 2.2. Features 2.3. Release Information 2.4. Device Family Support 2.5. Performance and Resource Utilization 2.6. IP Core and Design Example Support Levels
3. IP Architecture and Functional Description x
3.1. Architecture 3.2. Functional Description 3.3. Avalon-ST TX/RX 3.4. Interrupts 3.5. Completion Timeout 3.6. Hot Plug 3.7. Power Management 3.8. Configuration Output Interface (COI) 3.9. Configuration Intercept Interface (EP Only) 3.10. Hard IP Reconfiguration Interface 3.11. PHY Reconfiguration Interface 3.12. Page Request Service (PRS) (EP Only)
3.1. Architecture x
3.1.1. Clocks 3.1.2. Refclk 3.1.3. Reset
3.2. Functional Description x
3.2.1. PMA/PCS 3.2.2. Data Link Layer 3.2.3. Transaction Layer
3.3. Avalon-ST TX/RX x
3.3.1. TLP Header and Data Alignment 3.3.2. Avalon-ST RX 3.3.3. Avalon-ST TX 3.3.4. Tag Allocation 3.3.5. Completion Buffer Size
3.3.2. Avalon-ST RX x
3.3.2.1. RX Flow Control
3.3.3. Avalon-ST TX x
3.3.3.1. TX Flow Control
3.4. Interrupts x
3.4.1. Legacy Interrupts 3.4.2. MSI 3.4.3. MSI-X
3.10. Hard IP Reconfiguration Interface x
3.10.1. Configuration Registers Access
3.10.1. Configuration Registers Access x
3.10.1.1. Direct User Avalon-MM Interface (Byte Access) 3.10.1.2. Debug Register Interface Access (Dword Access)
4. Advanced Features x
4.1. Virtualization Support 4.2. TLP Bypass Mode 4.3. Precision Time Measurement (PTM) 4.4. Scalable IOV
4.1. Virtualization Support x
4.1.1. Single Root I/O Virtualization (SR-IOV) 4.1.2. VirtIO
4.1.1. Single Root I/O Virtualization (SR-IOV) x
4.1.1.1. SR-IOV Implementation
4.1.1.1. SR-IOV Implementation x
4.1.1.1.1. Functional Level Reset (FLR)
4.1.2. VirtIO x
4.1.2.1. VirtIO Implementation 4.1.2.2. VirtIO Registers
4.2. TLP Bypass Mode x
4.2.1. Register Settings for TLP Bypass mode 4.2.2. Avalon-MM usage for TLP Bypass Mode 4.2.3. Transmit Interface 4.2.4. Receive Interface 4.2.5. Configuration TLP 4.2.6. Malformed TLP 4.2.7. ECRC
4.2.1. Register Settings for TLP Bypass mode x
4.2.1.1. TLPBYPASS_ERR_EN (Address 0x14194) 4.2.1.2. TLPBYPASS_ERR_STATUS (Address 0x14190)
5. Interfaces x
5.1. Overview 5.2. Clocks and Resets 5.3. Serial Data Interface 5.4. Avalon-ST Interface 5.5. Interrupt Interface 5.6. Hard IP Status Interface 5.7. Error Interface 5.8. 10-bit Tag Support Interface 5.9. Completion Timeout Interface 5.10. Power Management Interface 5.11. Hot Plug Interface (RP Only) 5.12. Configuration Output Interface 5.13. Configuration Intercept Interface (EP Only) 5.14. Hard IP Reconfiguration Interface 5.15. PHY Reconfiguration Interface 5.16. Page Request Service (PRS) Interface (EP Only) 5.17. FLR Interface Signals 5.18. PTM Interface Signals 5.19. VF Error Flag Interface Signals 5.20. VirtIO PCI Configuration Access Interface Signals
5.4. Avalon-ST Interface x
5.4.1. Avalon-ST RX Interface Signals 5.4.2. Avalon-ST TX Interface Signals
5.5. Interrupt Interface x
5.5.1. Legacy Interrupt Interface Signals 5.5.2. MSI Pending Bits Interface Signals
6. Parameters x
6.1. Top-Level Settings 6.2. Core Parameters
6.2. Core Parameters x
6.2.1. Avalon Parameters 6.2.2. Base Address Registers 6.2.3. PCI Express and PCI Capabilities Parameters 6.2.4. Device Identification Registers 6.2.5. Configuration, Debug and Extension Options
6.2.3. PCI Express and PCI Capabilities Parameters x
6.2.3.1. Device Capabilities 6.2.3.2. Link Capabilities 6.2.3.3. Legacy Interrupt Pin Register 6.2.3.4. MSI Capabilities 6.2.3.5. MSI-X Capabilities 6.2.3.6. Slot Capabilities 6.2.3.7. Latency Tolerance Reporting (LTR) 6.2.3.8. Process Address Space ID (PASID) 6.2.3.9. Device Serial Number Capability 6.2.3.10. Page Request Service (PRS) 6.2.3.11. Access Control Service (ACS) Capabilities 6.2.3.12. Power Management 6.2.3.13. Vendor Specific Extended Capability (VSEC) Registers 6.2.3.14. Precision Time Measurement (PTM) 6.2.3.15. Address Translation Services (ATS) 6.2.3.16. TLP Processing Hints (TPH) 6.2.3.17. VirtIO Parameters
7. Testbench x
7.1. FASTSIM Mode Support 7.2. Generating Tile Files 7.3. Endpoint Testbench 7.4. Test Driver Module 7.5. Root Port BFM 7.6. BFM Procedures and Functions
7.5. Root Port BFM x
7.5.1. BFM Memory Map 7.5.2. Configuration Space Bus and Device Numbering 7.5.3. Configuration of Root Port and Endpoint 7.5.4. Issuing Read and Write Transactions to the Application Layer
7.6. BFM Procedures and Functions x
7.6.1. ebfm_barwr Procedure 7.6.2. ebfm_barwr_imm Procedure 7.6.3. ebfm_barrd_wait Procedure 7.6.4. ebfm_barrd_nowt Procedure 7.6.5. ebfm_cfgwr_imm_wait Procedure 7.6.6. ebfm_cfgwr_imm_nowt Procedure 7.6.7. ebfm_cfgrd_wait Procedure 7.6.8. ebfm_cfgrd_nowt Procedure 7.6.9. BFM Configuration Procedures 7.6.10. BFM Shared Memory Access Procedures 7.6.11. BFM Log and Message Procedures 7.6.12. Verilog HDL Formatting Functions
7.6.9. BFM Configuration Procedures x
7.6.9.1. ebfm_cfg_rp_ep Procedure 7.6.9.2. ebfm_cfg_decode_bar Procedure
7.6.10. BFM Shared Memory Access Procedures x
7.6.10.1. Shared Memory Constants 7.6.10.2. shmem_write Task 7.6.10.3. shmem_read Function 7.6.10.4. shmem_display Verilog HDL Function 7.6.10.5. shmem_fill Procedure 7.6.10.6. shmem_chk_ok Function
7.6.11. BFM Log and Message Procedures x
7.6.11.1. ebfm_display Verilog HDL Function 7.6.11.2. ebfm_log_stop_sim Verilog HDL Function 7.6.11.3. ebfm_log_set_suppressed_msg_mask Task 7.6.11.4. ebfm_log_set_stop_on_msg_mask Verilog HDL Task 7.6.11.5. ebfm_log_open Verilog HDL Function 7.6.11.6. ebfm_log_close Verilog HDL Function
7.6.12. Verilog HDL Formatting Functions x
7.6.12.1. himage1 7.6.12.2. himage2 7.6.12.3. himage4 7.6.12.4. himage8 7.6.12.5. himage16 7.6.12.6. dimage1 7.6.12.7. dimage2 7.6.12.8. dimage3 7.6.12.9. dimage4 7.6.12.10. dimage5 7.6.12.11. dimage6 7.6.12.12. dimage7
8. Troubleshooting/Debugging x
8.1. Hardware 8.2. Debug Toolkit
8.1. Hardware x
8.1.1. Debugging Link Training Issues 8.1.2. Debugging Data Transfer and Performance Issues
8.1.1. Debugging Link Training Issues x
8.1.1.1. Generic Tools and Utilities 8.1.1.2. Signal Tap II Logic Analyzer 8.1.1.3. Additional Debug Tools
8.1.1.3. Additional Debug Tools x
8.1.1.3.1. Using the Hard IP Reconfiguration Interface 8.1.1.3.2. Enable and Read LCRC and ECRC Error Count
8.1.2. Debugging Data Transfer and Performance Issues x
8.1.2.1. Advanced Error Reporting (AER) 8.1.2.2. Second-Level Debug Tools
8.2. Debug Toolkit x
8.2.1. Overview 8.2.2. Enabling the F-Tile Debug Toolkit 8.2.3. Launching the F-Tile Debug Toolkit 8.2.4. Using the F-Tile Debug Toolkit 8.2.5. Using the F-Tile Link Inspector
8.2.4. Using the F-Tile Debug Toolkit x
8.2.4.1. Main View 8.2.4.2. Toolkit Parameters 8.2.4.3. Channel Parameters 8.2.4.4. Eye Viewer
8.2.4.2. Toolkit Parameters x
8.2.4.2.1. F-Tile Information 8.2.4.2.2. Event Counter 8.2.4.2.3. P0/P1 Configuration Space
8.2.4.3. Channel Parameters x
8.2.4.3.1. General PHY 8.2.4.3.2. TX Path 8.2.4.3.3. RX Path
A. Configuration Space Registers x
A.1. Configuration Space Registers A.2. Configuration Space Registers for Virtualization A.3. Intel-Defined VSEC Capability Registers
A.1. Configuration Space Registers x
A.1.1. Register Access Definitions A.1.2. PCIe Configuration Header Registers A.1.3. PCI Express Capability Structures A.1.4. Physical Layer 16.0 GT/s Extended Capability Structure A.1.5. MSI-X Registers
A.2. Configuration Space Registers for Virtualization x
A.2.1. SR-IOV Virtualization Extended Capabilities Registers Address Map A.2.2. PCIe Configuration Registers for Each Virtual Function
A.2.2. PCIe Configuration Registers for Each Virtual Function x
A.2.2.1. Alternative Routing ID (ARI) Capability Structure A.2.2.2. TLP Processing Hint (TPH) Capability Structure A.2.2.3. Address Translation Services (ATS) Capability Structure A.2.2.4. Access Control Services (ACS) Capability Structure
A.2.2.1. Alternative Routing ID (ARI) Capability Structure x
A.2.2.1.1. ARI Enhanced Capability Header Register (Offset 0x0) A.2.2.1.2. ARI Capability and Control Register (Offset 0x4)
A.2.2.2. TLP Processing Hint (TPH) Capability Structure x
A.2.2.2.1. TPH Requester Enhanced Capability Header (Offset 0x0) A.2.2.2.2. TPH Requester Capability Register (Offset 0x4) A.2.2.2.3. TPH Requester Control Register (Offset 0x8)
A.2.2.3. Address Translation Services (ATS) Capability Structure x
A.2.2.3.1. ATS Enhanced Capability Header (Offset 0x0) A.2.2.3.2. ATS Capability Register and ATS Control Register (Offset 0x4)
A.2.2.4. Access Control Services (ACS) Capability Structure x
A.2.2.4.1. ACS Extended Capability Header (Offset 0x0) A.2.2.4.2. ACS Capability Register (Offset 0x4) A.2.2.4.3. ACS Control Register (Offset 0x6) A.2.2.4.4. Egress Control Vector (Offset 0x8)
A.3. Intel-Defined VSEC Capability Registers x
A.3.1. Intel-Defined VSEC Capability Header (Offset 00h) A.3.2. Intel-Defined Vendor Specific Header (Offset 04h) A.3.3. Intel Marker (Offset 08h) A.3.4. JTAG Silicon ID (Offset 0x0C - 0x18) A.3.5. User Configurable Device and Board ID (Offset 0x1C - 0x1D) A.3.6. General Purpose Control and Status Register (Offset 0x30) A.3.7. Uncorrectable Internal Error Status Register (Offset 0x34) A.3.8. Uncorrectable Internal Error Mask Register (Offset 0x38) A.3.9. Correctable Internal Error Status Register (Offset 0x3C) A.3.10. Correctable Internal Error Mask Register (Offset 0x40)
B. Implementation of Address Translation Services (ATS) in Endpoint Mode x
B.1. Sending Translated/Untranslated Requests B.2. Sending a Page Request Message from the Endpoint (EP) to the Root Complex (RC) B.3. Invalidating Requests/Completions
C. Packets Forwarded to the User Application in TLP Bypass Mode x
C.1. EP TLP Bypass Mode (Upstream) C.2. RC TLP Bypass Mode (Downstream)
E. Bifurcated Endpoint Support for Independent Resets x
E.1. PCI Express Resets E.2. Supported Configurations
E.2. Supported Configurations x
E.2.1. Configuration Type A E.2.2. Configuration Type B E.2.3. Configuration Type C
1. Acronyms
2. Introduction
3. IP Architecture and Functional Description
4. Advanced Features
5. Interfaces
6. Parameters
7. Testbench
8. Troubleshooting/Debugging
9. F-Tile Avalon Streaming Intel FPGA IP for PCI Express User Guide Archives
10. Revision History for the F-Tile Avalon Streaming Intel FPGA IP for PCI Express User Guide
A. Configuration Space Registers
B. Implementation of Address Translation Services (ATS) in Endpoint Mode
C. Packets Forwarded to the User Application in TLP Bypass Mode
D. Root Port Enumeration
E. Bifurcated Endpoint Support for Independent Resets

A.1. Configuration Space Registers

In addition to accessing the Endpoint's configuration space registers by sending Configuration Read/Write TLPs via the Avalon® -ST interface, the application logic can also gain read access to these registers via the Configuration Output Interface (tl_cfg*). Furthermore, the Hard IP Reconfiguration Interface (a User Avalon® -MM interface) also provides read/write access to these registers.

For signal timings on the User Avalon® -MM interface, refer to the Avalon® Interface Specifications document.

The table PCIe Configuration Space Registers describes the registers for each PF. To calculate the address for a particular register in a particular PF, add the offset for that PF from the table Configuration Space Offsets to the byte address for that register as given in the table PCIe Configuration Space Registers.

Table 122.  Configuration Space Offsets
Registers User Avalon® -MM Offsets
Physical function 0 0x0000
Physical function 1 0x1000
Physical function 2 0x2000
Physical function 3 0x3000
Physical function 4 0x4000
Physical function 5 0x5000
Physical function 6 0x6000
Physical function 7 0x7000
Port Configuration and Status Register 0x14000
Debug (DBI) Register 0x14200
Completion Timeout Register 0x90000
Table 123.  PCIe Configuration Space Registers for x16/x8/x4 Controllers
Byte Address Hard IP Configuration Space Register Corresponding Section in PCIe Specification

x16 = 0x000 : 0x03F

x8 = 0x000 : 0x03F

x4) = 0x000 : 0x03F

PCI Header Type 0/1 Configuration Registers Type 0/1 Configuration Space Header

x16 = 0x040 : 0x047

x8 = 0x040 : 0x047

x4 = 0x040 : 0x047

Power Management PCI Power Management Capability Structure

x16 = 0x050 : 0x067

x8 = 0x050 : 0x067

x4 = 0x050 : 0x067

MSI Capability

MSI Capability Structure, see also PCI Local Bus Specification

x16 = 0x070 : 0xAB

x8 = 0x070 : 0xAB

x4 = 0x070 : 0xAB

PCI Express Capability PCI Express Capability Structure

x16 = 0x0B0 : 0xBB

x8 = 0x0B0 : 0xBB

x4 = 0x0B0 : 0xBB

 MSI-X Capability MSI-X Capability Structure, see also PCI Local Bus Specification

x16 = 0x0BC : 0x0FC

x8 = 0x0BC : 0x0FC

x4 = 0x0BC : 0x0FC

Reserved N/A

x16 = 0x100 : 0x147

x8 = 0x100 : 0x147

x4 = 0x100 : 0x147

Advanced Error Reporting (AER) Advanced Error Reporting Capability Structure

x16, x8, x4 = 0x148 - 0x163

 Virtual Channel Capability Virtual Channel Capability Structure

x16, x8, x4 = 0x164 - 0x173

Device Serial Number Capability

Device Serial Number Capability Structure

x16, x8, x4: 0x174 - 0x17B

Alternative Routing-ID Implementation (ARI) ARI Capability Structure

x16 = 0x184 - 0x1B3

x8 = 0x184 - 0x1A3

x4 = 0x184 - 0x1A3

Secondary PCI Express Extended Capability Header

 PCI Express Extended Capability

x16 = 0x1B4 - 0x1E3

x8 = 0x1A4 - 0x1CB

x4 = 0x1A4 - 0x1C7

 Physical Layer 16.0 GT/s Extended Capability Physical Layer 16.0 GT/s Extended Capability Structure

x16 = 0x1E4 - 0x22B

x8 = 0x1CC - 0x1F3

x4 = 0x1C8 - 0x1DF

 Margining Extended Capability Margining Extended Capability Structure

x16 = 0x22C - 0x26B

x8 = 0x1F4 - 0x233

 SR-IOV Capability SR-IOV Capability Structure

x16 = 0x26C - 0x2F7

x8 = 0x234 - 0x2BF

x4 = 0x1E0 - 0x26B

 TLP Processing Hints (TPH) Capability TLP Processing Hints (TPH) Capability Structure

x16 = 0x2F8 - 0x2FF

x8 = 0x2C0 - 0x2C7

x4 = 0x26C - 0x273

 Address Translation Services (ATS) Capability Address Translation Services Extended Capability (ATS) in Single Root I/O Virtualization and Sharing Specification

x16 = 0x308 - 0x 313

x8 = 0x2D0 - 0x2DB

x4 = 0x27C - 0x287

 Access Control Services (ACS) Capability Access Control Services (ACS) Capability

x16 = 0x314 - 0x323

x8 = 0x2DC - 0x2EB

x4 = 0x288 - 0x297

 Page Request Services (PRS) Capability Page Request Services (PRS) Capability

x16 = 0x324 - 0x32B

x8 = 0x2EC - 0x2F3

x4 = 0x298 - 0x29F

 Latency Tolerance Reporting (LTR) Capability Latency Tolerance Reporting (LTR) Capability

x16 = 0x32C - 0x333

x8 = 0x2F4 - 0x2FB

x4 = 0x2A0 - 0x2A7

 Process Address Space (PASID) Capability Process Address Space (PASID) Capability Structure

x16 = 0x46C - 0x477

x8 = 0x434 - 0x43F

x4 = 0x3E0 - 0x3EB

 Data Link Feature Extended Capability Data Link Feature Extended Capability

x16 = 0x478 - 0x483

x8 = 0x440 - 0x44B

PTM Capability Structure

Precision Time Management (PTM) Capability

x16 = 0x484 - 0x4EB

x8 = 0x44C - 0x4B3

PTM Requestor Capability Structure (VSEC)

N/A
Note:
  • x16 core is mapped to port 0 ->x16(Port0)
  • x8 core is mapped to port1 -> x8(Port1)
  • x4 cores are mapped to port2 and port3 -> x4(Port2,3)

Section Content
Register Access Definitions
PCIe Configuration Header Registers
PCI Express Capability Structures
Physical Layer 16.0 GT/s Extended Capability Structure
MSI-X Registers

Level Two Title