R-tile Avalon® Streaming Intel® FPGA IP for PCI Express* User Guide

Download PDF
ID 683501
Date 3/28/2022
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. About the R-tile Avalon® Streaming Intel® FPGA IP for PCI Express 2. IP Architecture and Functional Description 3. Advanced Features 4. Interfaces 5. Parameters 6. R-Tile Avalon® Streaming Intel® FPGA IP for PCI Express* User Guide Archives 7. Document Revision History for the R-tile Avalon® Streaming Intel FPGA IP for PCI Express User Guide A. Configuration Space Registers B. Root Port Enumeration C. Implementation of Address Translation Services (ATS) in Endpoint Mode D. Packets Forwarded to the User Application in TLP Bypass Mode
1. About the R-tile Avalon® Streaming Intel® FPGA IP for PCI Express x
1.1. Overview 1.2. Features 1.3. Release Information 1.4. Device Family Support 1.5. Performance and Resource Utilization 1.6. IP Core Support Levels
1.1. Overview x
1.1.1. Standards and Specifications Compliance
2. IP Architecture and Functional Description x
2.1. Overview 2.2. Bias Temperature Instability (BTI) Protection Mode 2.3. PCIe Hard IP Mode 2.4. PIPE Direct Mode
2.3. PCIe Hard IP Mode x
2.3.1. Clocking 2.3.2. Reset 2.3.3. PCI Express Protocol Stack
2.3.3. PCI Express Protocol Stack x
2.3.3.1. PMA/PCS 2.3.3.2. Data Link Layer Overview 2.3.3.3. Transaction Layer Overview
2.4. PIPE Direct Mode x
2.4.1. Clocking 2.4.2. Reset 2.4.3. PIPE Layer
3. Advanced Features x
3.1. PCIe Port Bifurcation and PHY Channel Mapping 3.2. Virtualization Support 3.3. TLP Bypass Mode
3.2. Virtualization Support x
3.2.1. SR-IOV Support 3.2.2. VirtIO Support
3.2.1. SR-IOV Support x
3.2.1.1. SR-IOV Supported Features List 3.2.1.2. Implementation 3.2.1.3. VF to PF Mapping 3.2.1.4. Function Level Reset (FLR)
3.2.1.2. Implementation x
3.2.1.2.1. VF Error Flag Interface (for x16/x8 Cores Only)
3.2.2. VirtIO Support x
3.2.2.1. VirtIO Supported Features List 3.2.2.2. Overview 3.2.2.3. Parameters 3.2.2.4. VirtIO PCI Configuration Access Interface 3.2.2.5. Registers
3.2.2.5. Registers x
3.2.2.5.1. VirtIO Common Configuration Capability Register (Address: 0x012) 3.2.2.5.2. VirtIO Common Configuration BAR Indicator Register (Address: 0x013) 3.2.2.5.3. VirtIO Common Configuration BAR Offset Register (Address: 0x014) 3.2.2.5.4. VirtIO Common Configuration Structure Length Register (Address 0x015) 3.2.2.5.5. VirtIO Notifications Capability Register (Address: 0x016) 3.2.2.5.6. VirtIO Notifications BAR Indicator Register (Address: 0x017) 3.2.2.5.7. VirtIO Notifications BAR Offset Register (Address: 0x018) 3.2.2.5.8. VirtIO Notifications Structure Length Register (Address: 0x019) 3.2.2.5.9. VirtIO Notifications Notify Off Multiplier Register (Address: 0x01A) 3.2.2.5.10. VirtIO ISR Status Capability Register (Address: 0x02F) 3.2.2.5.11. VirtIO ISR Status BAR Indicator Register (Address: 0x030) 3.2.2.5.12. VirtIO ISR Status BAR Offset Register (Address: 0x031) 3.2.2.5.13. VirtIO ISR Status Structure Length Register (Address: 0x032) 3.2.2.5.14. VirtIO Device Specific Capability Register (Address: 0x033) 3.2.2.5.15. VirtIO Device Specific BAR Indicator Register (Address: 0x034) 3.2.2.5.16. VirtIO Device Specific BAR Offset Register (Address 0x035) 3.2.2.5.17. VirtIO Device Specific Structure Length Register (Address: 0x036) 3.2.2.5.18. VirtIO PCI Configuration Access Capability Register (Address: 0x037) 3.2.2.5.19. VirtIO PCI Configuration Access BAR Indicator Register (Address: 0x038) 3.2.2.5.20. VirtIO PCI Configuration Access BAR Offset Register (Address: 0x039) 3.2.2.5.21. VirtIO PCI Configuration Access Structure Length Register (Address: 0x03A) 3.2.2.5.22. VirtIO PCI Configuration Access Data Register (Address: 0x03B)
3.3. TLP Bypass Mode x
3.3.1. Overview 3.3.2. Register Settings for the TLP Bypass Mode 3.3.3. Hard IP Reconfiguration Interface 3.3.4. Avalon® Streaming Interface
3.3.2. Register Settings for the TLP Bypass Mode x
3.3.2.1. TLPBYPASS_ERR_EN (Address 0x1314) 3.3.2.2. TLPBYPASS_ERR_STATUS (Address 0x1310)
3.3.4. Avalon® Streaming Interface x
3.3.4.1. Configuration TLP 3.3.4.2. Transmit Interface 3.3.4.3. Receive Interface 3.3.4.4. Malformed TLP 3.3.4.5. ECRC
4. Interfaces x
4.1. Overview 4.2. Clocks and Resets 4.3. Serial Data Interface 4.4. PCI Express Mode 4.5. PIPE Direct Mode
4.2. Clocks and Resets x
4.2.1. Clocks 4.2.2. Resets
4.4. PCI Express Mode x
4.4.1. Avalon® Streaming Interface 4.4.2. Precision Time Measurement (PTM) Interface (Endpoint Only) 4.4.3. Interrupt Interface 4.4.4. Hard IP Reconfiguration Interface 4.4.5. Error Interface 4.4.6. Completion Timeout Interface 4.4.7. Configuration Intercept Interface 4.4.8. Power Management Interface 4.4.9. Hard IP Status Interface 4.4.10. Page Request Services (PRS) Interface (Endpoint Only) 4.4.11. Function-Level Reset (FLR) Interface (Endpoint Only) 4.4.12. SR-IOV VF Error Flag Interface (Endpoint Only) 4.4.13. General Purpose VSEC Interface
4.4.1. Avalon® Streaming Interface x
4.4.1.1. TLP Header and Data Alignment for the Avalon® streaming RX and TX Interfaces 4.4.1.2. Credit Control 4.4.1.3. Avalon® Streaming RX Interface 4.4.1.4. Avalon® Streaming TX Interface 4.4.1.5. Tag Allocation
4.4.1.2. Credit Control x
4.4.1.2.1. Credit Interface - Data and Header 4.4.1.2.2. Credit Initialization 4.4.1.2.3. Credit Update/Release 4.4.1.2.4. RX Flow Control Interface 4.4.1.2.5. TX Flow Control Interface
4.4.1.4. Avalon® Streaming TX Interface x
4.4.1.4.1. Avalon® Streaming TX Interface tx_st_ready_o Behavior
4.4.1.5. Tag Allocation x
4.4.1.5.1. Completion Buffer Size
4.4.3. Interrupt Interface x
4.4.3.1. Legacy Interrupts 4.4.3.2. MSI 4.4.3.3. MSI-X
4.4.3.3. MSI-X x
4.4.3.3.1. Implementing MSI-X Interrupts
4.4.8. Power Management Interface x
4.4.8.1. D3Hot Entry 4.4.8.2. D3Hot Exit 4.4.8.3. D3Cold Entry 4.4.8.4. D3Cold Exit
4.4.8.2. D3Hot Exit x
4.4.8.2.1. D3Hot Exit Initiated by Host 4.4.8.2.2. D3Hot Exit Initiated by EP
4.5. PIPE Direct Mode x
4.5.1. Data Signals 4.5.2. Message Channel 4.5.3. Deskew Channel 4.5.4. PIPE Direct Reset Sequence 4.5.5. PIPE Direct Speed Change 4.5.6. PIPE Lane Reset Staggering 4.5.7. Unused Lanes in PIPE Direct Mode
4.5.1. Data Signals x
4.5.1.1. Transmit Signals 4.5.1.2. Receive Signals
4.5.3. Deskew Channel x
4.5.3.1. MAC to PHY (M2P) Signals 4.5.3.2. PHY to MAC (P2M) Signals
5. Parameters x
5.1. Top-Level Settings 5.2. Core Parameters
5.2. Core Parameters x
5.2.1. Avalon Parameters 5.2.2. Base Address Registers 5.2.3. PCI Express and PCI Capabilities Parameters
5.2.3. PCI Express and PCI Capabilities Parameters x
5.2.3.1. Device Capabilities 5.2.3.2. VirtIO Parameters 5.2.3.3. Link Capabilities 5.2.3.4. Legacy Interrupt Pin Register 5.2.3.5. MSI Capabilities 5.2.3.6. MSI-X Capabilities 5.2.3.7. Slot Capabilities 5.2.3.8. Latency Tolerance Reporting (LTR) 5.2.3.9. Process Address Space ID (PASID) 5.2.3.10. Device Serial Number Capability 5.2.3.11. Page Request Service (PRS) 5.2.3.12. Access Control Service (ACS) 5.2.3.13. Power Management 5.2.3.14. Vendor Specific Extended Capability (VSEC) Registers 5.2.3.15. TLP Processing Hints (TPH) 5.2.3.16. Address Translation Services (ATS) Capabilities 5.2.3.17. Precision Time Management (PTM)
C. Implementation of Address Translation Services (ATS) in Endpoint Mode x
C.1. Sending Translated/Untranslated Requests C.2. Sending a Page Request Message from the Endpoint (EP) to the Root Complex (RC) C.3. Invalidating Requests/Completions
D. Packets Forwarded to the User Application in TLP Bypass Mode x
D.1. EP TLP Bypass Mode (Upstream) D.2. RC TLP Bypass Mode (Downstream)
1. About the R-tile Avalon® Streaming Intel® FPGA IP for PCI Express
2. IP Architecture and Functional Description
3. Advanced Features
4. Interfaces
5. Parameters
6. R-Tile Avalon® Streaming Intel® FPGA IP for PCI Express* User Guide Archives
7. Document Revision History for the R-tile Avalon® Streaming Intel FPGA IP for PCI Express User Guide
A. Configuration Space Registers
B. Root Port Enumeration
C. Implementation of Address Translation Services (ATS) in Endpoint Mode
D. Packets Forwarded to the User Application in TLP Bypass Mode

4.5.5. PIPE Direct Speed Change

In the PIPE Direct Data mode, the clock for the RX datapath is sourced from the PHY recovered clock (pipe_direct_pld_rx_clk_out_o). The PHY recovered clock changes frequency when the PHY trains from Gen1 to Gen5. During the PIPE Direct RX rate change, the following sequence needs to be adhered to.

The soft IP controller first changes the rate or width if required. The R-tile Avalon Streaming IP only asserts lnX_pipe_direct_pclkchangeok_o after the Soft IP controller has made the changes. The Soft IP controller asserts lnX_pipe_direct_pclkchangeack_i when the change is complete and stable. After the Soft IP controller asserts lnX_pipe_direct_pclkchangeack_i, the R-tile Avalon Streaming IP responds by asserting lnX_pipe_direct_phystatus_o for one cycle and deasserting lnX_pipe_direct_pclkchangeok_o at the same time as lnX_pipe_direct_phystatus_o. The Soft IP controller deasserts lnX_pipe_direct_pclkchangeack_i when lnX_pipe_direct_pclkchangeok_o is sampled low.

As a reference, the following two timing diagrams illustrate the speed change from Gen1 to Gen5.
Note: Although the diagrams below illustrate a speed change from Gen1 to Gen5, the overall sequence applies to all speed changes. However, the final value of ln0_pipe_direct_rate_i in Step 1 varies depending on what the final speed is.
Figure 38. PIPE Direct Speed Change (Part 1)
Figure 39. PIPE Direct Speed Change (Part 2)
The steps shown in the diagrams are:
  1. The Soft IP controller changes the PIPE per-channel rate signal (ln0_pipe_direct_rate_i) to the IP from Gen1 to Gen5.
  2. The IP deasserts the PIPE RX reset status signal (ln_pipe_direct_reset_status_n_o) for each channel.
  3. The PIPE per-channel PCLK change OK and ACK signals (ln0_pipe_direct_pclkchangeok_o, ln0_pipe_direct_pclkchangeack_i) are asserted.
  4. The IP deasserts the PIPE per-channel RX CDR lock-to-reference signal (ln0_pipe_direct_cdrlockstatus_o).
  5. The IP sends the PIPE per-channel PHY status pulse (ln0_pipe_direct_phystatus_o) to the Soft IP controller. Also, the IP deasserts pclkchangeok (ln0_pipe_direct_pclkchangeok_o) and the Soft IP controller deasserts pclkchangeack (ln0_pipe_direct_pclkchangeack_i).
  6. The PIPE per-channel TX data (ln0_pipe_direct_txdata_i) transfer from the Soft IP controller to the IP begins (at Gen5 rate).
  7. The IP asserts the PIPE per-channel RX CDR lock-to-data signal (ln0_pipe_direct_cdrlock2data_o).
  8. The PIPE per-channel RX output clock (ln0_pipe_direct_pld_rx_clk_out_o) from the IP to the Soft IP controller becomes active.
  9. The PIPE per-channel RX data (ln0_pipe_direct_pipe_rxdata_o) transfer from the IP to the Soft IP controller begins (at Gen5 rate).

PIPE Direct TX Datapath and PIPE Direct RX Datapath provide an illustration of the PIPE Direct mode TX and RX datapath signals.

Figure 40. PIPE Direct TX Datapath
Note: At Gen1 and Gen2 speeds, only the 10 LSB bits from the lower segment of LnX_pipe_direct_txdata bus contain valid data. Bits [63:10] are don't-cares.
Figure 41. PIPE Direct RX Datapath
Note: At Gen1 and Gen2 speeds, only the 10 LSB bits in the upper and lower segments of the LnX_pipe_direct_rxdata_o bus contain valid data. Bits [31:10] and [63:42] are don't-cares.
Level Two Title