Arria V Avalon-ST Interface for PCIe Solutions User Guide

ID 683733
Date 6/03/2020
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents

9.3. Receive Buffer Reordering

The PCI, PCI-X and PCI Express protocols include ordering rules for concurrent TLPs. Ordering rules are necessary for the following reasons:

  • To guarantee that TLPs complete in the intended order
  • To avoid deadlock
  • To maintain computability with ordering used on legacy buses
  • To maximize performance and throughput by minimizing read latencies and managing read/write ordering
  • To avoid race conditions in systems that include legacy PCI buses by guaranteeing that reads to an address do not complete before an earlier write to the same address

PCI uses a strongly-ordered model with some exceptions to avoid potential deadlock conditions. PCI-X added a relaxed ordering (RO) bit in the TLP header. It is bit 5 of byte 2 in the TLP header, or the high-order bit of the attributes field in the TLP header. If this bit is set, relaxed ordering is permitted. If software can guarantee that no dependencies exist between pending transactions, you can safely set the relaxed ordering bit.

The following table summarizes the ordering rules from the PCI specification. In this table, the entries have the following meanings:

  • Columns represent the first transaction issued.
  • Rows represent the next transaction.
  • At each intersection, the implicit question is: should this row packet be allowed to pass the column packet? The following three answers are possible:
    • Yes: the second transaction must be allowed to pass the first to avoid deadlock.
    • Y/N: There are no requirements. A device may allow the second transaction to pass the first.
    • No: The second transaction must not be allowed to pass the first.

The following transaction ordering rules apply to the table below.

  • A Memory Write or Message Request with the Relaxed Ordering Attribute bit clear (b’0) must not pass any other Memory Write or Message Request.
  • A Memory Write or Message Request with the Relaxed Ordering Attribute bit set (b’1) is permitted to pass any other Memory Write or Message Request.
  • Endpoints, Switches, and Root Complex may allow Memory Write and Message Requests to pass Completions or be blocked by Completions.
  • Memory Write and Message Requests can pass Completions traveling in the PCI Express to PCI directions to avoid deadlock.
  • If the Relaxed Ordering attribute is not set, then a Read Completion cannot pass a previously enqueued Memory Write or Message Request.
  • If the Relaxed Ordering attribute is set, then a Read Completion is permitted to pass a previously enqueued Memory Write or Message Request.
  • Read Completion associated with different Read Requests are allowed to be blocked by or to pass each other.
  • Read Completions for Request (same Transaction ID) must return in address order.
  • Non-posted requests cannot pass other non-posted requests.
  • CfgRd0CfgRd0 can pass IORd or MRd.
  • CfgWr0 can IORd or MRd.
  • CfgRd0 can pass IORd or MRd.
  • CfrWr0 can pass IOWr.
Table 76.  Transaction Ordering Rules

Can the Row Pass the Column?

Posted Req

Non Posted Req

Completion

Memory Write or Message Req

Read Request

I/O or Cfg Write Req

Spec

Hard IP

Spec

Hard IP

Spec

Hard IP

Spec

Hard IP

P

Posted Req

No

Y/N

No

No

Yes

Yes

Yes

Yes

Y/N

Yes

No

No

NP

Read Req

No

No

Y/N

No

Y/N

No

Y/N

No

Non-Posted Req with data

No

No

Y/N

No

Y/N

No

Y/N

No

Cmpl

Cmpl

No

Y/N

No

No

Yes

Yes

Yes

Yes

Y/N

No

No

No

I/O or Configuration Write Cmpl

Y/N

No

Yes

Yes

Yes

Yes

Y/N

No

As the table above indicates, the RX datapath implements an RX buffer reordering function that allows Posted and Completion transactions to pass Non-Posted transactions (as allowed by PCI Express ordering rules) when the Application Layer is unable to accept additional Non-Posted transactions.

The Application Layer dynamically enables the RX buffer reordering by asserting the rx_mask signal. The rx_mask signal blocks non-posted Req transactions made to the Application Layer interface so that only posted and completion transactions are presented to the Application Layer.

Note: MSI requests are conveyed in exactly the same manner as PCI Express memory write requests and are indistinguishable from them in terms of flow control, ordering, and data integrity.