Intel Acceleration Stack for Intel® Xeon® CPU with FPGAs Core Cache Interface (CCI-P) Reference Manual

Download PDF
ID 683193
Date 11/04/2019
Public
Document Table of Contents
Document Table of Contents x
1. Acceleration Stack for Intel® Xeon® CPU with FPGAs Core Cache Interface (CCI-P) Reference Manual
1. Acceleration Stack for Intel® Xeon® CPU with FPGAs Core Cache Interface (CCI-P) Reference Manual x
1.1. About this Document 1.2. Introduction 1.3. CCI-P Interface 1.4. AFU Requirements 1.5. Intel® FPGA Basic Building Blocks 1.6. Device Feature List 1.7. Document Revision History for Intel® Acceleration Stack for Intel® Xeon® CPU with FPGAs Core Cache Interface (CCI-P) Reference Manual
1.1. About this Document x
1.1.1. Intended Audience 1.1.2. Conventions 1.1.3. Related Documentation 1.1.4. Acronym List for Acceleration Stack for Intel® Xeon® CPU with FPGAs Core Cache Interface (CCI-P) Reference Manual 1.1.5. Acceleration Glossary
1.2. Introduction x
1.2.1. FPGA Interface Manager (FIM) 1.2.2. Intel® FPGA Interface Unit (FIU) 1.2.3. Memory and Cache Hierarchy
1.2.2. Intel® FPGA Interface Unit (FIU) x
1.2.2.1. FIU for Intel® FPGA PAC 1.2.2.2. FIU for Intel Integrated FPGA Platform 1.2.2.3. Comparison of FIU Capabilities
1.3. CCI-P Interface x
1.3.1. Signaling Information 1.3.2. Read and Write to Main Memory 1.3.3. Interrupts 1.3.4. UMsg 1.3.5. MMIO Accesses to I/O Memory 1.3.6. CCI-P Tx Signals 1.3.7. Tx Header Format 1.3.8. CCI-P Rx Signals 1.3.9. Multi-Cache Line Memory Requests 1.3.10. Byte Enable Memory Request ( Intel® FPGA PAC D5005) 1.3.11. Additional Control Signals 1.3.12. Protocol Flow 1.3.13. Ordering Rules 1.3.14. Timing Diagram 1.3.15. CCI-P Guidance
1.3.2. Read and Write to Main Memory x
1.3.2.1. Reading from Main Memory 1.3.2.2. Writing to Main Memory
1.3.8. CCI-P Rx Signals x
1.3.8.1. Rx Header and Rx Data Format
1.3.10. Byte Enable Memory Request ( Intel® FPGA PAC D5005) x
1.3.10.1. Mixing Byte Enable and Full Cache Line Accesses
1.3.12. Protocol Flow x
1.3.12.1. Upstream Requests 1.3.12.2. Downstream Requests
1.3.13. Ordering Rules x
1.3.13.1. Memory Requests 1.3.13.2. MMIO Requests
1.3.13.1. Memory Requests x
1.3.13.1.1. Memory Write Fence 1.3.13.1.2. Memory Consistency Explained
1.4. AFU Requirements x
1.4.1. Mandatory AFU CSR Definitions 1.4.2. AFU Discovery Flow 1.4.3. AFU_ID
1. Acceleration Stack for Intel® Xeon® CPU with FPGAs Core Cache Interface (CCI-P) Reference Manual

1.6. Device Feature List

This section defines a Device Feature List (DFL) structure that creates a linked list of features within MMIO space, thus providing an extensible way of adding and enumerating features. A feature region (sometimes referred to as a “feature”) is a group of related CSRs. For example, two different features of a DMA engine can be queue management and QoS functions. You can group queue management and QoS functions into two different feature regions. A Device Feature Header (DFH) register marks the start of the feature region.

The software can walk through the DFL to enumerate the following:

  • AFU
    • An AFU is compliant to the CCI-P interface and connected directly to the CCI-P Port. Must implement mandatory AFU registers, including AFU ID.
    • The AFU DFH must be located at MMIO address 0x0.
  • Private features
    • These are a linked list of features within the AFU, which provides a way of organizing functions within an AFU. It is the AFU developer’s responsibility to enumerate and manage them.
    • They are not required to implement a ID.
  • Intel® FPGA Basic Building Blocks
    • Are special features within the AFU, which are meant to be reusable building blocks (design once, reuse many times). Software visible Intel® FPGA Basic Building Blocks typically come with a corresponding software service to enumerate and configure the Intel® FPGA Basic Building Blocks, and possibly provide a higher-level software interface to the Intel® FPGA Basic Building Blocks.
    • Do not have strong hardware interface requirements like an AFU, but must have well defined architectural semantics from a software point of view.
    • Must implement the mandatory DFH registers when visible.
    • Must implement a GUID only for software-visible Intel® FPGA Basic Building Blocks.

The following figure shows an example of an AFU’s feature hierarchy made up of BBBs and private features.

Figure 26. Example Feature Hierarchy
A Device Feature Header (DFH) register (shown in below) marks the start of the feature region.
Table 44.  Device Feature Header CSR
Device Feature Header
Bit Description
63:60 Feature Type
4’h1 – AFU 4’h2 – BBB 4’h3 – Private Features
59:52 Reserved
51:48

AFU Minor version number

User defined value

 Reserved
47:41 Reserved
40

End of List

1’b0 There is another feature header beyond this (see “Next DFH Byte Offset”)

1’b1 This is the last feature header for this AFU

39:16

Next DFH Byte offset

Next DFH Address = Current DFH Address + Next DFH Byte offset

Also used as indication for the maximum size of MMIO region occupied by this feature. For last feature, this offset points to the beginning of the unallocated MMIO region, if any (or beyond the end of the MMIO space).

Refer to the example in Table 45.

15:12

AFU Major VersionNumber

User defined

Feature Revision Number

User defined
11:0

CCI-P Version Number

Use the CCIP_VERSION_NUMBER parameter from ccip_if_pkg.sv

Feature ID

Contains user defined ID to identify features within an AFU

Table 45.  Next DFH Byte Offset Example
Feature DFH Address EOL Next DFH Byte Offset
0 0x0 0x0 0x100
1 0x100 0x0 0x180
2-Last Feature 0x280 0x1 0x80
Unallocated MMIO space, no DFH 0x300 N/A N/A

A DFH with the type set to BBB must be followed by the mandatory BBB registers listed below.

Table 46.  Mandatory BBB DFH Register Map
Byte Address offset within DFH Register Name
0x0000 DFH Type=BBB
0x0008 BBB_ID_L
0x0010 BBB_ID_H
Table 47.  BBB_ID_L CSR Definition
Register Name BB_ID_L
Bit Attribute Description
63:0 RO Lower 64-bits of the BBB_ID ID
Table 48.  BBB_ID_H CSR Definition
Register Name BB_ID_H
Bit Attribute Description
63:0 RO Upper 64-bits of the BBB_ID ID

The BBB_ID is an GUID, similar in concept to an AFU_ID. It is defined so that each BBB has a unique identifier. This allows the software to identify the software service associated with the BBB hardware.

The figure below shows how a logical feature hierarchy (shown on left-hand side) can be expressed using DFH registers defined in this section.

Figure 27. Device Feature Conceptual View
Level Two Title