Intel Stratix 10 H-Tile PCIe Link Hardware Validation
Stratix® 10 Hard IP for
PCI Express® IP core includes a programmed I/O (PIO) design example to help you
understand the usage for this IP. The PIO design example transfers data from the host memory
to the local memory on a target device, which in this case is an Intel FPGA. This design
example is appropriate for low-bandwidth applications. In the
hardware validation process, this design example serves as an Endpoint (EP) that interacts
with the host, which is the Root Port. The same design example can be used in your hardware to
validate if a
PCIe® link is working as expected. The
Stratix® 10 GX FPGA Development Kit with H-Tile device is used in the
hardware validation process. This validation is performed on both the Linux* and Windows*
Hardware Validation System Block Diagram
Figure 1. Hardware Validation System Simplified Block Diagram
figure above shows the simplified block diagram of the hardware validation system. The Root
Port in this system is the host PC, which interfaces with the PIO design example to perform
memory reads and memory writes on an on-chip memory without the use of DMA.
The three main components of the design example are:
Device-Under-Test (DUT): This refers to the
PCIe® Hard IP (HIP), which you can configure.
PIO Application: This takes in the
Avalon®-ST data and converts it to the
before sending it to the slave.
The slave in this case is an on-chip memory with a size that matches the
DUT's BAR size.
For the host (or Root Port), a driver and application software are provided
for the validation process. After the driver is installed, the hardware will be initialized
and ready for transactions. Data is then packetized, and can be received by the DUT. In the
validation process, the application software performs Memory Writes to the on-chip memory
followed by Memory Reads from the on-chip memory. It then compares the written data with the
read data. A matching dataset indicates a successful validation.
Hardware and Software Requirements
sections describe the hardware and software used to perform the hardware
Stratix® 10 GX Development Kit
Hewlett Packard Z620 workstation with CentOS* 6.0
Hewlett Packard Z620 workstation with Windows* 7
Stratix® 10 GX FPGA Development Kit
Quartus® Prime Pro Edition version 18.0
CentOS 6.0 with Linux Kernel version 2.6.32-696
Linux driver generated together with the example design
PCIe® Windows Demo Driver
Quick Start Guide
Stratix® 10 GX FPGA Development Kit on
hardware setup to run the reference design, follow these steps:
Stratix® 10 GX FPGA Development Kit
into the x16 PCIe slot of the motherboard.
Connect the USB cable to the Micro USB Blaster connector on the development
Connect the power adapter (packaged together with the development board)
to the power supply jack.
Connect the Auxiliary power from the motherboard to the development
Power on the workstation.
Turn on the power for the
Stratix® 10 GX FPGA
Development Kit. The hardware system is now ready for programming.
Quartus® Prime Pro Edition software, create a new project (go to
File, and choose New Project Wizard).
Specify the Directory, Name, and Top-Level Entity.
For Project Type, accept the default value, Empty project. Click
For Add Files, click Next.
For Family, Device & Board Settings, select
Stratix® 10 for Family. Then
select the Target Device for your design from the list of Available Devices.
In the IP Catalog, locate and add the
Stratix® 10 Hard IP for
In the New IP Variant dialogue box, specify a name for your IP.
On the IP Settings tabs, specify the parameters for your IP variation.
On the Example Designs tab, make the following selections:
For Available Example Designs, select PIO.
For Example Design Files, turn on the Simulation and
If you have selected a x16 configuration, for Select Simulation Root Complex BFM,
choose the appropriate Bus Functional Model (BFM):
Intel FPGA BFM: for all configurations up to Gen3 x8. This BFM supports x16
configurations by downtraining to x8.
Third-party BFM: for x16 configurations if you want to simulate all 16 lanes.
Refer to AN-811: Using the Avery BFM for PCI Express Gen3x16 Simulation on
Stratix® 10 Devices for information about simulating with the
For Generated HDL Format, only Verilog is available in the current release.
For Target Development Kit, select the appropriate option. Note: If you select
None, the generated design example targets the device
specified. If you intend to test the design in hardware, make the appropriate pin
assignments in the .qsf file.
Select Generate Example Design to create a design example that you
can simulate and download to hardware. If you select one of the
development boards, the device on that board overwrites the device previously selected in
Quartus® Prime project if the devices are different. When the prompt asks
you to specify the directory for your example design, accept the default directory, /
Figure 4. Design Example Configuration
Close the IP Parameter Editor followed by the project.
Compiling the Design Example and Programming the Device
Navigate to <project_dir>/pcie_s10_hip_ast_0_example_design/ and
On the Processing menu, select Start Compilation.
After compilation, program the targeted device with the Programmer.
After successfully programming the device, reboot the workstation.
Hardware validation can be performed on either the Linux or Windows
Hardware Validation on the Linux Platform
Linux Kernel Driver
kernel driver is generated together with the design example at
./software/user/example under the design example generation directory.
You can use this driver to perform the following tests:
A PCIe link test that performs 100 writes and reads
Memory space DWORD reads and writes
Configuration space DWORD reads and writes
In addition, you can use the driver to change the value of the following parameters:
The bus, device and function (BDF) numbers to specify the selected device
Installing the Linux Kernel Driver
install the kernel driver, do the following:
Navigate to ./software/kernel/Linux
under the design example generation directory.
Change the permissions on the install, load, and unload files with the
$ chmod 777 install load
Install the driver with the following command:
$ sudo ./install
Verify the driver installation with the following command:
$ lsmod | grep intel_fpga_pcie_drv
An example of a returned result is “intel_fpga_pcie_drv 17792 0”.
PCIe® configuration space
information with the following command:
The following figure shows an example of
PCIe® configuration space information.
Figure 5. An example of
configuration space information
Verify that Linux recognizes the
design example with the following command:
lspci -d 1172:000 -v | grep intel_fpga_pcie_drv
expected result returned should be “Kernel driver in use: intel_fpga_pcie_drv” as shown in
the following figure.
Figure 6. Verify the Linux kernel driver in use
Verify the BDF of the
Stratix® 10 GX
Development Kit with the following command:
$ lspci -d 1172:000
below shows an example of the BDF results.
Figure 7. BDF information of the
Stratix® 10 GX
Running the Design Example Application
Navigate to ./software/user/example
under the design example directory.
Compile the design example application with the following command:
Run the test with the following command:
By default, you can run the Intel FPGA IP
PCIe® link test in manual or automatic mode. In this application note, the Intel
PCIe® link test is run in automatic mode. The
following figure shows an example of passing results with 0 write or read error.
Figure 8. Passing results for automatic mode with 0 write or read error
Hardware Validation on the Windows Platform
Installing the Windows driver
describes the steps to set up the Windows driver for hardware validation on the Windows
Download the driver from the following alterawiki link:
Open the Windows Device Manager and scan for hardware changes.
Select the Intel FPGA listed as an unknown
device and point to the
appropriate 32- or 64-bit driver (altera_pcie_win_driver.intf) in the
After the driver loads successfully, a new device named Altera PCI API
Device should appear in the Windows Device Manager.
You can determine the BDF number for the Altera PCI API Device
listed in the Windows Device Manager by doing the following:
Expand the tab Altera PCI API Driver under the devices.
Right click on Altera PCI API Device and select
Take note of the bus, device and function numbers for the device. Figure 10 shows an
example of the bus, device and function numbers for the new
Figure 9. An example of BDF numbers for a new
PCIe® device in Device
Running the Windows demo driver with design example
software application to test the
PCI Express® design example
Stratix® 10 FPGA Development Kit is available on both 32-
and 64-bit Windows platforms. This program performs the following tasks:
Print the configuration space, lane rate and lane width.
Write 0x00000000 to the specified BAR at offset 0x00000000 to initialize
the memory and read it back.
Write 0xABCD1234 at offset 0x00000000 of the specified BAR, then read it
back and compare.
If successful, the test program displays the message "PASSED".
Follow these steps to perform the
link test with the Windows demo driver with the design example in hardware:
In the Interop_software folder, run
When prompted, enter the bus, device and function numbers for your
Altera PCI API Driver as shown in the following
Figure 10. Bus, device and function numbers and BAR number entries into the
Note: The bus, device and
function numbers for your hardware setup may be different.
Enter the BAR number (0 - 5) that you specified during the
parameterization of the
Stratix® 10 Hard IP for
The Interop software application will proceed with the
PCIe® link test.
If the test is successful, you will see the message "PASSED" as shown in
the figure below.
Figure 11. "PASSED" message to indicate a successful link test
Note: There is a known
issue with this Windows demo driver. The lane rate and link width are incorrectly
displayed as Lane Rate = 0 and Link Width = 00. This issue will be fixed in the future
release of the Windows demo driver.