R-Tile Avalon® Streaming Intel® FPGA IP for PCI Express* Design Example User Guide

Download PDF
ID 683544
Date 9/26/2022
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. About the R-Tile Avalon® Streaming Intel® FPGA IP for PCI Express 2. Quick Start Guide 3. R-Tile Avalon® Streaming Intel® FPGA IP for PCI Express* Design Example User Guide Archives 4. Document Revision History for the R-Tile Avalon® Streaming Intel FPGA IP for PCI Express Design Example User Guide
1. About the R-Tile Avalon® Streaming Intel® FPGA IP for PCI Express x
1.1. Functional Description for the Programmed Input/Output (PIO) Design Example 1.2. Functional Description for the Single Root I/O Virtualization (SR-IOV) Design Example 1.3. Hardware and Software Requirements
1.1. Functional Description for the Programmed Input/Output (PIO) Design Example x
1.1.1. Credit Initialization Sequence
1.1.1. Credit Initialization Sequence x
1.1.1.1. Credit Value Initialization and Return 1.1.1.2. Credit Distribution
2. Quick Start Guide x
2.1. Directory Structure 2.2. Generating the Design Example 2.3. Simulating the Design Example 2.4. Root Port BFM 2.5. Compiling the Design Example 2.6. Installing the Linux Kernel Driver 2.7. Running the Design Example
2.3. Simulating the Design Example x
2.3.1. Steps to Run Simulations 2.3.2. Testbench
2.3.1. Steps to Run Simulations x
2.3.1.1. Siemens EDA QuestaSim* Simulator 2.3.1.2. VCS* Simulator 2.3.1.3. VCS* MX Simulator 2.3.1.4. Xcelium* Simulator
2.3.2. Testbench x
2.3.2.1. Testbench Modules 2.3.2.2. Test Driver Module 2.3.2.3. PIO Design Example Testbench 2.3.2.4. SR-IOV Design Example Testbench
2.4. Root Port BFM x
2.4.1. BFM Memory Map 2.4.2. Configuration Space Bus and Device Numbering 2.4.3. Configuration of Root Port and Endpoint 2.4.4. Issuing Read and Write Transactions to the Application Layer 2.4.5. BFM Procedures and Functions
2.4.5. BFM Procedures and Functions x
2.4.5.1. ebfm_barwr Procedure 2.4.5.2. ebfm_barwr_imm Procedure 2.4.5.3. ebfm_barrd_wait Procedure 2.4.5.4. ebfm_barrd_nowt Procedure 2.4.5.5. ebfm_cfgwr_imm_wait Procedure 2.4.5.6. ebfm_cfgwr_imm_nowt Procedure 2.4.5.7. ebfm_cfgrd_wait Procedure 2.4.5.8. ebfm_cfgrd_nowt Procedure 2.4.5.9. BFM Configuration Procedures 2.4.5.10. BFM Shared Memory Access Procedures 2.4.5.11. BFM Log and Message Procedures 2.4.5.12. Verilog HDL Formatting Functions
2.4.5.9. BFM Configuration Procedures x
2.4.5.9.1. ebfm_cfg_rp_ep Procedure 2.4.5.9.2. ebfm_cfg_decode_bar Procedure
2.4.5.10. BFM Shared Memory Access Procedures x
2.4.5.10.1. Shared Memory Constants 2.4.5.10.2. shmem_write Task 2.4.5.10.3. shmem_read Function 2.4.5.10.4. shmem_display Verilog HDL Function 2.4.5.10.5. shmem_fill Procedure 2.4.5.10.6. shmem_chk_ok Function
2.4.5.11. BFM Log and Message Procedures x
2.4.5.11.1. ebfm_display Verilog HDL Function 2.4.5.11.2. ebfm_log_stop_sim Verilog HDL Function 2.4.5.11.3. ebfm_log_set_suppressed_msg_mask Task 2.4.5.11.4. ebfm_log_set_stop_on_msg_mask Verilog HDL Task 2.4.5.11.5. ebfm_log_open Verilog HDL Function 2.4.5.11.6. ebfm_log_close Verilog HDL Function
2.4.5.12. Verilog HDL Formatting Functions x
2.4.5.12.1. himage1 2.4.5.12.2. himage2 2.4.5.12.3. himage4 2.4.5.12.4. himage8 2.4.5.12.5. himage16 2.4.5.12.6. dimage1 2.4.5.12.7. dimage2 2.4.5.12.8. dimage3 2.4.5.12.9. dimage4 2.4.5.12.10. dimage5 2.4.5.12.11. dimage6 2.4.5.12.12. dimage7
2.7. Running the Design Example x
2.7.1. Running the PIO Design Example 2.7.2. Running the SR-IOV Design Example
1. About the R-Tile Avalon® Streaming Intel® FPGA IP for PCI Express
2. Quick Start Guide
3. R-Tile Avalon® Streaming Intel® FPGA IP for PCI Express* Design Example User Guide Archives
4. Document Revision History for the R-Tile Avalon® Streaming Intel FPGA IP for PCI Express Design Example User Guide

2.7.2. Running the SR-IOV Design Example

Follow the steps below to test the SR-IOV design example on hardware:
  1. Navigate to ./software/user/example under the design example directory.
  2. Compile the design example application by running the command:

    make

    Then perform the following steps:

    1. Run the link test application by running the command:

      sudo ./intel_fpga_pcie_link_test

    2. Select Option 1: Manually Select the Device
    3. Enter the BDF of the Physical Function for which the virtual functions are allocated.
    4. Enter BAR 0 to proceed to the test menu.
      **********************************************************
Intel FPGA PCIe Link Test
Version 2.0
0: Automatically select a device
1: Manually select a device
*********************************************************
> 1
Enter bus number, in hex:
> 38
Enter device number, in hex:
> 0
Enter function number, in hex:
> 0
BDF is 0x3800
B:D.F, in hex, is 38:0.0
Enter BAR number (-1 for none):
> 0
Opened a handle to BAR 0 of a device with BDF 0x3800

*********************************************************
0: Link test - 100 writes and reads
 1: Write memory space
 2: Read memory space
 3: Write configuration space
 4: Read configuration space
 5: Change BAR for PIO
 6: Change device
 7: Enable SRIOV
 8: Do a link test for every enabled virtual function
    belonging to the current device
 9: Quit program
*********************************************************
>
    5. Enter Option 7 to enable SR-IOV for the current device.
    6. Enter 16 as the number of virtual functions to be enabled for the current device.
      *********************************************************
 0: Link test - 100 writes and reads
 1: Write memory space
 2: Read memory space
 3: Write configuration space
 4: Read configuration space
 5: Change BAR for PIO
 6: Change device
 7: Enable SRIOV
 8: Do a link test for every enabled virtual function
    belonging to the current device
 9: Quit program
*********************************************************
> 7
Enter the number of VFs to enable for the current device (0-31):
> 16
Enabled 16 VFs.
Type 'lspci -d 1172:' in a new terminal to determine newly enabled devices' BDFs.
[root@localhost ~]# lspci -d 1172:
38:00.0 Unassigned class [ff00]: Altera Corporation Device 0000 (rev 01)
38:00.2 Unassigned class [ff00]: Altera Corporation Device 0000
38:00.3 Unassigned class [ff00]: Altera Corporation Device 0000
38:00.4 Unassigned class [ff00]: Altera Corporation Device 0000
38:00.5 Unassigned class [ff00]: Altera Corporation Device 0000
38:00.6 Unassigned class [ff00]: Altera Corporation Device 0000
38:00.7 Unassigned class [ff00]: Altera Corporation Device 0000
38:01.0 Unassigned class [ff00]: Altera Corporation Device 0000
38:01.1 Unassigned class [ff00]: Altera Corporation Device 0000
38:01.2 Unassigned class [ff00]: Altera Corporation Device 0000
38:01.3 Unassigned class [ff00]: Altera Corporation Device 0000
38:01.4 Unassigned class [ff00]: Altera Corporation Device 0000
38:01.5 Unassigned class [ff00]: Altera Corporation Device 0000
38:01.6 Unassigned class [ff00]: Altera Corporation Device 0000
38:01.7 Unassigned class [ff00]: Altera Corporation Device 0000
38:02.0 Unassigned class [ff00]: Altera Corporation Device 0000
38:02.1 Unassigned class [ff00]: Altera Corporation Device 0000
    7. Enter Option 8 to perform a link test for every enabled virtual function allocated within the physical function. The link test application performs 100 memory writes with a single dword of data each and then reads back the data. At the end of the testing, the application prints the number of virtual functions that failed the test.
      Testing VF with BDF 0x380d...
Doing 100 writes and 100 reads..
Number of write errors:       0
Number of read errors:        0
Number of dword mismatches:   0
Testing VF with BDF 0x380e...
Doing 100 writes and 100 reads..
Number of write errors:       0
Number of read errors:        0
Number of dword mismatches:   0
Testing VF with BDF 0x380f...
Doing 100 writes and 100 reads..
Number of write errors:       0
Number of read errors:        0
Number of dword mismatches:   0
Testing VF with BDF 0x3810...
Doing 100 writes and 100 reads..
Number of write errors:       0
Number of read errors:        0
Number of dword mismatches:   0
Testing VF with BDF 0x3811...
Doing 100 writes and 100 reads..
Number of write errors:       0
Number of read errors:        0
Number of dword mismatches:   0
Test failed for 0 VFs out of 16 VFs
    8. Exit the application and run the command

      lspci -d 1172: | grep -c Altera

      to verify the enumeration of PFs and VFs. The expected result is the sum of PFs and VFs.

      In case you enable the 16 VFs for only one of the physical functions, you would have 18 devices reported, 2 PFs + 16 VFs of PF0. You can also repeat the same process for the second function included in the design example and you would have a total of 34 devices reported, 2 PFs + 32 VFs. 

      [root@localhost ~]# lspci -d 1172: | grep -c Altera
Level Two Title