10Gbps Ethernet Accelerator Functional Unit (AFU) Design Example User Guide: Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA

ID 683422
Date 4/30/2019

3.3. Running 10GbE External Loopback Test in Single Intel® PAC System

The setup and output from the commands in the external loopback test are similar to the internal loopback test. The only difference is that the traffic loopback is established after the Intel® PAC’s QSFP+ network port.

  1. Loopback the generated network traffic at the Intel® PAC’s external QSFP+ network port. You can accomplish this loopback in several ways:
    • installing a QSFP+ optical module loopback adapter, or
    • installing a QSFP+ optical module with MPO connection and looping back through:
      • an inserted fiber loopback plug, or
      • external network equipment
  2. Load the AF for the 10GbE AFU example (if the AF is not already loaded).
    $ sudo fpgaconf hw/samples/eth_e2e_e10/bin/eth_e2e_e10.gbs
  3. cd $OPAE_PLATFORM_ROOT/hw/samples/eth_e2e_e10/sw
  4. Run the following steps on your Intel® PAC:
    1. Complie the library and application using the command:
      $ make
    2. To configure the transceiver channel into 10G mode, write 10 to the following sysfs entry:
      $ sudo sh -c "echo 10 > /sys/class/fpga/intel-fpga-dev.<instance_id>\

      <instance_id> represents the consecutive numbering of device, fme, and hssi instances.

      For example:
      sudo sh -c "echo 10 > /sys/class/fpga/intel-fpga-dev.0\
    3. To allow non-root users to access the 10GbE AFU instance, you can provide read and write privileges to the port (/dev/intel-fpga-port.\*) where \* denotes the respective socket. For example, to provide read and write privileges on Port 0:
      $ sudo chmod 666 /dev/intel-fpga-port.0
    4. To resolve library dependency:
    5. To disable the internal loopback on B:D:F - 00:0a:0b,
      $ ./pac_hssi_e10 -b 00 -d 0a -f 0b --channel=0\
      You must disable loopback on all the channels that are used in the test.
    6. To trigger the DFE (Decision feedback equalizer):
      sudo sh -c "echo 1 > /sys/class/fpga/intel-fpga-dev.0/intel-fpga-fme.0/intel-pac-hssi.2.auto/hssi_mgmt/dfe_kickstart"
      Verify that some of the DFE tap values are non-zero. This ensures that the script run is successful.
      cat /sys/class/fpga/intel-fpga-dev.0/intel-fpga-fme.0/intel-pac-hssi.2.auto/hssi_mgmt/dfe_kickstart
    7. To clear PHY, transmit, and receive statistics:
      $ ./pac_hssi_e10 -b 00 -d 0a -f 0b --channel=0 --action=stat_clear
      Sample output:
      Cleared TX stats on channel 0
      Cleared RX stats on channel 0
    8. To transmit 0x1000 packets:
      $ ./pac_hssi_e10 -b 00 -d 0a -f 0b --channel=0 --action=pkt_send
      Sample output:
      Sent 0x10000 packets on channel 0
      Note: After programming the eth_e2e_e10 AFU, the initial send of packets may drop the first packet. Subsequent packet sends do not drop any packets.
    9. To get PHY, transmit and receive statistics:
      $ ./pac_hssi_e10 -b 00 -d 0a -f 0b --channel=0 --action=stat
    Note: After every hot plug/unplug of the cables, you must trigger the DFE as discussed above after disabling internal loopback.
    For more details, refer to the README file located in the sw subdirectory to:

    To run this example on a virtual machine:

    1. Program the eth_e2e_e10 AFU and configure the transceiver channel to 10G mode from the Host machine by referencing the previous substeps.
    2. Disable internal loopback and trigger DFE from the Host by refereccing the previous substeps.
    3. Follow the steps in the Running the OPAE in a Virtualized Environment section of the Intel® Acceleration Stack Quick Start Guide for Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA to create a virtual function and attach the virtual function to a virtual machine.
    4. Run the external loopback test on the virtual machine.