Intel Acceleration Stack Quick Start Guide: Intel FPGA Programmable Acceleration Card D5005

ID 683394
Date 7/09/2021
Public
Document Table of Contents
Give Feedback

7.2. Configuring the VF Port on the Host

By default, the PF controls the AFU port. The following procedure transfers AFU control to the VF. After the transfer to VF control, applications running on the VM can access the AFU.

In a multi-card system, if you want to configure the VF on only a single PCIe device, run below command to find the device mapping for the specific PCIe:
ls -l /sys/class/fpga/intel-fpga-dev.*
Sample output:
/sys/class/fpga/intel-fpga-dev.0 -> ../../devices/pci0000:36/0000:36:00.0/0000:37:00.0/fpga/intel-fpga-dev.0
/sys/class/fpga/intel-fpga-dev.1 -> ../../devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/fpga/intel-fpga-dev.1

To target PCIe B:D.F (AF:00.0) and B:D.F (37:00.0) in the following commands, use instance id 1 and 0 instead of * respectively.

  1. Run the following commands to export the required paths:
    export port_path=$(find /sys/class/fpga/intel-fpga-dev.* \
     -maxdepth 1 -follow -iname intel-fpga-port.*)
    export link_path=$(readlink -m /$port_path/../)
    export pci_path=$link_path/../../
  2. Release the port controlled by the PF using the fpgaport tool:
    sudo fpgaport release /dev/intel-fpga-fme.* 0
  3. Enable SR-IOV and VFs. Each VF has 1 AFU Port:
    sudo sh -c "echo 1 > $pci_path/sriov_numvfs"
  4. Find the additional Device number for the VF Device:
    lspci -nn | grep :0b2[bc]
    Sample output:
    04:00.0 Processing accelerators [1200]: Intel Corporation Device [8086:0b2b]
    04:00.1 Processing accelerators [1200]: Intel Corporation Device [8086:0b2c]

    lspci shows an additional Device number, 0b2c. This is the VF Device you assign to a VM. The original Bus and Device numbers for the PF remain 0b2b.

    Note that the Domain:Bus:Device.Function (BDF) notation for the VF device is: 000:04:00.1

  5. Load the vfio-pci driver:
    sudo modprobe vfio-pci
  6. Unbind the VF Device from its driver:
    sudo sh -c "echo 0000:04:00.1 > \
    /sys/bus/pci/devices/0000:04:00.1/driver/unbind"
  7. Find the Vendor ID and Device ID for the VF Device:
    lspci -n -s 04:00.1

    Expected output:

    04:00.1 1200: 8086:0b2c
  8. Bind the VF to the vfio-pci driver: Use the Vendor ID and Device ID from previous step.
    sudo sh -c "echo 8086 0b2c > \
    /sys/bus/pci/drivers/vfio-pci/new_id"