Multi Channel DMA Intel® FPGA IP for PCI Express Design Example User Guide

Download
ID 683517
Date 4/29/2022
Public
Document Table of Contents

3.5.2.3.1.3. Set the Boot Parameters

Follow the steps below to modify the default hugepages setting in the grub files:
  1. Edit the /etc/default/grub file

    Append the highlighted parameters to the GRUB_CMDLINE_LINUX line in the /etc/default/grub file

    GRUB_CMDLINE_LINUX=" rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb default_hugepagesz=1G hugepagesz=1G hugepages=40 panic=1 intel_iommu=on iommu=pt

    This is what the file looks like after the edit:

    GRUB_TIMEOUT=5

    GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"

    GRUB_DEFAULT=saved

    GRUB_DISABLE_SUBMENU=true

    GRUB_TERMINAL_OUTPUT="console"

    GRUB_CMDLINE_LINUX=" rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb default_hugepagesz=1G hugepagesz=1G hugepages=40 panic=1 intel_iommu=on iommu=pt

    GRUB_DISABLE_RECOVERY="true"

    In the case of memory allocation failure at the time of Virtual Function creation, add the following boot parameters: "pci=hpbussize=10,hpmemsize=2M,nocrs,realloc=on"

    To bind the device to vfio-pci and use IOMMU, enable the following parameter: intel_iommu=on

    To use UIO and not enable the IOMMU lookup, add the following parameter: iommu=pt

    To use the AMD platform and the UIO driver, add the following parameter at boot time: iommu=soft

  2. Generate GRUB configuration files.

    To check whether the boot system is legacy or EFI-based, check the existence of the following file:

    $ls -al /sys/firmware/efi

    If this file is present, the boot system is EFI-based. Otherwise, it is a legacy system.
    1. In case of a legacy system, execute the following command:

      $ grub2-mkconfig -o /boot/grub2/grub.cfg

    2. In case of an EFI-based system, execute the following command:

      $ grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg

  3. Reboot the system.
  4. Verify the changes above:

    $ cat /proc/cmdline

  5. Set the huge pages:

    $ echo 40 > /proc/sys/vm/nr_hugepages

  6. If host supports multiple NUMAs, follow the following steps:
    1. Check how many NUMAs enabled on host.
      $lscpu | grep NUMA
      NUMA node(s): 2
      NUMA node0 CPU(s): 0-15, 32-47
      NUMA node1 CPU(s): 16-31, 48-63

      In this example, we have 2 NUMAs. If only one NUMA is present, ignore this step:

    2. Check which device is provisioned:
      $cat /sys/class/pci_bus/<your device>\
      /device/numa_node
    3. Enable the Huge pages, which ever NUMA, device is located:
      $echo 40> /sys/devices/system/node/node<nodenum>\
      /hugepages/hugepages-1048576kB/nr_hugepages
    4. Configure thread sequence based on which NUMA device is located, for example:
      #define THREAD_SEQ "0-15" in software/user/cli/\
      perfq_app/perfq_app.h 

Did you find the information on this page useful?

Characters remaining:

Feedback Message