AN-811: Using the Avery BFM for PCI Express Gen3x16 Simulation on Intel Stratix 10 Devices
AN-811: Using the Avery BFM for PCI Express Gen3x16 Simulation on Intel Intel Stratix 10 Devices
Intel® Stratix® 10 devices support PCI Express Hard IP modes up to Gen3x16. Simulating Gen3x16 requires using a third-party root complex bus functional model (BFM). This document describes how to set up a simulation using a third-party BFM. The application note focuses on an Avery BFM and is targeted for the Mentor ModelSim and Synopsys VCS simulators. You can adapt these steps to other third-party BFMs and other simulation software. Third-party BFMs are only supported for Gen3x16 simulation. For modes up to Gen3x8, follow the instructions in the Stratix 10 Avalon-ST Interface for PCIe Solutions User Guide.
Software Requirements
To perform this simulation, you must have the following:
- Intel® Quartus® Prime Pro Edition software version 17.1 or later software
- Mentor Graphics® ModelSim® SE software version 10.3c or later or Synopsys® VCS software version J-2014.12 or later
- Avery BFM software version 1.8d or later
To obtain a download account for the Avery BFM, contact an Avery sales representative.
Design Components
High-level Overview of Steps
Create the Example Design
Follow the steps in the Stratix 10 Avalon-ST Interface for PCIe Solutions User Guide to parameterize a Gen3x16 Avalon-ST Stratix 10 Hard IP for PCI Express instance.
- Select PIO from the Select design drop-down menu in the Example Designs tab.
- Select the Simulation option.
-
Select Third-party BFM
from the Select simulation Root Complex
BFM menu. This menu selection is only available when you select
the Gen3x16 mode in the IP
Settings tab.
Figure 3. Example Designs tab
-
Click Generate Example Design and choose a destination
folder for the design.
This folder location will be referred to as the <Example_Design_Directory>. The default folder name for the Intel® Stratix® 10 example design is pcie_s10_ast_0_example_design.
Setup the Simulation Environment
Download and Extract the Avery Simulation Scripts
- Download the Avery simulation scripts referenced below.
- Copy the Avery_sim_script.zip file to the <Example_Design_ Directory>/pcie_example_design_tb/pcie_example_design_tb directory.
- Navigate to the <Example_Design_ Directory>/pcie_example_design_tb/pcie_example_design_tb directory and unzip Avery_sim_script.zip.
Archive Element | Description |
---|---|
pcie_example_design_tb.sv | This file is a replacement for the top-level simulation RTL file generated by the Intel® Quartus® Prime Pro Edition software. It removes the instantiation of the Intel FPGA root complex BFM, and adds the Avery root complex BFM. Additionally it changes the file format from Verilog to System Verilog, which is important for integrating with the Avery BFM in the VCS simulation. |
apci_top_rc.sv | This file is the top-level wrapper for the Avery root complex BFM. |
vcs/ | This folder contains:
|
modelsim/ | This folder contains:
|
Create the Avery BFM File List
VCS
The file is called vcs/avery_files_vcs.f. Open this file in a text editor and confirm that the file paths match those in your setup. By default, you should not have to make any changes to this file.
ModelSim
The file is called modelsim/avery_files_ms.f. Open this file in a text editor and confirm that the file paths match those in your setup. By default, you should not have to make any changes to this file.
Create the Example Design File List
VCS
However, many file names are uniquely generated when you create the example design and they must be transferred to the example design file list, so you must add the file names yourself.
- In a text editor, open vcs/pcie_example_design_tb.f.
- Open <Example_Design_Directory>/pcie_example_design_tb/pcie_example_design_tb/sim/synopsys/vcs/vcs_setup.sh. This file contains a list of all design files that need to be compiled for simulation.
- Scroll through vcs_setup.sh until you locate the file list. The first file in this list should be altera_primitives.v and the last should be pcie_example_design_tb.v.
- Copy this list into the marked area of
vcs/pcie_example_design_tb.f.
Figure 4. Marked Area of vcs/pcie_example_design_tb.f
- Close vcs_setup.sh.
- Remove the trailing \ character from the end of every line in the copied file list of the pcie_example_design_tb.f.
- Remove the following final four files from the list:
- altpcie_s10_tbed_hwtcl.v
- altpcied_s10_hwtcl.sv
- DUT_pcie_tb_ip.v
- pcie_example_design_tb.v
- In place of these files add $QSYS_SIMDIR/pcie_example_design_tb.sv. This is the new top-level file provided with the Avery simulation scripts that instantiates the Avery BFM.
- Save and close pcie_example_design_tb.f.
- In a text editor, open the Avery_sim_script/vcs/vcstest.sh.
- In line 8, specify your Intel® Quartus® Prime Pro Edition software installation path using QUARTUS_INSTALL_DIR environment variable. For example, export QUARTUS_INSTALL_DIR="/tools/acds/17.1/240/linux64/quartus".
- Save and close vcstest.sh.
ModelSim
An example TCL script is included with the Avery simulation scripts (modelsim/msim_setup_avery.tcl). However, many file names are uniquely generated when you create the example design, so you must modify this TCL script to incorporate the correct file names.
- Open the modelsim/msim_setup_avery.tcl in a text editor.
- Open the auto-generated msim_setup.tcl file in a text editor. By default this file is located at <Example_Design_Directory>/pcie_example_design_tb/pcie_example_design_tb/sim/mentor
- Locate the alias com sections of both msim_setup_avery.tcl and msim_setup.tcl. These sections are responsible for compiling all necessary design files.
- Copy the alias com section of msim_setup.tcl and paste it into the alias com section of msim_setup_avery.tcl.
- Close msim_setup.tcl.
- Remove the following final four files from the alias com section of
msim_setup_avery.tcl:
- altpcie_s10_tbed_hwtcl.v
- altpcied_s10_hwtcl.sv
- DUT_pcie_tb_ip.v
- pcie_example_design_tb.v
- Save and close msim_setup_avery.tcl.
Configure the Avery BFM
VCS
If you wish to run the simulation in serial mode or to make any other configuration changes, open vcs/vcstest.sh.
To run the simulation in serial mode, add +define+APCI_NEW_PHY immediately before +plusarg_save in the vcs command in Line 12.
ModelSim
- Open modelsim/mentor.do in a text editor.
- Modify TOP_LEVEL_NAME to match your project. By default, an example design generated by Platform Designer will have a top level module name of pcie_example_design_tb.pcie_example_design_tb. Only modify this variable if you have changed the name of the top-level file.
- Modify QSYS_SIMDIR to match the path to your project’s simulation directory. This can be either an absolute path or a relative path. The default value assumes that you unzipped the Avery simulation files to the <Example_Design_Directory>/pcie_example_design_tb/pcie_example_design_tb directory. If you unzipped them elsewhere or made any other changes to the structure or naming of the project, then you must change this path.
- Modify AVERY_PCIE to match the path to the Avery BFM.
- Modify QUARTUS_INSTALL_DIR to match the path to your Intel® Quartus® Prime Pro Edition software installation.
- Modify AVERY_PLI to match the path to the Avery PLI library.
- Modify USER_DEFINED_ELAB_OPTIONS. By default, these options reference the PLI library and instruct elaboration to wait for an available Avery license. Modify this variable only if you need to make changes to these options.
- Modify USER_DEFINED_COMPILE_OPTIONS. By default, PIPE simulation is enabled. To enable serial simulation, append +define+APCI_NEW_PHY to the user compile options. For example, set the variable to "+define+APCI_DUMP_WLF+define+APCI_NEW_PHY".
Testbench Top-level File
It has the same name as the old top-level, but instantiates the Avery BFM in place of the Intel FPGA BFM. Additionally, the file format is changed from Verilog to System Verilog to facilitate integration with the Avery BFM.
For the VCS simulator, you should have added the new top-level to the file list in VCS. For the ModelSim simulator, the script, mentor.do, compiles the new top-level separately after compiling all other design files.
Run the Simulation
VCS
- In a terminal with Intel® Quartus® Prime, VCS, and Avery resources, navigate to <Example_Design_Directory>/pcie_example_design_tb/pcie_example_design_tb/Avery_sim_script/vcs .
- Execute the command, /bin/sh vcstest.sh.
ModelSim
- In a terminal with Intel® Quartus® Prime, ModelSim SE, and Avery resources, navigate to <Example_Design_Directory>/pcie_example_design_tb/pcie_example_design_tb/Avery_sim_script/modelsim.
- Execute the command, vsim -c -do
mentor.do.
Note: Omitting the -c option opens the ModelSim GUI.
View Results
After running the simulation, the simulator should output results indicating whether or not the test passed, as well as the RX and TX bandwidth.

You can also view the waveforms by opening apci_top.vpd (VCS) or vsim.wlf (ModelSim).

Document Revision History
Date | Version | Changes |
---|---|---|
January 2018 | 2018.01.23 | Made the following changes:
|
May 2017 | 2017.05.30 | Made the following changes:
|
May 2017 | 2017.05.08 | Initial release. |