Intel^® Stratix^® 10 devices support PCI Express Hard IP modes up to Gen3x16. Simulating Gen3x16 requires using a third-party root complex bus functional model (BFM). This document describes how to set up a simulation using a third-party BFM. The application note focuses on an Avery BFM and is targeted for the Mentor ModelSim and Synopsys VCS simulators. You can adapt these steps to other third-party BFMs and other simulation software. Third-party BFMs are only supported for Gen3x16 simulation. For modes up to Gen3x8, follow the instructions in the Stratix 10 Avalon-ST Interface for PCIe Solutions User Guide.

To perform this simulation, you must have the following:

• Intel^® Quartus^® Prime Pro Edition software version 17.1 or later software
• Mentor Graphics^® ModelSim^® SE software version 10.3c or later or Synopsys^® VCS software version J-2014.12 or later
• Avery BFM software version 1.8d or later

To obtain a download account for the Avery BFM, contact an Avery sales representative.

1. Download the Avery simulation scripts referenced below.
2. Copy the Avery_sim_script.zip file to the <Example_Design_ Directory>/pcie_example_design_tb/pcie_example_design_tb directory.
3. Navigate to the <Example_Design_ Directory>/pcie_example_design_tb/pcie_example_design_tb directory and unzip Avery_sim_script.zip.

The file is called vcs/avery_files_vcs.f. Open this file in a text editor and confirm that the file paths match those in your setup. By default, you should not have to make any changes to this file.

The file is called modelsim/avery_files_ms.f. Open this file in a text editor and confirm that the file paths match those in your setup. By default, you should not have to make any changes to this file.

However, many file names are uniquely generated when you create the example design and they must be transferred to the example design file list, so you must add the file names yourself.

1. In a text editor, open vcs/pcie_example_design_tb.f.
2. Open <Example_Design_Directory>/pcie_example_design_tb/pcie_example_design_tb/sim/synopsys/vcs/vcs_setup.sh. This file contains a list of all design files that need to be compiled for simulation.
3. Scroll through vcs_setup.sh until you locate the file list. The first file in this list should be altera_primitives.v and the last should be pcie_example_design_tb.v.
4. Copy this list into the marked area of vcs/pcie_example_design_tb.f.
   Figure 4. Marked Area of vcs/pcie_example_design_tb.f

   
5. Close vcs_setup.sh.
6. Remove the trailing \ character from the end of every line in the copied file list of the pcie_example_design_tb.f.
7. Remove the following final four files from the list:
   • altpcie_s10_tbed_hwtcl.v
   • altpcied_s10_hwtcl.sv
   • DUT_pcie_tb_ip.v
   • pcie_example_design_tb.v
   These files all instantiate the Intel FPGA root complex BFM and will be replaced by Avery files.
8. In place of these files add $QSYS_SIMDIR/pcie_example_design_tb.sv. This is the new top-level file provided with the Avery simulation scripts that instantiates the Avery BFM.
9. Save and close pcie_example_design_tb.f.
10. In a text editor, open the Avery_sim_script/vcs/vcstest.sh.
11. In line 8, specify your Intel^® Quartus^® Prime Pro Edition software installation path using QUARTUS_INSTALL_DIR environment variable. For example, export QUARTUS_INSTALL_DIR="/tools/acds/17.1/240/linux64/quartus".
12. Save and close vcstest.sh.

An example TCL script is included with the Avery simulation scripts (modelsim/msim_setup_avery.tcl). However, many file names are uniquely generated when you create the example design, so you must modify this TCL script to incorporate the correct file names.

1. Open the modelsim/msim_setup_avery.tcl in a text editor.
2. Open the auto-generated msim_setup.tcl file in a text editor. By default this file is located at <Example_Design_Directory>/pcie_example_design_tb/pcie_example_design_tb/sim/mentor
3. Locate the alias com sections of both msim_setup_avery.tcl and msim_setup.tcl. These sections are responsible for compiling all necessary design files.
4. Copy the alias com section of msim_setup.tcl and paste it into the alias com section of msim_setup_avery.tcl.
5. Close msim_setup.tcl.
6. Remove the following final four files from the alias com section of msim_setup_avery.tcl:
   • altpcie_s10_tbed_hwtcl.v
   • altpcied_s10_hwtcl.sv
   • DUT_pcie_tb_ip.v
   • pcie_example_design_tb.v
   These files all instantiate the Intel FPGA root complex BFM and will be replaced by Avery files.
7. Save and close msim_setup_avery.tcl.

If you wish to run the simulation in serial mode or to make any other configuration changes, open vcs/vcstest.sh.

To run the simulation in serial mode, add +define+APCI_NEW_PHY immediately before +plusarg_save in the vcs command in Line 12.

1. Open modelsim/mentor.do in a text editor.
2. Modify TOP_LEVEL_NAME to match your project. By default, an example design generated by Platform Designer will have a top level module name of pcie_example_design_tb.pcie_example_design_tb. Only modify this variable if you have changed the name of the top-level file.
3. Modify QSYS_SIMDIR to match the path to your project’s simulation directory. This can be either an absolute path or a relative path. The default value assumes that you unzipped the Avery simulation files to the <Example_Design_Directory>/pcie_example_design_tb/pcie_example_design_tb directory. If you unzipped them elsewhere or made any other changes to the structure or naming of the project, then you must change this path.
4. Modify AVERY_PCIE to match the path to the Avery BFM.
5. Modify QUARTUS_INSTALL_DIR to match the path to your Intel^® Quartus^® Prime Pro Edition software installation.
6. Modify AVERY_PLI to match the path to the Avery PLI library.
7. Modify USER_DEFINED_ELAB_OPTIONS. By default, these options reference the PLI library and instruct elaboration to wait for an available Avery license. Modify this variable only if you need to make changes to these options.
8. Modify USER_DEFINED_COMPILE_OPTIONS. By default, PIPE simulation is enabled. To enable serial simulation, append +define+APCI_NEW_PHY to the user compile options. For example, set the variable to "+define+APCI_DUMP_WLF+define+APCI_NEW_PHY".

It has the same name as the old top-level, but instantiates the Avery BFM in place of the Intel FPGA BFM. Additionally, the file format is changed from Verilog to System Verilog to facilitate integration with the Avery BFM.

For the VCS simulator, you should have added the new top-level to the file list in VCS. For the ModelSim simulator, the script, mentor.do, compiles the new top-level separately after compiling all other design files.

1. In a terminal with Intel^® Quartus^® Prime, VCS, and Avery resources, navigate to <Example_Design_Directory>/pcie_example_design_tb/pcie_example_design_tb/Avery_sim_script/vcs .
2. Execute the command, /bin/sh vcstest.sh.

1. In a terminal with Intel^® Quartus^® Prime, ModelSim SE, and Avery resources, navigate to <Example_Design_Directory>/pcie_example_design_tb/pcie_example_design_tb/Avery_sim_script/modelsim.
2. Execute the command, vsim -c -do mentor.do.
   Note: Omitting the -c option opens the ModelSim GUI.

After running the simulation, the simulator should output results indicating whether or not the test passed, as well as the RX and TX bandwidth.

Figure 5. Simulator Results

You can also view the waveforms by opening apci_top.vpd (VCS) or vsim.wlf (ModelSim).

Figure 6. ModelSim WaveformLink-training process up to Gen3x16 using the Avery root complex BFM