The Intel^® Arria^® 10 GX FPGA Development Kit Reference Platform targets a subset of the hardware features available in the Intel^® Arria^® 10 GX FPGA Development Kit.

Figure 1. Hardware Features of the Intel^® Arria^® 10 GX FPGA Development Kit

Features of the a10_ref Reference Platform:

• OpenCL Host

  The a10_ref Reference Platform uses a PCIe-based host that connects to the Intel^® Arria^® 10 PCIe Gen3x8 hard IP core.

• OpenCL Global Memory

  The hardware provides one 2-gigabyte (GB) DDR4 SDRAM daughtercard that is mounted on the HiLo connector (J14 in Figure 1).

• FPGA Programming via one of the following methods:
  • Partial Reconfiguration (PR) over PCIe* .
  • External cable and the Intel^® Arria^® 10 GX FPGA Development Kit's on-board Intel^® FPGA Download Cable II interface.
  • External Intel FPGA Download Cable II interface connected to a 10-pin JTAG header.
• Guaranteed Timing

  The a10_ref Reference Platform relies on the Intel^® Quartus^® Prime Pro Edition compilation flow to provide guaranteed timing closure. The timing-clean a10_ref Reference Platform is preserved in the form of a precompiled post-fit netlist (that is, the base.qdb Intel^® Quartus^® Prime Database Export File). The Intel^® FPGA SDK for OpenCL™ Offline Compiler imports this preserved post-fit netlist into each OpenCL kernel compilation.

• OpenCL Host Pipe

  Using direct memory access (DMA) in Intel^® Arria^® 10 PCIe* Gen3 x8 hard IP core a10gx_hostch board variant has a direct host to kernel and kernel to host pipe.

To compile your OpenCL kernel for a specific board variant, include the -board=<board_name> option in your aoc command (for example, aoc -board=a10gx myKernel.cl).

Dependencies

• Intel^® Arria^® 10 PCIe* hard IP core
• Parameter Settings section of the Intel^® Arria^® 10 Avalon^® -MM DMA Interface for PCIe Solutions User Guide

The kernel driver uses the Vendor ID, Subsystem Vendor ID and the Subsystem Device ID to identify the boards it supports. The SDK's programming flow checks the Device ID to ensure that it programs a device with a .aocx Intel^® FPGA SDK for OpenCL™ Offline Compiler executable file targeting that specific device.

The version ID for the a10_ref Reference Platform is A0C7C1E6 in 17.1 release.

Before communicating with any part of the FPGA system, the host first reads from this version_id register to confirm the following:

• The PCIe can access the FPGA fabric successfully
• The address map matches the map in the MMD software

Update the VERSION_ID parameter in the version_id component to a new value with every slave addition or removal from the PCIe BAR 4 bus, or whenever the address map changes.

The two header files that describe the hardware design to the software are in the following locations:

• For Windows systems, the header files are in the INTELFPGAOCLSDKROOT\board\a10_ref\source\include folder, where INTELFPGAOCLSDKROOT is the path to the SDK installation.
• For Linux systems, the header files are in the INTELFPGAOCLSDKROOT/board/a10_ref/linux64/driver directory.

This driver is installed using the Intel^® FPGA SDK for OpenCL™ install utility.

The a10_ref Reference Platform

• For Windows systems, the driver is in the <path_to_al0pciedk>\windows64\driver folder.

  The kernel driver, the WinDriver application programming interface (API), is a third-party driver from Jungo Connectivity Ltd. For more information about the WinDriver, refer to the Jungo Connectivity Ltd. website or contact a Jungo Connectivity representative.

• For Linux, an open-source MMD-compatible kernel driver is in the <path_to_al0pciedk>/linux64/driver directory. The table below highlights some of the files that are available in this directory.

Hardware Considerations

The instantiation process exports the DMA controller slave ports (that is, rd_dts_slave and wr_dts_slave) and master ports (that is, rd_dcm_master and wr_dcm_master) into the PCIe module. Two additional master ports, dma_rd_master and dma_wr_master, are exported for DMA read and write operations, respectively. For the DMA interface to function properly, all these ports must be connected correctly in the board.qsys Platform Designer system, where the PCIe hard IP is instantiated.

At the start of DMA transfer, the DMA Descriptor Controller reads from the DMA descriptor table in user memory, and stores the status and the descriptor table into a FIFO address. There are two FIFO addresses: Read Descriptor FIFO address and Write Descriptor FIFO address. After storing the descriptor table into a FIFO address, DMA transfers into the FIFO address can occur. The dma_rd_master port, which moves data from user memory to the device, must connect to the rd_dts_slave and wr_dts_slave ports. Because the dma_rd_master port connects to DDR4 memory also, the locations of the rd_dts_slave and wr_dts_slave ports in the address space must be defined in the hw_pcie_dma.h file.

The rd_dcm_master and wr_dcm_master ports must connect to the txs port. At the end of the DMA transfer, the DMA controller writes the MSI data and the done status into the user memory via the txs slave. The txs slave is part of the PCIe* hard IP in board.qsys.

All modules that use DMA must connect to the dma_rd_master and dma_wr_master ports. For DDR4 memory connection, Intel^® recommends implementing an additional pipeline to connect the two 256-bit PCIe DMA ports to the 512-bit memory slave. For more information, refer to the DDR4 Connection to PCIe* Host section.

Software Considerations

The MMD layer uses DMA to transfer data if it receives a data transfer request that satisfies both of the following conditions:

• A transfer size that is greater than 1024 bytes
• The starting addresses for both the host buffer and the device offset are aligned to 64 bytes

Two different modules generate the signal for the MSI line. The DMA controller in the PCIe hard IP core generates the DMA's MSI. The PCI Express interrupt request (IRQ) module (that is, the INTELFPGAOCLSDKROOT/board/a10_ref/hardware/a10gx/ip/irq_controller directory) generates the kernel interface's MSI.

For more information on the PCI Express IRQ module, refer to Handling PCIe Interrupts webpage.

Hardware Considerations

In INTELFPGAOCLSDKROOT/board/a10_ref/hardware/a10gx/board.qsys, the DMA MSI is connected internally; however, you must connect the kernel interface interrupt manually. For the kernel interface interrupt, the PCI Express IRQ module is instantiated as pcie_irq_0 in board.qsys. The kernel interface interrupts connections are as follows:

• The kernel_irq_to_host port from the OpenCL Kernel Interface (kernel_interface) connects to the interrupt receiver, which allows the OpenCL kernels to signal the PCI Express IRQ module to send an MSI.
• The PCIe hard IP's msi_intfc port connects to the MSI_Interface port in the PCI Express IRQ module. The kernel interface interrupt receives the MSI address and the data necessary to generate the interrupt via msi_intfc.
• The IRQ_Gen_Master port on the PCI Express IRQ module, which is used to write the MSI, connects to the txs port on the PCIe hard IP.
• The IRQ_Read_Slave and IRQ_Mask_Slave ports connect to the pipe_stage_host_ctrl module on Bar 4. After receiving an MSI, the user driver can read the IRQ_Read_Slave port to check the status of the kernel interface interrupt, and read the IRQ_Mask_Slave port to mask the interrupt.

Software Considerations

The interrupt service routine in the Linux driver checks which module generates the interrupt. For the DMA's MSI, the driver reads the DMA descriptor table's status bit in local memory, as specified in the Read DMA Example section of the Intel^® Arria^® 10 Avalon-MM DMA Interface for PCIe Solutions User Guide. For kernel interface's MSI, the driver reads the interrupt line sent by the kernel interface.

The interrupt service routine involves the following tasks:

1. Check DMA status on the DMA descriptor table.
2. Read the kernel status from the IRQ_READ_SLAVE port on the PCI Express IRQ module.
3. If a kernel interrupt was triggered, mask the interrupt by writing to the IRQ_MASK_SLAVE port on the PCI Express IRQ module. Then, execute the kernel interrupt service routine.
4. If a DMA interrupt was triggered, reset the DMA descriptor table and execute the DMA interrupt service routine.
5. If applicable, unmask a masked kernel interrupt.

You can only use PR when the static board interface, generated during base compilations, matches the static region of the design that is used to compile the OpenCL kernel's PR region.

For Windows MMD implementation, the INTELFPGAOCLSDKROOT\board\a10_ref\source\host\mmd\acl_pcie_config.cpp file contains the MMD code that communicates with the PR configuration controller within the static region of the design. The program_core_with_PR_file function within the acl_pcie_config.cpp file requires a handle to the PR bitstream and the length of the PR bitstream in order to perform the PR operation.

For Linux driver implementation, the INTELFPGAOCLSDKROOT/board/a10_ref/linux64/driver/aclpci_pr.c file includes the main host driver routine that communicates with the PR configuration controller within the static region of the design. The aclpci_pr function within the acl_pci_pr.c file requires the following information in order to perform the PR operation:

• A handle to the board
• A handle to the PR bitstream
• The length of the PR bitstream

After verifying that the device is opened, the bitstream is of adequate length, and the PCIe endpoint of the device is reachable, the aclpci_pr function writes 0x1 to the PR IP status register. Then, the aclpci_pr function writes the complete bitstream, 32 bits at a time, to the PR IP. After the bitstream transfer is complete, the aclpci_pr function performs a read operation to the PR IP status register to verify whether PR is successful. A return value of 0x14 indicates a successful PR operation; any other return value indicates an error.

To override the default reconfiguration mechanism, set the ACL_PCIE_USE_JTAG_PROGRAMMING environment variable, as shown below:

• For Windows, type set ACL_PCIE_USE_JTAG_PROGRAMMING=1 at the command prompt.
• For Linux, type export ACL_PCIE_USE_JTAG_PROGRAMMING=1 at the command prompt.

Setting ACL_PCIE_USE_JTAG_PROGRAMMING specifies that JTAG full-chip configuration is the default mechanism for reconfiguring the device.

The Intel^® Arria^® 10 GX FPGA Development Kit Reference Platform automatically tries to detect the cable by default when programming the FPGA via the Intel FPGA Download Cable.

You can set the ACL_PCIE_JTAG_CABLE or ACL_PCIE_JTAG_DEVICE_INDEX environment variables to disable the auto-detect feature and use values that you define.

Cable autodetect is useful when you have multiple devices connected to a single host and PR cannot be used to program the FPGA.

The memory-mapped device (MMD) uses in-system sources and probes to identify the cable connected to the target board. You must instantiate the cade_id register block and connect it to Bar 4 with the correct address map. You must also instantiate board_in_system_sources_probes_cade_id, which is an in-system sources and probe component, and connect it to cade_id register.

The MMD must be updated to take in the relevant changes. Add the scripts/find_jtag_cable.tcl script to your custom platform.

When the FPGA is being programmed via the Intel FPGA Download Cable, the MMD invokes quartus_stp to execute the find_jtag_cable.tcl script. The script identifies the cable and index number which is then used to program the FPGA through the quartus_pgm command.

The streaming interface makes use of DMA.

These ports must be connected to the DMA descriptor controller in the host channel IP.

Base address offset must match the address mentioned in the board.qsys file, since these addresses are used by the internal descriptor controller of a10gx board variant.

For one FIFO, the in port should be connected to the stout port of the host channel top, while the out port is exported. For the second FIFO, out port should be connected to the stin port of the host channel top, while the in port is exported.

In the board_spec.xml, the host channel ports are IO ports.

In the current version of the a10_ref Reference Platform, all Platform Designer components related to the DDR4 global memory are now part of the INTELFPGAOCLSDKROOT/board/a10_ref/hardware/a10gx/acl_ddr4_a10.qsys Platform Designer subsystem within board.qsys. In addition, the location of the clock domain crossings has changed to increase the number of blocks operating in the slower PCIe* domain. With this modified structure, you can add multiple memories with different clock domains to the system.

If you have a Custom Platform that is ported from a previous version of the a10_ref Reference Platform, you have the option to modify your Custom Platform as described above. This modification is not mandatory.

Dependencies

DDR4 external memory interfaces

For more information on the DDR4 external memory interface IP, refer to the DDR2, DDR3, and DDR4 SDRAM Board Design Guidelines section in External Memory Interface Handbook Volume 2: Design Guidelines.

The DDR4 IP core has one bank where its width and address configurations match those of the DDR4 SDRAM. Intel^® tunes the other parameters such as burst size, pending reads, and pipelining. These parameters are customizable for an end application or board design.

The Avalon^® master interfaces from the OpenCL Memory Bank Divider component connect to their respective memory controllers. The Avalon^® slave connects to the PCIe* and DMA IP core. Implementations of appropriate clock crossing and pipelining are based on the design floorplan and the clock domains specific to the computing card. The OpenCL Memory Bank Divider section in the Intel^® FPGA SDK for OpenCL™ Custom Platform Toolkit User Guide specifies the connection details of the snoop and memorg ports.

The INTELFPGAOCLSDKROOT/board/a10_ref/hardware/a10gx/board.qsys Platform Designer system uses a custom UniPHY Status to AVS IP component to aggregate different UniPHY status conduits into a single Avalon^® slave port named s. This slave port connects to the pipe_stage_host_ctrl component so that the PCIe host can access it.

A clock crosser is necessary because the kernel interface for the compiler must be clocked in the kernel clock domain. In addition, the width, address width, and burst size characteristics of the kernel interface must match those specified in the OpenCL Memory Bank Divider connecting to the host. Appropriate pipelining also exists between the clock crosser and the memory controller.

The OpenCL Kernel Interface component exports an Avalon^® master interface (kernel_cra) that connects to this slave port. The OpenCL Kernel Interface component also generates the kernel reset (kernel_reset) that resets all logic in the kernel clock domain.

The Intel^® Arria^® 10 FPGA Development Kit Reference Platform has one DDR4 memory bank. As a result, the Reference Platform instantiates the OpenCL Kernel Interface component and sets the Number of global memory systems parameter to 1.

Examples of design complexities:

• Designing a robust reset sequence
• Establishing a design floorplan
• Managing global routing
• Pipelining

Optimizations of these design complexities occur in tandem with one another to meet timing and board hardware optimization requirements.

These clock domains include:

• 250 MHz PCIe* clock
• 300 MHz DDR4 clock
• 50 MHz general clock (config_clk)
• 125 MHz kernel reference clock
• Kernel clock that can have any clock frequency

With the exception of the kernel clock, the a10_ref Reference Platform is responsible for the timing closure of these clocks. However, because the board design must clock cross all interfaces in the kernel clock domain, the board design also has logic in the kernel clock domain. It is crucial that this logic is minimal and achieves an F_max higher than typical kernel performance.

These reset drivers include:

• The por_reset_counter in the INTELFPGAOCLSDKROOT/board/a10_ref/hardware/a10gx/board.qsys Platform Designer system implements the power-on-reset. The power-on-reset resets all the hardware on the device by issuing a reset for a number of cycles after the FPGA completes configuration.
• The PCIe^® bus issues a perst reset that resets all hardware on the device.
• The OpenCL™ Kernel Interface component issues the kernel_reset that resets all logic in the kernel clock domain.

The power-on-reset and the perst reset are combined into a single global_reset; therefore, there are only two reset sources in the system (that is, global_reset and kernel_reset). However, these resets are explicitly synchronized across the various clock domains, resulting in several reset interfaces.

Important Considerations Regarding Resets

• Synchronizing resets to different clock domains might cause several high fan-out resets.

  Platform Designer automatically synchronizes resets to the clock domain of each connected component. In doing so, the Platform Designer instantiates new reset controllers with derived names that might change when the design changes. This name change makes it difficult to make and maintain global clock assignments to some of the resets. As a result, for each clock domain, there are explicit reset controllers. For example, global_reset drives reset_controller_pcie and reset_controller_ddr4. However, they are synchronized to the PCIe and DDR4 clock domains, respectively.

• Resets and clocks must work together to propagate reset to all logic.

  Resetting a circuit in a given clock domain involves asserting the reset over a number of clock cycles. However, your design may apply resets to the PLLs that generate the clocks for a given clock domain. This means a clock domain can hold in reset without receiving the clock edge that is necessary for synchronous resets. In addition, a clock holding in reset might prevent the propagation of a reset signal because it is synchronized to and from that clock domain. Avoid such situations by ensuring that your design satisfies the following criteria:

  • Generate the global_reset signal off the free-running config_clk.
  • The ddr4_calibrate IP resets the External Memory Interface controller separately.

• Apply resets to both reset interfaces of a clock-crossing bridge or FIFO component.

  FIFO content corruption might occur if only part of a clock-crossing bridge or a dual-clock FIFO component is reset. These components typically provide a reset input for each clock domain; therefore, reset both interfaces or none at all. For example, in the a10_ref Reference Platform, kernel_reset resets all the kernel clock-crossing bridges between DDR on both the m0_reset and s0_reset interfaces.

Dependencies

• Partial Reconfiguration
• Chip Planner
• Logic Lock Plus regions

Intel^® performed the following tasks iteratively to derive the floorplan of the a10_ref Reference Platform:

1. Compile a design without any region or floorplanning constraints.

   Intel^® recommends that you compile the design with several seeds.

2. Examine the placement of the IP cores (for example, PCIe* , DDR4, Avalon^® interconnect pipeline stages and adapters) for candidate locations, as determined by the Intel^® Quartus^® Prime Pro Edition software's Fitter. In particular, Intel^® recommends examining the seeds that meet or almost meet the timing constraints.

For the a10_ref Reference Platform, the PCIe* I/O is located in the lower left corner of the Intel^® Arria^® 10 FPGA. The DDR4 I/O is located on the top part of the left I/O column of the device. Because the placements of the PCIe and DDR4 IP components tend to be close to the locations of their respective I/Os, you can apply Logic Lock Plus regions to constrain the IP components to those candidate regions.

Figure 2. Floorplan of the Intel^® Arria^® 10 FPGA Development Kit Reference Platform

As shown in this Chip Planner view of the floorplan, the two Logic Lock Plus regions are spread out between the PCIe I/O and the top region of the left I/O column (that is, the DDR4 I/O area).

• The largest Logic Lock Plus region (Region 1) covers the PCIe I/O and contains most of the static board interface logic.
• Regions 2 contains an Avalon interconnect pipeline stage that bridges the PCIe I/O and DDR4 I/O regions. The Avalon interconnect pipeline stages also help improve the timing closure rate of the static board interface part of the design.

You must create a dedicated Logic Lock Plus region for the OpenCL™ kernel system. Furthermore, do not place kernel logic in the board's Logic Lock Plus regions (that is, static region). The static region and the OpenCL kernel system region (that is, PR region) do not overlap each other. As shown in Figure 2, the logic for the boardtest.cl OpenCL kernel, that is, the scatter area, can be placed anywhere except within the seven Logic Lock Plus regions.

Intel^® recommends the following strategies to maximize the available FPGA resources for the OpenCL kernel system to improve kernel routability:

• The OpenCL kernel system PR region should cover the entire device except the Logic Lock Plus regions of the board.
• The size of a Logic Lock Plus region should be just large enough to contain the board logic and to meet timing constraints of the board clocks. Oversized Logic Lock Plus regions consume FPGA resources unnecessarily.
• Avoid creating tightly-packed Logic Lock Plus regions that cause very high logic utilization and high routing congestion.

  High routing congestion within the Logic Lock Plus regions might decrease the Fitter's ability to route OpenCL kernel signals through the regions.

In the case where the board clocks are not meeting timing and the critical path is between the Logic Lock Plus regions (that is, across region-to-region gap), insert back-to-back pipeline stages on paths that cross the gap. For example, if the critical path is between Region 1 and Region 2, lock down the first pipeline stage (an Avalon-MM Pipeline Bridge component) to Region 1, lock down the second pipeline stage to Region 2, and connect the two pipeline stages directly. This technique ensures that pipeline registers are on both sides of the region-to-region gap, thereby minimizing the delay of paths crossing the gap.

Refer to the Pipelining section for more information.

There is no restriction on the placement location of the OpenCL™ kernel on the device. As a result, the kernel clocks and kernel reset must distribute high fan-out signals globally.

In the Platform Designer, you can implement pipelines via an Avalon^® -MM Pipeline Bridge component by setting the following pipelining parameters within the Avalon^® -MM Pipeline Bridge dialog box:

• Select Pipeline command signals
• Select Pipeline response signals
• Select both Pipeline command signals and Pipeline response signals

Examples of Pipeline Implementation

• Signals that traverse long distances because of the floorplan's shape or the region-to-region gaps require additional pipelines.

  The DMA at the bottom of the FPGA must connect to the DDR4 memory at the top of the FPGA. To achieve timing closure of the board interface logic at a DDR4 clock speed of 300 MHz, additional pipeline stages between the OpenCL™ Memory Bank Divider component and the DDR4 controller IP are necessary. In the Intel^® Arria^® 10 GX FPGA Development Kit Reference Platform's board.qsys Platform Designer system, the pipeline stages are named pipe_stage_ddr4a_dimm_*.

  The middle pipeline stage, pipe_stage_ddr4a_dimm, combines both the direct kernel DDR4 accesses and the accesses through the OpenCL Memory Bank Divider. The multistage pipeline approach ensures that the kernel entry point to the pipeline is neither geared towards the OpenCL Memory Bank Divider, which is close to the PCIe* IP core, nor the DDR4 IP core, which is at the very top of the FPGA.

The driver within the a10_ref Reference Platform can detect a failed calibration via the Uniphy Status to AVS IP, and retrigger calibration through the ddr4_calibrate IP block.

Dependencies

To ensure that the device functions properly during and after PR reprogramming, following these rules:

• Place a freeze wrapper around the PR region. The freeze wrapper holds the critical control outputs from the PR region in a known, inactive state during the reprogramming of the logic inside the PR region.

  The INTELFPGAOCLSDKROOT/board/a10_ref/hardware/a10gx/ip/freeze_wrapper.v file implements the freeze wrapper, where INTELFPGAOCLSDKROOT is the path to the SDK installation.

• Hold the kernel_reset_n signal, which is routed using Global Clock resources, in a logic 1 (deasserted) state during reprogramming of the PR region. When programming completes, assert the kernel_reset_n signal (that is, set it to the low state) before disabling the freeze wrapper. Asserting the kernel_reset_n signal resets all logic in the PR region to a known state. This assertion step is necessary because the state of all flipflops in the PR region is undefined after PR programming. The logic in the freeze_wrapper.v file implements the required behavior for the reset and freeze signals.

In both PR reprogramming and full-chip JTAG programming, the PLL is dynamically reconfigured by default after FPGA configuration completes. This default dynamic PLL reconfiguration step is unnecessary after full-chip programming because the correct PLL settings are already part of the .sof file programmed onto the FPGA over JTAG.

Both the SDK and the Custom Platform contribute to the implementation of the SDK's guaranteed timing closure feature.

The SDK provides the IP to generate the kernel clock, and a post-flow script that ensures this clock is configured with a safe operating frequency confirmed by timing analysis. The Custom Platform developer imports a post-fit netlist that has already achieved timing closure on all non-kernel clocks.

The REF_CLK_RATE parameter specifies the frequency of the reference clock that connects to the kernel PLL (pll_refclk). For the a10_ref Reference Platform, the REF_CLK_RATE frequency is 125 MHz.

The KERNEL_TARGET_CLOCK_RATE parameter specifies the frequency that the Intel^® Quartus^® Prime Pro Edition software attempts to achieve during compilation. The board hardware contains some logic that the kernel clock clocks. At a minimum, the board hardware includes the clock crossing hardware. To prevent this logic from limiting the Fmax achievable by a kernel, the KERNEL_TARGET_CLOCK_RATE must be higher than the frequency that a simple kernel can achieve on your device. For the Intel^® Arria^® 10 GX FPGA Development Kit that the a10_ref Reference Platform targets, the KERNEL_TARGET_CLOCK_RATE is 400 MHz.

In the import revision compilation, the compilation script import_compile.tcl invokes the INTELFPGAOCLSDKROOT/board/a10_ref/hardware/a10gx/scripts/post_flow.tcl Tcl script in the a10_ref Reference Platform after every Intel^® Quartus^® Prime Pro Edition software compilation using quartus_cdb.

The post_flow.tcl script also determines the kernel clock and configures it to a functional frequency.

Dependencies

Intel^® Quartus^® Prime Pro Edition compiler

Intel^® Quartus^® Prime software provides several mechanisms for preserving the placement and routing of some previously compiled logic and importing this logic into a new compilation. For Intel^® Arria^® 10 devices, the previously compiled logic is imported into the compilation flow.

The Intel^® Quartus^® Prime Pro Edition compilation flow can preserve the placement and routing of the board interface partition via the exported Intel^® Quartus^® Prime Archive File. The base.qdb file contains all the database files for the base compilation of root_partition. The a10_ref Reference Platform is configured with the project revisions and partitioning that are necessary to implement the compilation flow. By default, the SDK invokes the Intel^® Quartus^® Prime Pro Edition software on the top revision. This revision is configured to import and restore the base.qdb file, which has been pre-compiled and exported from a base revision compilation.

When developing your Custom Platform from the a10_ref Reference Platform, it is essential to maintain the flat.qsf, base.qsf, top.qsf, and top_synth.qsf Intel^® Quartus^® Prime Settings Files.

The a10_ref Reference Platform includes two additional partitions: the Top partition and the kernel_system partition. The Top partition contains all logic, and the kernel_system partition contains the logic in the PR region. The PR region is specified by the following assignments:

set_instance_assignment -name PARTIAL_RECONFIGURATION_PARTITION ON -to freeze_wrapper_inst|kernel_system_inst

The order of the application of time constraints is based on the order of appearance of the top.sdc and top_post.sdc files in the top.qsf file.

One noteworthy constraint in the a10_ref Reference Platform is the multicycle constraint for the kernel reset in the top_post.sdc file. Using global routing saves routing resources and provides more balanced skew. However, the delay across the global route might cause recovery timing issues that limit kernel clock speed. Therefore, it is necessary to include a multicycle path on the global reset signal.

For each hardware design, your Custom Platform also requires a board_spec.xml file for each hardware design that describes the hardware.

The following sections describe the implementation of these files for the Intel^® Arria^® 10 GX FPGA Development Kit Reference Platform.

Details of each field in the board_env.xml file are available in the Creating the board_env.xml File section of the Intel^® FPGA SDK for OpenCL™ Custom Platform Toolkit User Guide.

In the a10_ref Reference Platform, Intel^® uses the bin folder for Windows dynamic link libraries (DLLs), the lib directory for delivering libraries, and the libexec directory for delivering the SDK utility executables. This directory structure allows the PATH environment variable to point to the location of the DLLs (that is, bin) in isolation of the SDK utility executables.

Device

The device section contains the name of the device model file available in the INTELFPGAOCLSDKROOT/share/models/dm directory of the SDK and in the board spec.xml file. The used_resources element accounts for all logic outside of the kernel partition. The value of used_resources for alms equals the difference between the total number of adaptive logic modules (ALMs) used in final placement and the total number of ALMs available to the kernel partition. You can derive this value from the Partition Statistic section of the Fitter report after a compilation. Consider the following ALM categories within an example Fitter report:

+----------------------------------------------------------------------------------+
; Fitter Partition Statistics                                                      ;
+----------------------+-----------------+-----------------------------------------+
; Statistic            ; l               ; freeze_wrapper_inst|kernel_system_inst  ;
+----------------------+-----------------+-----------------------------------------+
; ALMs needed [=A-B+C] ; 0 / 427200 (0%) ; 0 / 385220 (0%)                         ;

The value of used_resources equals the total number of ALMs in l minus the total number of ALMs in freeze wrapper inst|kernel_system_inst. In the example above, used_resources = 427200 - 385220 = 41980 ALMs.

You can derive used_resources for rams and dsps in the same way using M20Ks and DSP blocks, respectively. The used_resources value for ffs is four times the used_resources value for alms because there are two primary and two secondary logic registers per ALM.

Global Memory

In the board_spec.xml file, there is one global_mem section for DDR memory. Assign the string DDR to the name attribute of the global_mem element. The board instance in Platform Designer provides all of these interfaces. Therefore, the string board is specified in the name attribute of all the interface elements within global_mem.

• DDR

  Because DDR memory serves as the default memory for the board that the a10_ref Reference Platform targets, its address attribute begins at zero. Its config_addr is 0x018 to match the memorg conduit used to connect to the corresponding OpenCL Memory Bank Divider for DDR.

  Attention: The width and burst sizes must match the parameters in the OpenCL Memory Bank Divider for DDR (memory_bank_divider).

Interfaces

The interfaces section describes kernel clocks, reset, CRA, and snoop interfaces. The OpenCL Memory Bank Divider for the default memory (in this case, memory_bank_divider) exports the snoop interface described in the interfaces section. The width of the snoop interface should match the width of the corresponding streaming interface.

In the order from the longest to the shortest configuration time, the three FPGA programming methods are as follows:

• To replace both the FPGA periphery and the core while maintaining the programmed state after power cycling, use Flash programming.
• To replace both the FPGA periphery and the core, use the Intel^® Quartus^® Prime Programmer command-line executable (quartus_pgm) to program the device via cables such as the Intel^® FPGA Download Cable (formerly USB-Blaster).
• To replace only the kernel portion of the device, use PR.

The default FPGA programming flow is to use PR over PCIe* . The Partial Reconfiguration Controller IP instantiates PR over PCIe* using the following IP parameter settings:

The 50 MHz config_clk clocks the Partial Reconfiguration Controller IP. The Avalon^® -MM interface connects to the host control bus on PCIe* BAR4. Using PCIe* Gen3x8 under these configuration settings, the duration of partial reconfiguration of the PR region is about 1.6 seconds.

You cannot use PR if there is a mismatch between the hash within the .aocx file and the hash in the static region of the current image on the FPGA. In this case, program the FPGA via Intel^® FPGA Download Cable by invoking quartus_pgm instead. If the .aocx file is not PR compatible with the current image on the FPGA, the Intel^® Quartus^® Prime Programmer displays the following message:

aocl program acl0 boardtest.aocx
aocl program: Running program from <path_to_a10_ref>/linux64/libexec
Reprogramming device with handle 1
MMD INFO : [acla10_ref0] PR base and import compile IDs do not match
MMD INFO : [acla10_ref0] PR base ID currently configured is 0x7d056bf2
MMD INFO : [acla10_ref0] PR import compile expects ID to be 0x30242eb9
mmd program_device: Board reprogram failed

Only use quartus_pgm via Intel^® FPGA Download Cable if you use a cable to connect the board and the host computer. Cabling is a point of potential failure, and it does not scale well to large deployments. If possible, reserve the quartus_pgm programming approach for development and testing purposes only.

If PR fails, an attempt is automatically made to detect the Intel^® FPGA Download Cable and do a full JTAG programming.

The source codes of an MMD library that demonstrates good performance are available in the INTELFPGAOCLSDKROOT/board/a10_ref/source/host/mmd directory. Refer to the Host-to-Device MMD Software Implementation section in the Stratix V Network Reference Platform Porting Guide for more information.

For more information on the MMD API functions, refer to the MMD API Descriptions section of the Intel^® FPGA SDK for OpenCL™ Custom Platform Toolkit User Guide.

Windows

The install.bat script is in the <your_custom_platform>\windows64\libexec directory, where <your_custom_platform> points to the top-level directory of your Custom Platform. This install.bat script triggers the install executable from Jungo Connectivity Ltd. to install the WinDriver on the host machine.

Linux

The install script is located in the <your_custom_platform>/linux64/libexec directory. This install script first compiles the kernel module in a temporary location and then performs the necessary setup to enable automatic driver loading after reboot.

Windows

The uninstall.bat script is located in the <your_custom_platform>\windows64\libexec directory, where <your_custom_platform> points to the top-level directory of your Custom Platform. This uninstall.bat script triggers the uninstall executable from Jungo Connectivity Ltd. to uninstall the WinDriver on the host machine.

Linux

The uninstall script is located in the <your_custom_platform>/linux64/libexec directory. This uninstall script removes the driver module from the Linux kernel.

Without an argument, the utility returns the overall information of all the devices installed in a host machine. If a specific device name is provided as an argument (that is, aocl diagnose <device_name> ), the diagnose utility runs a memory transfer test and then reports the host-device transfer performance.

You can run the diagnose utility for multiple devices (that is, aocl diagnose <device_name1> <device_name2> <device_name3> ). If you want to run the diagnose utility for all devices, use the all option (that is aocl diagnose all).

Memory Module Not Plugged-in or a Loose Connection on the Board

aocl diagnose: Running diagnose from 
aocl diagnose: failed 32 times. First error below:
Vendor: Intel Corporation
MMD INFO: [acla10_ref0] uniphy(s) did not calibrate. Expected 0 but read 2
MMD INFO: If there are more failures than Uniphy controllers connected, 
MMD INFO: ensure the uniphy_status core is correctly parameterized.

Solution

Confirm that you have connected the memory board, power cable, and USB cable correctly as shown in Configuring and installing the Intel Arria 10 GX FPGA Development Kit board. If you have confirmed your connections and continue to get this error, the memory board might not be seated correctly in the HiLo connector.

Error While Loading Shared Libraries

When you execute diagnose all utility, you might see errors as shown in the following example:

$ aocl diagnose all
hld/board/a10_ref/linux64/libexec/diagnose: error while loading shared libraries: 
libaltera_a10_ref_mmd.so: cannot open shared object file: No such file or directory

Solution

Ensure that you have set the PATH environment variable correctly.

export PATH=$PATH:$INTELFPGAOCLSDKROOT/bin:$QUARTUS_DIR/bin

The list-devices utility is similar to the diagnose utility. It first verifies the installation of the kernel driver and then lists all the devices.