Khronos Compatibility

Intel® FPGA SDK for OpenCL™ is based on a published Khronos Specification and is supported by many vendors who are part of the Khronos group. Intel FPGA SDK for OpenCL has passed the Khronos Conformance Testing Process. It conforms to the OpenCL 1.0 standard and provides both the OpenCL 1.0 and OpenCL 2.0 headers by the Khronos Group.

Attention: The SDK currently does not support all OpenCL 2.0 application programming interfaces (APIs). If you use the OpenCL 2.0 headers and make a call to an unsupported API, the call will return an error code to indicate that the API is not fully supported.

The Intel FPGA SDK for OpenCL host runtime conforms with the OpenCL platform layer and API with some clarifications and exceptions, which can be found at the Support Statuses of OpenCL Features section of the Intel FPGA SDK for OpenCL Programming Guide.

Other Related Links:

• For more information on OpenCL, visit the Kronos Group OpenCL Overview page.
• Current conformance status can be found at the Kronos Group Adopter Program page.
• For more information on the OpenCL 1.0 standard, refer to The OpenCL Specification by Khronos.
  

OpenCL Extensions

Channels (I/Os or Kernel)

The Intel® FPGA SDK for OpenCL™ channel extension provides a mechanism for passing data to kernels and synchronizing kernels with high efficiency and low latency. Use the following links for more information on how to implement, use, and emulate channels:

• Implementing the Intel FPGA SDK for OpenCL Channels Extension
• Using Channels with Kernel Copies
• HTML Report: Kernel Design Concepts - Channels
• Transferring Data Via Intel FPGA SDK for OpenCL Channels or OpenCL Pipes
• Requirement for Multiple Command Queues in Channels or Pipes Implementation

Note: If you want to leverage the capabilities of channels but have the ability to run your kernel program using other SDKs, implement OpenCL pipes. For more information on pipes, see the following section on pipes.

Pipes

Intel FPGA SDK for OpenCL provides preliminary support for OpenCL pipe functions, which are part of the OpenCL Specification version 2.0. They provide a mechanism for passing data to kernels and synchronizing kernels with high efficiency and low latency.

The Intel FPGA SDK for OpenCL implementation of pipes is not fully conformant to the OpenCL Specification version 2.0. The goal of the SDK's pipe implementation is to provide a solution that works seamlessly on a different OpenCL 2.0-conformant device. To enable pipes for Intel FPGA products, your design must meet certain requirements.

See the following links for more information on how to implement OpenCL pipes:

• Implementing OpenCL Pipes
• Transferring Data Via Intel FPGA SDK for OpenCL Channels or OpenCL Pipes
• Requirement for Multiple Command Queues in Channels or Pipes Implementation

In a multistep design flow, you can assess the functionality of your OpenCL™ kernel by executing it on one or multiple emulation devices on an x86-64 Windows* or Linux* host. The compilation of the design for emulation takes seconds to generate an .aocx file and allows you to iterate on your design more effectively without having to go through the lengthy hours required for the full compilation.

For Linux systems, the emulator offers symbolic debug support. Symbolic debug allows you to locate the origins of functional errors in your kernel code.

The link below has an overview of the design flow for OpenCL kernels and illustrates the different stages for which you can emulate your kernel.

Multistep Intel® FPGA SDK for OpenCL Design Flow

The Emulating and Debugging Your OpenCL Kernel section from the  Programming Guide contains more details on the differences between kernel operation on hardware and emulation.

Other Related Links:

• Emulating and Debugging Your OpenCL Kernel
• Emulating I/O channels
• Verifying Host Runtime Functionality via Emulation (Windows)
• Verifying Host Runtime Functionality via Emulation (Linux)

With the Intel® FPGA SDK for OpenCL™ Offline Compiler technology, you do not need to change your kernel to fit it optimally into a fixed hardware architecture. Instead, the offline compiler customizes the hardware architecture automatically to accommodate your kernel requirements.

In general, you should optimize a kernel that targets a single compute unit first. After you optimize this compute unit, increase the performance by scaling the hardware to fill the remainder of the FPGA. The hardware footprint of the kernel correlates with the time it takes for hardware compilation. Therefore, the more optimizations you can perform with a smaller footprint (that is, a single computing unit), the more hardware compilations you can perform in a given amount of time.

OpenCL Optimization for Intel FPGAs

To optimize the implementation of your design and get the maximum performance, understand your theoretical maximum performance and understand what your limitations are. Follow these steps:

1. Start with a simple known good functional implementation.
2. Use an emulator to validate the functionality.
3. Remove or minimize the pipeline stalls that are reported with the optimization report.
4. Plan memory access for optimal memory bandwidth.
5. Use a profiler to debug performance issues.

The Profiler gives more insight into the system performance, which gives you direction to further optimize the algorithm in usage of the memory.
Remember that for FPGAs, the more resources that can be allocated, the more unrolling, parallelization, and higher performance can be attained.

Helpful Reports and Resources for Optimization

There are a number of system generated reports available to users. These reports give insight into the code, resource usage, and hints on where to focus to further improve the performance:

• Loop Analysis Report of an OpenCL Design Example
• Verifying Information on Memory Replication and Stalls
• Reviewing Area Information
• HTML Report: Area Report Messages

Memory Optimization

Understanding memory systems is crucial to efficiently implement an application using OpenCL.

Global Memory Interconnect

Unlike a GPU, an FPGA can build any custom load-store unit (LSU) that is most optimal for your application. As a result, your ability to write OpenCL code that selects the ideal LSU types for your application might help improve the performance of your design significantly.

For more information, refer to the Global Memory Interconnect section of the Intel FPGA SDK for the OpenCL Best Practices Guide.

Local Memory

Local memory is a complex system. Unlike typical GPU architecture where there are different levels of caches, an FPGA implements local memory in dedicated memory blocks inside the FPGA. For more information, refer to the Local Memory section of the Intel FPGA SDK for OpenCL Best Practices Guide.

There are a number of ways memory used can be optimized for improving the overall performance. For more information on some of the key techniques, refer to the Allocating Aligned Memory section of the Intel FPGA SDK for OpenCL Best Practices Guide. 

For more information on the strategies to improve memory access efficiency, refer to the Strategies for Improving Memory Access Efficiency section of the Intel FPGA SDK for OpenCL Best Practices Guide.

Pipelines

Understanding pipelines is crucial for leveraging the best performance of your implementation. Efficient use of pipelines directly improves the performance throughput. For more details, refer to the Pipelines section of the Intel FPGA SDK for OpenCL Best Practices Guide.

For more information on data transfer, refer to the Transferring Data Via Intel FPGA SDK for OpenCL Channels or OpenCL Pipes section of the Intel FPGA SDK for OpenCL Best Practices Guide.

Stall, Occupancy, Bandwidth

Profile your kernel to identify performance bottlenecks. For more information on how profiling information helps you identify poor memory or channel behaviors that lead to unsatisfactory kernel performance, refer to the Profiling Your Kernel to Identify Performance Bottlenecks section of the Intel FPGA SDK for OpenCL Best Practices Guide.

Loop Optimization

Some techniques for optimizing the loops are: 

• Nested Loops
• Loops in a Single Work-Item Kernel
• Loop Unrolling

For some tips on removing loop-carried dependencies in various scenarios for a single work item kernel, refer to the Removing Loop-Carried Dependency section of the Intel FPGA SDK for OpenCL Best Practices Guide.

For more information on optimizing floating-point operations, refer to the Optimizing Floating-Point Operations section of the Intel FPGA SDK for OpenCL Best Practices Guide.

Area Optimization

Area usage is an important design consideration if your OpenCL kernels are executable on FPGAs of different sizes. When you design your OpenCL application, Intel recommends that you follow certain design strategies for optimizing hardware area usage.

Optimizing kernel performance generally requires additional FPGA resources. In contrast, area optimization often results in decreased performance. During kernel optimization, Intel recommends that you run multiple versions of the kernel on the FPGA board to determine the kernel programming strategy that generates the best size versus performance trade-off.

For more information on strategies for optimizing FPGA area usage, refer to the Strategies for Optimizing FPGA Area Usage section of the Intel FPGA SDK for OpenCL Best Practices Guide. 

Reference Design Examples

Some design examples that illustrate the optimization techniques are as follow: 

• Matrix Multiplication Design Example

This example shows the optimization of the fundamental matrix multiplication operation using loop tiling to take advantage of the data reuse inherent in the computation.

This example illustrates:

    - Single-precision floating-point optimizations

    - Local memory buffering

    - Compile optimizations (loop unrolling, num_simd_work_items attribute)

    - Floating-point optimizations

    - Multiple device execution

• Time-Domain FIR Filter Design Example

This design example implements the time-domain finite impulse response (FIR) filter benchmark from the HPEC Challenge Benchmark Suite. For more information, refer to the Time-Domain Finite Impulse Response Filter Bank page. 

This design is a great example of how FPGAs can provide far better performance than a GPU architecture for floating-point FIR filters.

This example illustrates:

                - Single-precision floating-point optimizations

                - Efficient 1D sliding window buffer implementation        

                - Single work-item kernel optimization methods

• Video Downscaling Design Example

This design example implements a video downscaler that takes 1080p input video and outputs 720p video at 110 frames per second. This example uses multiple kernels to efficiently read from and write to global memory.

This example illustrates

                - Kernel channels

                - Multiple simultaneous kernels

                - Kernel-to-kernel channels

                - Sliding window design pattern

                - Memory access pattern optimizations

• Optical Flow Design Example

This design example is an OpenCL implementation of the Lucas Kanade optical flow algorithm. A dense, non-iterative, and non-pyramidal version with a window size of 52x52 is shown to run at over 80 frames per second on the Cyclone® V SoC Development Kit.

This example illustrates:

                - Single work-item kernel

                - Sliding window design pattern

                - Resource usage reduction techniques

                - Visual output

Training

Online training specific to OpenCL optimization with design examples are available at:

• OpenCL Optimization Techniques: Image Processing Algorithm Example
• OpenCL Optimization Techniques: Secure Hash Algorithm Example

References

• Intel FPGA SDK for OpenCL Best Practices Guide

In a multistep design flow, if the estimated kernel performance from emulation is acceptable, you can chose to collect information about how your design performs while executing on the FPGA.

You can instruct the Intel® FPGA SDK for OpenCL™ Offline Compiler to instrument performance counters in the Verilog code in the .aocx file with the -profile option. During execution, the Intel FPGA SDK for OpenCL Profiler measures and reports performance data that are collected during the OpenCL kernel execution on the FPGA. You can then review the performance data in the Profiler GUI.

The Profiling Your OpenCL Kernel section of the Intel FPGA SDK for OpenCL Programming Guide contains more information on how to profile your kernel.

How to Analyze Profiling Data

Profiling information helps you identify poor memory or channel behaviors that lead to unsatisfactory kernel performance. The Profile Your Kernel to Identify Performance Bottlenecks section of the Intel FPGA SDK for OpenCL Best Practices Guide contains more in-depth information on the Dynamic Profiler GUI and how to interpret profiling data such as stall, bandwidth, cache hits, and so on. It also contains Profiler analysis of several OpenCL design example scenarios.

You can now significantly reduce the system latency of your systems using host channels that allows streaming data from the host to stream directly into the FPGA kernel through the PCIe* interface while bypassing the memory controller. The FPGA kernel can begin processing the data immediately and does not have to wait for the data transfer to complete. Host channels are supported in the OpenCL runtime application programming interfaces (APIs) and include emulation support.

For more details on host channels and emulation support, refer to the Emulating I/O Channels section of the Intel® FPGA SDK for OpenCL™ Programming Guide. 

Profiling allows you to learn where your program spent its time and what are the different functions that are called. This information shows you which part of your program is running slower than you expected that might need a rewrite for faster program execution. It can also tell you which functions are being called more or less often than you expected.

gprof

The gprof is an open-source tool available in Linux* operating systems for profiling the source code. It works on time-based sampling. During intervals the program counter is interrogated to decide at which point in the code the execution has arrived.

To use the gprof, recompile the source code using the compiler profiling flag -pg

Run the executables to generate the files containing profiling information:

A specific file named “gmon.out” containing all the information that the gprof tool requires to produce a human-readable profiling data is generated. So, now use the gprof tool in the following way:

$ gprof source code gmon.out > profile_data.txt

profile_data.txt is the file that contains the information that the gprof tool uses to produce human-readable profiling data. This contains two parts: flat profile and call graph.

The flat profile shows how much time your program spent in each function, and how many times that function was called.

The call graph shows, for each function, which functions called it, which other functions it called, and how many times. There is also an estimate of how much time was spent in the subroutines of each function.

More information on the usage of gprof for profiling is available on the GNU website.

Intel® VTune™ Amplifier

The Intel® VTune™ Amplifier used for profiling helps you speed up and optimize execution of your code on Linux embedded platforms, Android*, or Windows* systems providing the following types of analysis:

• Performance analysis: Find serial and parallel code bottlenecks, analyze algorithm choices, and GPU engine usage, and understand where and how your application can benefit from available hardware resources
• Intel Energy Profiler analysis: Analyze power events and identify those that waste energy

For more information on the Intel V-tune Amplifier, visit the Getting Started with Intel VTune Amplifier 2018 for Linux OS website.

OpenCL™ host pipelined multithread provides a framework to achieve high throughput for algorithms where a large number of input data needs to be processed and the process for each data needs to be done in sequential order. One of the best applications of this framework is in heterogeneous platforms where high-throughput hardware or platform is used to accelerate the most time-consuming part of the application. Remaining parts of the algorithm must run in a sequential order on other platforms such as CPUs, to either prepare the input data for the accelerated task or to use the output of that task to prepare the final output. In this scenario, although the performance of the algorithm is partially accelerated, the total system throughput is much lower because of the sequential nature of the original algorithm.

In this AN 831: Intel FPGA SDK for OpenCL Host Pipelined Multithread Application Note, a new pipelined framework for high-throughput design is proposed. This framework is optimal for processing large input data through algorithms where data dependency forces sequential execution of all stages or tasks of the algorithm. 

FPGAs are highly used in the acceleration space. OpenCL has a specific way to be used by the CPU to offload task to FPGA. The file attached below contains the common initialization steps needed for the host code to launch the FPGA kernel. Download the file containing initialization steps here.

The init() function can be called from the main() function to initialize the FPGA. The code first finds the device upon which the kernel will run, and then programs it with the aocx supplied in the same directory as the host execuatable. After the initialization steps in the code, the user must set the kernel arguments according to their designs needs. 

There is also a cleanup() function which frees the resources after executing the kernel. 

For more information on profiling, refer to these sections in the Intel® FPGA SDK for OpenCL™ Programming Guide:

• Profiling Enqueued and Autorun Kernels
• Profile Data Acquisition
• Multiple Autorun Profiling Calls

There are certain environment variables that can be set to get more debug information while running the host application. These are Intel® FPGA SDK for OpenCL™ specific environment variables, which can help diagnose problems with custom platform designs. The following table lists all of these environment variables as well as describes them in detail.

The diagnostic tool for Intel FPGA SDK for OpenCL helps diagnose and resolve various installation/setup issues, hardware and software issues that come up while working with Intel FPGA SDK for OpenCL. The tool performs installation tests, device tests and link tests. For more information about the tool, refer to this presentation. To use the tool, download from here. 

Due to a loop in the host program, users may experience the OpenCL™ system slowing down while running it. To know more details about such a scenario, refer to the Debugging Your OpenCL System That is Gradually Slowing Down section of the Intel® FPGA SDK for OpenCL Programming Guide

The Intel Code Builder for OpenCL is a software development tool available as part of the Intel FPGA SDK for OpenCL. It provides a set of Microsoft* Visual Studio and Eclipse plug-ins that enable capabilities for creating, building, debugging, and analyzing Windows* and Linux* applications accelerated with OpenCL. For more information, refer to the Developing/Debugging OpenCL Applications Using Intel Code Builder for OpenCL section of the Intel FPGA SDK for OpenCL Programming Guide.

Intel® Arria® 10 Devices

• Search the Knowledge Database (OpenCL™)

Intel® Stratix® 10 Devices

• Search the Knowledge Database (OpenCL)

Here are some additional links from the Intel FPGA Community for specific issues related to design and run stages:   

• Intel FPGA Community for OpenCL™ issues
• Emulating Altera OpenCL Kernels
• Intel FPGA Community for Emulator