Intel® FPGA AI Suite
Find out how Intel® FPGA AI Suite can add FPGA AI to embedded systems and datacenters.
"The ease-of-use of the Intel® FPGA AI Suite and the Intel® Distribution of OpenVINO™ toolkit enabled Stryker* to develop optimized Intel® FPGA IP for deep learning inference. The inference IP was successfully integrated into an Intel® FPGA using Intel® Quartus® Prime Software. The example designs provided with the suite enabled the team to quickly evaluate different algorithms for different image sources. Intel® FPGA AI Suite and the Intel® Distribution of OpenVINO toolkit enable data scientists and FPGA engineers to seamlessly work together to develop optimized deep learning inference for medical applications."
— Stryker Engineering Team
Intel FPGAs enable real-time, low-latency, and low-power deep learning inference combined with the following advantages:
- I/O flexibility
- Ease of integration into custom platforms
- Long lifetime
Intel FPGA AI Suite was developed with the vision of ease-of-use of artificial intelligence (AI) inference on Intel FPGAs. The suite enables FPGA designers, machine learning engineers, and software developers to create optimized FPGA AI platforms efficiently.
Utilities in the Intel FPGA AI Suite speed up FPGA development for AI inference using familiar and popular industry frameworks such as TensorFlow* or PyTorch* and OpenVINO toolkit, while also leveraging robust and proven FPGA development flows with the Intel Quartus Prime Software.
The Intel FPGA AI Suite tool flow works with the OpenVINO toolkit, an open-source project to optimize inference on a variety of hardware architectures. The OpenVINO toolkit takes Deep Learning models from all the major Deep Learning frameworks (such as TensorFlow, PyTorch, Keras*) and optimizes them for inference on a variety of hardware architectures, including various CPUs, CPU+GPU, and FPGAs.
Find out how Intel FPGA AI Suite can add FPGA AI to embedded systems and data centers.
Figure 1: Intel FPGA AI Suite Development Flow
Devices supported: Intel® Agilex™ FPGA, Intel® Cyclone® 10 GX FPGA, Intel® Arria® 10 FPGA
Tested networks and activation functions1:
- ResNet-50, MobileNet v1/v2/v3, YOLO v3, TinyYOLO v3, UNET
- ReLU, 2D Conv, BatchNorm, EltWise Mult, Fully Connected, Clamp, pReLU, SoftMax
FPGA AI Inference Development Flow
The AI inference development flow is shown in Figure 1. The flow seamlessly combines a hardware and software workflow into a generic end-to-end AI workflow. The steps are as follows:
1. Model Optimizer in the OpenVINO toolkit creates intermediate representation network files (.xml) and weights and biases files (.bin).
2. Intel FPGA AI Suite compiler is used to:
- Provide estimated area or performance metrics for a given architecture file or produce an optimized architecture file. (Architecture refers to inference IP parameters such as size of PE array, precisions, activation functions, interface widths, window sizes, etc.)
- Compile network files into a .bin file with network partitions for FPGA and CPU (or both) along with weights and biases.
3. The compiled .bin file is imported by the user inference application at runtime.
- Runtime application programming interfaces (APIs) include Inference Engine API (runtime partition CPU and FPGA, schedule inference) and FPGA AI (DDR memory, FPGA hardware blocks).
- Reference designs demonstrate the basic operations of importing .bin and running inference on FPGA with supporting host CPUs (x86 and Arm* processors).
System Level Architectures
Intel FPGA AI Suite is flexible and configurable for a variety of system-level use cases. Typical ways to incorporate the FPGA AI Suite IP into a system are listed in Figure 2. The use cases span different verticals from optimized embedded platforms ranging from applications with host CPUs (Intel® Core™ processors, Arm processors) to data center environments with Intel® Xeon® processors and also to host-less applications (or soft processors such as Nios® V processors).
Figure 2: Typical Intel FPGA AI Suite System Topologies
Intel FPGA AI Suite includes reference designs with pre-built FPGA design examples for initial evaluation and further development for different system-level architectures:
- Terasic* DE10-Agilex Development Board
- Intel Arria 10 SoC Development Kit