FPGA AI Suite: SoC Design Example User Guide

ID 768979
Date 3/29/2024
Document Table of Contents

6.3.4. Resolving Input Rate Mismatches Between the FPGA AI Suite IP and the Streaming Input

When designing a system, the stream buffer rate should be matched to the FPGA AI Suite IP inferencing rate, so that the input data does not arrive faster than the IP can process it.

The SoC design example has safeguards in the Nios® subsystem for when the input data rate exceeds the FPGA AI Suite processing rate.

To prevent input buffer overflow (potentially writing to memory still being processed by the FPGA AI Suite IP), the Nios® subsystem has a buffer dropping technique built into it. If the subsystem detects that the FPGA AI Suite IP is falling behind, it starts dropping input buffers to allow the IP to catch up.

Using mailbox commands, the host application can check the queue depth level of the Nios® subsystem and see if the subsystem needs to drop input data.

Depending on the buffer processing requirements of a design, dropping input data might not be considered a failure. It is up to you to ensure that the IP inference rate meets the needs of the input data.

If buffer dropping is not desired, you can try to alleviate buffer dropping and increase FPGA AI Suite IP performance with the following options:

  • Configure a higher performance .arch file (IP configuration), which requires more FPGA resource. The .arch can be customized for the target machine learning graphs.
  • Increase the system clock-speed.
  • Reduce the size of the machine learning network, if possible.
  • Implement multiple instances of the FPGA AI Suite IP and multiplex input data between them.