6.3.5. The Layout Transform IP as an Application-Specific Block
The layout transformation IP in the S2M design is provided as RTL source as an example layout transformation within a video inferencing application.
The flexibility of the Intel® FPGA AI Suite and the scope of projects it can support means that a layout transformation IP cannot serve all inference applications.
Each target application typically requires its own layout transformation module to be designed. System architectures need to budget for this design effort within their project.
Input data to the Intel® FPGA AI Suite IP must be formatted in memory so that the data matches the structure of the IP PE array and uses FP16 values.
The structure of the PE array is defined by the architecture file and the c_vector parameter setting describes the number of layers required for the input buffer. Typical c_vector values are 8, 16, and 32.
The S2M design demonstrates an example of video streaming. Pixel data is sent through the layout-transform as RGB pixels, where each color is considered an input channel of data.
In the first example where c_vector is set to 8, the first pixel of RGB is placed on the input stream filling the first 3 channels, but there are 5 more channels remaining that must be initialized. These are filled with zero (represented by the white squares). This padded stream is then fed into the Nios® subsystem.
This example layout transform does not support input folding. Input folding is an input preprocessing step that reduces the amount of zero padding in the c_vector. This folding then enables more efficient use of the dot product engine in the Intel® FPGA AI Suite IP. The efficiency gains can be significant depending on the graph and C_VEC. For more details, refer to "Input Folding" in Intel® FPGA AI Suite IP Reference Manual .