FPGA AI Suite: IP Reference Manual

ID 768974
Date 3/29/2024
Document Table of Contents

2.5.5. Output Tensor In-Memory Format

The output tensor in-memory format is similar to the input tensor in-memory format described in Input Feature Tensor In-Memory Format. However, the output tensor is padded to the nearest multiple of KVEC rather than CVEC, with the padding being done at the boundaries between FPGA AI Suite IP outputs rather than strictly at the edge of the logical tensor output.

While the logical tensor output might be a single tensor, the FPGA AI Suite IP might compute this output as one single output or by slicing the output into smaller pieces.

Each scenario can result in a different output layout, with the first resulting in padding only on the boundary of the tensor as shown in the following figure:
Figure 5. Output Tensor In-Memory Layout Without Slicing
The second scenario results in padding within the logical tensor as well as on the boundary of the logical tensor as shown in the following figure:
Figure 6. Output In-Memory Layout With Slicing

To enable the plugin to determine the appropriate output layout and offsets, the compiler provides a DlaRuntimeOutputConfiguration object. One of the fields of this object is the output_tensor_mapping field, which provides a mapping from FPGA AI Suite IP tensor output to logical tensor output.

To show how the IP output tensor data is mapped to the logical output tensor for a compiled graph, the FPGA AI Suite compiler creates the following CSV files:
  • output_transform_dump_<graph-name>.csv

    This file describes the tensor shape and offsets.

  • output_transform_mapping_<graph-name>.csv

    This file shows the element-wise mapping of the FPGA AI Suite output tensor to the logical output tensor. This mapping is in the inverse of the input mapping described in Input Transform Mapping. Refer to the example in that topic for a description of the transform mapping CSV file.