Video and Image Processing Suite User Guide

ID 683416
Date 4/04/2022
Document Table of Contents

8.2. 2D FIR Filter Precision

The 2D FIR II IP core does not lose calculation precision during the FIR calculation.
  • You may parameterize the input data to between 4-20 bits per color per pixel, and the IP core treats this data as unsigned integer data. You may enable optional guard bands at the input to keep the data inside a reduced range of values.
  • You may parameterize the coefficient data up to a total width of 32 bits per coefficient. The coefficients may be signed or unsigned and contain up to 24 fractional bits.
  • You may parameterize the output data to between 4-20 bits per color per pixel, and the selected output data width may be different from the input data width.

To convert from the full precision result of the filtering to the selected output precision, the IP core first rounds up the value to remove the required number of fraction bits. Then the IP core saturates the value. You may select how many fraction bits should be preserved in the final output using the 2D FIR II parameter editor. As with the input data, the output data is treated as unsigned, so the IP core clips any negative values that result from the filtering to 0. Any values greater than the maximum value that can be represented in the selected number of bits per color per pixel are clipped to this maximum value.