Video and Image Processing Suite User Guide

ID 683416
Date 4/04/2022
Document Table of Contents

8.5. Result to Output Data Type Conversion

After calculation, the fixed point type of the results must be converted to the integer data type of the output.

The conversion is performed in four stages, in the following order:

  1. Result scaling—scaling is useful to quickly increase the color depth of the output.
    • The available options are a shift of the binary point right –16 to +16 places.
    • Scaling is implemented as a simple shift operation so it does not require multipliers.
  2. Removal of fractional bits—if any fractional bits exist, you can choose to remove them through these methods:
    • Truncate to integer—fractional bits are removed from the data; equivalent to rounding towards negative infinity.
    • Round - Half up—round up to the nearest integer. If the fractional bits equal 0.5, rounding is towards positive infinity.
    • Round - Half even—round to the nearest integer. If the fractional bits equal 0.5, rounding is towards the nearest even integer.
  3. Conversion from signed to unsigned— if any negative numbers exist in the results and the output type is unsigned, you can convert to unsigned through these methods:.
    • Saturate to the minimum output value (constraining to range).
    • Replace negative numbers with their absolute positive value.
  4. Constrain to range—if any of the results are beyond a specific range, logic to saturate the results to the minimum and maximum output values is automatically added. The specific range is the specified range of the output guard bands, or if unspecified, the minimum and maximum values allowed by the output bits per pixel.