Video and Image Processing Suite User Guide

ID 683416
Date 2/12/2021

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents

12.2.1. Predefined Conversions

Predefined conversions only support unsigned input and output data.

If you select signed input or output data, the predefined conversion produces incorrect results. When using a predefined conversion, the precision of the coefficients and summands must still be defined.

Predefined conversions are only defined for input and output bits per pixel per color plane equal to 8, 10, and 12. You must manually scale the summands accordingly when using a different bits per color plane value. If you use different input and output bits per pixel per color plane, you must also shift the results by the correct number of binary places to compensate. For example, to convert from 10-bit CbCrY' to 8-bit Computer B'G'R', select the conversion preset for 10-bit CbCrY' to 10-bit computer B'G'R'. The summands are already scaled for a 10-bit input so they remain unchanged. Change the output bits per color plane value from 10 to 8 on the parameter editor and follow the instructions of the warning message to shift the results by the correct number of binary places (2 places to the left).

Note: Always check the matrix of coefficients after applying a predefined conversion or after custom modifications. If the differences between the desired floating-point coefficient values and their actual fixed-point quantized values indicate an unacceptable loss of precision, you must increase the number of integer and/or fractional bits to fix the problem.