Video and Vision Processing Suite Intel® FPGA IP User Guide

ID 683329
Date 5/08/2024
Document Table of Contents
1. About the Video and Vision Processing Suite 2. Getting Started with the Video and Vision Processing IPs 3. Video and Vision Processing IPs Functional Description 4. Video and Vision Processing IP Interfaces 5. Video and Vision Processing IP Registers 6. Video and Vision Processing IPs Software Programming Model 7. Protocol Converter Intel® FPGA IP 8. 1D LUT Intel® FPGA IP 9. 3D LUT Intel® FPGA IP 10. AXI-Stream Broadcaster Intel® FPGA IP 11. Bits per Color Sample Adapter Intel FPGA IP 12. Black Level Correction Intel® FPGA IP 13. Black Level Statistics Intel® FPGA IP 14. Chroma Key Intel® FPGA IP 15. Chroma Resampler Intel® FPGA IP 16. Clipper Intel® FPGA IP 17. Clocked Video Input Intel® FPGA IP 18. Clocked Video to Full-Raster Converter Intel® FPGA IP 19. Clocked Video Output Intel® FPGA IP 20. Color Plane Manager Intel® FPGA IP 21. Color Space Converter Intel® FPGA IP 22. Defective Pixel Correction Intel® FPGA IP 23. Deinterlacer Intel® FPGA IP 24. Demosaic Intel® FPGA IP 25. FIR Filter Intel® FPGA IP 26. Frame Cleaner Intel® FPGA IP 27. Full-Raster to Clocked Video Converter Intel® FPGA IP 28. Full-Raster to Streaming Converter Intel® FPGA IP 29. Genlock Controller Intel® FPGA IP 30. Generic Crosspoint Intel® FPGA IP 31. Genlock Signal Router Intel® FPGA IP 32. Guard Bands Intel® FPGA IP 33. Histogram Statistics Intel® FPGA IP 34. Interlacer Intel® FPGA IP 35. Mixer Intel® FPGA IP 36. Pixels in Parallel Converter Intel® FPGA IP 37. Scaler Intel® FPGA IP 38. Stream Cleaner Intel® FPGA IP 39. Switch Intel® FPGA IP 40. Tone Mapping Operator Intel® FPGA IP 41. Test Pattern Generator Intel® FPGA IP 42. Unsharp Mask Intel® FPGA IP 43. Video and Vision Monitor Intel FPGA IP 44. Video Frame Buffer Intel® FPGA IP 45. Video Frame Reader Intel FPGA IP 46. Video Frame Writer Intel FPGA IP 47. Video Streaming FIFO Intel® FPGA IP 48. Video Timing Generator Intel® FPGA IP 49. Vignette Correction Intel® FPGA IP 50. Warp Intel® FPGA IP 51. White Balance Correction Intel® FPGA IP 52. White Balance Statistics Intel® FPGA IP 53. Design Security 54. Document Revision History for Video and Vision Processing Suite User Guide

37.3.5. 422 and 420 Chroma Sampled Data Scaling

The Intel FPGA streaming video protocol that the scaler IP uses to transmit and receive video fields allows for the chroma sampling of the video to vary from frame to frame. For most of the IPs in the Video and Vision Processing suite, support for variable chroma sampling adds little or no extra resource cost. Variable chroma sampling of the video is always on and not gated by any parameters.

However, support for scaling 444, 422 and 420 chroma sampling on a field-variable basis adds significant ALM and memory cost to the resource footprint of the scaler. The scaler includes three parameters (444 chroma sampling, 422 chroma sampling and 420 chroma sampling) to allow you to select which chroma sampling formats the IP should support. Intel recommends that you only turn on chroma samplings formats that your scaler expects to receive in your system.

When the scaler processes 444 sampled data, each color plane is scaled independently, and each color plane has the same input and output dimensions. When the scaler processes 422 sampled data, each color plane is still processed independently (using the same filters as the 444 case), but the width of the chroma planes at the input and output is half that of the luma plane. Likewise, for 420 sampled data, each color plane is scaled independently, and the width and height of the chroma planes is half that of the luma plane. In all cases, the scaling ratio applied to each color plane must be the same. The scaler requires that the input and output field width are even for 422 sampled data, and the input and output width and height are even for 420 sampled data.

For 420 sampled data, each video line contains data from either the blue chroma (Cb) or red chroma (Cr) color plane. To supply both N lines of Cb and Cr data to an N tap vertical scaling filter, 2N lines of video data must be stored in line buffer, which is twice the storage required to process 444 or 422 sampled data. To reduce the on-chip memory footprint of the scaler with 420 chroma sampling enabled, you can turn on Mirror 420 chroma data. When on, the size of the line buffer is reduced back the N lines (for an N tap filter) and the IP mirrors N/2 lines of Cb and Cr data available in the buffer to fill the available taps. This parameter might give a small reduction in the quality of the scaled results for 420 chroma sampled data. You must assess any reduction in memory usage against the potential reduction in quality.

For 420 sampled data, the IP packs two luma samples into the three color planes that make up a standard pixel for either 444 or 422 chroma sampling. The scaler applies the horizontal scaling algorithm to the luma plane independently of the chroma planes. To allow for the double packing of the luma samples the scaler must, by default, implement the luma scaling with processing for twice the number of pixels in parallel you specify. However, you might not require the full processing bandwidth of the interface using 420 sampling. For example, if you have an HDMI 2.0 interface that supports up to 4k60 processing in both 444 and 420 sampling modes, the 420 mode only uses half the bandwidth allowed by the interface. In this example, the scaler need not use double the pixels in parallel to process the luma data and you should turn on the Half rate 420 parameter. This parameter specifies that the IP does not require internal doubling of pixels in parallel for processing the luma plane.