Video and Vision Processing Suite Intel® FPGA IP User Guide

ID 683329
Date 5/08/2024
Document Table of Contents
1. About the Video and Vision Processing Suite 2. Getting Started with the Video and Vision Processing IPs 3. Video and Vision Processing IPs Functional Description 4. Video and Vision Processing IP Interfaces 5. Video and Vision Processing IP Registers 6. Video and Vision Processing IPs Software Programming Model 7. Protocol Converter Intel® FPGA IP 8. 1D LUT Intel® FPGA IP 9. 3D LUT Intel® FPGA IP 10. AXI-Stream Broadcaster Intel® FPGA IP 11. Bits per Color Sample Adapter Intel FPGA IP 12. Black Level Correction Intel® FPGA IP 13. Black Level Statistics Intel® FPGA IP 14. Chroma Key Intel® FPGA IP 15. Chroma Resampler Intel® FPGA IP 16. Clipper Intel® FPGA IP 17. Clocked Video Input Intel® FPGA IP 18. Clocked Video to Full-Raster Converter Intel® FPGA IP 19. Clocked Video Output Intel® FPGA IP 20. Color Plane Manager Intel® FPGA IP 21. Color Space Converter Intel® FPGA IP 22. Defective Pixel Correction Intel® FPGA IP 23. Deinterlacer Intel® FPGA IP 24. Demosaic Intel® FPGA IP 25. FIR Filter Intel® FPGA IP 26. Frame Cleaner Intel® FPGA IP 27. Full-Raster to Clocked Video Converter Intel® FPGA IP 28. Full-Raster to Streaming Converter Intel® FPGA IP 29. Genlock Controller Intel® FPGA IP 30. Generic Crosspoint Intel® FPGA IP 31. Genlock Signal Router Intel® FPGA IP 32. Guard Bands Intel® FPGA IP 33. Histogram Statistics Intel® FPGA IP 34. Interlacer Intel® FPGA IP 35. Mixer Intel® FPGA IP 36. Pixels in Parallel Converter Intel® FPGA IP 37. Scaler Intel® FPGA IP 38. Stream Cleaner Intel® FPGA IP 39. Switch Intel® FPGA IP 40. Tone Mapping Operator Intel® FPGA IP 41. Test Pattern Generator Intel® FPGA IP 42. Unsharp Mask Intel® FPGA IP 43. Video and Vision Monitor Intel FPGA IP 44. Video Frame Buffer Intel® FPGA IP 45. Video Frame Reader Intel FPGA IP 46. Video Frame Writer Intel FPGA IP 47. Video Streaming FIFO Intel® FPGA IP 48. Video Timing Generator Intel® FPGA IP 49. Vignette Correction Intel® FPGA IP 50. Warp Intel® FPGA IP 51. White Balance Correction Intel® FPGA IP 52. White Balance Statistics Intel® FPGA IP 53. Design Security 54. Document Revision History for Video and Vision Processing Suite User Guide

3. Video and Vision Processing IPs Functional Description

Video and vision processing IPs conform to the Intel FPGA streaming video protocol.

Reset Behavior

IPs employ a synchronous reset and system resets must have a minimum duration of 256 clock cycles. In accordance with the AXI specification, all TVALID and TREADY signals from components drive low during reset and for at least one cycle after you deassert reset.

TUSER usage

The protocol specifies a TUSER width of TDATA/8 where TDATA is at least 16 bits and is always divisible by 8. The 2 LSBs of TUSER indicate whether a packet is control (full variants only) or data (bit 1) and indicate the start of a new field of video (bit 0). Intel video and vision processing IPs do not drive any unused bits (bit 2 upwards). Intel Quartus Prime optimizes them away during synthesis. The IPs ignore and do not propagate any data you drive on bits 2 and upwards of TUSER


The protocol specifies that an input interface can wait for TVALID to be asserted before asserting the corresponding TREADY. However, Intel video and vision processing IP sinks assert TREADY independently of whether the input TVALID is asserted. If a third-party IP drives a video and vision processing IP sink and it does not respect this AXI rule for sources, the video pipe still operates correctly.

Figure 2. Example video processing pipeline

The figure shows a typical video processing pipeline comprising video ingress and egress over HDMI, frame storage to DDR, and various video processing functions controlled by a processor.

If you turn off Lite mode for IPs, the pipeline includes the protocol converters to convert from lite mode, otherwise the IPs do not require them.

Video data passes along the pipeline in different formats in different places. The HDMI in connectivity IP passes clocked video to the clocked video to full-raster converter IP. That IP outputs a streaming full-raster format.

The full-raster to streaming video converter converts streaming full-raster data to Intel FPGA streaming video data packets. Then (optionally) the IP converts to full variants with additional metapackets by the protocol converter IP. The video data remains in this format until the end of the pipeline when the IPs perform reverse conversions.

Figure 3. Conversion of Intel clocked Video to full-raster video data.
Figure 4. Conversion of full-raster to streaming video data.

Figure 5. Conversion of lite to full Intel FPGA streaming video data.The figure shows the optional conversion from lite to full variants as performed by the protocol converter.

The Intel FPGA streaming video protocol states that IPs transmit video fields in packets of pixel data. One packet carries each line of video, with the start of field indicated by tuser[0]. The protocol converter supplements the pixel data packets with image information packets and end of field packets. The IP gains image information packet information from the protocol converter’s control registers

Most IPs update their behavior after the end of the current field, switching to any new control settings if required. Full variant IPs detect the end of the current field by the presence of the end of field packet. Lite variants IPs, without the benefit of metapackets, need to count the number of lines and compare this count with the value in the IMG_INFO_HEIGHT register. Alternatively, the IPs wait until they detect tuser[0], marking the start of the next field.