Video and Vision Processing Suite Intel® FPGA IP User Guide

ID 683329
Date 5/08/2024
Document Table of Contents
1. About the Video and Vision Processing Suite 2. Getting Started with the Video and Vision Processing IPs 3. Video and Vision Processing IPs Functional Description 4. Video and Vision Processing IP Interfaces 5. Video and Vision Processing IP Registers 6. Video and Vision Processing IPs Software Programming Model 7. Protocol Converter Intel® FPGA IP 8. 1D LUT Intel® FPGA IP 9. 3D LUT Intel® FPGA IP 10. AXI-Stream Broadcaster Intel® FPGA IP 11. Bits per Color Sample Adapter Intel FPGA IP 12. Black Level Correction Intel® FPGA IP 13. Black Level Statistics Intel® FPGA IP 14. Chroma Key Intel® FPGA IP 15. Chroma Resampler Intel® FPGA IP 16. Clipper Intel® FPGA IP 17. Clocked Video Input Intel® FPGA IP 18. Clocked Video to Full-Raster Converter Intel® FPGA IP 19. Clocked Video Output Intel® FPGA IP 20. Color Plane Manager Intel® FPGA IP 21. Color Space Converter Intel® FPGA IP 22. Defective Pixel Correction Intel® FPGA IP 23. Deinterlacer Intel® FPGA IP 24. Demosaic Intel® FPGA IP 25. FIR Filter Intel® FPGA IP 26. Frame Cleaner Intel® FPGA IP 27. Full-Raster to Clocked Video Converter Intel® FPGA IP 28. Full-Raster to Streaming Converter Intel® FPGA IP 29. Genlock Controller Intel® FPGA IP 30. Generic Crosspoint Intel® FPGA IP 31. Genlock Signal Router Intel® FPGA IP 32. Guard Bands Intel® FPGA IP 33. Histogram Statistics Intel® FPGA IP 34. Interlacer Intel® FPGA IP 35. Mixer Intel® FPGA IP 36. Pixels in Parallel Converter Intel® FPGA IP 37. Scaler Intel® FPGA IP 38. Stream Cleaner Intel® FPGA IP 39. Switch Intel® FPGA IP 40. Tone Mapping Operator Intel® FPGA IP 41. Test Pattern Generator Intel® FPGA IP 42. Unsharp Mask Intel® FPGA IP 43. Video and Vision Monitor Intel FPGA IP 44. Video Frame Buffer Intel® FPGA IP 45. Video Frame Reader Intel FPGA IP 46. Video Frame Writer Intel FPGA IP 47. Video Streaming FIFO Intel® FPGA IP 48. Video Timing Generator Intel® FPGA IP 49. Vignette Correction Intel® FPGA IP 50. Warp Intel® FPGA IP 51. White Balance Correction Intel® FPGA IP 52. White Balance Statistics Intel® FPGA IP 53. Design Security 54. Document Revision History for Video and Vision Processing Suite User Guide

38.3. Stream Cleaner IP Functional Description

Intel Video and Vision IPs follow the Intel Video and Vision Protocol. In full mode, this protocol consists of metapackets of a specific length and data packets of variable length. For example, an image information packet is four beats in length; end-of-field packets and timestamp packets are two and three beats in length, respectively. For legal packet ordering, refer to the Intel FPGA Video and Vision Protocol Specification
Figure 90. Packet ordering rulesThe figure summarizes the control packet ordering rules

A field must start with an image information packet, followed by 0 to n video data packets and then 0 to m auxiliary and timestamp packets. The field is then terminated by an end-of-field packet. After the end-of-field packet and before the image information packet, the protocol allows any number of auxiliary packets. If in any way this protocol is not adhered to, the video and vision IPs do not work correctly.

All IPs receive and transmit according to the protocol. It is difficult to produce a stream that does not comply with the protocol. In most cases, you do not require a Stream Cleaner IP in the video processing pipeline. However, in situations such as when switching may occur midpacket (crash switching), the output stream may produce broken control packets that do not match their required length, or packet orders which are not permitted by the protocol.

Figure 91. Stream Cleaner fixing broken packet example

In the figure, the Intel FPGA streaming video input receives the first beat of an end-of-field packet followed by the first beat of an image information packet. The end-of-field is therefore missing its second beat. Detecting this fault, the Stream Cleaner pauses the input stream by dropping axi_vid_in_tready low and inserts the second beat of the end-of-field before continuing the input stream by raising axi_vid_in_tready. The packet is now legal, although the data in the second beat is incorrect.

Figure 92. An example of a packet order that violates the Intel FPGA Streaming Video Protocol SpecificationThe figure shows the IP detecting a protocol error.

An end-of-field packet is followed by another end-of-field packet, which is an erroneous sequence. The Intel FPGA Streaming Video Protocol Specification states that an end-of-field packet must be followed by either an image information packet or an auxiliary control packet. The IP drops output axi_vid_in_tvalid low to prevent it from passing the data on to the next component. axi_vid_in_tvalid goes high again when a valid packet is on the output of the Stream Cleaner.

Figure 93. An example of a broken packet and a packet order that violates the Intel FPGA Streaming Video Protocol Specification.The figure shows a protocol error and a packet error.

A broken image information packet (shaded red) is followed by a complete image information packet. The first image information packet is broken due to missing its final beat. The Stream Cleaner produces a fixed packet (shaded green) by recreating the missing beat using the data from the previous beat. It then ensures the axi_vid_out_tlast signal goes high.

The Intel FPGA Streaming Video Protocol Specification states that an image information packet is followed by

  • A data packet (with the tuser[0] start-of-field signal high)
  • A timestamp packet
  • A custom auxiliary packet
  • An end-of-field packet.

The Stream Cleaner discards the second image information packet by lowering the axi_vid_out_tvalid signal. The axi_vid_out_tvalid signal raises again when a packet adhering to the specification passes through.

The Stream Cleaner only supports user auxiliary Packets of ID range 16-31 and requires them to be 4 beats in length.