Video and Image Processing Suite User Guide

RSS

UG-VIPSUITE | 2021.02.12

Version Information

Updated for:
Intel® Quartus® Prime Design Suite 19.4

1. About the Video and Image Processing Suite

The Intel^® Video and Image Processing (VIP) Suite is available in the DSP library of the Intel^® Quartus^® Prime software. You can configure the IPs to the required number of bits per symbols, symbols per pixel, symbols in sequence or parallel, and pixels in parallel.

The VIP Suite offers the following IPs:

2D FIR II Intel^® FPGA IP
Avalon-ST Video Monitor Intel^® FPGA IP (Available only in Platform Designer (Standard) edition)
Avalon-ST Video Stream Cleaner Intel^® FPGA IP
Chroma Resampler II Intel^® FPGA IP
Clipper II Intel^® FPGA IP
Clocked Video Input II Intel^® FPGA IP
Clocked Video Output II Intel^® FPGA IP
Color Plane Sequencer II Intel^® FPGA IP
Color Space Converter II Intel^® FPGA IP
Configurable Guard Bands Intel^® FPGA IP
Control Synchronizer Intel^® FPGA IP (Available only in Platform Designer (Standard) edition)
Deinterlacer II Intel^® FPGA IP
Frame Buffer II Intel^® FPGA IP
Gamma Corrector II Intel^® FPGA IP
Interlacer II Intel^® FPGA IP
Mixer II Intel^® FPGA IP
Scaler II Intel^® FPGA IP
Switch II Intel^® FPGA IP
Test Pattern Generator II Intel^® FPGA IP
Trace System Intel^® FPGA IP (Available only in Platform Designer (Standard) edition)
Warp Lite IP

These IPs transmit and receive video according to the Avalon streaming video standard. Most IPs receive and transmit video data according to the same Avalon streaming video configuration, but some explicitly convert from one Avalon streaming video configuration to another. For example, you can use the Color Plane Sequencer II IP to convert from 1 pixel in parallel to 4.

All VIP IPs require even frame widths when using 4:2:2 data; odd frame widths create unpredictable results or distorted images. The Clipper II IP requires even clip start offsets and the Mixer II IP requires even offsets when using 4:2:2 data.

The signal names are standard Avalon streaming signals, and so by default, not enumerated. Some IPs may have additional signals.

All IPs in the VIP Suite support pixels in parallel, with the exception of Control Synchronizer, and Avalon-ST Video Monitor IP. Most of the IPs support 8 pixels in parallel and 8K resolutions.

Table 1. Minimum and Maximum Supported X and Y Resolutions
VIP IP Cores	Minimum Input and Output Resolution in Pixels (Width × Height)	Maximum Input and Output Resolution in Pixels (Width × Height)
Clocked Video Input II	32 × 32	8192 × 8192
Clocked Video Output II	32 × 32	8192 × 8192
Control Synchronizer	32 × 32	1920 × 1080
Deinterlacer II	32 × 32	4096 × 2160 ¹
Warp Lite IP	128 × 128	1920 × 1080
Other IP	32 × 32	8192 × 8192

¹ Deinterlacer II can pass through progressive video up to a maximum resolution of 4096 × 2160.

1.1. Release Information

Release information for the Video and Image Processing Suite.

Intel^® FPGA IP versions match the Intel^® Quartus^® Prime Design Suite software versions until v19.1. Starting in Intel^® Quartus^® Prime Design Suite software version 19.2, Intel^® FPGA IP has a new versioning scheme.

The Intel^® FPGA IP version (X.Y.Z) number can change with each Intel^® Quartus^® Prime software version. A change in:

X indicates a major revision of the IP. If you update the Intel^® Quartus^® Prime software, you must regenerate the IP.
Y indicates the IP includes new features. Regenerate your IP to include these new features.
Z indicates the IP includes minor changes. Regenerate your IP to include these changes.

Table 2. Release Information
Item	Description
Version	19.4
Release Date	December 2019
Ordering Code	IPS-VIDEO (Video and Image Processing Suite)

Intel verifies that the current version of the Intel^® Quartus^® Prime software compiles the previous version of each IP core, if this IP core was included in the previous release. Intel reports any exceptions to this verification in the Intel FPGA IP Release Notes. Intel does not verify compilation with IP core versions older than the previous release.

1.2. Device Family Support

Table 3. Device Family SupportThe table lists the device support information for the Video and Image Processing Suite IP cores. The Warp Lite IP only supports Intel^® Arria^® 10 and Intel^® Stratix^® 10 devices.
Device Family	Support
Intel^® Stratix^® 10	Final
Intel^® Arria^® 10	Final
Intel^® Cyclone^® 10 LP	Final
Intel^® Cyclone^® 10 GX	Final
Intel^® MAX^® 10	Final
Arria^® II GX/GZ	Final
Arria^® V GX/GT/SX/ST	Final
Arria^® V GZ	Final
Cyclone^® IV E/GX	Final
Cyclone^® V	Final
Stratix^® IV	Final
Stratix^® V	Final
Other device families	No support

The following terms define device support levels for Intel FPGA IP cores:

Advance support—the IP core is available for simulation and compilation for this device family. Timing models include initial engineering estimates of delays based on early post-layout information. The timing models are subject to change as silicon testing improves the correlation between the actual silicon and the timing models. You can use this IP core for system architecture and resource utilization studies, simulation, pinout, system latency assessments, basic timing assessments (pipeline budgeting), and I/O transfer strategy (data-path width, burst depth, I/O standards tradeoffs).
Preliminary support—the IP core is verified with preliminary timing models for this device family. The IP core meets all functional requirements, but might still be undergoing timing analysis for the device family. It can be used in production designs with caution.
Final support—the IP core is verified with final timing models for this device family. The IP core meets all functional and timing requirements for the device family and can be used in production designs.

1.3. Latency

You can use the latency information to predict the approximate latency between the input and the output of your video processing pipeline.

The latency is described using one or more of the following measures:

the number of progressive frames
the number of interlaced fields
the number of lines when less than a field of latency
a number of cycles

When you measure the latency, assume that other functions on the data path are not stalling the IP; (the output ready signal is high).

Table 4. Video and Image Processing Suite LatencyThe table lists the approximate latency from the video data input to the video data output for typical usage modes of the Video and Image Processing IPs.
IP Core	Mode	Latency
2D FIR Filter Latency	Filter size: `N` × `N`	(N–1) lines + (n cycles)
Mixer II	All modes	n cycles
Avalon-ST Video Stream Cleaner	All modes	n cycles
Chroma Resampler II	Input format: 4:2:2; Output format: 4:4:4	n cycles
Chroma Resampler II	Input format: 4:2:0; Output format: 4:4:4 or 4:2:2	1 line + n cycles
Clipper II	All modes	n cycles
Clocked Video Input II	All modes	n cycles
Clocked Video Output II	All modes	n cycles
Color Plane Sequencer II	All modes	n cycles
Color Space Converter II	All modes	n cycles
Control Synchronizer	All modes	n cycles
Deinterlacer II	Method: Bob Frame buffering: None	n cycles
	Method: Weave Frame buffering: None	1 field + n cycles
	Method: Motion-adaptive Frame buffering: None Output frame rate: As input field rate	2 lines + n cycles
	Method: Motion-adaptive, video-over-film mode Frame buffering: 3 input fields are buffered Output frame rate: same as the input field rate	1 field + 2 lines, or 2 lines 40% to 60% (depending on phasing) of the time, the core performs a weave forward so there is no initial field of latency.
Frame Buffer II	All modes	1 frame + n cycles
Gamma Corrector II	All modes	n cycles
Interlacer II	All modes	n cycles
Scaler II	Scaling algorithm: Polyphase Number of vertical taps: `N`	(N –1) lines + n cycles
Switch II	All modes	n cycles
Test Pattern Generator II	Not applicable.
Warp Lite IP	All modes	Up to 100 lines. The amount varies across the frame depending on the level of distortion specified.

1.4. In-System Performance and Resource Guidance

The performance and resource data provided for your guidance.

Note: Run your own synthesis and f_MAX trials to confirm the listed IP cores meet your system requirements.

Table 5. Performance and Resource Data Using Intel^® Arria^® 10 DevicesThe following data are obtained through a 4K test design example using an Intel^® Arria^® 10 device (10AX115S3F45E2SGE3).
The general settings for the design is 10 bits per color plane; 2 pixels in parallel. The target f_MAX is 300 MHz.
IP Core	Configuration	ALMs	M20K	DSP Blocks
Mixer II	Number of color planes in parallel = 3 Inputs = 2 Output = 1 Internal Test Pattern Generator	1,890	0	12
Clocked Video Input II	Number of color planes in parallel = 3 Sync signals = On separate wires Pixel FIFO size = 4096 pixels Use control port = On	855	14	0
Clocked Video Output II	Number of color planes in parallel = 3 Sync signals = On separate wires Pixel FIFO size = 4096 pixels Use control port = On Run-time configurable video modes = 4	2,251	12	0
Color Space Converter II	Run-time control = On Color model conversion = RGB to YCbCr	865	1	12
Frame Buffer II	Maximum frame size = 4096 × 2160 Number of color planes in parallel = 3 Avalon-MM master ports width = 512 Read/write FIFO depth = 32 Frame dropping = On Frame repeating = On	2,232	33	2
Clipper II	Number of color planes in parallel = 3 Enable run-time control of clipping parameters = On	963	0	0
Warp Lite	Bits per symbol = 10 Maximum width = 1920 Maximum distortion = 30 Paired with VFB = 0	6,530	395	21

1.5. Stall Behavior and Error Recovery

The Video and Image Processing Suite IPs do not continuously process data. Instead, they use flow-controlled Avalon streaming interfaces, which allow them to stall the data while they perform internal calculations.

During control packet processing, the IPs might stall frequently and read or write less than once per clock cycle. During data processing, the IPs generally process one input or output per clock cycle. The IPs have some stalling cycles. Typically, the stalling cycles are for internal calculations between rows of image data and between frames/fields.

When stalled, an IP indicates that it is not ready to receive or produce data. The time spent in the stalled state varies between IPs and their parameterizations. In general, it is a few cycles between rows and a few more between frames.

If data is not available at the input when required, all of the IPs stall and do not output data. With the exceptions of the Deinterlacer and Frame Buffer in double or triple-buffering mode, none of the IPs overlap the processing of consecutive frames. The first sample of frame F + 1 is not input until after the IPs produce the last sample of frame F.

When the IPs receive an endofpacket signal unexpectedly (early or late), the IPs recover from the error and prepare for the next valid packet (control or data).

IP Core	Stall Behavior	Error Recovery
2D FIR Filter II	Has a delay of a little more than N–1 lines between data input and output in the case of a N×N 2D FIR Filter. Delay caused by line buffering internal to the IP.	An error condition occurs if an `endofpacket` signal is received too early or too late for the run-time frame size. In either case, the 2D FIR Filter always creates output video packets of the configured size. If an input video packet has a late `endofpacket` signal, then the extra data is discarded. If an input video packet has an early `endofpacket` signal, then the video frame is padded with an undefined combination of the last input pixels.
Mixer II	All modes stall for a few cycles after each output frame and between output lines. Between frames, the IP processes non-image data packets from its input layers in sequential order. The IP may exert backpressure during the process until the image data header has been received for all its input. During the mixing of a frame, the IP: Reads from the background input for each non-stalled cycle. Reads from the input ports associated with layers that currently cover the background image. Because of pipelining, the foreground pixel of layer N is read approximately N active cycles after the corresponding background pixel has been read. If the output is applying backpressure or if one input is stalling, the pipeline stalls and the backpressure propagates to all active inputs. When alpha blending is enabled, one data sample is read from each alpha port once each time that a whole pixel of data is read from the corresponding input port. There is no internal buffering in the IP, so the delay from input to output is just a few clock cycles and increases linearly with the number of inputs.	The Mixer II IP core processes video packets from the background layer until the end of packet is received. Receiving an `endofpacket` signal too early for the background layer—the IP core enters error mode and continues writing data until it has reached the end of the current line. The `endofpacket` signal is then set with the last pixel sent. Receiving an `endofpacket` signal early for one of the foreground layers or for one of the alpha layers—the IP core stops pulling data out of the corresponding input and pads the incomplete frame with undefined samples. Receiving an `endofpacket` signal late for the background layer, one or more foreground layers, or one or more alpha layers—the IP core enters error mode. This error recovery process maintains the synchronization between all the inputs and is started once the output frame is completed. A large number of samples may have to be discarded during the operation and backpressure can be applied for a long time on most input layers. Consequently, this error recovery mechanism could trigger an overflow at the input of the system.
Avalon-ST Video Stream Cleaner	All modes stall for a few cycles between frames and between lines.	Receiving an early `endofpacket` signal—the IP core stalls its input but continues writing data until it has sent an entire frame. Not receiving an `endofpacket` signal at the end of a frame—the IP core discards data until it finds end-of-packet.
Chroma Resampler II	All modes stall for a few cycles between frames and between lines. Latency from input to output varies depending on the operation mode of the IP core. The only modes with latency of more than a few cycles are 4:2:0 to 4:2:2 and 4:2:0 to 4:4:4—corresponding to one line of 4:2:0 data The quantities of data input and output are not equal because this is a rate-changing function. Always produces the same number of lines that it accepts—but the number of samples in each line varies according to the subsampling pattern used. When not stalled, always processes one sample from the more fully sampled side on each clock cycle. For example, the subsampled side pauses for one third of the clock cycles in the 4:2:2 case or half of the clock cycles in the 4:2:0 case.	Receiving an early `endofpacket` signal—the IP core stalls its input but continues writing data until it has sent an entire frame. Not receiving an `endofpacket` signal at the end of a frame—the IP core discards data until it finds end-of-packet.
Clipper II	Stalls for a few cycles between lines and between frames. Internal latency is less than 10 cycles. During the processing of a line, it reads continuously but only writes when inside the active picture area as defined by the clipping window.	Receiving an early `endofpacket` signal—the IP core stalls its input but continues writing data until it has sent an entire frame. Not receiving an `endofpacket` signal at the end of a frame—the IP core discards data until it finds end of packet.
Clocked Video Input II ²	Dictated by incoming video. If its output FIFO is empty, during horizontal and vertical blanking periods the IP core does not produce any video data.	If an overflow is caused by a downstream core failing to receive data at the rate of the incoming video, the Clocked Video Input sends an `endofpacket` signal and restart sending video data at the start of the next frame or field.
Clocked Video Output II	Dictated by outgoing video. If its input FIFO is full, during horizontal and vertical blanking periods the IP stalls and does not take in any more video data.	Receiving an early `endofpacket` signal— the IP resynchronizes the outgoing video data to the incoming video data on the next start of packet it receives. Receiving a late `endofpacket`— the IP resynchronizes the outgoing video data to the incoming video immediately.
Color Plane Sequencer II	Stalls for a few cycles between frames and user/control packets The Avalon-ST Video transmission settings (color planes in sequence/parallel, number of color planes and number of pixels per beat) determine the throughput for each I/O. The slowest interface limits the overall rate of the others	Processes video packets until the IP core receives an `endofpacket` signal on either inputs. Frame dimensions taken from the control packets are not used to validate the sizes of the input frames.. When receiving an `endofpacket` signal on either `din0` or `din1`; the IP core terminates the current output frame. When both inputs are enabled and the `endofpacket` signals do not line up, extra input data on the second input is discarded until the end of packet is signaled.
Color Space Converter II	Only stalls between frames and not between rows. It has no internal buffering apart from the registers of its processing pipeline—only a few clock cycles of latency.	Processes video packets until the IP core receives an `endofpacket` signal—the control packets are not used. Any mismatch of the `endofpacket` signal and the frame size is propagated unchanged to the next IP core.
Control Synchronizer	Stalls for several cycles between packets. Stalls when it enters a triggered state while it writes to the Avalon-MM Slave ports of other IP cores. If the slaves do not provide a wait request signal, the stall lasts for no more than 50 clock cycles. Otherwise the stall is of unknown length.	Processes video packets until the IP core receives an `endofpacket` signal—the image width, height and interlaced fields of the control data packets are not compared against the following video data packet. Any mismatch of the `endofpacket` signal and the frame size of video data packet is propagated unchanged to the next IP core.
Deinterlacer II	Stores input video fields in the external memory and concurrently uses these input video fields to construct deinterlaced frames. Stalls up to 50 clock cycles for the first output frame. Additional delay of one line for second output frame because the IP core generates the last line of the output frame before accepting the first line of the next input field. Delay of two lines for the following output frames, which includes the one line delay from the second output frame. For all subsequent fields, the delay alternates between one and two lines.	Bob and Weave configurations always recover from an error caused by illegal control or video packets. Motion adaptive modes require the embedded stream cleaner to be enabled to fully recover from errors.
Frame Buffer II	May stall frequently and read or write less than once per clock cycle during control packet processing. During data processing at the input or at the output, the stall behavior of the IP core is largely decided by contention on the memory bus.	Does not rely on the content of the control packets to determine the size of the image data packets. Any early or late `endofpacket` signal and any mismatch between the size of the image data packet and the content of the control packet are propagated unchanged to the next IP core. Does not write outside the memory allocated for each non-image and image Avalon-ST video packet—packets are truncated if they are larger than the maximum size defined at compile time.
Gamma Corrector II	Stalls only between frames and not between rows. Has no internal buffering aside from the registers of its processing pipeline— only a few clock cycles of latency	Processes video packets until the IP core receives an `endofpacket` signal—non-image packets are propagated but the content of control packets is ignored. Any mismatch of the `endofpacket` signal and the frame size is propagated unchanged to the next IP core.
Interlacer II	Alternates between propagating and discarding a row from the input port while producing an interlaced output field—the output port is inactive every other row. The delay from input to output is a few clock cycles when pixels are propagated.	Receiving `endofpacket` signal later than expected—discards extra data. Receiving an early `endofpacket` signal—the current output field is interrupted as soon as possible and may be padded with a single undefined pixel.
Scaler II	The ratio of reads to writes is proportional to the scaling ratio and occurs on both a per-pixel and a per-line basis. The frequency of lines where reads and writes occur is proportional to the vertical scaling ratio. For example, scaling up vertically by a factor of 2 results in the input being stalled every other line for the length of time it takes to write one line of output; scaling down vertically by a factor of 2 results in the output being stalled every other line for the length of time it takes to read one line of input. In a line that has both input and output active, the ratio of reads and writes is proportional to the horizontal scaling ratio. For example, scaling from 64×64 to 128×128 causes 128 lines of output, where only 64 of these lines have any reads in them. For each of these 64 lines, there are two writes to every read. The internal latency of the IP core depends on the scaling algorithm and whether any run time control is enabled. The scaling algorithm impacts stalling as follows: Bilinear mode: a complete line of input is read into a buffer before any output is produced. At the end of a frame there are no reads as this buffer is drained. The exact number of possible writes during this time depends on the scaling ratio. Polyphase mode with N _v vertical taps: N _v – 1 lines of input are read into line buffers before any output is ready. The scaling ratio depends on the time at the end of a frame where no reads are required as the buffers are drained. Enabling run-time control of resolutions affects stalling between frames: With no run-time control: about 10 cycles of delay before the stall behavior begins, and about 20 cycles of further stalling between each output line. With run-time control of resolutions: about additional 25 cycles of delay between frames.	Receiving an early `endofpacket` signal at the end of an input line—the IP core stalls its input but continues writing data until it has sent one further output line. Receiving an early `endofpacket` signal part way through an input line—the IP core stalls its input for as long as it takes for the open input line to complete; completing any output line that may accompany that input line. Then continues to stall the input, and writes one further output line. Not receiving an `endofpacket` signal at the end of a frame—the IP core discards extra data until it finds an end of packet.
Switch II	Only stalls its inputs when performing an output switch. Before switching its outputs, the IP core synchronizes all its inputs and the inputs may be stalled during this synchronization.	—
Test Pattern Generator II	All modes stall for a few cycles after a field control packet, and between lines. When producing a line of image data, the IP core produces one sample output on every clock cycle, but it can be stalled without consequences if other functions down the data path are not ready and exert backpressure.	—
Warp Lite	All modes produce a few cycles of stall if the line store fills. The line store size ensures it shouldn’t fill unless a mismatch occurs between input and output rates.	The Warp Lite IP has a built in stream cleaner to handle error conditions. Receiving an `endofpacket` signal too early causes the stream cleaner to pad the packet up to the frame size specified by the preceding control packet. Receiving an `endofpacket` signal too late causes the stream cleaner to clip the packet to the frame size specified by the preceding control packet. Receiving a video packet without a preceding control packet causes the stream cleaner to discard the video packet.

² For CVI II IP, the error recovery behavior varies depending on the Platform Designer parameters. Refer to the Clocked Video Interface IP for more information.

2. Avalon Streaming Video

The Video and Image Processing IPs conform to the Avalon streaming video standard of data transmission.

This standard is a configurable protocol layer that sits on top of the Intel Avalon streaming standard. The standard comprises video packets, control packets, and/or user packets.

Note: Before you start using the VIP IPs, you must fully understand this protocol layer. The IPs transmit and receive all video data in this format.

The individual video formats supported (i.e. NTSC, 1080p, UHD 4K) depend primarily on the configuration of the Avalon streaming video standard and the clock frequency. The IPs may transmit pixel information either in sequence or in parallel, in RGB or YCbCr color spaces, and under a variety of different chroma samplings and bit depths, depending on which is the most suitable for the end application. The Avalon streaming video protocol adheres to the Avalon streaming standard packet data transfers, with backpressure and a ready latency of 1.

Figure 1. Avalon Streaming Video SignalsThe figure shows two VIP IPs and the Avalon streaming video signals used for data transfer. The Avalon streaming optional channel signal is always unused.

Figure 2. Avalon Streaming Video Packet Transmission (Symbols in Parallel)The figure shows an example transmission of twelve “symbols” using the Avalon streaming video configuration of the Avalon streaming specification.

A “ready latency” of 1 is used for Avalon streaming video. The example shows the receiving video sink drops its ready signal in cycle 3, to indicate that it is not ready to receive any data in cycles 4 or 5. The video source responds by extending its valid, endofpacket and data signals into cycle 6. As the ready signal returns high in cycle 5, the video source data in cycle 6 is safely registered by the sink.

The symbols D0, D1… can be pixel color plane data from an Avalon streaming video image packet or data from a control packet or a user packet. The type of packet is determined by the lowest 4 bits of the first symbol transmitted.

Table 6. Avalon Streaming Packet Type Identifiers
Type Identifier D0[3:0]	Description
0x0 (0)	Video data packet
0x1–0x8 (1–8)	User data packet
0x9–0xC (9–12)	Reserved
0xD (13)	Clocked Video data ancillary user packet
0xE (14)	Reserved
0xF (15)	Control packet

2.1. Avalon-ST Video Configuration Types

The Avalon-ST video protocol also allows for symbols to be transmitted in sequence. The start of the same transaction transmitted in “symbols in sequence” configuration. The symbols themselves are unchanged, but are transmitted consecutively rather than being grouped together.

Most color spaces require more than one symbol to represent each pixel. For example, three symbols are needed for each of the red, green and blue planes of the RGB color space.

Figure 3. Avalon-ST Video Packet Transmission (Symbols in Series)

Avalon-ST Video allows for multiple pixels to be transmitted in parallel. When the number of pixels transmitted in parallel is greater than one, the optional Avalon-ST empty signal is added to the data transfer interface between the IP cores.

The figure below shows 4 pixels, each comprising 3 symbols (or color planes), being transmitted in parallel.

Figure 4. Avalon-ST Video Packet Transmission (Pixels in Parallel)This figure illustrates the end of a video packet containing 1366 pixels, where the final beat of the transaction is not fully populated with pixel data - because 1366 is indivisible by 4, the number of pixels in parallel being used in the Avalon-ST Video configuration. The Avalon-ST standard handles this situation by populating the empty signal with the number of invalid (or empty) symbols in the final beat of transmission. The number of invalid symbols given in the empty signal must be a multiplication of the number of symbols per pixel in all circumstances.

2.2. Avalon-ST Video Packet Types

The three different types of Avalon-ST Video packets are defined, then a description is given of the Avalon-ST Video expected ordering and the meaning of these packets when transmitted or received by Avalon-ST Video compliant cores.

2.2.1. Avalon-ST Video Control Packets

A control packet is identified when the low nibble of the first symbol is set to decimal 15 (0xF). The Avalon-ST Video protocol further defines that any other symbol data transmitted in the first cycle (or beat) of the transmission is ignored.

An Avalon-ST Video control packet comprises the identifier nibble, and 9 other nibbles which indicate the height, width and interlacing information of any subsequent Avalon-ST Video packets.

Figure 5. Avalon-ST Video Control PacketThe figure below shows the structure of a control packet in an Avalon-ST Video configuration of two 10 bit symbols (color planes) per pixel and two pixels in parallel. Observe the symbol-alignment of the control packet nibbles; the remaining bits in each symbol are undefined.

Note: Most VIP IP cores do not drive the empty signal for control packets because extra data is always ignored. Nevertheless, a value of two indicating that the last pixel of the final beat is invalid would be tolerated.

Table 7. Avalon-ST Video Control Packet Nibble DecodingThe Height and Width are given in pixels and the Interlacing nibble is decoded.
Nibble	Description
D₀	Width[15:12]
D₁	Width[11:8]
D₂	Width[7:4]
D₃	Width[3:0]
D₄	Height[15:12]
D₅	Height[11:8]
D₆	Height[7:4]
D₇	Height[3:0]
D₈	Interlacing[3:0]

When the Interlacing nibble indicates an interlaced field, the height nibble gives the height of the individual fields and not that of the equivalent whole frame—for example, a control packet for 1080i video would show a height of 540, not 1080.

Table 8. Avalon-ST Video Control Packet Interlaced Nibble Decoding
Interlaced/Progressive	Interlacing[3]	Interlacing[2]	Interlacing[1]	Interlacing[0]	Description
Interlaced	1	1	0	0	Interlaced F1 field, paired with the following F0 field
			0	1	Interlaced F1 field, paired with the preceding F0 field
			1	x	Interlaced F1 field, pairing don’t care
		0	0	0	Interlaced F0 field, paired with the preceding F1 field
			0	1	Interlaced F0 field, paired with the following F1 field
			1	x	Interlaced F0 field, pairing don’t care
Progressive	0	x	0	1	Progressive frame, deinterlaced from an f1 field
			0	0	Progressive frame, deinterlaced from an f0 field
			1	x	Progressive frame

2.2.2. Avalon-ST Video Video Packets

A video packet is identified when the low nibble of the first symbol is set to decimal 0.

The Avalon-ST Video protocol further defines that any other symbol data transmitted in the first cycle (or beat) of the transmission is ignored. Uncompressed and rasterized, the pixel data is transmitted in the symbols that follow in subsequent cycles, starting with the top-left pixel.

Avalon-ST Video packets support RGB and YCbCr color spaces, with 4:4:4 or 4:2:2 chroma sub-sampling, with or without an optional alpha (transparency) channel. The 4:2:0 color space is only handled by the Clocked Video interfaces and the chroma resampler. For the other VIP IP cores, the 4:2:0 color space should be converted to or from 4:2:2 for processing.

Color channel data for RGB video packets is transmitted in the order of Blue, Green, Red. You can observe this order directly on the bus for symbols in sequence Avalon-ST configurations. For symbols (and pixels) in parallel configurations, the blue symbol occupies the least significant symbol position (D₀) as shown the figure below, with the (x,y) raster position shown in brackets.

Figure 6. Avalon-ST RGB Video Packet

For 4:4:4 YCbCr data, chroma sample Cb is in the least significant position (or first symbol to be transmitted in a symbols in sequence configuration), Cr is the mid symbol, with the luminance (Y) occupying the most significant symbol position. The figure below shows an example with symbols in parallel.

Figure 7. Avalon-ST YCbCr 4:4:4 Video Packet

For 4:2:2 YCbCr video, with sub-sampled chroma, the Cr and Cb symbols alternate, such that each Luma symbol is associated with either a Cr or Cb symbol as shown in the figure below.

Figure 8. Avalon-ST YCbCr 4:2:2 Video Packet

For video with an alpha layer, the alpha channel occupies the first (least significant) symbol with the remaining color channels following as per the usual ordering with Blue or Cb occupying the next symbol after the alpha as shown in the figure below.

Figure 9. Avalon-ST YCbCr 4:2:2 Video Packet with Alpha Channel

2.2.3. Avalon-ST Video User Packets

The Avalon-ST protocol may use the user packet types to transmit any user data, such as frame identification information, audio, or closed caption data.

The Avalon-ST Video protocol only requires for the payload data to begin on the second cycle of transmission (the first cycle must contain the user packet identification nibble). The content or length of these packets are ignored.

Figure 10. Avalon-ST User Packet (Start)The figure below shows the start of an example user packet transmission. If a user packet passes through a VIP IP core which reduces the number of bits per pixel, then data is lost.

2.3. Avalon-ST Video Operation

Most Avalon-ST Video compliant VIP IP cores require an Avalon-ST control packet to be received before any video packets, so that line buffers and other sub-components can be configured.

Intel recommends that every video frame (or field, in the case of interlaced video) is preceded by a control packet. User packets may be presented in any order and may be re-ordered by some configurations of the VIP IP cores (e.g. the Deinterlacer II IP core when configured with 1 field of buffering). However, Intel recommends that the user packets precede the control packet.

Note: Some VIP IP cores, like Frame Buffer II, require a control packet to initialize storage. To ensure correct operation across all VIP components, Intel mandates that at least one control packet must be sent to an IP core prior to any video packets.

Figure 11. Avalon-ST Recommended Packet Ordering

The VIP IP cores always transmit a control packet before any video packet, and the user packets either follow or precede this control packet, depending upon the function of the IP core. When a VIP IP core receives an Avalon-ST Video control packet, the IP core decodes the height, width, and interlacing information from that packet and interprets any following Avalon-ST Video packets as being video of that format until it receives another control packet.

Most IP cores handle user packets, simply passing them through, or in the case of the Frame Buffer II IP core, writing and then reading them to memory. For IP cores that change the number of bits per symbol or symbols per pixel, additional padding is introduced to the user data.

All IP cores transmit a control packet before sending a video packet, even if no control packet has been received.

Stalling behavior (behavior when either a core is ready but there is no valid input data, or when a core has valid output data but the receiving core is not ready to receive it) varies according to the different cores. However, stalls propagate up and down the pipeline except where they can be absorbed through buffering within the cores themselves.

2.4. Avalon-ST Video Error Cases

The Avalon-ST protocol accepts certain error cases.

If a video packet has a different length to the one implied by the preceding control packet’s height, width and interlacing fields, it is termed as an early end of packet or late end of packet. All the VIP IP cores are able to accept this type of erroneous video, but the resultant output video may exhibit cropped or stretched characteristics. For example, the Deinterlacer II IP core has an integral stream cleaner which allows it to accept such packets when configured in complex motion adaptive modes.

If an Avalon-ST Video packet violates the Avalon-ST protocol in some way, for example by not raising startofpacket or endofpacket, this is a more serious error case and often results in a video pipeline locking up due to a freeze of the Avalon-ST bus.

3. Clocked Video

Most IP cores in the Video and Image Processing Suite transmit and receive video according to the Avalon-ST video standard.

The Clocked Video Input II (CVI II) IP core converts clocked video into the Avalon-ST Video control and data packets and the Clocked Video Output II (CVO II) IP core converts Avalon-ST video packets into clocked video. These two IP cores interface between Avalon-ST Video cores and video interface standards such as BT.656 and others as used in Displayport, Serial Digital Interface (SDI), and High-Definition Multimedia Interface (HDMI).

3.1. Video Formats

The Clocked Video IP cores create and accept clocked video formats.

The IP cores create and accept the following formats:

Video with synchronization information embedded in the data (in BT656 or BT1120 format)
Video with separate synchronization (H sync, V sync) signals

The BT656 and BT1120 formats use time reference signal (TRS) codes in the video data to mark the places where synchronization information is inserted in the data.

Figure 12. Time Reference Signal FormatThe TRS codes are made up of values that are not present in the video portion of the data, and they take the format shown in the figure below.

3.1.1. Embedded Synchronization Format: Clocked Video Output

For the embedded synchronization format, the CVO IP cores insert the horizontal and vertical syncs and field into the data stream during the horizontal blanking period.

The IP cores create a sample for each clock cycle on the vid_data bus.

There are two extra signals only used when connecting to the SDI IP core. They are vid_trs, which is high during the 3FF sample of the TRS, and vid_ln, which produces the current SDI line number. These are used by the SDI IP core to insert line numbers and cyclical redundancy checks (CRC) into the SDI stream as specified in the 1.5 Gbps HD-SDI and 3 Gbps 3G-SDI standards.

The CVO IP cores insert any ancillary packets (packets with a type of 13 or 0xD) into the output video during the vertical blanking. The IP cores begin inserting the packets on the lines specified in its parameters or mode registers (ModeN Ancillary Line and ModeN F0 Ancillary Line). The CVO IP cores stop inserting the packets at the end of the vertical blanking.

3.1.1.1. Clocked Video Output and SDI II TX Interface

The clocked video interface from the CVO II IP core expects the SDI II TX core to pull data and its attributes by asserting the CVO II IP core's read enable signal (sdi_cvo_rden).

Figure 13. Clocked Video Output II with SDI II TX Interface Timing Diagram

Figure 14. Clocked Video Output II with SDI II TX Interface Block Diagram

The rate that the SDI II TX core uses to pull the data depends on the SDI standard. The table below describes the officially supported cadences.

Note: The IP core does not support SD-SDI (20 bits) mode for multi-rate designs with SDI Resampler.

Table 9. sdi_cvo_rden (tx_dataout_valid) Cadence
SDI Standard	tx_dataout_valid Cadence from SDI II TX Core
SD-SDI (10 bit)	1H 4L 1H 5L
SD-SDI (20 bit)	1H 10L
HD-SDI	1H 1L
3G-SDI	Always H
6G-SDI	Always H
12G-SDI	Always H

3.1.2. Embedded Synchronization Format: Clocked Video Input

The CVI IP cores support both 8 and 10-bit TRS and XYZ words.

When in 10-bit mode, the IP cores ignore the bottom 2 bits of the TRS and XYZ words to allow easy transition from an 8-bit system.

Table 10. XYZ Word FormatThe XYZ word contains the synchronization information and the relevant bits of its format.
Bits	10-bit	8-bit	Description
Unused	[5:0]	[3:0]	These bits are not inspected by the CVI IP cores.
H (sync)	6	4	When 1, the video is in a horizontal blanking period.
V (sync)	7	5	When 1, the video is in a vertical blanking period.
F (field)	8	6	When 1, the video is interlaced and in field 1. When 0, the video is either progressive or interlaced and in field 0.
Unused	9	7	These bits are not inspected by the CVI IP cores.

For the embedded synchronization format, the vid_datavalid signal indicates a valid BT656 or BT1120 sample. The CVI IP cores only read the vid_data signal when vid_datavalid is 1.

Figure 15. Vid_datavalid Timing

The CVI IP cores extract any ancillary packets from the Y channel during the vertical blanking. Ancillary packets are not extracted from the horizontal blanking.

Clocked Video Input IP core—The extracted packets are produced through the CVI IP core's Avalon-ST output with a packet type of 13 (0xD).
Clocked Video Input II IP core— The extracted packets are stored in a RAM in the IP core, which can be read through the control interface.

3.1.3. Separate Synchronization Format

The separate synchronization format uses separate signals to indicate the blanking, sync, and field information.

The CVO IP cores create horizontal and vertical syncs and field information through their own signals. The CVO IP cores create a sample for each clock cycle on the vid_data bus. The vid_datavalid signal indicates when the vid_data video output is in an active picture period of the frame.

Table 11. Clocked Video Input and Output Signals for Separate Synchronization Format Video
Signal Name	Description
`vid_h_sync`	When 1, the video is in a horizontal synchronization period.
`vid_v_sync`	When 1, the video is in a vertical synchronization period.
`vid_f`	When 1, the video is interlaced and in field 1. When 0, the video is either progressive or interlaced and in field 0.
`vid_h`	When 1, the video is in a horizontal blanking period, (only for Clocked Video Output IP core).
`vid_v`	When 1, the video is in a vertical blanking period, (only for Clocked Video Output IP core).
`vid_de`	When asserted, the video is in an active picture period (not horizontal or vertical blanking). This signal must be driven for correct operation of the IP cores. Note: Only for Clocked Video Input IP cores.
`vid_datavalid`	Clocked Video Output IP cores: When asserted, the video is in an active picture period (not horizontal or vertical blanking) Clocked Video Input IP cores: Tie this signal high if you are not oversampling your input video.

Figure 16. Separate Synchronization Signals Timing Diagram

The CVI IP cores only read the vid_data, vid_de, vid_h_sync, vid_v_sync, and vid_f signals when vid_datavalid is 1. This allows the CVI IP cores to support oversampling where the video clock is running at a higher rate than the pixel clock.

3.1.4. Video Locked Signal

The vid_locked signal indicates that the clocked video stream is active.

When the vid_locked signal has a value of 1, the CVI IP cores take the input clocked video signals as valid, and read and process them as normal. When the signal has a value of 0 (if for example the video cable is disconnected or the video interface is not receiving a signal):

Clocked Video Input IP core: The IP core takes the input clocked video signals as invalid and do not process them.
Clocked Video Input II IP core: The vid_clk domain registers of the IP core are held in reset and no video is processed. The control and Avalon-ST Video interfaces are not held in reset and will respond as normal. The vid_locked signal is synchronized internally to the IP core and is asynchronous to the vid_clk signal.

If the vid_locked signal goes invalid while a frame of video is being processed, the CVI IP cores end the frame of video early.

3.1.5. Clocked Video and 4:2:0 Chroma Subsampling

Other than the Chroma Resampler II IP core, none of the VIP IP cores offer explicit support for clocked video with 4:2:0 subsampling.

When processing 4:2:0 streams, you need to use chroma resampler to convert to 4:2:2 or 4:4:4 before or after any video processing is performed. The video streams can be converted back to 4:2:0 for transmission from the pipeline with another chroma resampler.

Figure 17. Pipeline with Color Space and Subsampling Adaptive Interfaces Block DiagramThe figure below shows how the chroma resampler handles the conversion.

The connectivity cores (SDI, HDMI and DisplayPort) present data at their interfaces in accordance with their respective standards, not in line with the AV-ST mappings. Before the data is processed, it must be arranged in an Avalon-ST compliant manner. The input and output preparation areas present a gray area in the pipeline where video packets are Avalon-ST video compliant but the arrangement of data within the packet may not match the expectations of the processing blocks.

3.1.5.1. Avalon-ST Video Control Packets for 4:2:0 Video

When the Chroma Resampler II IP core receives or transmits an Avalon-ST video carrying a frame in 4:2:0 format, the control packet of the associated frame will have a horizontal resolution that is half the actual resolution of the frame.

The triplet of 2 luma and 1 chroma values in 4:2:0 represent 2 pixels but they are carried in a single “pixel” of symbols in the AV-ST domain. Because the triplet is counted as a single pixel, the value of the horizontal resolution carried in control packets will be half that of the resolution the data actually represents.

For example, a UHD frame received in 4:4:4 will have a control packet specifying 3840x2160. The same resolution frame received in 4:2:0 will have a control packet indicating 1920x2160.

The chroma resampler automatically adjusts control packets depending on the conversion being applied.

When converting from 4:2:0 to either 4:2:2 or 4:4:4, the horizontal width will be doubled.
When converting from 4:4:4 or 4:2:2 down to 4:2:0 the horizontal width is halved.

If you choose to create your own 4:2:0 processing blocks, the half horizontal width control packet requirement must be met.

3.1.5.2. 4:2:0 Clocked Video

The CVI II and CVO II IP cores are agnostic of the video standard being driven through them.

Figure 18. CVI/CVO Map DataFigure below shows how a CVI II IP core, configured for 1 pixel in parallel with 3 color planes per pixel, maps all input pixels to the same, 3 color plane output pixel.

For 4:2:2, the “empty” data on the unused input data line becomes a color plane of “empty” data at the output of the CVI II. Likewise, the 4:2:0 triplet gets mapped into a single 3 color plane pixel at the output. This has a significant impact on handling 4:2:0.

The CVI II IP core automatically creates control packets where the horizontal width is half the real frame width. To select its timing parameters, the CVO II IP core compares the control packet dimensions against those held in the mode banks. To match a 4:2:0 control packet, the mode bank width must be recorded as half of the actual frame dimension so that it matches the control packet. If the full width is entered to the mode bank, the correct timing parameters will not be matched.

3.1.5.3. Resampling 4:2:0

When the 4:2:0 samples brought into the pipeline, they should be resampled to 4:2:2 or 4:4:4 to be compatible with the other processing IP cores.

The table below summarizes the conversions required to move from each sampling scheme to a specific pipeline sampling scheme. For systems requiring color space conversion and chroma resampling, the order of chroma resampler and color space converter in the system is determined by whether the pipeline target is RGB or YCbCr.

Table 12. Conversions Required to Present an Input Video Stream in the Designated Pipeline Format
Input Format	Pipeline Format	Conversion
RGB	RGB	None
YCbCr 4:4:4	RGB	Color Space Conversion
YCbCr 4:2:2	RGB	Chroma Resampling Color Space Conversion
YCbCr 4:2:0	RGB	Chroma Resampling Color Space Conversion
YCbCr 4:4:4	YCbCr 4:4:4	None
YCbCr 4:2:2	YCbCr 4:4:4	Chroma Resampling
YCbCr 4:2:0	YCbCr 4:4:4	Chroma Resampling
RGB	YCbCr 4:4:4	Color Space Conversion
YCbCr 4:4:4	YCbCr 4:2:2	Chroma Resampling
YCbCr 4:2:2	YCbCr 4:2:2	None
YCbCr 4:2:0	YCbCr 4:2:2	Chroma Resampling
RGB	YCbCr 4:2:2	Color Space Conversion Chroma Resampling

The Chroma Resampler II IP core makes assumptions about the arrangement of pixel sub-samples for its resampling. It could be because that the connectivity core does not supply pixels with the sub-samples in this order. If this is the case, then use a color plane sequencer to rearrange the sub-samples into the correct order.

Refer to the Chroma Resampler II IP Core for the expected sample ordering.

4. VIP Run-Time Control

All the Video and Image Processing IP cores have an optional simple run-time control interface that comprises a set of control and status registers, accessible through an Avalon Memory-Mapped (Avalon-MM) slave port.

All the IP cores have an optional simple run-time control interface that comprises a set of control and status registers, accessible through an Avalon-MM slave port. A run-time control configuration has a mandatory set of three registers for every IP core, followed by any function-specific registers.

Table 13. Video and Image Processing IP Core Run-time Control Registers
Address	Data		Description
0	Bits 31:1 = X	Bit 0 = `Go`	Control register
1	Bits 31:1 = X	Bit 1 = `Status`	Status register
2	Core specific		Interrupt register

Figure 19. Video and Image Processing Suite IP Cores BehaviorThe figure below illustrates the behavior of the Go and Status bits for every IP core when run-time control is configured, together with the steady-state running behavior that is always present.

Note: The Test Pattern Generator II and Mixer II IP cores deviate from this behavior. These IP cores start transmitting video before receiving any Avalon-ST Video packets.

When you enable run-time control, the Go bit gets deasserted by default. If you do not enable run-time control, the Go is asserted by default.

Every IP core retains address 2 in its address space to be used as an interrupt register. However this address is often unused because only some of the IP cores require interrupts.

5. Getting Started

The Video and Image Processing Suite IP cores are installed as part of the Intel^® Quartus^® Prime Standard Edition installation process.

5.1. IP Catalog and VIP Parameter Editor

The Video and Image Processing Suite IP cores are available only through the Platform Designer IP Catalog in the Intel^® Quartus^® Prime. The Platform Designer IP Catalog (Tools > Platform Designer ) and parameter editor help you easily customize and integrate IP cores into your project. You can use the Platform Designer IP Catalog and parameter editor to select, customize, and generate files representing your custom IP variation.

Double-click on any IP core name to launch the parameter editor and generate files representing your IP variation. The parameter editor prompts you to specify your IP variation name, optional ports, architecture features, and output file generation options. The parameter editor generates a top-level .qsys file representing the IP core in your project. Alternatively, you can define an IP variation without an open Intel^® Quartus^® Prime project. When no project is open, select the Device Family directly in IP Catalog to filter IP cores by device.

Use the following features to help you quickly locate and select an IP core:

Search to locate any full or partial IP core name in IP Catalog.
Right-click an IP core name in IP Catalog to display details about supported devices, installation location, and links to documentation.

Upgrading VIP Designs

In the Intel^® Quartus^® Prime software, if you open a design from previous versions that contains VIP components in a Platform Designer system, you may get a warning message with the title "Upgrade IP Components". This message is just letting you know that VIP components within your Platform Designer system need to be updated to their latest versions, and to do this the Platform Designer system must be regenerated before the design can be compiled within the Intel^® Quartus^® Prime software. The recommended way of doing this with a VIP system is to close the warning message and open the design in Platform Designer so that it is easier to spot any errors or potential errors that have arisen because of the design being upgraded.

VIP IP Interoperability

The IP in the VIP Suite interoperate with each other. Their streaming interfaces use the same standard of data transmission layered on top of Avalon streaming interface. You can connect them together in Platform Designer and create a chain to build a video processing pipeline. The automatic insertion of Avalon streaming adapters in Platform Designer is disabled for video IP cores. Inserting width adapters breaks the semantics of the Avalon streaming video protocol. You must manually insert the Color Space Sequencer IP to perform data width conversion.

5.1.1. Specifying IP Core Parameters and Options

Follow these steps to specify IP core parameters and options.

In the Platform Designer IP Catalog (Tools > IP Catalog), locate and double-click the name of the IP core to customize. The parameter editor appears.
Specify a top-level name for your custom IP variation. This name identifies the IP core variation files in your project. If prompted, also specify the target FPGA device family and output file HDL preference. Click OK.
Specify parameters and options for your IP variation:
- Optionally select preset parameter values. Presets specify all initial parameter values for specific applications (where provided).
- Specify parameters defining the IP core functionality, port configurations, and device-specific features.
- Specify options for generation of a timing netlist, simulation model, testbench, or example design (where applicable).
- Specify options for processing the IP core files in other EDA tools.
Note: The Video and Image Processing IP cores support only ModelSim* simulation software.
Click Finish to generate synthesis and other optional files matching your IP variation specifications. The parameter editor generates the top-level .qsys IP variation file and HDL files for synthesis and simulation. Some IP cores also simultaneously generate a testbench or example design for hardware testing.
To generate a simulation testbench, click Generate > Generate Testbench System. Generate Testbench System is not available for some IP cores that do not provide a simulation testbench.
To generate a top-level HDL example for hardware verification, click Generate > HDL Example. Generate > HDL Example is not available for some IP cores.

The top-level IP variation is added to the current Intel^® Quartus^® Prime project. Click Project > Add/Remove Files in Project to manually add a .qsys ( Intel^® Quartus^® Prime Standard Edition) or .ip ( Intel^® Quartus^® Prime Pro Edition) file to a project. Make appropriate pin assignments to connect ports.

5.2. Installing and Licensing IP Cores

The Intel^® Quartus^® Prime software installation includes the Intel FPGA IP library. This library provides useful IP core functions for your production use without the need for an additional license. Some Intel^® FPGA IP IP functions in the library require that you purchase a separate license for production use. The OpenCore^® feature allows evaluation of any Intel FPGA IP core in simulation and compilation in the Intel^® Quartus^® Prime software. Upon satisfaction with functionality and performance, visit the Self Service Licensing Center to obtain a license number for any Intel FPGA product.

The Intel^® Quartus^® Prime software installs IP cores in the following locations by default:

Figure 20. IP Core Installation Path

Table 14. IP Core Installation Locations
Location	Software	Platform
`<drive>`:\intelFPGA_pro\quartus\ip\altera	Intel^® Quartus^® Prime Pro Edition	Windows
`<drive>`:\intelFPGA\quartus\ip\altera	Intel^® Quartus^® Prime Standard Edition	Windows
`<home directory>`:/intelFPGA_pro/quartus/ip/altera	Intel^® Quartus^® Prime Pro Edition	Linux*
`<home directory>`:/intelFPGA/quartus/ip/altera	Intel^® Quartus^® Prime Standard Edition	Linux

5.2.1. Intel FPGA IP Evaluation Mode

The free Intel^® FPGA IP Evaluation Mode allows you to evaluate licensed Intel^® FPGA IP cores in simulation and hardware before purchase. Intel^® FPGA IP Evaluation Mode supports the following evaluations without additional license:

Simulate the behavior of a licensed Intel^® FPGA IP core in your system.
Verify the functionality, size, and speed of the IP core quickly and easily.
Generate time-limited device programming files for designs that include IP cores.
Program a device with your IP core and verify your design in hardware.

Intel^® FPGA IP Evaluation Mode supports the following operation modes:

Tethered—Allows running the design containing the licensed Intel^® FPGA IP indefinitely with a connection between your board and the host computer. Tethered mode requires a serial joint test action group (JTAG) cable connected between the JTAG port on your board and the host computer, which is running the Intel^® Quartus^® Prime Programmer for the duration of the hardware evaluation period. The Programmer only requires a minimum installation of the Intel^® Quartus^® Prime software, and requires no Intel^® Quartus^® Prime license. The host computer controls the evaluation time by sending a periodic signal to the device via the JTAG port. If all licensed IP cores in the design support tethered mode, the evaluation time runs until any IP core evaluation expires. If all of the IP cores support unlimited evaluation time, the device does not time-out.
Untethered—Allows running the design containing the licensed IP for a limited time. The IP core reverts to untethered mode if the device disconnects from the host computer running the Intel^® Quartus^® Prime software. The IP core also reverts to untethered mode if any other licensed IP core in the design does not support tethered mode.

When the evaluation time expires for any licensed Intel^® FPGA IP in the design, the design stops functioning. All IP cores that use the Intel^® FPGA IP Evaluation Mode time out simultaneously when any IP core in the design times out. When the evaluation time expires, you must reprogram the FPGA device before continuing hardware verification. To extend use of the IP core for production, purchase a full production license for the IP core.

You must purchase the license and generate a full production license key before you can generate an unrestricted device programming file. During Intel^® FPGA IP Evaluation Mode, the Compiler only generates a time-limited device programming file (<project name> _time_limited.sof) that expires at the time limit.

Figure 21. Intel^® FPGA IP Evaluation Mode Flow

Note: Refer to each IP core's user guide for parameterization steps and implementation details.

Intel^® licenses IP cores on a per-seat, perpetual basis. The license fee includes first-year maintenance and support. You must renew the maintenance contract to receive updates, bug fixes, and technical support beyond the first year. You must purchase a full production license for Intel^® FPGA IP cores that require a production license, before generating programming files that you may use for an unlimited time. During Intel^® FPGA IP Evaluation Mode, the Compiler only generates a time-limited device programming file (<project name> _time_limited.sof) that expires at the time limit. To obtain your production license keys, visit the Self-Service Licensing Center.

The Intel^® FPGA Software License Agreements govern the installation and use of licensed IP cores, the Intel^® Quartus^® Prime design software, and all unlicensed IP cores.

6. VIP Connectivity Interfacing

Avalon-ST Video expects pixel subsamples to be arranged in particular orders, depending on the sampling method selected.

While the Color planes transmitted in parallel and Number of color planes parameters define an interface that is capable of carrying the sampling methods, they do not enforce the transmission of particular sub-samples in particular symbols of the Avalon-ST video packet.

You have to understand the arrangement of the color planes on entry to the pipeline and any reconfiguration of this order performed by the components within the pipeline. This is a particular concern around the connectivity points. The connectivity IP cores present data arranged according to their respective standards. When connected to a clocked video component, the clocked video components will package the data as it is presented to the IP core. They do not re-arrange it. In simple terms, on each clock cycle during the active video, the Clocked Video Input (CVI) II IP samples the entire data bus and divides the samples into pixels according to the Number of color planes, Bits per color plane, and Pixels in parallel parameters used to configure the module.

Figure 22. Variable Interpretation of Pixels on a Clocked Video Data Bus Based on Parameterization

If the configuration selected were PiP=1 and CP=4, but a 10-bit RGB signal were being fed on Data [29:30], the output pixels will still be 40 bits in size, the unused data bits having been sampled.

The converse function of the Clocked Video Output (CVO) II IP drives the entire data bus on each clock cycle. To drive 10-bit RGB on Data[29:0] in the PiP=1 and CP=4 configuration, the VIP pipeline would have to generate 40-bit pixels containing the 30 data bits and 10 null bits.

6.1. Avalon-ST Color Space Mappings

The Avalon-ST expected arrangement differs for the various supported color spaces.

Figure 23. Expected Order of Chroma Samples for Avalon-ST Video

The CVI and CVO blocks offer a simple mechanism to present data in the AV-ST video format but they do not offer functions to remap the many different data mappings the connectivity cores present.

6.1.1. Interfacing with High-Definition Multimedia Interface (HDMI)

The order that the HDMI core presents chroma samples differs from the Avalon-ST expectations for YCbCr 4:4:4 and 4:2:2 sampling schemes.

Figure 24. Intel HDMI IP Core Chroma Sampling

For YCbCr 4:4:4, it is necessary to perform the translation shown in the figure below to meet the Avalon-ST requirements. If the system only handles YCbCr, then you can use the Color Plane Sequencer II IP core to perform this remapping. If the system handles both RGB and YCbCr, then you need to use the Color Space Converter II IP core to convert between RGB and YCbCr and also to remap the color plane ordering for YCbCr 4:4:4.

Figure 25. Remapping HDMI 4:4:4 to Avalon-ST Video 4:4:4

YCbCr 4:2:2 require two potential remappings. Symbols 2 and 1 carry the upper 8 bits of the chroma and luma samples. If the system supports 10- or 12-bit depth, the additional bits are carried together in symbol 0. To support 10 or 12 bit, YCbCr 4:2:2 requires the recombining of these lower bits with the upper bits in 2 distinct symbols as shown in the figure below.

Figure 26. Remapping HDMI YCbCr 4:2:2 to Avalon-ST YCbCr 4:2:2

At present, a Mux instantiated between the HDMI IP core and the clocked video IP core implements the recombining of the 4:2:2 MSBs with LSBs. Future clocked video IP cores will support this remapping internally.

Note: For 8-bit inputs, recombination is not necessary.

The positioning of the luma and chroma in the upper 2 symbols is at odds with the Avalon-ST requirement for the chroma in the bottom symbol and luma in the symbol above. The luma samples are in the correct place but the chroma samples must be remapped from the upper symbol to lower symbol.

If the system only handles YCbCr, then you can use the Color Plane Sequencer II IP core to remap the symbols. If the system handles both RGB and YCbCr, then you need to use the Color Space Converter II IP core to remap the chroma samples.

6.1.2. Interfacing with DisplayPort

YCbCr 4:4:4 requires the same translation as the HDMI IP core.

Figure 27. Remapping DisplayPort 4:4:4 to Avalon-ST Video 4:4:4

Figure 28. Intel DisplayPort IP Core Chroma Sampling

If the system only handles YCbCr, then you can use the Color Plane Sequencer II IP core to perform the remapping. If the system handles both RGB and YCbCr, then you need to use the Color Space Converter II IP core to convert between RGB and YCbCr and also to remap the color plane ordering for YCbCr 4:4:4.

6.1.3. Interfacing with Serial Digital Interface (SDI)

The succession of SDI standards offers a range of mappings of video samples onto SDI symbols.

The SDI-clocked video interface deals in the SDI symbols, not the video samples. This means that only certain of the SDI mappings are directly compatible with Avalon-ST video requirements. The figure below shows the 8- and 10-bit YCbCr 4:2:2 mappings.

Figure 29. The SDI Sample Mappings Directly Compatible with Avalon-ST Video

You can transport other mappings into the Avalon-ST domain but to rearrange the samples into a compatible format, immediate remapping into the Avalon-ST format is required. The clocked video units do not perform any remapping function.

Figure 30. SMPTE-372 Dual Link Carrying Dual StreamsYCbCr 4:2:2 delivered as SMPTE-372 dual-link format over 2 20 bit buses can be configured either as a single pixel of 4 parallel color planes or 2 pixels of 2 parallel color planes.

If a single pixel configuration is used, then the horizontal resolution would be correct in the Avalon-ST control packet. If the 2 pixel solution were selected, the Avalon-ST control packet would report a width 2x the actual horizontal resolution (since an entire extra line is carried on Link B but it’s pixels would be counted as part of Link As). In both cases, the Avalon-ST control packet would report a vertical resolution ½ the actual resolution since 2 lines have been handled as 1. Any remapping logic has to account for this.

Figure 31. Clocked Video Input Interpretations of SMPTE-372 Dual Link 10-bit RGBA Mapping For 10-bit RGBA carried on a SMPTE-372 dual link remapping is required since 2 cycles of samples are required to assemble complete RGB pixel.

In this case using 1 pixel, 4 color planes in parallel will mean that the control packet reflects the true dimensions of the incoming image. If 2 pixels with 2 color planes is configured, then the control packet will report a horizontal width 2x the actual width.

The 12-bit data mappings also require processing to recombine the relevant SDI symbols into the correct pixel components.

Figure 32. Clocked Video Input Interpretations of SMPTE-372 Dual Link 12-bit RGBA Mapping The12-bit RGB mapping matches the 12-bit YCbCr 4:4:4 mapping if you replace R with Cr, G with Y and B with Cb.

Figure 33. Clocked Video Input Interpretations of SMPTE-372 Dual Link 12-bit YCbCr Mapping The 12 bit 4:2:2 mapping with optional alpha channel also spreads pixel components across multiple SDI symbols requiring remapping to an Avalon-ST Video compliant arrangement.

If you select 2 pixel in parallel configuration, then the control packets from a CVI will report 2x actual width. Going into the CVO, the packets would need to report 2x the actual width.

6.1.4. Unsupported SDI Mappings

The key requirement for compatibility with the clocked video units is that the TRS code words be presented in the 10 LSBs of the clocked video interface in the order of 3FF, 000, 000, XYZ.

The CVI can only identify streams demarked in this way. The CVO can only generate streams demarked in this way.

This means that the SMPTE-425 Level B mappings are not directly supported by the clocked video units. Because both Level B-DL and Level B-DS mappings modify the arrival order of TRS codes, these streams must be demultiplexed before clocked video inputs. After clocked video outputs, the streams must be multiplexed together.

Figure 34. Clocked Video Requirements of TRS CodesThe figure below shows that the clocked video units do not handle dual link and dual stream mappings in a single stream (SMPTE-425 level B). Separation of streams must be handled before entry to the CVI. Dual CVO outputs must be recombined into a single stream after the CVO.

6.1.5. 12G-SDI

The CVI II and CVO II IP cores support two-sample interleave (2SI) format variant of the 12G-SDI specification. The IP cores do not support square divisions quad split format.

You can enable the support for 12G-SDI by turning on the Support 6G and 12G-SDI parameter in the CVI II and CVO II IP parameter editors. Turning on the Support 6G and 12G-SDI parameter enables the SDI Resampler block within a CVI II or CVO II instance. The SDI resampler enables the support for 12G-SDI format.

The SDI Resampler also supports 6G-, 3G-, HD-, and SD-SDI formats. Conversion to Avalon-ST video format is possible only for configurations with 4 pixels in parallel. With the presence of the SDI Resampler, the CVI II and CVO II IPs are able to provide seamless switching between 12G-, 6G-, 3G-, HD-, and SD-SDI formats.

Note: The Support 6G and 12G-SDI parameter is available only for embedded sync modes..

6.1.5.1. 12G-SDI with Clocked Video Input II

You can configure the CVI II IP core to allow 12G-SDI packets pass into the Avalon-ST video domain.

The 12G-SDI option uses the two-sample interleave (2SI) pattern, which transmits two lines simultaneously, The simultaneous transmission of two lines means that the active width received by the CVI II IP core in between two sets of horizontal SYNC is two times the actual width. The CVI II IP core applies the appropriate scaling before reporting to the Active Sample Count status register and the Avalon-ST video control packet. The 2SI pattern also requires that the clocked video input to be remapped to raster scan order for processing in a VIP pipeline. The CVI II IP core performs the remapping.

Figure 35. Clocked Video Capture of 12G-SDI Mapping and Resampling (Remapping) for Avalon-ST Video

When a 6G-SDI packet is carried over the 12G link, half of the 80-bit data bus will be unused. The SDI Resampler within the CVI II IP core discards the null pixels, splits the pixel components, and accumulates successive components to recombine the original image format into Avalon-ST format.

Note: The SDI Resampler is only enabled when you turn on the Support 6G and 12G-SDI parameter with the pixels in parallel fixed to 4 in the CVI II or CVO II Intel FPGA IP parameter editor.

Figure 36. Clocked Video Capture of 6G-SDI Mapping Transported Over 12G-SDI Link and Required Remapping

When a 3G-SDI packet is carried over a 12G-SDI link, the CVI II IP core captures 3 null pixels for every active pixel. The SDI Resampler discards these null pixels to reestablish the 3G-SDI sample order in the Avalon-ST video frame, as shown in the figure below.

Figure 37. Clocked Video Capture of 3G-SDI Mapping Transported Over 12G-SDI Link

Table 15. SDI Resampler with CVI II IP ParametersThe table shows the differences in the CVI II IP core with the presence of the SDI Resampler.
Parameters	Support 6G and 12G-SDI Off	Support 6G and 12G-SDI On
SDI Resampler	Not available	Present
Pixels in parallel	1	4
Standard width range	1–16	3
Multi-rate Mode	3G-SDI, HD-SDI, SD-SDI	12G-SDI, 6G-SDI, 3G-SDI, HD-SDI, SD-SDI
Square Division Quad format	Not supported	Not supported
2 Sample-interleave (2SI) format	Not supported	Supported
12G-SDI with 8 streams interleaved	Not supported	Supported
12G-SDI with 16 streams interleaved	Not supported	Not supported
6G-SDI with 4 streams interleaved	Not supported	Not supported
6G-SDI with 8 streams interleaved	Not supported	Supported
3G-SDI Level A	Supported	Supported
3G-SDI Level B	Not supported	Not supported
HD-SDI	Supported	Supported
SD-SDI	Supported	Supported
4:4:4	Not supported	Not supported
4:2:2	Supported	Supported
4:2:0	Not supported	Not supported
Minimum active frame size	32x32	3G-SDI, HD-SDI, SD-SD: 32x32 12G-SDI, 6G-SDI: 32x64
Active width supported	Modulo 1	12G-SDI, 6G-SDI, 3G-SDI, HD-SDI: Modulo 4 SD-SD: Modulo 8
Active height supported	Modulo 1	Modulo 1
Clipping	Supported	Not supported
Padding	Supported	Not supported

6.1.5.2. 12G-SDI with Clocked Video Output II

The mapping and reordering of the pixels for clocked video outputs are the reverse of the clocked video inputs.

Table 16. SDI Resampler with CVO II IP ParametersThe table shows the differences in the CVO II IP core with the presence of the SDI Resampler.
Parameters	Support 6G and 12G-SDI Off	Support 6G and 12G-SDI On
SDI Resampler	Not available	Present
Pixels in parallel	1	4
Standard width range	1–16	3
Multi-rate Mode	3G-SDI, HD-SDI, SD-SDI	12G-SDI, 6G-SDI, 3G-SDI, HD-SDI, SD-SDI
Square Division Quad format	Not supported	Not supported
2 Sample-interleave (2SI) format	Not supported	Supported
12G-SDI with 8 streams interleaved	Not supported	Supported
12G-SDI with 16 streams interleaved	Not supported	Not supported
6G-SDI with 4 streams interleaved	Not supported	Not supported
6G-SDI with 8 streams interleaved	Not supported	Supported
3G-SDI Level A	Supported	Supported
3G-SDI Level B	Not supported	Not supported
HD-SDI	Supported	Supported
SD-SDI	Supported	Supported
4:4:4	Not supported	Not supported
4:2:2	Supported	Supported
4:2:0	Not supported	Not supported
Minimum active frame size	32x32	Fixed resolution: 32x32 Note: For switching resolutions, if the frame is smaller than the combined capacity of the CVO IP core’s FIFOs and pipelines, the behavior is undetermined. Use the set of standard resolutions defined in the CTA-861-G specifications.
Active width supported	Modulo 1	12G-SDI, 6G-SDI, 3G-SDI, HD-SDI: Modulo 4 SD-SDI: Modulo 8
Active height supported	Modulo 1	12G-SDI, 6G-SDI: Modulo 2 3G-SDI, HD-SDI, SD-SD: Modulo 1
Dynamic control over `Go` bit	Supported	Cannot be disabled once enabled.
Low latency mode	Supported	Not supported

Note: In simulation, you can create artificial short bursts of just a few frames before switching either video standard or video resolution, or both. If two back-to-back switches occur very rapidly e.g. switching from A-to-B-to-C or A-to-B-to-A, the clocked video output’s behavior is nondeterministic.

7. Clocked Video Interface IPs

The Clocked Video Interface IPs convert clocked video formats (such as BT656, BT1120, and DVI) to Avalon streaming video; and vice versa. You can configure these IP at run time using an Avalon memory-mapped target interface.

The Clocked Video Input II IP:

Converts clocked video formats (such as BT656, BT1120, and DVI) to Avalon streaming video.
Provides clock crossing capabilities to allow video formats running at different frequencies to enter the system.
Strips incoming clocked video of horizontal and vertical blanking, leaving only active picture data.

The Clocked Video Output II IP:

Converts data from the flow controlled Avalon streaming video protocol to clocked video.
Formats Avalon streaming video into clocked video. Inserts horizontal and vertical blanking and generates horizontal and vertical synchronization information using the Avalon streaming video control and active picture packets.
Provides clock crossing capabilities to allow video formats running at different frequencies to be created from the system.

7.1. Supported Features for Clocked Video Output II IP

The Clocked Video Output II IP support the following features.

HDMI SD/HD
HDMI 4K
3G-SDI
12G-SDI
Note: The IP does not support Low Latency Mode if you turn on the Use 6G and 12G-SDI parameter.
Low latency mode
Note: The IP does not support Low Latency Mode if you turn on the Use 6G and 12G-SDI parameter.
Full frame mode

7.2. Control Port

To configure a clocked video IP using an Avalon memory-mapped target interface, turn on Use control port in the parameter editor.

Initially, the IP is disabled and does not transmit any data or video. However, the Clocked Video Input IP still detects the format of the clocked video input and raise interrupts; and the Clocked Video Output IP still accepts data on the Avalon streaming video interface if the input FIFO buffer has space.

The sequence for starting the output of the IP:

Write a 1 to Control register bit 0.
Read Status register bit 0. When this bit is 1, the IP starts transmitting data or video. The transmission starts on the next start of frame or field boundary.
Note: For the Clocked Video Input IP, the frame or field matches the Field order parameter settings.

The sequence for stopping the output of the IP:

Write a 0 to Control register bit 0.
Read Status register bit 0. When this bit is 0, the IP stops transmitting data. The transmission ends on the next start of frame or field boundary.
Note: For Clocked Video Input IP, the frame or field matches the Field order parameter settings.

The starting and stopping of the IP synchronize to a frame or field boundary.

Table 17. Synchronization Settings for Clocked Video Input IPThe table lists the output of the IP with the different Field order settings.
Video Format	Field Order	Output
Interlaced	F1 first	Start, F1, F0, ..., F1, F0, Stop
Interlaced	F0 first	Start, F0, F1, ..., F0, F1, Stop
Interlaced	Any field first	Start, F0 or F1, ... F0 or F1, Stop
Progressive	F1 first	No output
Progressive	F0 first	Start, F0, F0, ..., F0, F0, Stop
Progressive	Any field first	Start, F0, F0, ..., F0, F0, Stop

7.3. Clocked Video Input IP Format Detection

The IP detects the format of the incoming clocked video and use it to create the Avalon streaming video control packet. The IP also provides this information in a set of registers.

Table 18. Format DetectionThe IP can detect different aspects of the incoming video stream.
Format	Description
Picture width (in samples)	The IP counts the total number of samples per line, and the number of samples in the active picture period. One full line of video is required before the IP can determine the width.
Picture height (in lines)	The IP counts the total number of lines per frame or field, and the number of lines in the active picture period. One full frame or field of video is required before the IP can determine the height.
Interlaced/Progressive	The IP detects whether the incoming video is interlaced or progressive. If it is interlaced, separate height values are stored for both fields. One full frame or field of video and a line from a second frame or field are required before the IP can determine whether the source is interlaced or progressive.
Standard	The IP provides the contents of the `vid_std` bus through the Standard register. When connected to the `rx_std` signal of an SDI IP instance, for example, these values can be used to report the standard (SD, HD, 3G, 6G, or 12G) of the incoming video. Note: In 3G, 6G, and 12G modes, the IP only supports YCbCr input color format with either 8 or 10 bits for each component.

7.3.1. Format Detection in Clocked Video Input II

After reset, if the CVI II IP has not determined the format of the incoming video, it uses the values you specify in the Avalon-ST Video Initial/Default Control Packet section in the parameter editor.

When the IP detects a resolution, it uses the resolution to generate the Avalon streaming viideo control packets until a new resolution is detected.

When the resolution valid bit in the Status register is 1, the Active Sample Count, F0 Active Line Count, F1 Active Line Count, Total Sample Count, F0 Total Line Count, F1 Total Line Count, and Standard registers are valid and contain readable values. The interlaced bit of the Status register is also valid and you can read it.

7.3.2. Interrupts

The Clocked Video Input IP produces a single interrupt line.

Table 19. Internal InterruptsThe table lists the internal interrupts of the interrupt line.
IP Core	Internal Interrupts	Description
Clocked Video Input II IP	Status update interrupt	Triggers when the stable bit, the resolution valid bit, the overflow sticky bit, or the picture drop sticky bit of the `Status` register changes value.
Clocked Video Input II IP	End of field/frame interrupt	If the synchronization settings are set to Any field first, triggers on the falling edge of the v sync. If the synchronization settings are set to F1 first, triggers on the falling edge of the F1 v sync. If the synchronization settings are set to F0 first, triggers on the falling edge of the F0 v sync. You can use this interrupt to trigger the reading of the ancillary packets from the control interface before the packets are overwritten by the next frame.

You can independently enable these interrupts using bits [2:1] of the Control register. You can read the interrupt values using bits [2:1] of the Interrupt register. Writing 1 to either of these bits clears the respective interrupt.

7.4. Clocked Video Output IP Video Modes

The video frame is described using the mode registers that you access through the Avalon memory-mapped control port.

If you turn off Use control port in the parameter editor for the IP, the output video format always has the format specified in the parameter editor.

You can configure the IP to support between 1 to 13 different modes and each mode has a bank of registers that describe the output frame.

When the IP receives a new control packet on the Avalon streaming video input, it searches the mode registers for a mode that is valid. The valid mode must have a field width and height that matches the width and height in the control packet.

The Video Mode Match register shows the selected mode:

If the IP finds a matching mode, it completes the current frame; duplicating data if needed before commencing output with the new settings at the beginning of the next frame.
If the IP does not find a matching mode, the video output format is unchanged.

If a new control packet is encountered before the expected end of frame, the IP completes the timing of the current frame with the remaining pixels taking the value of the last pixel output. The IP changes modes to the new packet at the end of this frame, unless you enable the Low Latency mode. During this period, when the FIFO buffer fills, the IP back-pressures the input until it is ready to transmit the new frame.

You must enable the Go bit to program the mode control registers. The sync signals, controlled by the mode control registers, reside in the video clock domain. The register control interface resides in the streaming clock domain. Enabling the Go bit, indicating that both clocks are running, avoids situations where a write in the streaming side cannot be issued to the video clock side because the video clock isn't running.

Figure 38. Progressive Frame ParametersThe figure shows how the register values map to the progressive frame format.

Figure 39. Interlaced Frame ParametersThe figure shows how the register values map to the interlaced frame format.

You can only write to the mode register of a mode bank if that mode is marked as invalid. To reconfigure mode 1:

Write 1 to the Bank Select register.
Write 0 to the Mode N Valid configuration register.
Write to the Mode N configuration registers, the Clocked Video Output II IP mirrors these writes internally to the selected bank.
Write 1 to the Mode N Valid register. The mode is now valid and you can select it.

You can configure a currently-selected mode in this way without affecting the video output of the IP.

If multiple modes match the resolution, the function selects the lowest mode. For example, the function selects Mode1 over Mode2 if both modes match. To allow the function to select Mode2, invalidate Mode1 by writing a 0 to its mode valid register. Invalidating a mode does not clear its configuration.

Figure 40. Mode Bank Selection

7.4.1. Interrupts

The Clocked Video Output IP produces a single interrupt line.

This interrupt line is the OR of the following internal interrupts:

Status update interrupt—triggers when the IP updates the Video Mode Match register when you select a new video mode.
Locked interrupt—triggers when the outgoing video SOF aligns to the incoming SOF.

You can independently enable both interrupts using bits [2:1] of the Control register. You can read their values using bits [2:1] of the Interrupt register. Writing 1 to either of these bits clears the respective interrupt.

7.5. Clocked Video Output II Latency Mode

The Clocked Video Output II IP provides a low latency mode..

You can enable the low latency mode by setting the Low latency mode parameter to 1 in the Clocked Video Output II parameter editor. In the low latency mode, when the IP sees an early end of frame, or a change of resolution, the IP immediately updates the selected mode, resets the internal counters, and starts transmitting the new frame. These changes happen even if the timing is only part way through the previous frame.

If you choose not to enable the low latency mode, an early end of packet or change of resolution pauses reading of the FIFO buffer until the timing of the current frame is completed. Only at this point, the IP updates any new timings and starts transmitting new frames. In this mode, if the IP receives a partial video frame , when the FIFO buffer has filled, it generates backpressure until the current frame (including vertical sync) has completed.

7.6. Generator Lock

Generator lock (Genlock) locks the timing of video outputs to a reference source. Sources that are locked to the same reference can be cleanly switched between on a frame boundary.

For Clocked Video Input II IP, the refclk_div signal is a pulse on the rising edge of the H sync which a PLL can align its output clock to.

Clocked Video Input II IP

For the Clocked Video Input II IP, the SOF signal produces a pulse on the rising edge of the V sync. For interlaced video, the pulse is only produced on the rising edge of the F0 field, not the F1 field. The IP indicates a start of frame by a rising edge on the SOF signal (0 to 1).

7.7. Underflow and Overflow

Moving between the domain of clocked video and the flow controlled Avalon streaming video can cause flow problems. The Clocked Video Interface IP contains a FIFO buffer that accommodates any bursts in the flow data when set to a large enough value. The FIFO buffer can accommodate any bursts as long as the input or output rate of the upstream or downstream Avalon streaming video components is equal to or higher than that of the incoming/outgoing clocked video.

Underflow

The FIFO buffer can accommodate any bursts if the output rate of the downstream Avalon streaming video components is equal to or higher than that of the outgoing clocked video. If not , the FIFO underflows. If underflow occurs, the CVO IP continues to produce video and resynchronizing the startofpacket for the next image packet, from the Avalon streaming video interface with the start of the next frame. You can detect the underflow by looking at bit 2 of the Status register. This bit is sticky and if an underflow occurs, it stays at 1 until the bit is cleared by writing a 1 to it.

Overflow

The FIFO buffer can accommodate any bursts if the input rate of the upstream Avalon streaming video components is equal to or higher than that of the incoming clocked video. If not, the FIFO overflows. If overflow occurs, the IP produces an early endofpacket signal to complete the current frame. It then waits for the next start of frame (or field) before resynchronizing to the incoming clocked video and beginning to produce data again. The overflow is recorded in bit [9] of the Status register. This bit is sticky, and if an overflow occurs, it stays at 1 until the bit is cleared by writing a 0 to it. In addition to the overflow bit, you can read the current level of the FIFO buffer from the Used Words register.

The IP uses the height and width parameters at the point the frame was completed early in the control packet of the subsequent frame. If you are reading back the detected resolution, these unusual resolution values can make the IP seem to be operating incorrectly. When actually, the downstream system is failing to service the CVI IP at the necessary rate.

7.8. Timing Constraints

You need to constrain the Clocked Video Input II and Clocked Video Output II IPs.

For these IPs, the .sdc files are automatically included by their respective .qip files. After adding the Platform Designer system to your design in the Intel^® Quartus^® Prime software, verify that the alt_vip_cvi_core.sdc or alt_vip_cvo_core.sdc has been included.

When you apply the .sdc file, you may see some warning messages similar to the format below:

Warning: At least one of the filters had some problems and could not be matched.
Warning: * could not be matched with a keeper.

You should expect these warnings, because in certain configurations the Intel^® Quartus^® Prime software optimizes unused registers and they no longer remain in your design.

Intel recommends that you place a frame buffer in any CVI to CVO system. Because the CVO II IP generates sync signals for a complete frame, even when video frames end early, the CVO II IP can continually generate backpressure to the CVI II IP so that it keeps ending packets early.

Placing a frame buffer may not be appropriate if the system requires latency lower than 1 frame. In this case, enable the Low Latency mode when you configure the CVO II IP.

7.9. Handling Ancillary Packets

The Clocked Video Interface IPs use active format description (AFD) extractor and inserter examples to handle ancillary packets.

Ancillary Packets (Clocked Video Input II IP)

When you turn on the Extract Ancillary Packets parameter in embedded sync mode, the Clocked Video Input IP extracts any ancillary packets that are present in the Y channel of the incoming video's vertical blanking. The ancillary packets are stripped of their TRS code and placed in a RAM. You can access these packets by reading from the Ancillary Packet register. The packets are packed end to end from their Data ID to their final user word.

The RAM is 16 bits wide—two 8-bit ancillary data words are packed at each address location. The first word is at bits 0–7 and the second word is at bits 8–15. A word of all 1's indicates that no further ancillary packets are present and can appear in either the first word position or the second word position.

Figure 41. Ancillary Packet RegisterThe figure shows the position of the ancillary packets. The different colors indicate different ancillary packets.

Use the Depth of ancillary memory parameter to control the depth of the ancillary RAM. If available space is insufficient for all the ancillary packets, excess packets will be lost. The IP fills the ancillary RAM from the lowest memory address to the highest during each vertical blanking period—the IP overwrites the packets from the previous blanking periods. To avoid missing ancillary packets, the ancillary RAM should be read every time the End of field/frame interrupt register triggers.

AFD Inserter (Clocked Video Output IP)

When the output of the AFD Inserter connects to the input of the Clocked Video Output IP, the AFD Inserter inserts an Avalon streaming video ancillary data packet into the stream after each control packet. The AFD Inserter sets the DID and SDID of the ancillary packet to make it an AFD packet (DID = 0x41, SDID = 0x5). The contents of the ancillary packet are controlled by the AFD Inserter register map.

You can get the AFD Extractor from <install_dir>\ip\altera\clocked_video_output\afd_example.

Table 20. AFD Inserter Register Map
Address	Register	Description
0	Control	When bit 0 is 0, the core discards all packets. When bit 0 is 1, the core passes through all non-ancillary packets.
1	—	Reserved.
2	—	Reserved.
3	AFD	Bits 0-3 contain the active format description code.
4	AR	Bit 0 contains the aspect ratio code.
5	Bar data flags	Bits 0-3 contain the bar data flags to insert.
6	Bar data value 1	Bits 0-15 contain bar data value 1 to insert.
7	Bar data value 2	Bits 0-15 contain bar data value 2 to insert.
8	AFD valid	When bit 0 is 0, an AFD packet is not present for each image packet. When bit 0 is 1, an AFD packet is present for each image packet.

7.10. Modules for Clocked Video Input II IP Core

The architecture for the Clocked Video Input II IP differs from the previous clocked video input IP.

Figure 42. Block Diagram for Clocked Video Input II IPThe figure shows a block diagram of the Clocked Video Input II IP architecture.

Table 21. Modules for Clocked Video Input II IPThe table below describes the modules in the Clocked Video Input II IP architecture.
Modules	Description
Sync_conditioner	In embedded sync mode, this module extracts the embedded syncs from the video data and produces `h_sync`, `v_sync`, `de`, and `f` signals. The module also extracts any ancillary packets from the video and writes them into a RAM in the control module. In separate sync modes, this module converts the incoming sync signals to active high and produces `h_sync`, `v_sync`, `de`, and `f` signals. If you turn on the Extract field signal parameter, the `f` signal is generated based on the position of the V-sync. If the rising edge of the V-sync occurs when `h_sync` is high, the `f` signal is set to 1, otherwise it is set to 0.
Resolution_detection	This module uses the `h_sync`, `v_sync`, `de`, and `f` signals to detect the resolution of the incoming video. The resolution consists of: width of the line width of the active picture region of the line (in samples) height of the frame (or fields in the case of interlaced video) height of the active picture region of the frame or fields (in lines) The IP writes the resolutions into a RAM in the control module. The resolution detection module also produces some additional information. It detects whether the video is interlaced by looking at the `f` signal. It detects whether the video is stable by comparing the length of the lines. If two outputs of the last three lines have the same length. the video is considered stable. Finally, it determines if the resolution of the video is valid by checking that the width and height of the various regions of the frame has not changed.
Write_buffer_fifo	This module writes the active picture data, marked by the `de` signal, into a FIFO that is used to cross over into the `is_clk` clock domain. If you set the Color plane transmission format parameter to Parallel for the output, the write_buffer_fifo also converts any incoming sequential video, marked by the `hd_sdn` signal, into parallel video before writing it into the FIFO buffer. The `Go` bit of the `Control` register must be 1 on the falling edge of the `v_sync` signal before the write_buffer_fifo module starts writing data into the FIFO buffer. If an overflow occurs because of insufficient room in the FIFO buffer, the module stops writing active picture data into the FIFO buffer. It waits for the start of the next frame before attempting to write in video data again.
Control	This module provides the register file that is used to control the IP through an Avalon memory-mapped target interface. It also holds the RAM that contains the detected resolution of the incoming video and the extracted auxiliary packet which is read by the av_st_output module, to form the control packets, and can also be read from the Avalon memory-mapped target interface. The RAM provides the clock crossing between the `vid_clk` and `is_clk` clock domains.
Av_st_output	This module creates the control packets, from the detected resolution read from the control module, and the video packets, from the active picture data read from the write_buffer_fifo module. The IP sends the packets to the Video Output Bridge which turns them into Avalon streaming video packets.
Sdi_resampler	This module is present only when you turn on the Support 6G and 12G-SDI parameter in the parameter editor. It reformats the video stream from 2SI 6G- and 12G-SDI formats into the Avalon streaming video raster format and reformats the video stream from 3G-, HD-, and SD-SDI formats into the Avalon streaming video format. Note: This module reformats the SDI video streams only for configurations with 4 pixels in parallel.

7.11. Clocked Video Input II Signals, Parameters, and Registers

7.11.1. Clocked Video Input II Interface Signals

Table 22. Clocked Video Input II Signals
Signal	Direction	Description
`main_reset_reset`	Input	The IP core asynchronously resets when you assert this signal. You must deassert this signal synchronously to the rising edge of the clock signal.
`main_clock_clk`	Input	The main system clock. The IP core operates on the rising edge of this signal.
`dout_data`	Output	`dout` port Avalon-ST `data` bus. This bus enables the transfer of pixel data out of the IP core.
`dout_endofpacket`	Output	`dout` port Avalon-ST `endofpacket` signal. This signal is asserted when the IP core is ending a frame.
`dout_ready`	Input	`dout` port Avalon-ST `ready` signal. The downstream device asserts this signal when it is able to receive data.
`dout_startofpacket`	Output	`dout` port Avalon-ST `startofpacket` signal. This signal is asserted when the IP core is starting a new frame.
`dout_valid`	Output	`dout` port Avalon-ST `valid` signal. This signal is asserted when the IP core produces data.
`dout_empty`	Output	`dout` port Avalon-ST `empty` signal. This signal has a non-zero value only when you set the Number of pixels in parallel parameter to be greater than 1. This signal specifies the number of pixel positions which are empty at the end of the `dout_endofpacket` signal.
`status_update_int`	Output	`control` slave port Avalon-MM interrupt signal. When asserted, the status registers of the IP core have been updated and the master must read them to determine what has occurred. Note: Present only if you turn on Use control port.
`vid_clk`	Input	Clocked video clock. All the video input signals are synchronous to this clock.
`vid_data`	Input	Clocked video data bus. This bus enables the transfer of video data into the IP core.
`vid_de`	Input	Clocked video data enable signal. The driving core asserts this signal to indicate that the data on `vid_data` is part of the active picture region of an incoming video. This signal must be driven for correct operation of the IP core. Note: For separate synchronization mode only.
`vid_datavalid`	Input	Enabling signal for the CVI II IP core. The IP core only reads the `vid_data`, `vid_de`, `vid_h_sync`, `vid_v_sync`, `vid_std`, and `vid_f` signals when `vid_datavalid` is 1. This signal allows the CVI II IP core to support oversampling during when the video runs at a higher rate than the pixel clock. Note: If you are not oversampling your input video, tie this signal high.
`vid_locked`	Input	Clocked video locked signal. Assert this signal when a stable video stream is present on the input. Deassert this signal when the video stream is removed. When 0, this signal triggers an early end of output frame packet and does not reset the internal registers. When this signal recovers after 0, if the system is not reset from outside, the first frame may have leftover pixels from the lock-lost frame,
`vid_f`	Input	Clocked video field signal. For interlaced input, this signal distinguishes between field 0 and field 1. For progressive video, you must deassert this signal. Note: For separate synchronization mode only.
`vid_v_sync`	Input	Clocked video vertical synchronization signal. Assert this signal during the vertical synchronization period of the video stream. Note: For separate synchronization mode only.
`vid_h_sync`	Input	Clocked video horizontal synchronization signal. Assert this signal during the horizontal synchronization period of the video stream. Note: For separate synchronization mode only.
`vid_hd_sdn`	Input	Clocked video color plane format selection signal . This signal distinguishes between sequential (when low) and parallel (when high) color plane formats. Note: For run-time switching of color plane transmission formats mode only.
`vid_std`	Input	Video standard bus. Can be connected to the `rx_std` signal of the SDI IP core (or any other interface) to read from the `Standard` register.
`vid_color_encoding`	Input	This signal is captured in the `Color Pattern` register and does not affect the functioning of the IP core. It provides a mechanism for control processors to read incoming color space information if the IP core (e.g. HDMI RX core) driving the CVI II does not provide such an interface. Tie this signal to low if no equivalent signal is available from the IP core driving CVI II.
`vid_bit_width`	Input	This signal is captured in the `Color Pattern` register and does not affect the functioning of the IP core. It provides a mechanism for control processors to read incoming video bit width information if the IP core (e.g. HDMI RX core) driving the CVI II does not provide such an interface. Tie this signal to low if no equivalent signal is available from the IP core driving CVI II.
`vid_total_sample_count`	Input	The IP core creates this signal if you do not turn on the Extract the total resolution parameter. The CVI II IP core operates using this signal as the total horizontal resolution instead of an internally detected version.
`Vid_total_line_count`	Input	The IP core creates this signal if you do not turn on the Extract the total resolution parameter. The CVI II IP core operates using this signal as the total vertical resolution instead of an internally detected version.
`sof`	Output	Start of frame signal. A change of 0 to 1 indicates the start of the video frame as configured by the SOF registers. Connecting this signal to a CVO IP core allows the function to synchronize its output video to this signal.
`sof_locked`	Output	Start of frame locked signal. When asserted, the `sof` signal is valid and can be used.
`refclk_div`	Output	A single cycle pulse in-line with the rising edge of the h sync.
`clipping`	Output	Clocked video clipping signal. A signal corresponding to the clipping bit of the `Status` register synchronized to `vid_clk`. This signal is for information only and no action is required if it is asserted.
`padding`	Output	Clocked video padding signal. A signal corresponding to the padding bit of the `Status` register synchronized to `vid_clk`. This signal is for information only and no action is required if it is asserted.
`overflow`	Output	Clocked video overflow signal. A signal corresponding to the overflow sticky bit of the `Status` register synchronized to `vid_clk`. This signal is for information only and no action is required if it is asserted. Note: Present only if you turn on Use control port.
`vid_hdmi_duplication[3:0]`	Input	If you select Remove duplicate pixels in the parameter, this 4-bit bus is added to the CVI II interface. You can drive this bus based on the number of times each pixel is duplicated in the stream (HDMI-standard compliant).

Table 23. Control Signals for CVI II IP Cores
Signal	Direction	Description
`av_address`	Input	`control` slave port Avalon-MM address bus. Specifies a word offset into the slave address space. Note: Present only if you turn on Use control port.
`av_read`	Input	`control` slave port Avalon-MM read signal. When you assert this signal, the `control` port drives new data onto the read data bus. Note: Present only if you turn on Use control port.
`av_readdata`	Output	`control` slave port Avalon-MM read data bus. These output lines are used for read transfers. Note: Present only if you turn on Use control port.
`av_waitrequest`	Output	`control` slave port Avalon-MM wait request bus. This signal indicates that the slave is stalling the master transaction. Note: Present only if you turn on Use control port.
`av_write`	Input	`control` slave port Avalon-MM `write` signal. When you assert this signal, the `control` port accepts new data from the write data bus. Note: Present only if you turn on Use control port.
`av_writedata`	Input	`control` slave port Avalon-MM write data bus. These input lines are used for write transfers. Note: Present only if you turn on Use control port.
`av_byteenable`	Input	`control` slave port Avalon-MM `byteenable` bus. These lines indicate which bytes are selected for write and read transactions.

7.11.2. Clocked Video Input II Parameter Settings

Table 24. Clocked Video Input II Parameter Settings
Parameter	Value	Description
Bits per pixel per color plane	4–20, Default = 8	Select the number of bits per pixel (per color plane).
Number of color planes	1–4, Default = 3	Select the number of color planes.
Color plane transmission format	Sequence Parallel	Specify whether to transmit the color planes in sequence or in parallel. If you select multiple pixels in parallel, then select Parallel.
Number of pixels in parallel	1, 2, 4, or 8	Specify the number of pixels transmitted or received in parallel.
Field order	Field 0 first Field 1 first Any field first	Specify the field to synchronize first when starting or stopping the output.
Enable matching data packet to control by clipping	On or Off	When there is a change in resolution,the control packet and video data packet transmitted by the IP core mismatch. Turn on this parameter if you want to clip the input video frame to match the resolution sent in control packet. When the current input frame resolution is wider and/or taller than the one specified in the control packet, then the IP core clips them to match the control packet dimensions.
Enable matching data packet to control by padding	On or Off	Turn on this parameter if you also want to pad the incoming frame if it is smaller and/or shorter than the resolution specified in the control packet. Note: This parameter is available only when you turn on Enable matching data packet to control by clipping. Depending on the size of the mismatch, padding operation could lead to frame drops at the input.
Overflow handling	On or Off	Turn this parameter if you want the to finish the current frame (with dummy pixel data) based on the resolution specified in the control packet if overflow happens. The IP core waits for the FIFO to become empty before it starts the padding process. By default (turned off), if an overflow is encountered, current frame is terminated abruptly. Note: Depending on size of the frame left to finish and the back pressure from downstream IP, overflow handling operation could lead to frame drops at the input.
Sync signals	Embedded in video On separate wires	Specify whether to embed the synchronization signal in the video stream or provide on a separate wire.
Support 6G and 12G-SDI	On or Off	Turn on to enable 6G-SDI or 12G-SDI support for CVI II IP core. Turning on this option will fix the number of pixels in parallel to 4. Note: This option is available only when you select the Embedded in video option for the Sync signals parameter.
Allow color planes in sequence input	On or Off	Turn on if you want to allow run-time switching between sequential and parallel color plane transmission formats. The format is controlled by the `vid_hd_sdn` signal.
Extract field signal	On or Off	Turn on to internally generate the field signal from the position of the V sync rising edge.
Use vid_std bus	On or Off	Turn on if you want to use the video standard, `vid_std`. Note: Platform Designer always generates the `vid_std` signal even when you turn off this parameter. The IP core samples and stores this signal in the `Standard` register to be read back for software control. If not needed, leave this signal disconnected.
Width of vid_std bus	External sync: 1–16, Default = 1 Embedded sync: 3, Default = 3	Specify the width of the `vid_std` bus, in bits.
Extract ancillary packets	On or Off	Turn on to extract the ancillary packets in embedded sync mode.
Depth of the ancillary memory	0–4096, Default = 0	Specify the depth of the ancillary packet RAM, in words.
Extract the total resolution	On or Off	Turn on to extract total resolution from the video stream.
Enable HDMI duplicate pixel removal	No duplicate pixel removal Remove duplicate pixel	Specify whether to enable a block to remove duplicate pixels for low rate resolutions. When you select Remove duplicate pixel, the IP core generates an additional 4-bit port to connect to the HDMI IP core. This port extracts the duplication factor from the HDMI IP core. Note: The CVI II IP core currently supports only duplication factors of 0 (no duplication) or 1 (each pixel transmitted twice).
Interlaced or progressive	Progressive Interlaced	Specify the format to be used when no format is automatically detected.
Width	32–8192, Default = 1920	Specify the image width to be used when no format is automatically detected.
Height – frame/field 0	32–8192, Default = 1080	Specify the image height to be used when no format is automatically detected.
Height – field 1	32–8192, Default = 480	Specify the image height for interlaced field 1 to be used when no format is automatically detected.
Pixel FIFO size	32–4096, Default = 2048	Specify the required FIFO depth in pixels.
Video in and out use the same clock	On or Off	Turn on if you want to use the same signal for the input and output video image stream clocks.
Use control port	On or Off	Turn on to use the optional stop/go control port.

7.11.3. Clocked Video Input II Control Registers

Table 25. Clocked Video Input II Registers
Address	Register	Description
0	`Control`	Bit 0 of this register is the `Go` bit. Setting this bit to 1 causes the CVI II IP core to start data output on the next video frame boundary. Bits 3, 2, and 1 of the `Control` register are the interrupt enables: Setting bit 1 to 1, enables the status update interrupt. Setting bit 2 to 1, enables the end of field/frame video interrupt.
1	`Status`	Bit 0 of this register is the `Status` bit. This bit is asserted when the CVI IP core is producing data. Bits 6–1 of the `Status` register are unused. Bit 7 is the interlaced bit. When asserted, the input video stream is interlaced. Bit 8 is the stable bit. When asserted, the input video stream has had a consistent line length for two of the last three lines. Bit 9 is the overflow sticky bit. When asserted, the input FIFO has overflowed. The overflow sticky bit stays asserted until a 1 is written to this bit. Bit 10 is the resolution bit. When asserted, indicates a valid resolution in the sample and line count registers. Bit 11 is the `vid_locked` bit. When asserted, indicates current signal value of the `vid_locked` signal. Bit 12 is the clipping bit. When asserted, input video frame/field is being clipped to match the resolution specified in the control packet. Note: Present only when you turn on Enable matching data packet to control by clipping. Bit 13 is the padding bit. When asserted, input video frame/field is being padded to match the resolution specified in the control packet. Note: Present only when you turn on Enable matching data packet to control by padding. Bit 14 is the picture drop sticky bit. When asserted, indicates one or more picture(s) has been dropped at input side. It stays asserted until a 1 is written to this bit. Bits 21–15 give the picture drop count. When picture drop sticky bit is asserted, this drop count provides the number of frame/field dropped at the input. Count resets when you clear the picture drop sticky bit. Note: Both picture drop sticky and picture drop count bit are present only when you turn on Enable matching data packet to control by padding and/or Overflow handling.
2	`Interrupt`	Bits 2 and 1 are the interrupt status bits: When bit 1 is asserted, the status update interrupt has triggered. When bit 2 is asserted, the end of field/frame interrupt has triggered. The interrupts stay asserted until a 1 is written to these bits.
3	`Used Words`	The used words level of the input FIFO.
4	`Active Sample Count`	The detected sample count of the video streams excluding blanking.
5	`F0 Active Line Count`	The detected line count of the video streams F0 field excluding blanking.
6	`F1 Active Line Count`	The detected line count of the video streams F1 field excluding blanking.
7	`Total Sample Count`	The detected sample count of the video streams including blanking.
8	`F0 Total Line Count`	The detected line count of the video streams F0 field including blanking.
9	`F1 Total Line Count`	The detected line count of the video streams F1 field including blanking.
10	`Standard`	The contents of the `vid_std` signal.
11-13	Reserved	Reserved for future use.
14	`Color Pattern`	Bits 7–0 are for color encoding—captures the value driven on the `vid_color_encoding` input. Bits 15–8 are for bit width—captures the value driven on the `vid_bit_width` input.
15	`Ancillary Packet`	Start of the ancillary packets that have been extracted from the incoming video.
15 + Depth of ancillary memory		End of the ancillary packets that have been extracted from the incoming video.

7.12. Clocked Video Output II Signals, Parameters, and Registers

7.12.1. Clocked Video Output II Interface Signals

Table 26. Clocked Video Output II Signals
Signal	Direction	Description
`main_reset_reset`	Input	The IP core asynchronously resets when you assert this signal. You must deassert this signal synchronously to the rising edge of the clock signal.
`main_clock_clk`	Input	The main system clock. The IP core operates on the rising edge of this signal.
`din_data`	Input	`din` port Avalon-ST `data` bus. This bus enables the transfer of pixel data into the IP core.
`din_endofpacket`	Input	`din` port Avalon-ST `endofpacket` signal. This signal is asserted when the downstream device is ending a frame.
`din_ready`	Output	`din` port Avalon-ST `ready` signal. This signal is asserted when the IP core function is able to receive data.
`din_startofpacket`	Input	`din` port Avalon-ST `startofpacket` signal. Assert this signal when the downstream device is starting a new frame.
`din_valid`	Input	`din` port Avalon-ST `valid` signal. Assert this signal when the downstream device produces data.
`din_empty`	Input	`din` port Avalon-ST empty signal. This signal has a non zero value only when you set the Number of pixels in parallel parameter to be greater than 1. This signal specifies the number of pixel positions which are empty at the end of the `din_endofpacket` signal.
`underflow`	Output	Clocked video underflow signal. A signal corresponding to the underflow sticky bit of the `Status` register synchronized to `vid_clk`. This signal is for information only and no action is required if it is asserted. Note: Present only if you turn on Use control port.
`status_update_int`	Output	`control` slave port Avalon-MM interrupt signal. When asserted, the status registers of the IP core have been updated and the master must read them to determine what has occurred. Note: Present only if you turn on Use control port.
`vid_clk`	Input	Clocked video clock. All the video output signals are synchronous to this clock.
`vid_data`	Output	Clocked video data bus. This bus transfers video data out of the IP core.
`vid_datavalid`	Output	Clocked video data valid signal. Assert this signal when a valid sample of video data is present on `vid_data`. Note: This signal is equivalent to the CVI II IP core's `vid_de` signal.
`vid_f`	Output	Clocked video field signal. For interlaced input, this signal distinguishes between field 0 and field 1. For progressive video, this signal is unused. Note: For separate synchronization mode only.
`vid_h`	Output	Clocked video horizontal blanking signal. This signal is asserted during the horizontal blanking period of the video stream. Note: For separate synchronization mode only.
`vid_h_sync`	Output	Clocked video horizontal synchronization signal. This signal is asserted during the horizontal synchronization period of the video stream. Note: For separate synchronization mode only.
`vid_ln`	Output	Clocked video line number signal. Used with the SDI IP core to indicate the current line number when the `vid_trs` signal is asserted. Note: For embedded synchronization mode only.
`vid_mode_change`	Output	Clocked video mode change signal. This signal is asserted on the cycle before a mode change occurs.
`vid_std`	Output	Video standard bus. Can be connected to the `tx_std` signal of the SDI IP core (or any other interface) to read from the `Standard` register.
`vid_trs`	Output	Clocked video time reference signal (TRS) signal. Used with the SDI IP core to indicate a TRS, when asserted. Note: For embedded synchronization mode only.
`vid_v`	Output	Clocked video vertical blanking signal. This signal is asserted during the vertical blanking period of the video stream. Note: For separate synchronization mode only.
`vid_v_sync`	Output	Clocked video vertical synchronization signal. This signal is asserted during the vertical synchronization period of the video stream. Note: For separate synchronization mode only.
`vcoclk_div`	Output	A divided down version of `vid_clk` (`vcoclk`). Setting the `Vcoclk Divider` register to be the number of samples in a line produces a horizontal reference on this signal. A PLL uses this horizontal reference to synchronize its output clock. Note: Present only if you turn on Use control port.
`vid_sof`	Output	Start of frame signal. A rising edge (0 to 1) indicates the start of the video frame as configured by the SOF registers. Note: Present only if you turn on Use control port.
`vid_sof_locked`	Output	Start of frame locked signal. When asserted, the `vid_sof` signal is valid and can be used. Note: Present only if you turn on Use control port.
`sof`	Input	Start of frame signal. A rising edge (0 to 1) indicates the start of the video frame as configured by the SOF registers. Connecting this signal to a CVI IP core allows the output video to be synchronized to this signal. Note: Present only if you turn on Accept synchronization inputs.
`sof_locked`	Input	Start of frame locked signal. When asserted, the `sof` signal is valid and can be used. Note: Present only if you turn on Accept synchronization inputs.
`sdi_cvo_rden`	Input	This signal connects to the SDI II IP core's `tx_dataout_valid` output signal. This signal indicates to the CVO II IP core to advance the state of the clocked video data and attributes. Note: This signal is only available when you select the Embedded in video option for the Sync signals parameter. It will not be present for external sync interfaces such as DisplayPort and HDMI.

Table 27. Control Signals for CVO II IP Cores
Signal	Direction	Description
`av_address`	Input	`control` slave port Avalon-MM address bus. Specifies a word offset into the slave address space. Note: Present only if you turn on Use control port.
`av_read`	Input	`control` slave port Avalon-MM read signal. When you assert this signal, the `control` port drives new data onto the read data bus. Note: Present only if you turn on Use control port.
`av_readdata`	Output	`control` slave port Avalon-MM read data bus. These output lines are used for read transfers. Note: Present only if you turn on Use control port.
`av_waitrequest`	Output	`control` slave port Avalon-MM wait request bus. This signal indicates that the slave is stalling the master transaction. Note: Present only if you turn on Use control port.
`av_write`	Input	`control` slave port Avalon-MM `write` signal. When you assert this signal, the `control` port accepts new data from the write data bus. Note: Present only if you turn on Use control port.
`av_writedata`	Input	`control` slave port Avalon-MM write data bus. These input lines are used for write transfers. Note: Present only if you turn on Use control port.
`av_byteenable`	Input	`control` slave port Avalon-MM `byteenable` bus. These lines indicate which bytes are selected for write and read transactions.

7.12.2. Clocked Video Output II Parameter Settings

Table 28. Clocked Video Output II Parameter Settings
Parameter	Value	Description
Image width/Active pixels	32–8192, Default = 1920	Specify the image width by choosing the number of active pixels.
Image height/Active lines	32–8192, Default = 1200	Specify the image height by choosing the number of active lines.
Bits per pixel per color plane	4–20, Default = 8	Select the number of bits per pixel (per color plane).
Number of color planes	1–4, Default = 3	Select the number of color planes.
Color plane transmission format	Sequence Parallel	Specify whether to transmit the color planes in sequence or in parallel. If you select multiple pixels in parallel, then select Parallel.
Allow output of channels in sequence	On or Off	Turn on if you want to allow run-time switching between sequential formats, such as NTSC, and parallel color plane transmission formats, such as 1080p. The format is controlled by the `ModeXControl` registers. Turn off if you are using multiple pixels in parallel.
Number of pixels in parallel	1, 2, 4, or 8	Specify the number of pixels transmitted or received in parallel. Note: Number of pixels in parallel are only supported if you select On separate wires for the Sync signals parameter.
Interlaced video	On or Off	Turn off to use progressive video.
Sync signals	Embedded in video On separate wires	Specify whether to embed the synchronization signal in the video stream or to provide the synchronization signal on a separate wire. Embedded in video: You can set the active picture line, horizontal blanking, and vertical blanking values. On separate wires: You can set horizontal and vertical values for sync, front porch, and back porch.
Support 6G and 12G-SDI	On or Off	Turn on to enable 6G-SDI or 12G-SDI support for CVO II IP core. Turning on this option will fix the number of pixels in parallel to 4. Note: This option is available only when you select the Embedded in video option for the Sync signals parameter.
Default SDI video standard	SD HD 3GA (3G-SDI Level A) 6GB (6G-SDI with 8 streams interleaved) 12GA (12G-SDI with 8 streams interleaved)	Specify the default SDI video standard. Note: This option is available only when you select the Embedded in video option for the Sync signals parameter. 6GB and 12GA options are available only when you turn on the Support 6G and 12G-SDI parameter.
Active picture line	32–65536, Default = 0	Specify the start of active picture line for Frame.
Frame/Field 1: Ancillary packet insertion line	32–65536, Default = 0	Specify the line where ancillary packet insertion starts.
Embedded syncs only - Frame/Field 1: Horizontal blanking	32–65536, Default = 0	Specify the size of the horizontal blanking period in pixels for Frame/Field 1.
Embedded syncs only - Frame/Field 1: Vertical blanking	32–65536, Default = 0	Specify the size of the vertical blanking period in pixels for Frame/Field 1.
Separate syncs only - Frame/Field 1: Horizontal sync	32–65536, Default = 44	Specify the size of the horizontal synchronization period in pixels for Frame/Field 1.
Separate syncs only - Frame/Field 1: Horizontal front porch	32–65536, Default = 88	Specify the size of the horizontal front porch period in pixels for Frame/Field 1.
Separate syncs only - Frame/Field 1: Horizontal back porch	32–65536, Default = 148	Specify the size of the horizontal back porch in pixels for Frame/Field 1.
Separate syncs only - Frame/Field 1: Vertical sync	32–65536, Default = 5	Specify the number of lines in the vertical synchronization period for Frame/Field 1.
Separate syncs only - Frame/Field 1: Vertical front porch	32–65536, Default = 4	Specify the number of lines in the vertical front porch period in pixels for Frame/Field 1.
Separate syncs only - Frame/Field 1: Vertical back porch	32–65536, Default = 36	Specify the number of lines in the vertical back porch in pixels for Frame/Field 1.
Interlaced and Field 0: F rising edge line	32–65536, Default = 0	Specify the line when the rising edge of the field bit occurs for Interlaced and Field 0.
Interlaced and Field 0: F falling edge line	32–65536, Default = 0	Specify the line when the falling edge of the field bit occurs for Interlaced and Field 0.
Interlaced and Field 0: Vertical blanking rising edge line	32–65536, Default = 0	Specify the line when the rising edge of the vertical blanking bit for Field 0 occurs for Interlaced and Field 0.
Interlaced and Field 0: Ancillary packet insertion line	32–65536, Default = 0	Specify the line where ancillary packet insertion starts.
Embedded syncs only - Field 0: Vertical blanking	32–65536, Default = 0	Specify the size of the vertical blanking period in pixels for Interlaced and Field 0.
Separate syncs only - Field 0: Vertical sync	32–65536, Default = 0	Specify the number of lines in the vertical synchronization period for Interlaced and Field 0.
Separate syncs only - Field 0: Vertical front porch	32–65536, Default = 0	Specify the number of lines in the vertical front porch period for Interlaced and Field 0.
Separate syncs only - Field 0: Vertical back porch	32–65536, Default = 0	Specify the number of lines in the vertical back porch period for Interlaced and Field 0.
Pixel FIFO size	32–(memory limit), Default = 1920	Specify the required FIFO depth in pixels, (limited by the available on-chip memory).
FIFO level at which to start output	0–(memory limit), Default = 1919	Specify the fill level that the FIFO must have reached before the output video starts.
Video in and out use the same clock	On or Off	Turn on if you want to use the same signal for the input and output video image stream clocks.
Use control port	On or Off	Turn on to use the optional Avalon-MM control port.
Generate synchronization outputs	On or Off	When you turn on Use control port, this option becomes available. Turning on this option generates the `vid_vcoclk_div`, `vid_sof`, and `vid_sof_locked` output signals. You can use these signals to generate timing reference signals to synchronize video.
Accept synchronization inputs	On or Off	When you turn on Generate synchronization outputs, this option becomes available. Turning on this option generates the `sof` and `sof_locked` input signals. These signals enable the CVO II IP core to align the synchronization outputs to within 1 line of inputs.
Set vco_clk_divider increment to pixels in parallel	On or Off	When you turn on Generate synchronization outputs, this option becomes available. Turning on this option enables you to set `vco_clk_divider` to increment in 2 different modes: Increment in single counts until the divider value is reached. Increment in steps equal to the value specified in the `Number of pixels in parallel` parameter. The second increment allows `vco_clk_div` to keep step with the internal counters for the video output but introduces variation in the number of cycles between `divclk` pulses depending on the `divclk` value and how the value in the `Number of pixels in parallel` divides.
Low latency mode	0–1, Default = 0	Select 0 for regular completion mode. Each output frame initiated completes its timing before a new frame starts. Select 1 for low latency mode. The IP core starts timing for a new frame immediately.
Run-time configurable video modes	1–13, Default = 1	Specify the number of run-time configurable video output modes that are required when you are using the Avalon-MM control port. Note: This parameter is available only when you turn on Use control port.
Width of vid_std bus	External sync: 1–16, Default = 1 Embedded sync: 3, Default = 3	Select the width of the `vid_std` bus, in bits.

7.12.3. Clocked Video Output II Control Registers

Note: If you configure the design without enabling the control interface, the interrupt line (status_update_int) will not be generated. This is because the logic required to clear the interrupt will not be generated and therefore could not provide useful information.

Table 29. Clocked Video Output II RegistersThe rows in the table are repeated in ascending order for each video mode. All of the ModeN registers are write only.
Address	Register	Description
0	`Control`	Bit 0 of this register is the `Go` bit. Setting this bit to 1 causes the CVO IP core to start video data output. Bit 2 of the `Control` register is the `Clear Underflow Register` bit. When bit 2 of the `Status` register is set, a 1 should be written to this register to clear the underflow. Bits 4 and 3 of the `Control` register are the `Genlock` control bits. Setting bit 3 to 1 enables the synchronization outputs: `vid_sof`, `vid_sof_locked`, and `vcoclk_div`. Setting bit 4 to 1, while bit 3 is 1, enables frame locking. The IP core attempts to align its `vid_sof` signal to the `sof` signal from the CVI IP core. Bits 9 and 8 of the `Control` register are the interrupt enables, matching the position of the interrupt registers at address 2. Setting bit 8 to 1 enables the status update interrupt. Setting bit 9 to 1 enables the locked interrupt.
1	`Status`	Bit 0 of this register is the `Status` bit. This bit is asserted when the CVO IP core is producing data. Bit 1 of the `Status` register is unused. Bit 2 is the underflow sticky bit. When bit 2 is asserted, the output FIFO has underflowed. The underflow sticky bit stays asserted until a 1 is written to bit 2 of the `Control` register Bit 3 is the frame locked bit. When bit 3 is asserted, the CVO IP core has aligned its start of frame to the incoming `sof` signal.
2	`Interrupt`	Bits 9 and 8 are the interrupt status bits: When bit 1 is asserted, the status update interrupt has triggered. When bit 2 is asserted, the locked interrupt has triggered. The interrupts stay asserted until a 1 is written to these bits.
3	`Video Mode Match`	Before any user specified modes are matched, this register reads back 0 indicating the default values are selected. Once a match has been made, the register reads back in a one-hot fashion, e.g. 0x0001=Mode0 0x00020=Mode5
4	`Bank Select`	Writes to the ModeN registers will be reflected to the mode bank selected by this register. Up to 13 banks are available depending on parameterization. Selection is by standard binary encoding.
5	`ModeN Control`	Video ModeN 1 Control. Bit 0 of this register is the `Interlaced` bit. Set to 1 for interlaced. Set to 0 for progressive. Bit 1 of this register is the sequential output control bit (only if the Allow output of color planes in sequence compile-time parameter is enabled). Setting bit 1 to 1, enables sequential output from the CVO IP core (NTSC). Setting bit 1 to 0, enables parallel output from the CVO IP core (1080p).
6	`ModeN Sample Count`	Video mode N sample count. Specifies the active picture width of the field.
7	`ModeN F0 Line Count`	Video mode N field 0/progressive line count. Specifies the active picture height of the field.
8	`ModeN F1 Line Count`	Video mode N field 1 line count (interlaced video only). Specifies the active picture height of the field.
9	`ModeN Horizontal Front Porch`	Video mode N horizontal front porch. Specifies the length of the horizontal front porch in samples.
10	`ModeN Horizontal Sync Length`	Video mode N horizontal synchronization length. Specifies the length of the horizontal synchronization length in samples.
11	`ModeN Horizontal Blanking`	Video mode N horizontal blanking period. Specifies the length of the horizontal blanking period in samples.
12	`ModeN Vertical Front Porch`	Video mode N vertical front porch. Specifies the length of the vertical front porch in lines.
13	`ModeN Vertical Sync Length`	Video mode 1 vertical synchronization length. Specifies the length of the vertical synchronization length in lines.
14	`Mode1 Vertical Blanking`	Video mode N vertical blanking period. Specifies the length of the vertical blanking period in lines.
15	`ModeN F0 Vertical Front Porch`	Video mode N field 0 vertical front porch (interlaced video only). Specifies the length of the vertical front porch in lines.
16	`ModeN F0 Vertical Sync Length`	Video mode N field 0 vertical synchronization length (interlaced video only). Specifies the length of the vertical synchronization length in lines.
17	`ModeN F0 Vertical Blanking`	Video mode N field 0 vertical blanking period (interlaced video only). Specifies the length of the vertical blanking period in lines.
18	`ModeN Active Picture Line`	Video mode N active picture line. Specifies the line number given to the first line of active picture.
19	`ModeN F0 Vertical Rising`	Video mode N field 0 vertical blanking rising edge. Specifies the line number given to the start of field 0's vertical blanking.
20	`ModeN Field Rising`	Video mode N field rising edge. Specifies the line number given to the end of Field 0 and the start of Field 1.
21	`ModeN Field Falling`	Video mode N field falling edge. Specifies the line number given to the end of Field 0 and the start of Field 1.
22	`ModeN Standard`	The value output on the `vid_std` signal.
23	`ModeN SOF Sample`	Start of frame sample register. The sample and subsample upon which the SOF occurs (and the `vid_sof` signal triggers): Bits 1–0 are the subsample value. Bits 15–2 are the sample value.
24	`ModeN SOF Line`	SOF line register. The line upon which the SOF occurs measured from the rising edge of the F0 vertical sync. This is a 13-bit register.
25	`ModeN Vcoclk Divider`	Number of cycles of `vid_clk` (`vcoclk`) before `vcoclk_div` signal triggers. This is a 14-bit register.
26	`ModeN Ancillary Line`	The line to start inserting ancillary data packets.
27	`ModeN F0 Ancillary Line`	The line in field F0 to start inserting ancillary data packets.
28	`ModeN H-Sync Polarity`	Specify positive or negative polarity for the horizontal sync. Bit 0 is 0 for falling edge pulses (negative polarity). Bit 0 is 1 for rising edge hsync pulses (positive polarity).
29	`ModeN V-Sync Polarity`	Specify positive or negative polarity for the vertical sync. Bit 0 is 0 for falling edge pulses (negative polarity). Bit 0 is 1 for rising edge vsync pulses (positive polarity).
30	`ModeN Valid`	Video mode valid. Set to indicate that this mode is valid and can be used for video output.

Note:

To ensure the vid_f signal rises at the Field 0 blanking period and falls at the Field 1, use the following equations:

F rising edge line ≥ Vertical blanking rising edge line
F rising edge line < Vertical blanking rising edge line + (Vertical sync + Vertical front porch + Vertical back porch)
F falling edge line < active picture line

8. 2D FIR II IP Core

The 2D FIR II IP core performs 2D convolution using matrices of specific coefficients.

The 2D FIR II IP core has 2 distinct modes of operation (user-defined at compile time).

Standard FIR mode
- In this mode, the IP core performs 2D finite impulse response filtering (convolution) using matrices of N×M coefficients, where N is a parameterizable number of horizontal taps (1 <= N <= 16) and M is a parameterizable number of vertical taps (1 <= M <= 16).
- You can set the coefficients used by the filter either as fixed parameters at compile time, or as run-time alterable values which you can edit through an Avalon-MM slave interface.
- With suitable coefficients, the filter can perform operations such as sharpening, smoothing, and edge detection.
Dedicated edge-adaptive sharpening mode
- You can use this mode to sharpen blurred edges in the incoming video.

8.1. 2D FIR Filter Processing

The 2D FIR II IP core calculates the output pixel values in 3 stages.

Kernel creation
An N×M array of input pixels is created around the input pixel at the same position in the input image as the position of the output pixel in the output image. This center pixel has (N-1)/2 pixels to its left and N/2 pixels to its right in the array, and (M-1)/2 lines above it and M/2 lines below it.

When the pixels to the left, right, above, or below the center pixel in the kernel extend beyond the edges of the image, then the filter uses either replication of the edge pixel or full data mirroring, according to the value of a compile time parameter.
Convolution
Each pixel in the N×M input array is multiplied by the corresponding coefficient in the N×M coefficient array and the results are summed to produce the filtered value.

The 2D FIR Filter II IP core retains full precision throughout the calculation of each filtered value, with all rounding and saturation to the required output precision applied as a final stage.
Rounding and saturation.
The resulting full precision filtered value as rounded and saturated according the output precision specification

8.2. 2D FIR Filter Precision

The 2D FIR II IP core does not lose calculation precision during the FIR calculation.

You may parameterize the input data to between 4-20 bits per color per pixel, and the IP core treats this data as unsigned integer data. You may enable optional guard bands at the input to keep the data inside a reduced range of values.
You may parameterize the coefficient data up to a total width of 32 bits per coefficient. The coefficients may be signed or unsigned and contain up to 24 fractional bits.
You may parameterize the output data to between 4-20 bits per color per pixel, and the selected output data width may be different from the input data width.

To convert from the full precision result of the filtering to the selected output precision, the IP core first rounds up the value to remove the required number of fraction bits. Then the IP core saturates the value. You may select how many fraction bits should be preserved in the final output using the 2D FIR II parameter editor. As with the input data, the output data is treated as unsigned, so the IP core clips any negative values that result from the filtering to 0. Any values greater than the maximum value that can be represented in the selected number of bits per color per pixel are clipped to this maximum value.

8.3. 2D FIR Coefficient Specification

You can either specify the filtering operation coefficients as fixed values that are not run-time editable, or you can opt to enable an Avalon-MM slave interface to edit the values of the coefficients at run time.

The 2D FIR Filter IP core requires a fixed point type to be defined for the coefficients. The user-entered coefficients (shown as white boxes in the parameter editor) are rounded to fit in the chosen coefficient fixed point type (shown as purple boxes in the parameter editor).

In run-time editable coefficient mode, you must enter the desired coefficient values through an Avalon-MM control slave interface at run time, and the coefficient values may be updated as often as once per frame.
Note: In this mode, the coefficient values will all revert to 0 after every reset, so coefficients must be initialized at least once on start-up.
To keep the register map as small as possible and to reduce complexity in the hardware, the number of coefficients that are edited at run time is reduced when any of the symmetric modes is enabled.
If there are T unique coefficient values after symmetry is considered then the register map will contain T addresses into which coefficients should be written, starting at address 7 and finishing at T+ 6.
Coefficient index 0 (as described in the symmetry section) should be written to address 7 with each successively indexed coefficient written at each following address. The updated coefficient set does not take effect until you issue a write to address 6 - any value may be written to address 6, it is just the action of the write that forces the commit.
The new coefficient set will then take effect on the next frame after the write to address 6 Note that the coefficient values written to the register map must be in pre-quantized form as the hardware cost to implement quantization on floating point values would be prohibitive.

Table 30. 2D FIR Filter II Coefficient Modes
Coefficient Mode	Description
Fixed Coefficient	In fixed coefficient mode, the values for the coefficients are specified using a comma separated CSV text file and there is no Avalon-MM control slave interface and the selected coefficient values take effect immediate effect at reset. Regardless of the symmetry mode, the text file must contain a full listing of all the coefficients in the N×M array i.e. the file must always contain N×M comma separated values. When the CSV file is parsed in Platform Designer to create the list of compile time coefficients, the values entered will be checked against the selected symmetry mode and warnings issued if the coefficients are not found to be symmetric across the selected axes. The values specified in the CSV file must be in their unquantized format – for example, if you want a value of 1.7 for a given coefficient, then the value in the file should be 1.75. When the file is parsed in Platform Designer, the coefficients will be automatically quantized according to the precision you specified. Note: The quantization process aims to select the closest value available in the given precision format. If you select the coefficients are selected arbitrarily without reference to the available precision, then the quantized value may differ from the desired value.
Run-time Editable Coefficient	In run-time editable coefficient mode, you must enter the desired coefficient values through an Avalon-MM control slave interface at run time, and the coefficient values may be updated as often as once per frame. Note: In this mode, the coefficient values will all revert to 0 after every reset, so coefficients must be initialized at least once on start-up. To keep the register map as small as possible and to reduce complexity in the hardware, the number of coefficients that are edited at run time is reduced when any of the symmetric modes is enabled. If there are T unique coefficient values after symmetry is considered then the register map will contain T addresses into which coefficients should be written, starting at address 7 and finishing at T+ 6. Coefficient index 0 (as described in the symmetry section) should be written to address 7 with each successively indexed coefficient written at each following address. The updated coefficient set does not take effect until you issue a write to address 6 - any value may be written to address 6, it is just the action of the write that forces the commit. The new coefficient set will then take effect on the next frame after the write to address 6. Note: The coefficient values written to the register map must be in pre-quantized format as the hardware cost to implement quantization on floating point values is prohibitive.

8.4. 2D FIR Filter Symmetry

The 2D FIR IP core supports symmetry modes for coefficient data.

The 2D FIR Filter II IP core provides 5 symmetry modes for you to select:

No symmetry
Horizontal symmetry
Vertical symmetry
Horizontal and vertical symmetry
Diagonal symmetry

8.4.1. No Symmetry

There are no axes of symmetry in the 2D coefficient array.

The number of horizontal taps (N) and the number of vertical taps (M) may both be even or odd numbers.

If run-time control of the coefficient data is enabled, the register map will include N×M addresses to allow the value of each coefficient to be updated individually. The coefficients are indexed within the register map in raster scan order.

8.4.2. Horizontal Symmetry

There is 1 axis of symmetry across a vertical line through the center tap in the 2D coefficient array.

In this case, the number of vertical taps (M) may be even or odd, but the number of horizontal taps (N) must be even. With horizontal symmetry enabled, there are only ((N+1)/2)×M unique coefficient values, with the remaining values in the array being mirrored duplicates.

With run-time control of the coefficients enabled, the register map only includes addresses to update the ((N+1)/2)×M unique coefficient values, indexed for an example 5×5 array as shown in the figure below.

8.4.3. Vertical Symmetry

There is 1 axis of symmetry across a horizontal line through the center tap in the 2D coefficient array.

In this case, the number of horizontal taps (N) may be even or odd, but the number of vertical taps (M) must be even. With vertical symmetry enabled, there are only N×((M+1)/2) unique coefficient values, with the remaining values in the array being mirrored duplicates.

With run-time control of the coefficients enabled, the register map only includes addresses to update the N×((M+1)/2) unique coefficient values, indexed for an example 5×5 array as shown in the figure below.

8.4.4. Horizontal and Vertical Symmetry

There are 2 axes of symmetry across a horizontal line and a vertical line through the center tap in the 2D coefficient array.

In this case, the number of horizontal taps (N) and the number of vertical taps (M) must be even. With horizontal and vertical symmetry enabled, there are only ((N+1)/2)×((M+1)/2) unique coefficient values, with the remaining values in the array being mirrored duplicates.

With run-time control of the coefficients enabled, the register map only includes addresses to update the ((N+1)/2)×((M+1)/2) unique coefficient values, indexed for an example 5×5 array as shown in the figure below.

8.4.5. Diagonal Symmetry

There are 4 axes of symmetry in the 2D coefficient array across a horizontal line, a vertical line, and 2 diagonal lines through the center tap.

In this case, the number of horizontal taps (N) and the number of vertical taps (M) must be even, and they must have same value (N = M). With diagonal symmetry enabled, there are only Tu = ((N+1)/2) unique coefficient values in either the horizontal or vertical directions, and a total of (Tu*(Tu+1))/2 unique coefficient values.

With run-time control of the coefficients enabled, the register map only includes addresses to update the (Tu *( Tu +1))/2 unique coefficient values, indexed for an example 5×5 array as shown in the figure below.

8.5. Result to Output Data Type Conversion

After calculation, the fixed point type of the results must be converted to the integer data type of the output.

The conversion is performed in four stages, in the following order:

Result scaling—scaling is useful to quickly increase the color depth of the output.
- The available options are a shift of the binary point right –16 to +16 places.
- Scaling is implemented as a simple shift operation so it does not require multipliers.
Removal of fractional bits—if any fractional bits exist, you can choose to remove them through these methods:
- Truncate to integer—fractional bits are removed from the data; equivalent to rounding towards negative infinity.
- Round - Half up—round up to the nearest integer. If the fractional bits equal 0.5, rounding is towards positive infinity.
- Round - Half even—round to the nearest integer. If the fractional bits equal 0.5, rounding is towards the nearest even integer.
Conversion from signed to unsigned— if any negative numbers exist in the results and the output type is unsigned, you can convert to unsigned through these methods:.
- Saturate to the minimum output value (constraining to range).
- Replace negative numbers with their absolute positive value.
Constrain to range—if any of the results are beyond a specific range, logic to saturate the results to the minimum and maximum output values is automatically added. The specific range is the specified range of the output guard bands, or if unspecified, the minimum and maximum values allowed by the output bits per pixel.

8.6. Edge-Adaptive Sharpen Mode

In edge-adaptive sharpen mode, the 2D FIR II IP core attempts to detect blurred edges and applies a sharpening filter only to the pixels that are determined to be part of a blurred edge.

8.6.1. Edge Detection

You can determine what constitutes a blurred edge by setting upper and lower blur thresholds (either at compile time or as run-time controlled values). The edge detection is applied independently in the horizontal and vertical directions according to the following steps:

The target pixel is compared to the pixels immediately to the left and right.
If both differences to the neighboring pixel are less than the upper blur threshold, the edge detection continues. Otherwise, the pixel is considered to be part of an edge that is already sharp enough that does not require further sharpening.
If the differences to the pixels to the left and right are below the upper blur threshold, the differences between the target pixel and its neighbors 1, 2, and 3 pixels to the left and right are calculated.
You may configure the range of pixels over which a blurred edge is detected.
- Setting the Blur search range register to 1 means that only the differences to the neighbors 1 pixel to the left and right are considered.
- Setting the register to 2 increases the search across the 2 neighboring pixels.
- Setting the register to 3 increases it to the full range across all 3 neighbors in each direction.
The value of the blur search range can be updated at run time if you turn on run-time control feature for the 2D FIR II IP core.
If the difference to any of the enabled neighboring pixels is greater than the lower blur threshold, the target pixel is tagged as a horizontal edge. Otherwise, target pixel is considered to be part of an area of flat color and left unaltered.

Note: The algorithm is described for horizontal edge detection, but the same algorithm is used for vertical detection, just replace left and right with above and below.

8.6.2. Filtering

Depending on whether each pixel is tagged as a horizontal and/or vertical edge, 1 of 4 different 3×3 filter kernels is applied (and the result divided by 16) to produce the final output pixel.

Note: In edge-adaptive sharpen, the filter is fixed at 3×3 taps and all parameters regarding coefficient precision are ignored. There is also no option to override the coefficients used in the filter kernels.

Figure 43. Edge-adaptive Sharpen Filtering

8.6.3. Precision

The edge-adaptive sharpen mode of the 2D FIR II IP core does not allow for a different setting in the bits per pixel per color plane.

The input bits per color plane is maintained at the output.
The output of the filtering kernel is rounded to the nearest integer (by adding 8 prior to the divide by 16).
Any negative values are clipped to 0 and any values greater than the maximum value that can be represented in the selected number of bits per color per pixel are clipped to this maximum value.

8.7. 2D FIR Filter Parameter Settings

Table 31. 2D FIR II Parameter Settings
Parameter	Value	Description
Number of color planes	1, 2, 3, 4	Select the number of color planes per pixel.
Color planes transmitted in parallel	On or Off	Select whether to send the color planes in parallel or in sequence (serially).
Number of pixels in parallel	1, 2, 4, 8	Select the number of pixels transmitted per clock cycle.
4:2:2 video data	On or Off	Turn on if the input data is 4:2:2 formatted, otherwise the data is assumed to be 4:4:4 formatted. Note: The IP core does not support odd heights or widths in 4:2:2 mode.
Maximum frame width	32-8192, Default = 1920	Specify the maximum frame width allowed by the IP core.
Maximum frame height	32-8192, Default = 1080	Specify the maximum frame height allowed by the IP core.
Input bits per pixel per color plane	4-20, Default = 8	Select the number of bits per color plane per pixel at the input.
Enable input guard bands	On or Off	Turn on to limit the range for each input color plane.
Lower input guard bands	0– 2^{(input bits per symbol)-1} Default = 0	Set the lower range limit to each input color plane. Values beneath this will be clipped to this limit.
Upper input guard bands	0– 2^{(input bits per symbol)-1} Default = 255	Set the upper range limit to each input color plane. Values above this will be clipped to this limit.
Output bits per pixel per color plane	4–20, Default = 8	Select the number of bits per color plane per pixel at the output.
Enable output guard bands	On or Off	Turn on to limit the range for each output color plane.
Lower output guard bands	0– 2^{(input bits per symbol)-1} Default = 0	Set the lower range limit to each output color plane. Values beneath this will be clipped to this limit.
Upper output guard bands	0– 2^{(input bits per symbol)-1} Default = 255	Set the upper range limit to each output color plane. Values above this will be clipped to this limit.
Filtering algorithm	STANDARD_FIR EDGE_ADAPTIVE_SHARPEN	Select the preferred FIR mode.
Enable edge data mirroring	On or Off	Turn on to enable full mirroring of data at frame/field edges. If you do not turn on this feature, the edge pixel will be duplicated to fill all filter taps that stray beyond the edge of the frame/field.
Vertical filter taps	1–16, Default = 8	Select the number of vertical filter taps.
Horizontal filter taps	1–16, Default = 8	Select the number of horizontal filter taps.
Vertically symmetric coefficients	On or Off	Turn on to specify vertically symmetric coefficients.
Horizontally symmetric coefficients	On or Off	Turn on to specify horizontally symmetric coefficients.
Diagonally symmetric coefficients	On or Off	Turn on to specify diagonally symmetric coefficients.
Blur search range	1–3, Default = 1	Select the number of pixels over which a blurred edge may be detected. This option is available only if you select EDGE_ADAPTIVE_SHARPEN. If you enable Run-time control, you may override this value using the Avalon-MM control slave interface at run time.
Rounding method	TRUNCATE ROUND_HALF_UP ROUND_HALF_EVEN	Select how fraction bits are treated during rounding. TRUNCATE simply removes the unnecessary fraction bits. ROUND_HALF_UP rounds to the nearest integer value, with 0.5 always being rounded up. ROUND_HALF_EVEN rounds 0.5 to the nearest even integer value.
Use signed coefficients	On or Off	Turn on to use signed coefficient values.
Coefficient integer bits	0–16, Default = 1	Select the number of integer bits for each coefficient value.
Coefficient fraction bits	0–24, Default = 7	Select the number of fraction bits for each coefficient value.
Move binary point right	–16 to +16, Default = 0	Specify the number of places to move the binary point to the right prior to rounding and saturation. A negative value indicates a shift to the left.
Run-time control	On or Off	Turn on to enable coefficient values to be updated at run-time through the Avalon-MM control slave interface. Note: When you turn on this parameter, the `Go` bit gets deasserted by default. When you turn off this parameter, the `Go` is asserted by default.
Fixed coefficients file (Unused if run-time updates of coefficients is enabled.)	User specified file (including full path to locate the file)	If you do not enable run-time control, you must specify a CSV containing a list of the fixed coefficient values.
Default upper blur limit (per color plane)	0– 2^{(input bits per symbol)-1} Default = 0	Sets the default upper blur threshold for blurred edge detection. This option is available only if you select EDGE_ADAPTIVE_SHARPEN. If you enable Run-time control, you may override this value using the Avalon-MM control slave interface at run time.
Default lower blur limit (per color plane)	0– 2^{(input bits per symbol)-1} Default = 0	Sets the default lower blur threshold for blurred edge detection. This option is available only if you select EDGE_ADAPTIVE_SHARPEN. If you enable Run-time control, you may override this value using the Avalon-MM control slave interface at run time.
Reduced control register readback	On or Off	If you turn on this parameter, the values written to register 3 and upwards cannot be read back through the control slave interface. This option reduces ALM usage. If you do not turn on this parameter, the values of all the registers in the control slave interface can be read back after they are written.
How user packets are handled	No user packets allowed Discard all user packets received Pass all user packets through to the output	If your design does not require the 2D FIR II IP core to propagate user packets, then you may select Discard all user packets received to reduce ALM usage. If your design guarantees there will never be any user packets in the input data stream, then you can further reduce ALM usage by selecting No user packets allowed. In this case, the 2D FIR II IP core may lock if it encounters a user packet.
Add extra pipelining registers	On or Off	Turn on to add extra pipeline stage registers to the data path. You must to turn on this option to achieve: Frequency of 150 MHz for Cyclone V devices Frequencies above 250 MHz for Arria V, Stratix V, or Intel^® Arria^® 10 devices
Video no blanking	On or Off	Turn on if the input video does not contain vertical blanking at its point of conversion to the Avalon-ST video protocol.

8.8. 2D FIR Filter Control Registers

Table 32. 2D FIR Filter II Control Register MapThe 2D FIR Filter II IP core filtering operation coefficients can either be specified as fixed values that are not run-time editable, or you can opt to enable an Avalon-MM slave interface to edit the values of the coefficients at run time. When a control slave interface is included, the IP core resets into a stopped state and must be started by writing a ‘1’ to the `Go` bit of the control register before any input data is processed.
Address	Register	Description
0	`Control`	Bit 0 of this register is the `Go` bit, all other bits are unused. Setting this bit to 0 causes the IP core to stop at the end of the next frame/field packet. When you enable run-time control, the `Go` bit gets deasserted by default. If you do not enable run-time control, the `Go` is asserted by default.
1	`Status`	Bit 0 of this register is the `Status` bit, all other bits are unused. The IP core sets this address to 0 between frames. The IP core sets this address to 1 when it is processing data and cannot be stopped.
2	`Interrupt`	This bit cannot be used because the IP core does not generate any interrupts.
3	`Blur search range`	Set this register to 1, 2, or 3 to override the default parameter setting for the edge detection range in edge-adaptive sharpen mode.
4	`Lower blur threshold`	This register updates the value of the lower blur threshold used in edge detection in edge-adaptive sharpen mode.
5	`Upper blur threshold`	This register updates the value of the upper blur threshold used in edge detection in edge-adaptive sharpen mode.
6	`Coefficient commit`	Writing any value to this register causes the coefficients currently in addresses 7 to (6+T) to be applied from the start of the next input.
7–(6+ T)	`Coefficient data`	Depending on the number of vertical taps, horizontal taps, and symmetry mode, T addresses are allocated to upload the T unique coefficient values required to fill the 2D coefficient array.

9. Mixer II IP Core

The Mixer II IP core mixes together multiple image layers.

The run-time control is partly provided by an Avalon-MM slave port with registers for the location, and on or off status of each foreground layer. The dimensions of each layer are then specified by Avalon-ST Video control packets.

Each foreground layer must be driven by a frame buffer or frame reader so that data can be provided at the correct time.
- If the Synchronize background to layer 0 parameter is enabled, the input connected to layer 0 is used as the background layer and does not have to be driven by a frame buffer or frame reader.
- If the Synchronize background to layer 0 parameter is disabled, all layers are treated as foreground layers and must be driven by a frame buffer or frame reader.
Each layer must fit within the dimensions of the background layer.
To display the layers correctly:
- The rightmost edge of each layer (width + X offset) must fit within the dimensions of the background layer.
- The bottom edge of each layer (height + Y offset) must fit within the dimensions of the background layer.

Note: If these conditions are not met for any layers, the Mixer II IP core will not display those layers. However, the corresponding inputs will be consumed and will not get displayed.

The Mixer II IP core has the following features:

Supports picture-in-picture mixing and image blending with per-pixel and static value alpha support.
Supports dynamic changing of location and size of each layer during run time.
Supports dynamic changing of layers positioning during run time.
Allows the individual layers to be switched on and off.
Supports up to 8 pixels in parallel.
Supports up to 20 inputs.
Includes built in test pattern generator as background layer if required.
Supports low-latency mode by using layer 0 as background layer.

The Mixer II IP core reads the control data in two steps at the start of each frame. The buffering happens inside the IP core so that the control data can be updated during the frame processing without unexpected side effects.

The first step occurs after the IP core processes and transmits all the non-image data packets of the background layer, and it has received the header of an image data packet of type 0 for the background. At this stage, the on/off status of each layer is read. A layer can be:

disabled (0),
active and displayed (1), or
consumed but not displayed (2)

The maximum number of image layers mixed cannot be changed dynamically and must be set in the parameter editor.

The IP core processes the non-image data packets of each active foreground layer, displayed or consumed, in a sequential order, layer 1 first. The IP core integrally transmits the non-image data packets from the background layer. The IP core treats the non-image data packets from the foreground layers differently depending on their type.

Control packets (type 15)— processed to extract the width and height of each layer and are discarded on the fly.
Other/user packets (types 1–14)—propagated unchanged.

The second step corresponds to the usual behavior of other Video and Image Processing IP cores that have an Avalon-MM slave interface. After the IP core has processed and/or propagated the non-image data packets from the background layer and the foreground layers, it waits for the Go bit to be set to 1 before reading the top left position of each layer.

Consequently, the behavior of the Mixer II IP core differs slightly from the other Video and Image Processing IP cores, as illustrated by the following pseudo-code:

go = 0;
while (true)
{
	status = 0;
	read_non_image_data_packet_from background_layer();
	read_control_first_pass(); // Check layer status 
											(disable/displayed/consumed)
	for_each_layer layer_id
	{ 
		// process non-image data packets for displayed or consumed 
															layers
		if (layer_id is not disabled)
		{
		
		handle_non_image_packet_from_foreground_layer(layer_id); 
		}
	}
	while (go != 1)
		wait;
	status = 1;
	read_control_second_pass(); // Copies top-left coordinates to 
														internal registers
	send_image_data_header();
	process_frame();
}

9.1. Alpha Blending

When you turn on Alpha Blending Enable in the Mixer II parameter editor, the IP core is able to mix layers with varying levels of translucency using an alpha value obtained from a run-time programmable register setting.

Each input (N) has an InputN alpha channel parameter that enables the extra per-pixel alpha value for input n. When you enable either of the Alpha blending modes, bit [3:2] in the Input control N registers control which one of these alpha values is used:

Fixed opaque alpha value
Static (run-time programmable) value (only when you select the Alpha Blending Enable parameter)
Per-pixel streaming value (only when you select the InputN alpha channel parameter for InputN)

Note: When you turn the InputN alpha channel parameter, the IP core adds an extra symbol per-pixel for that input. The least significant symbol is the alpha value. The control packet is composed of all symbols including alpha.

The valid range of alpha coefficients is 0 to 1, where 1 represents full translucence, and 0 represents fully opaque.

The Mixer II IP core determines the alpha value width based on your specification of the bits per pixel per color parameter. For n-bit alpha values, the coefficients range from 0 to 2ⁿ–1. The model interprets (2ⁿ– 1) as 1, and all other values as (Alpha value/2ⁿ. For example, 8-bit alpha value 255>1, 254>254/256, 253>253/256, and so on.

The value of an output pixel O_N, where N ranges from 1 to number of inputs minus 1, is derived from the following recursive formula:

O_N = (1 – a_N) p_N + a_N O_{N – 1}

O₀ = (1 – a₀) p₀ + a₀ p_background ,

where p_N is the input pixel for layer N, a_N is the alpha value for layer N, and p_background is the background pixel value.

The Mixer II IP core skips consumed and disabled layers.

Note: All input data samples must be in unsigned format. If the number of bits per pixel per color plane is N, then each sample consists of N bits of data, which are interpreted as an unsigned binary number in the range [0,2ⁿ–1]. The Mixer II IP core also transmits all output data samples in the same unsigned format.

9.2. Mixer II Parameter Settings

Table 33. Mixer II Parameter Settings
Parameter	Value	Description
Maximum output frame width	32-8192, Default = 1920	Specify the maximum image width for the layer background in pixels.
Maximum output frame height	32-8192, Default = 1080	Specify the maximum image height for the layer background in pixels.
Bits per color sample	4-20, Default = 8	Select the number of bits per pixel (per color plane).
Number of pixels in parallel	1, 2, 4, 8	Select the number of pixels transmitted every clock cycle.
Colorspace (used for background layer)	RGB YCbCr	Select the color space you want to use for the background test pattern layer.
4:2:2 support	On or Off	Turn on to enable 4:2:2 sampling rate format for the background test pattern layer. Turn off to enable 4:4:4 sampling rate. Note: The IP core does not support odd heights or widths in 4:2:2 mode.
Synchronize background to layer 0	On or Off	Turn on to synchronize the background layer to layer 0 (low-latency mode). Turn off to use the internal Test Pattern Generator as the background layer.
Layer Position Enable	On or Off	Turn on to enable the layer mapping. Turn off to disable the layer mapping functionality to save gates.
Number of inputs	1-20; Default = 4	Specify the number of inputs to be mixed.
Alpha Blending Enable (all layers)	On or Off	Turn on to allow the IP core to alpha blend using a run-time programmable register alpha value..
InputN alpha channel	On or Off	Turn on to allow the input streams to have an alpha channel.
Pattern	Color bars Uniform background	Select the pattern you want to use for the background test pattern layer.
R or Y	Default = 0	If you choose to use uniform background pattern, specify the individual R'G'B' or Y'Cb'Cr' values based on the color space you selected. The uniform values match the width of bits per pixel up to a maximum of 16 bits. Values beyond 16 bits are zero padded at the LSBs.
G or Cb	Default = 0
B or Cr	Default = 0
Register Avalon-ST ready signals	On or Off	Turn on to add pipeline. Adding pipeline increases the f_MAX value when required, at the expense of increased resource usage.
Reduced control register readback	On or Off	If you turn on this parameter, the values written to register 3 and upwards cannot be read back through the control slave interface. This option reduces ALM usage. If you do not turn on this parameter, the values of all the registers in the control slave interface can be read back after they are written.
Add extra register stages to data pipeline	1-20; Default = 0	Add extra register stages in the data pipeline to improve timing. Add one register stage every Nth data processing stage where N is the parameter value; setting a value of 0 disables this feature. There are as many data processing stages as there are inputs.
How user packets are handled	No user packets allowed Discard all user packets received Pass all user packets through the output	Select whether to allow user packets to be passed through the mixer.

9.3. Mixer II Control Registers

For efficiency reasons, the Video and Image Processing Suite IP cores buffer a few samples from the input stream even if they are not immediately processed. This implies that the Avalon-ST inputs for foreground layers assert ready high, and buffer a few samples even if the corresponding layer has been deactivated.

Table 34. Mixer II Control Register MapThe table describes the control register map for Mixer II IP core.
Address	Register	Description
0	`Control`	Bit 0 of this register is the `Go` bit, all other bits are unused. Setting this bit to 0 causes the IP core to stop the next time control information is read.
1	`Status`	Bit 0 of this register is the `Status` bit, all other bits are unused.
2	`Reserved`	Reserved for future use.
3	`Background Width`	Change the width of the background layer for the next and all future frames. Not available when the Synchronize background to layer 0 parameter is enabled.
4	`Background Height`	Changes the height of the background layer for the next and all future frames. Not available when the Synchronize background to layer 0 parameter is enabled.
5	`Uniform background Red/Y`	Specifies the value for R (RGB) or Y (YCbCr). If you choose to use uniform background pattern, specify the individual R'G'B' or Y'Cb'Cr' values based on the color space you selected. The uniform values match the width of bits per pixel up to a maximum of 16 bits. The IP core zero-pads values beyond 16 bits at the LSBs.
6	`Uniform background Green/Cb`	Specifies the value for G (RGB) or Cb (YCbCr). If you choose to use uniform background pattern, specify the individual R'G'B' or Y'Cb'Cr' values based on the color space you selected. The uniform values match the width of bits per pixel up to a maximum of 16 bits. The IP core zero-pads values beyond 16 bits at the LSBs.
7	`Uniform background Blue/Cr`	Specifies the value for B (RGB) or Cr (YCbCr). If you choose to use uniform background pattern, specify the individual R'G'B' or Y'Cb'Cr' values based on the color space you selected. The uniform values match the width of bits per pixel up to a maximum of 16 bits. The IP core zero-pads values beyond 16 bits at the LSBs.
8+5n	`Input X offset n`	X offset in pixels from the left edge of the background layer to the left edge of input n. Note: n represents the input number, for example input 0, input 1, and so on.
9+5n	`Input Y offset n`	Y offset in pixels from the top edge of the background layer to the top edge of input n. Note: n represents the input number, for example input 0, input 1, and so on.
10+5n	`Input control n`	Set to bit 0 to enable input n. Set to bit 1 to enable consume mode. Set to bits 3:2 to enable alpha mode. 00 – No blending, opaque overlay 01 – Use static alpha value (available only when you turn on the Alpha Blending Enable parameter.) 10 – Use alpha value from input stream (available only when you turn on the InputN alpha channel parameter.) 11 – Unused Note: n represents the input number, for example input 0, input 1, and so on.
11+5n	`Layer position n`	Specifies the layer mapping functionality for input n. Available only when you turn on the Layer Position Enable parameter. Note: n represents the input number, for example input 0, input 1, and so on.
12+5n	`Static alpha n`	Specifies the static alpha value for input n with bit width matching the bits per pixel per color plane parameter. Available only when you turn on the Alpha Blending Enable parameter. Note: n represents the input number, for example input 0, input 1, and so on.

9.4. Layer Mapping

When you turn on Layer Position Enable in the Mixer II parameter editor, the Mixer II allows a layer mapping to be defined for each input using the Layer Position control registers.

The layer positions determine whether an input is mixed in the background (layer 0) through to the foreground (layer N, where N is the number of inputs minus one) in the final output image.

Note: if there are any repeated values within the Layer Position registers (indicating that two inputs are mapped to the same layer), the input with the repeated layer position value will not be displayed and will be consumed.

If you turn off the Layer Position Enable parameter, the Mixer II IP core uses a direct mapping between the ordering of the inputs and the mixing layers. For example, Layer 0 will be mapped to Input 0, Layer 1 to Input 1, and so on.

9.5. Low-Latency Mode

When you turn on Synchronize background to layer 0 in the Mixer II parameter editor, the IP core operates in low-latency mode.

In low-latency mode, the Mixer II IP core synchronizes its operations to the incoming video on layer 0. Each video frame received on layer 0 generates a frame at the IP core's output. The dimensions of the output video matches the dimensions of the input frame.

For layer mapping in low-latency mode, the Mixer II IP core functions in the following behavior:

If you turn off the Layer Position Enable parameter when the IP core is in low-latency mode, the Mixer II IP core synchronizes its operation to input 0 (for this parameterization, input 0 will be mapped to layer 0).
If you turn on the Layer Position Enable parameter when the IP core is in low-latency mode, the Mixer IP core synchronizes its operation to the input that is mapped to layer 0 by its Layer position n control register.

10. Clipper II IP Core

The Clipper II IP core provides a means to select an active area from a video stream and discard the remainder.

You can specify the active region by providing the offsets from each border or a point to be the top-left corner of the active region along with the region's width and height.

The Clipper II IP core handles changing input resolutions by reading Avalon-ST Video control packets. An optional Avalon-MM interface allows the clipping settings to be changed at run time.

10.1. Clipper II Parameter Settings

Table 35. Clipper II Parameter Settings
Parameter	Value	Description
Maximum input frame width	32–8192, Default = 1920	Specify the maximum frame width of the clipping rectangle for the input field (progressive or interlaced).
Maximum input frame height	32–8192, Default = 1080	Specify the maximum height of the clipping rectangle for the input field (progressive or interlaced).
Bits per pixel per color sample	4–20, Default = 8	Select the number of bits per color plane.
Number of color planes	1–4, Default = 2	Select the number of color planes per pixel.
Number of pixels in parallel	1, 2, 4, 8	Select the number of pixels in parallel.
Color planes transmitted in parallel	On or Off	Select whether to send the color planes in parallel or serial. If you turn on this parameter, and set the number of color planes to 3, the IP core sends the R’G’B’s with every beat of data.
Run-time control	On or Off	Turn on if you want to specify clipping offsets using the Avalon-MM interface. Note: When you turn on this parameter, the `Go` bit gets deasserted by default. When you turn off this parameter, the `Go` is asserted by default.
Clipping method	OFFSETS RECTANGLE	Specify the clipping area as offsets from the edge of the input area or as a fixed rectangle.
Left offset	0–8192, Default = 0	Specify the x coordinate for the left edge of the clipping rectangle. 0 is the left edge of the input area. Note: The left and right offset values must be less than or equal to the input image width.
Top offset	0–8192, Default = 0	Specify the y coordinate for the top edge of the clipping rectangle. 0 is the top edge of the input area. Note: The top and bottom offset values must be less than or equal to the input image height.
Right offset	0–8192, Default = 0	Specify the x coordinate for the right edge of the clipping rectangle. 0 is the right edge of the input area. Note: The left and right offset values must be less than or equal to the input image width.
Bottom offset	0–8192, Default = 0	Specify the y coordinate for the bottom edge of the clipping rectangle. 0 is the bottom edge of the input area. Note: The top and bottom offset values must be less than or equal to the input image height.
Width	32–8192, Default = 32	Specify the width of the clipping rectangle. The minimum output width is 32 pixels.
Height	32–8192, Default = 32	Specify the height of the clipping rectangle. The minimum output height is 32 pixels.
Add extra pipelining registers	On or Off	Turn on this parameter to add extra pipeline stage registers to the data path. You must turn on this parameter to achieve: Frequency of 150 MHz for Cyclone III or Cyclone IV devices Frequencies above 250 MHz for Arria II, Stratix IV, or Stratix V devices

10.2. Clipper II Control Registers

Table 36. Clipper II Control Register MapThe control data is read once at the start of each frame and is buffered inside the Clipper II IP core, so the registers can be safely updated during the processing of a frame.
Address	Register	Description
0	`Control`	Bit 0 of this register is the `Go` bit, all other bits are unused. Setting this bit to 0 causes the IP core to stop the next time control information is read. When you enable run-time control, the `Go` bit gets deasserted by default. If you do not enable run-time control, the `Go` is asserted by default.
1	`Status`	Bit 0 of this register is the `Status` bit, all other bits are unused. The Clipper IP core sets this address to 0 between frames. It is set to 1 while the IP core is processing data and cannot be stopped.
2	`Interrupt`	This bit is not used because the IP core does not generate any interrupts.
3	`Left Offset`	The left offset, in pixels, of the clipping window/rectangle. Note: The left and right offset values must be less than or equal to the input image width.
4	`Right Offset or Width`	In clipping window mode, the right offset of the window. In clipping rectangle mode, the width of the rectangle. Note: The left and right offset values must be less than or equal to the input image width.
5	`Top Offset`	The top offset, in pixels, of the clipping window/rectangle. Note: The top and bottom offset values must be less than or equal to the input image height.
6	`Bottom Offset or Height`	In clipping window mode, the bottom offset of the window. In clipping rectangle mode, the height of the rectangle. Note: The top and bottom offset values must be less than or equal to the input image height.

11. Color Plane Sequencer II IP Core

The Color Plane Sequencer II IP core changes how color plane samples are transmitted across the Avalon-ST interface and rearranges the color patterns used to transmit Avalon-ST Video data packets.

A color pattern is a matrix that defines a pattern of color samples repeating over the length of an image.

The Color Plane Sequencer II IP core offers the following features:

Splits or duplicates a single Avalon-ST Video stream into two or, conversely, combines two input streams into a single stream.
Supports Avalon-ST Video streams with up to 4 pixels transmitted in parallel. A pixel may contain up to 4 color planes transmitted either in parallel or in sequence but not both
The input/output color patterns to rearrange the Avalon-ST Video streams between the inputs and outputs may be defined over two pixels, which covers all common use-cases.

11.1. Combining Color Patterns

You can configure the Color Plane Sequencer II IP core to combine two Avalon-ST Video streams into a single stream.

In this mode of operation, the IP core pulls in two input color patterns (one for each input stream) and arranges to the output stream color pattern in a user-defined way, in sequence or parallel, as long as it contains a valid combination of the input channels. In addition to this combination and rearrangement, color planes can also be dropped.

Figure 44. Example of Combining Color PatternsThe figure shows an example of combining and rearranging two color patterns.

11.2. Rearranging Color Patterns

You can configure the Color Plane Sequencer II IP core to rearrange the color pattern of a video packet, and drop or duplicate color planes.

Figure 45. Example of Rearranging Color PatternsThe figure shows an example that rearranges the color pattern of a video data packet which transmits color planes in sequence to transmit color planes in parallel.

11.3. Splitting and Duplicating

You can configure the Color Plane Sequencer II IP core to split a single Avalon-ST Video input stream into two Avalon-ST Video output streams.

In this mode of operation, the IP core arranges the color patterns of video data packets on the output streams in a user-defined way using any of the color planes of the input color pattern.

The color planes of the input color pattern are available for use on either, both, or neither of the outputs. This allows for splitting of video data packets, duplication of video data packets, or a mix of splitting and duplication. The output color patterns are independent of each other, so the arrangement of one output stream's color pattern places no limitation on the arrangement of the other output stream's color pattern.

Figure 46. Example of Splitting Color Patterns

CAUTION:

A deadlock may happen when a video design splits, processes independently, and then joins back the color planes, or when the sequencer splits the color planes in front of another VIP IP core. To avoid this issue, add small FIFO buffers at the output of the Color Plane Sequencer II IP core that are configured as splitters.

11.4. Handling of Subsampled Data

Besides fully sampled color patterns, the Color Plane Sequencer II IP core also supports 4:2:2 subsampled data.

To support 4:2:2 subsampled data, you must configure the IP core to use a 2-pixel pattern for the relevant input or output.

When specifying an input pattern over two pixels, the Color Plane Sequencer II IP core pulls two input pixels from the corresponding input before doing the rearrangement. Hence, you can configure the first pixel of the pattern with color planes "Y" and "Cb" and the second pixel of the pattern with color planes "Y" and "Cr".
When specifying an output pattern over two pixels, each rearrangement operation produces two output pixels. You may specify different color planes for the first and second pixel of the pattern.

You may use two-pixel patterns irrespective of the Avalon-ST Video transmission settings. They remain valid when pixels are transmitted in parallel or when color planes are transmitted sequentially.

The width of Avalon-ST Video control packets is automatically modified when handling subsampled data.

When using a 2-pixel pattern for input stream 0, the IP core halves the width of the input control packets if the output is using a single-pixel pattern.
When using a single pixel pattern for input stream 0, the IP doubles the width of the input control packets if the output is using a 2-pixel pattern.
Control packet widths are not modified when using a single-pixel or a 2-pixel pattern on both sides.

11.5. Handling of Non-Image Avalon-ST Packets

The Color Plane Sequencer II IP core also handles Avalon-ST Video packets other than video data packets to the output(s).

You can forward Avalon-ST Video packets other than video data packets to the output(s) with these options:

Avalon-ST Video control packets from input stream 1 are always dropped.
Avalon-ST Video control packets from input stream 0 may be either propagated or dropped depending on the IP parameterization but the last control packet received before the image packet on input stream 0 is always propagated on all enabled outputs and its width may be altered.
Input user packets can be dropped or forwarded to either or both outputs.

Note: When the color pattern of a video data packet changes from the input to the output side of a block, the IP core may pad the end of non-video user packets with extra data. Intel recommends that when you define a packet type where the length is variable and meaningful, you send the length at the start of the packet. User data is never truncated but there is no guarantee that the packet length will be preserved or even rounded up to the nearest number of output color planes.

11.6. Color Plane Sequencer Parameter Settings

Table 37. Color Plane Sequencer II Parameter Settings n refers to the input or output number.
Parameter	Value	Description
How user packets are handled	No user packets allowed Discard all user packets received Pass all user packets through to the output(s)	If your design does not require the IP core to propagate user packets, then you may select Discard all user packets received to reduce ALM usage. If your design guarantees there will never be any user packets in the input data stream, then you can further reduce ALM usage by selecting No user packets allowed. In this case, the IP core may lock if it encounters a user packet. When propagating user packets, you should specify how the packets are routed. Each input can be routed to either or both outputs independently.
Add extra pipelining registers	On or Off	Turn on to add extra pipeline stage registers to the data path.
Bits per color sample	4-20, Default = 8	Select the number of bits per color sample.
Number of inputs	1 or 2	Select the number of inputs.
Number of outputs	1 or 2	Select the number of outputs.
din_n: Add input fifo	On or Off	Turn on if you want to add a FIFO at the input to smooth the throughput burstiness.
din_n: Input fifo size	1–128, Default = 8	Specify the size (in powers of 2) of the input FIFO (in number of input beats).
din_n: Number of color planes	1–4, Default = 3	Select the number of color planes per pixel.
din_n: Color planes transmitted in parallel	On or Off	Select whether the color planes are in parallel or in sequence (serially).
din_n: Number of pixels in parallel	1, 2, 4	Specify the number of pixels received in parallel (per clock cycle).
din_n: Specify an input pattern over two pixels	On or Off	Turn on if you want to create an input color pattern using two consecutive input pixels instead of one.
din_n: Input pattern for pixel 0	—	Select a unique symbol name for each color plane of pixel 0. Each symbol may appear only once and must not be reused for pixel 1, or when specifying the color pattern for the other input.
din_n: Input pattern for pixel 1	—	Select a unique symbol name for each color plane of pixel 1. This parameter is only available if you turn on Specify an input pattern over two pixels.
dout_n: Add output fifo	On or Off	Turn on if you want to add a FIFO at the output to smooth the throughput burstiness.
dout_n: Output fifo size	1–128, Default = 8	Specify the size (in powers of 2) of the output FIFO (in number of output beats).
dout_n: Number of color planes	1–4, Default = 3	Select the number of color planes per pixel.
dout_n: Color planes transmitted in parallel	On or Off	Select whether to transmit the color planes in parallel or in sequence (serially).
dout_n: Number of pixels in parallel	1, 2, 4	Specify the number of pixels transmitted in parallel (per clock cycle).
dout_n: Propagate user packets from input 0	—	Select whether user packets from input 0 are propagated through output n. This parameter is only available if you turn on Pass all user packets through to the output(s).
dout_n: Propagate user packets from input 1	—	Select whether user packets from input 1 are propagated through output n. This parameter is only available if you turn on Pass all user packets through to the output(s) and Specify an input pattern over two pixels.
dout_n: Specify an output pattern over two pixels	On or Off	Turn on if you want to create an output color pattern using two consecutive output pixel instead of one.
dout_n: Output pattern for pixel 0	—	Select a valid symbol name for each color plane of pixel 0. The symbol must be defined on one of the input color patterns.
dout_n: Output pattern for pixel 1	—	Select a valid symbol name for each color plane of pixel 1. The symbol must be defined on one of the input color patterns. This parameter is only available if you turn on Specify an output pattern over two pixels.

12. Color Space Converter II IP Core

The Color Space Converter II IP core transforms video data between color spaces. The color spaces allow you to specify colors using three coordinate values.

You can configure this IP core to change conversion values at run time using an Avalon-MM slave interface.

The Color Space Converter II IP core offers the following features:

Provides a flexible and efficient means to convert image data from one color space to another.
Supports a number of predefined conversions between standard color spaces.
Allows the entry of custom coefficients to translate between any two three-valued color spaces.
Supports up to 4 pixels per transmission.

A color space is a method for precisely specifying the display of color using a three-dimensional coordinate system. Different color spaces are best for different devices, such as R'G'B' (red-green-blue) for computer monitors or Y'CbCr (luminance-chrominance) for digital television.

Color space conversion is often necessary when transferring data between devices that use different color space models. For example, to transfer a television image to a computer monitor, you are required to convert the image from the Y'CbCr color space to the R'G'B' color space. Conversely, transferring an image from a computer display to a television may require a transformation from the R'G'B' color space to Y'CbCr.

Different conversions may be required for standard definition television (SDTV) and high definition television (HDTV). You may also want to convert to or from the Y'IQ (luminance-color) color model for National Television System Committee (NTSC) systems or the Y'UV (luminance-bandwidth-chrominance) color model for Phase Alternation Line (PAL) systems.

12.1. Input and Output Data Types

The inputs and outputs of the Color Space Converter II IP core support signed or unsigned data and 4 to 20 bits per pixel per color plane. The IP cores also support minimum and maximum guard bands.

The guard bands specify ranges of values that must never be received by, or transmitted from the IP core. Using output guard bands allows the output to be constrained, such that it does not enter the guard bands.

12.2. Color Space Conversion

You convert between color spaces by providing an array of nine coefficients and three summands that relate the color spaces.

You can set these coefficients and summands at compile time, or you can enable the Avalon-MM slave interface to change them dynamically at run-time.

Given a set of nine coefficients [A0, A1, A2, B0, B1, B2, C0, C1, C2] and a set of three summands [S0, S1, S2], the IP cores calculate the output values for color planes 0, 1, and 2 (denoted dout_0, dout_1, and dout_2):

dout_0 = (A0 × din_0) + (B0 × din_1) + (C0 × din_2) + S0
dout_1 = (A1 × din_0) + (B1 × din_1) + (C1 × din_2) + S1
dout_2 = (A2 × din_0) + (B2 × din_1) + (C2 × din_2) + S2

Note: din_0, din_1, and din_2 are inputs read from color planes 0, 1, and 2.

The Color Space Converter II IP core supports the following predefined conversions that are available through the Platform Designer presets:

Computer B’G’R’ to CbCrY’: SDTV
CbCrY’: SDTV to Computer B’G’R’
Computer B’G’R’ to CbCrY’: HDTV
CbCrY’: HDTV to Computer B’G’R’
Studio B’G’R’ to CbCrY’: SDTV
CbCrY’: SDTV to Studio B’G’R’
Studio B’G’R’ to CbCrY’: HDTV
CbCrY’: HDTV to Studio B’G’R’
IQY' to Computer B'G'R'
Computer B'G'R' to IQY'
UVY' to Computer B'G'R'
Computer B'G'R' to UVY'

The values are assigned in the order indicated by the conversion name. For example, if you select Computer B’G’R’ to CbCrY’: SDTV, din_0 = B’, din_1 = G’, din_2 = R’, dout_0 = Cb’, dout_1 = Cr, and dout_2 = Y’.

If the channels are in sequence, din_0 is first, then din_1, and din_2. If the channels are in parallel, din_0 occupies the least significant bits of the word, din_1 the middle bits, and din_2 the most significant bits. For example, if there are 8 bits per sample and one of the predefined conversions inputs B’G’R’, din_0 carries B’ in bits 0–7, din_1 carries G’ in bits 8–15, and din_2 carries R’ in bits 16–23.

12.2.1. Predefined Conversions

Predefined conversions only support unsigned input and output data.

If you select signed input or output data, the predefined conversion produces incorrect results. When using a predefined conversion, the precision of the coefficients and summands must still be defined.

Predefined conversions are only defined for input and output bits per pixel per color plane equal to 8, 10, and 12. You must manually scale the summands accordingly when using a different bits per color plane value. If you use different input and output bits per pixel per color plane, you must also shift the results by the correct number of binary places to compensate. For example, to convert from 10-bit CbCrY' to 8-bit Computer B'G'R', select the conversion preset for 10-bit CbCrY' to 10-bit computer B'G'R'. The summands are already scaled for a 10-bit input so they remain unchanged. Change the output bits per color plane value from 10 to 8 on the parameter editor and follow the instructions of the warning message to shift the results by the correct number of binary places (2 places to the left).

Note: Always check the matrix of coefficients after applying a predefined conversion or after custom modifications. If the differences between the desired floating-point coefficient values and their actual fixed-point quantized values indicate an unacceptable loss of precision, you must increase the number of integer and/or fractional bits to fix the problem.

12.3. Result of Output Data Type Conversion

After the calculation, the fixed point type of the results must be converted to the integer data type of the output.

This conversion is performed in four stages, in the following order:

Result scaling—You can choose to scale up the results, increasing their range. This is useful to quickly increase the color depth of the output.
- The available options are a shift of the binary point right –16 to +16 places.
- This is implemented as a simple shift operation so it does not require multipliers.
Removal of fractional bits—If any fractional bits exist, you can choose to remove them:
- Truncate to integer—Fractional bits are removed from the data. This is equivalent to rounding towards negative infinity.
- Round-half up—Round up to the nearest integer. If the fractional bits equal 0.5, rounding is towards positive infinity.
- Round-half even. Round to the nearest integer. If the fractional bits equal 0.5, rounding is towards the nearest even integer.
Conversion from signed to unsigned—If any negative numbers can exist in the results and the output type is unsigned, you can choose how they are converted:
- Saturate to the minimum output value (constraining to range).
- Replace negative numbers with their absolute positive value.
Constrain to range—logic that saturates the results to the minimum and maximum output values is automatically added:
- If any of the results are not within the minimum and maximum values allowed by the output bits per pixel
- If any of the results are beyond the range specified by the output Guard bands (optional)

12.4. Color Space Converter Parameter Settings

Table 38. Color Space Converter II Parameter Settings
Parameter	Value	Description
General
Color planes transmitted in parallel	On or Off	Turn on to transmit the color planes in parallel.
Number of pixels in parallel	1, 2, 4, or 8	Specify the number of pixels transmitted or received in parallel.
Input data type: Input bits per pixel per color plane	4–20, Default = 8	Specify the number of input bits per pixel (per color plane).
Input data type: Signed	On or Off	Turn to specify the output as signed 2's complement.
Input data type: Guard bands	On or Off	Turn to use a defined input range.
Input data type: Max	-524288–1048575, Default = 255	Specify the input range maximum value.
Input data type: Min	-524288–1048575, Default = 0	Specify the input range minimum value.
Output data type: Bits per pixel per color plane	4–20, Default = 8	Select the number of output bits per pixel (per color plane).
Output data type: Signed	On or Off	Turn to specify the output as signed 2's complement.
Output data type: Guard bands	On or Off	Turn on to enable a defined output range.
Output data type: Max	-524288–1048575, Default = 255	Specify the output range maximum value.
Output data type: Min	-524288–1048575, Default = 0	Specify the output range minimum value.
How user packets are handled	No user packets allowed Discard all user packets received Pass all user packets through to the output	If you design does not require the IP core to propagate user packets, then you may select to discard all user packets to reduce ALM usage. If your design guarantees there will never be any user packets in the input data stream, then you further reduce ALM usage by selecting No user packets allowed. In this case, the IP core may lock if it encounters a user packet.
Conversion method	LSB or MSB	This parameter is enabled when input and output bits per sample per color plane differ and when user packets are propagated. When the propagation of user packets requires padding or truncation, the IP can do one of the following: Truncate or zero-pad the most significant bits Truncate or pad the least significant bits
Run-time control	On or Off	Turn on to enable runtime control of the conversion values.
Reduced control register readback	On or Off	If you do not turn on this parameter, the values of all the registers in the control slave interface can be read back after they are written. If you turn on this parameter, the values written to registers 3 and upwards cannot be read back through the control slave interface. This option reduces ALM usage.
Operands
Coefficient and summand fractional bits	0–31, Default = 8	Specify the number of fraction bits for the fixed point type used to store the coefficients and summands.
Coefficient precision: Signed	On or Off	Turn on to set the fixed point type used to store the constant coefficients as having a sign bit.
Coefficient precision: Integer bits	0–16, Default = 1	Specifies the number of integer bits for the fixed point type used to store the constant coefficients.
Summand precision: Signed	On or Off	Turn on to set the fixed point type used to store the constant summands as having a sign bit.
Summand precision: Integer bits	0–22, Default = 10	Specifies the number of integer bits for the fixed point type used to store the constant summands.
Coefficients and Summand Table A0, B0, C0, S0 A1, B1, C1, S1 A2, B2, C2, S2	12 fixed-point values	Each coefficient or summand is represented by a white cell with a gray cell underneath. The value in the white cell is the desired value, and is editable. The value in the gray cell is the actual value, determined by the fixed-point type specified. The gray cells are not editable. You can create a custom coefficient and summand set by specifying one fixed-point value for each entry.
Move binary point right	-16 to +16, Default = 0	Specify the number of places to move the binary point.
Remove fraction bits by	Round values - Half up Round values - Half even Truncate values to integer	Select the method of discarding fraction bits resulting from the calculation.
Convert from signed to unsigned by	Saturating to minimum value at stage 4 Replacing negative with absolute value	Select the method of signed to unsigned conversion for the results.

12.5. Color Space Conversion Control Registers

The width of each register in the Color Space Conversion control register map is 32 bits. To convert from fractional values, simply move the binary point right by the number of fractional bits specified in the user interface.

The control data is read once at the start of each frame and is buffered inside the IP core, so the registers can be safely updated during the processing of a frame.

Table 39. Color Space Converter II Control RegisterThe table below describes the control register map for Color Space Converter II IP core.
Address	Register	Description
0	`Control`	Bit 0 of this register is the `Go` bit, all other bits are unused. Setting this bit to 0 causes the IP core to stop the next time control information is read.
1	`Status`	Bit 0 of this register is the `Status` bit, all other bits are unused.
2	`Interrupts`	Unused.
3	`Coeff-commit`	Writing a 1 to this location commits the writing of coefficient data. You must make this write to swap the coefficients currently in use with the latest set written to the register map.
4	`Coefficient A0`	The coefficient and summand registers use integer, signed 2’s complement numbers. Refer to Color Space Conversion.
5	`Coefficient B0`
6	`Coefficient C0`
7	`Coefficient A1`
8	`Coefficient B1`
9	`Coefficient C1`
10	`Coefficient A2`
11	`Coefficient B2`
12	`Coefficient C2`
13	`Summand S0`
14	`Summand S1`
15	`Summand S2`

13. Chroma Resampler II IP Core

The Chroma Resampler II IP core resamples video data to and from common sampling formats.

The human eye is more sensitive to brightness than tone. Taking advantage of this characteristic, video transmitted in the Y’CbCr color space often subsamples the color components (Cb and Cr) to save on data bandwidth.

The Chroma Resampler II IP core allows you to change between 4:4:4 and 4:2:2 sampling rates where:

4:4:4 specifies full resolution in planes 1, 2, and 3 (Y, Cb and Cr respectively)
4:2:2 specifies full resolution in plane 1 and half width resolution in planes 2 and 3 (Y, Cb and Cr respectively)

Table 40. Chroma Resampler II Sampling Support for Algorithms
Sampling (4:2:2 to 4:4:4	Algorithm
Sampling (4:2:2 to 4:4:4	Nearest Neighbor	Bilinear	Filtered
Upsampling	Yes	Yes	Yes
Downsampling	Yes	Yes	Yes
Horizontal Resampling	Yes	No	No
Vertical Resampling	Yes	No	No

You can configure the Chroma Resampler II IP core to operate in one of two generalized modes: fixed mode and variable mode.

Table 41. Chroma Resampler Modes
Fixed mode	Variable mode
Both the input and output Avalon-ST Video interfaces are fixed to a set subsampling format, either 4:2:2 or 4:4:4. Select either a fixed 4:4:4 to 4:2:2 downsampling operation, or a fixed 4:2:2 to 4:4:4 upsampling operation. Enable fixed mode by setting the Variable 3 color interface parameter to NEITHER. Does not support 4:2:0 format and does not provide the option for run-time control of the subsampling on either interface.	Configure either the input or output Avalon-ST video interface as a variable subsampling interface. Has 3 color planes per pixel and may transmit data formatted as 4:4:4, 4:2:2 or 4:2:0 data, with the selected subsampling option set at run time through the Avalon-MM Slave control interface. Enable variable mode by setting the Variable 3 color interface parameter to INPUT or OUTPUT. Note: If you leave the Variable 3 color interface parameter by the default selection, the IP core retains a fixed subsampling format of either 4:4:4 or 4:2:2.

The variable mode is mainly to be use with interface standards, such as HDMI 2.0, which allow color space and subsampling to vary at run time. Because most of the VIP IP cores support only a fixed subsampling (either 4:4:4 or 4:2:2), the Chroma Resampler II IP core is allows a variable-to-fixed conversion (Variable 3 color interface = INPUT) at the input to the processing pipeline, and a fixed-to-variable conversion (Variable 3 color interface = OUTPUT) at the output of the processing pipeline

13.1. Chroma Resampler Algorithms

The Chroma Resampler II IP core supports 3 different resampling algorithms.

These three algorithms vary in the level of visual quality provided, the chroma siting, and the resources required for implementation:

Nearest Neighbor
Bilinear
Filtered

13.1.1. Nearest Neighbor

Nearest neighbor is the lowest quality resampling algorithm, with the lowest device resource usage.

For horizontal downsampling (4:4:4 to 4:2:2), it simply drops every other Cb and Cr sample.
For horizontal upsampling (4:2:2 to 4:4:4), it simply repeats each Cb and Cr sample.
For vertical downsampling (4:2:2 to 4:2:0) the chroma data from every other video line is discarded.
For vertical upsampling (4:2:0 to 4:2:2) the chroma data is repeated for two lines of luma data.

Figure 47. Nearest Neighbor for Horizontal ResamplingThe nearest neighbor algorithm uses left siting (co-siting) for the 4:2:2 chroma samples—both the Cb and Cr samples from the even indexed Y samples are retained during downsampling.

Figure 48. Nearest Neighbor for Vertical ResamplingThe nearest neighbor algorithm uses top siting (co-siting) for both the Cb and Cr planes, for example, the chroma data from lines 0, 2, 4, and so on is preserved in downsampling, while the data from lines 1, 3, 5, and so on is discarded.

13.1.2. Bilinear

The bilinear algorithm offers a middle point between visual image quality and device resource cost.

The figure and equations below show how the Chroma Resampler II IP core calculates the bilinear resampled chroma for both horizontal and vertical upsampling and downsampling. The bilinear algorithm uses center chroma siting for both the Cb and Cr samples in 4:2:2 format.

Figure 49. Bilinear Resampling

13.1.3. Filtered

The filtered algorithm is the most computationally expensive and device resource heavy algorithm, but it offers increased visual quality.

You can parameterize the filtered algorithm to use either left siting (co-siting) or center siting of the chroma data.

For downsampling conversions (4:4:4 to 4:2:2), the filtered algorithm applies an 8-tap Lanczos-2 resampling filter to generate the downsampled data. Different phase shifts are applied to the Lanczos-2 function when generating the coefficients, depending on the siting selected and whether the pixel index is even or odd. For left chroma siting, phase shifts of 0 and 0.5 are applied to the Lanczos-2 coefficients for the,even and odd indexed chroma samples respectively. For center chroma siting, the phases shifts are –0.25 and +0.25.
For upsampling conversions (4:2:2 to 4:4:4), the filtered algorithm applies a 4-tap Lanczos-2 resampling filter to generate the upsampled data. For left chroma siting phase shifts of 0 and 0.5 are applied to the Lanczos-2 coefficients for the even and odd indexed chroma samples respectively. For center chroma siting the phases shifts are -0.25 and +0.25.

You may also opt to enable luma adaption for upsampling conversions. This feature further increases device resource usage (and is the only chroma resampler mode to implement some logic in DSP blocks), but may reduce color bleed around edges when compared to the default filtered algorithm.

When you enable luma adaption, the differences between successive luma samples are computed and compared to an edge threshold to detect significant edges. In areas where edges with strong vertical components are detected the phase of the Lanczos-2 filter can be shifted by up to 0.25 to the left or right to weight the resulting chroma samples more heavily towards the more appropriate side of the edge.

13.2. Chroma Resampler Parameter Settings

Table 42. Chroma Resampler II Parameter Settings
Parameter	Value	Description
Horizontal resampling algorithm	NEAREST_NEIGHBOR BILINEAR FILTERED	Select the horizontal resampling algorithm to be used.
Horizontal chroma siting	LEFT CENTER	Select the horizontal chroma siting to be used. This option is only available for the filtered algorithm. The nearest neighbor algorithm forces left siting and bilinear algorithm forces center siting.
Enable horizontal luma adaptive resampling	On or Off	Turn on to enable horizontal luma-adaptive resampling. The parameter is only available for filtered upsampling.
Vertical resampling algorithm	NEAREST_NEIGHBOR BILINEAR FILTERED	Select the vertical resampling algorithm to be used.
Vertical chroma siting	LEFT CENTER	Select the vertical chroma siting to be used. This option is only available for the filtered algorithm. The nearest neighbor algorithm forces top siting and bilinear algorithm forces center siting.
Enable vertical luma adaptive resampling	On or Off	Turn on to enable vertical luma-adaptive resampling. The parameter is only available for filtered upsampling.
Maximum frame width	32–8192, Default = 1920	Specify the maximum frame width allowed by the IP core.
Maximum frame height	32–8192, Default = 1080	Specify the maximum frame height allowed by the IP core.
How user packets are handled	No user packets allowed Discard all user packets received Pass all user packets through to the output	If your design does not require the Chroma Resampler II IP core to propagate user packets, then you may select Discard all user packets received to reduce ALM usage. If your design guarantees that the input data stream will never have any user packets, then you can further reduce ALM usage by selecting No user packets allowed. In this case, the IP core may lock if it encounters a user packet.
Add extra pipelining registers	On or Off	Turn on to add extra pipeline stage registers to the data path. You must to turn on this option to achieve: Frequency of 150 MHz for Cyclone V devices Frequencies above 250 MHz for Arria V, Stratix V, Intel^® Arria^® 10, or Intel^® Cyclone^® 10 GX devices
Bits per color sample	4–20, Default = 8	Select the number of bits per color plane per pixel.
Number of color planes	1–4, Default = 2	Select the number of color planes per pixel.
Color planes transmitted in parallel	On or Off	Select whether to send the color planes in parallel or in sequence (serially).
Input pixels in parallel	1, 2, 4, 8, Default = 1	Select the number of pixels transmitted per clock cycle on the input interface.
Output pixels in parallel	1, 2, 4, 8, Default = 1	Select the number of pixels transmitted per clock cycle on the output interface.
Variable 3 color interface	NEITHER INPUT OUTPUT	Select which interface uses the variable subsampling 3 color interface.
Enable 4:4:4 input	On or Off	Turn on to select 4:4:4 format input data. Note: The input and output formats must be different. A warning is issued when the same values are selected for both.
Enable 4:2:2 input	On or Off	Turn on to select 4:2:2 format input data. Note: The input and output formats must be different. A warning is issued when the same values are selected for both. The IP core does not support odd heights or widths in 4:2:2 mode.
Enable 4:2:0 input	On or Off	Turn on to select 4:2:0 format input data. Note: The input and output formats must be different. A warning is issued when the same values are selected for both.
Enable 4:4:4 output	On or Off	Turn on to select 4:4:4 format output data. Note: The input and output formats must be different. A warning is issued when the same values are selected for both.
Enable 4:2:2 output	On or Off	Turn on to select 4:2:2 format output data. Note: The input and output formats must be different. A warning is issued when the same values are selected for both. The IP core does not support odd heights or widths in 4:2:2 mode.
Enable 4:2:0 output	On or Off	Turn on to select 4:2:0 format output data. Note: The input and output formats must be different. A warning is issued when the same values are selected for both.

13.3. Chroma Resampler Control Registers

Table 43. Chroma Resampler II Control Register MapThe Chroma Resampler II IP Core automatically includes an Avalon-MM control slave interface if you select GUI parameters that have either a variable format input or variable format output interface. For variable format interfaces, you select the required input or output subsampling format through the control slave. If both interfaces are fixed formats, then there are no configurable features so the control slave is omitted.

Note: As is the convention with all VIP Suite cores, when a control slave interface is included, the core resets into a stopped state and must be started by writing a ‘1’ to the `Go` bit of the control register before any input data is processed.
Address	Register	Description
0	`Control`	Bit 0 of this register is the `Go` bit, all other bits are unused. Setting this bit to 0 causes the IP core to stop at the end of the next frame/field packet.
1	`Status`	Bit 0 of this register is the `Status` bit, all other bits are unused. The Chroma Resampler II IP core sets this address to 0 between frames. It is set to 1 while the IP core is processing data and cannot be stopped.
2	`Interrupt`	This bit is not used because the IP core does not generate any interrupts.
3	`Selected subsampling`	Control the selected subsampling format on either the input or output interface (whichever is variable). Write 0 to select 4:2:0, 1 for 4:2:2, and 2 for 4:4:4.

14. Control Synchronizer IP Core

The Control Synchronizer IP core synchronizes the configuration change of IP cores with an event in a video stream. For example, the IP core can synchronize the changing of a position of a video layer with the changing of the size of the layer.

The Control Synchronizer IP core has the following ports:

Avalon Video Streaming Input and Output port—passes through Avalon-ST Video data, and monitors the data for trigger events.
Avalon Master port—writes data to the Avalon Slave control ports of other IP cores when the Control Synchronizer IP core detects a trigger event.
Avalon Slave port—sets the data to be written and the addresses that the data must be written to when the IP core detects a trigger event.
Avalon Slave Control port—disables or enables the trigger condition. You can configure the IP core before compilation to disable this port after every trigger event; disabling this port is useful if you want the IP core to trigger only on a single event.

The following events trigger the Control Synchronizer IP core:

the start of a video data packet.
a change in the width or height field of a control data packet that describes the next video data packet.

When the Control Synchronizer IP core detects a trigger event, the following sequence of events take place:

The IP core immediately stalls the Avalon-ST video data flowing through the IP core.
The stall freezes the state of other IP cores on the same video processing data path that do not have buffering in between.
The IP core then writes the data stored in its Avalon Slave register map to the addresses that are also specified in the register map.
After writing is complete, the IP core resumes the Avalon-ST video data flowing through it. This ensures that any cores after the Control Synchronizer IP core have their control data updated before the start of the video data packet to which the control data applies.
When all the writes from a Control Synchronizer IP core trigger are complete, an interrupt is triggered or is initiated, which is the “completion of writes” interrupt.

14.1. Using the Control Synchronizer IP Core

The example illustrates how the Control Synchronizer IP Core is set to trigger on the changing of the width field of control data packets.

In the following example, the Control Synchronizer IP Core is placed in a system containing the following IP cores:

Test Pattern Generator II
Frame Buffer II
Scaler II

The Control Synchronizer IP core must synchronize a change of the width of the generated video packets with a change to the scaler output size in the following conditions:

The scaler maintains a scaling ratio of 1:1 (no scaling)
The frame buffer is configured to drop and repeat making it impossible to calculate packets streamed into the frame buffer and streamed out to the scaler.
The scaler cannot be configured in advance of a certain video data packet.

The Control Synchronizer IP Core solves the problem through the following sequence of events:

Sets up the change of video width.
Figure 50. Change of Video Width
The test pattern generator changes the size of its Video Data Packet and Control Data Packet pairs to 320 width. It is not known when this change will propagate through the frame buffer to the scaler.
Figure 51. Changing Video Width
The Video Data Packet and Control Data Packet pair with changed width of 320 propagates through the frame buffer. The control synchronizer detects the change and triggers a write to the scaler. The control synchronizer stalls the video processing pipeline while it performs the write.
Figure 52. Test Pattern Generator Change
The scaler is reconfigured to output width 320 frames. The control synchronizer resumes the video processing pipeline. The scaling ratio maintains at 1:1.
Figure 53. Reconfigured Scaler II

14.2. Control Synchronizer Parameter Settings

Table 44. Control Synchronizer Parameter Settings
Parameter	Value	Description
Bits per pixel per color plane	4-20, Default = 8	Select the number of bits per pixel (per color plane).
Number of color planes	1–4, Default = 3	Select the number of color planes that are sent over one data connection. For example, a value of 3 for R'G'B' R'G'B' R'G'B' in serial.
Color planes are in parallel	On or Off	Turn on to set colors planes in parallel. Turn off to set colors planes in series.
Trigger on width change	On or Off	Turn on to start transfer of control data when there is a change in width value.
Trigger on height change	On or Off	Turn on to start transfer of control data when there is a change in height value.
Trigger on start of video data packet	On or Off	Turn on to start transfer of control data when the core receives the start of video data packet.
Require trigger reset via control port	On or Off	Turn on to disable the trigger once triggered. If you turn on this parameter, you need to enable the trigger using the control port.
Maximum number of control data entries	1–10, Default = 3	Specify the maximum number of control data entries that can be written to other cores.

14.3. Control Synchronizer Control Registers

Table 45. Control Synchronizer Register MapThe control data is read once at the start of each frame and is buffered inside the IP core, so the registers can be safely updated during the processing of a frame.

Note: The width of each register of the frame reader is 32 bits.
Address	Register	Description
0	`Control`	Bit 0 of this register is the `Go` bit. Setting this bit to 0 causes the IP core to start passing through data. Bit 1 of this register is the interrupt enable. Setting this bit to 1 enables the completion of writes interrupt.
1	`Status`	Bit 0 of this register is the `Status` bit, all other bits are unused.
2	`Interrupt`	Bit 1 of this register is the completion of writes interrupt bit, all other bits are unused. Writing a 1 to bit 1 resets the completion of writes interrupt.
3	`Disable Trigger`	Setting this register to 1 disables the trigger condition of the control synchronizer. Setting this register to 0 enables the trigger condition of the control synchronizer. When you turn on the Require trigger reset via control port parameter, this register value is automatically set to 1 every time the control synchronizer triggers.
4	`Number of writes`	This register sets how many write operations, starting with address and word 0, are written when the control synchronizer triggers.
5	`Address 0`	Address where word 0 must be written on trigger condition.
6	`Word 0`	The word to write to address 0 on trigger condition.
7	`Address 1`	Address where word 1 must be written on trigger condition.
8	`Word 1`	The word to write to address 1 on trigger condition.
9	`Address 2`	Address where word 2 must be written on trigger condition.
10	`Word 2`	The word to write to address 2 on trigger condition.
11	`Address 3`	Address where word 3 must be written on trigger condition.
12	`Word 3`	The word to write to address 3 on trigger condition.
13	`Address 4`	Address where word 4 must be written on trigger condition.
14	`Word 4`	The word to write to address 4 on trigger condition.
15	`Address 5`	Address where word 5 must be written on trigger condition.
16	`Word 5`	The word to write to address 5 on trigger condition.
17	`Address 6`	Address where word 6 must be written on trigger condition.
18	`Word 6`	The word to write to address 6 on trigger condition.
19	`Address 7`	Address where word 7 must be written on trigger condition.
20	`Word 7`	The word to write to address 7 on trigger condition.
21	`Address 8`	Address where word 8 must be written on trigger condition.
22	`Word 8`	The word to write to address 8 on trigger condition.
23	`Address 9`	Address where word 9 must be written on trigger condition.
24	`Word 9`	The word to write to address 9 on trigger condition.

15. Deinterlacer II IP Core

The Deinterlacer II IP core (4K HDR passthrough) provides deinterlacing algorithms.

Interlaced video is commonly used in television standards such as phase alternation line (PAL) and national television system committee (NTSC), but progressive video is required by LCD displays and is often more useful for subsequent image processing functions.

The features for the Deinterlacer II IP core include:

Run-time control and status registers to improve deinterlacing quality for mid-range motion-adaptive configurations
Run-time control registers to allow run-time switching between bob, weave, and motion adaptive modes.
Support for pass-through of progressive video at up to 4K resolutions and at a higher number of bits per pixel per color plane
Integration of a stream cleaner core and embedded chroma resamplers where necessary

15.1. Deinterlacing Algorithm Options

The Deinterlacer II IP core is highly configurable. When using the IP core, choose the deinterlacing algorithm first, based on your design goals.

When you have selected the appropriate algorithm, it should be easy for you to determine the other parameters.

Table 46. Deinterlacing Algorithm OptionsThe table below provides some guidelines for you to consider when choosing the appropriate deinterlacing algorithm. All configurations support 1, 2, or 4 pixels in parallel.
Deinterlacing Algorithm	Quality	DDR Usage	Area	Latency	Film or Cadenced Content	Symbols in Sequence	Native 4K HDR/SDR Passthrough
Vertical Interpolation ("Bob")	Low	None	Low	1 line	Not supported	Supported	Supported
Field Weaving ("Weave")	Low	Low	Low	1 field	Not supported	Supported	Supported
Motion Adaptive	Medium	Medium	Low	1 line	3:2 and 2:2 detect and correct configurable	Not supported	Not supported
Motion Adaptive High Quality	High	High	High	1 field and 2 lines	3:2 with video over film and 2:2 detect and correct configurable	Not supported	Supported

DDR Usage:

Low DDR usage—1 video field is read or written to DDR per output frame generated
Medium DDR usage—approximately 4 fields of video is read or written to DDR per output frame generated
High DDR usage—approximately 5 fields of video is read or written to DDR per output frame generated

Area:

Low area—approximately 1–2K ALMs, ≤25 M10Ks, no DSP usage
High area—approximately 15K ALMs, 44 DSPs

Quality:

Low—some blockiness, flickering, or weave artifacts may be seen, depending on the content
Medium—most content perceived as artifact-free, but some high frequency artifacts will be visible
High—some tuning and software control may be required using the register set available, and then all content should display well, with minimal artifacts

Note: All deinterlacer configurations assume a new frame is starting if the height of the current field is different from the previous field. This means that if NTSC deinterlacing support is required, you must use a clipper to clip incoming fields of 244 lines of F0 and 243 lines of F1 input video, so no height difference is detected.

15.2. Deinterlacing Algorithms

The Deinterlacer II IP core provides four deinterlacing algorithms.

Vertical Interpolation ("Bob")
Field weaving ("Weave")
Motion Adaptive
Motion Adaptive High Quality (Sobel edge interpolation)

15.2.1. Vertical Interpolation (Bob)

The bob algorithm produces output frames by filling in the missing lines from the current field with the linear interpolation of the lines above and below them.

All color spaces and bits per pixel per color plane are supported.

At the top of an F1 field or the bottom of an F0 field there is only one line available so it is just duplicated. The function only uses the current field, therefore if the output frame rate is the same as the input frame rate, the function discards half of the input fields.

You can set the output frame rate (through the Vertical Interpolation (Bob) deinterlacing behavior parameter) to one of these options:

Produce one frame for every field—interpolations are applied for each incoming field to create a new frame
Produce one frame for every F0 field or Produce one frame for every F1 field—half the input field rate by producing fields on F0 or F1 according to the selection mode

15.2.2. Field Weaving (Weave)

Weave deinterlacing creates an output frame by filling all of the missing lines in the current field with lines from the previous field.

All color spaces and bits per pixel per color plane are supported.

This option gives good results for still parts of an image but unpleasant artifacts in moving parts. The weave algorithm requires external memory. This makes it significantly more expensive in external RAM bandwidth than the bob algorithms, if external buffering is not otherwise required. However, this option does not require any line buffering, making it the smallest deinterlacer configuration in terms of ALMs used.

Note: Progressive segmented video, where each video frame splits into two fields, may not perfectly deinterlace with the weave deinterlacer, because it is necessary to detect which field pairs belong together. To enable the detection of the pairs, select 2:2 detector for the Cadence detect and correction parameter in motion adaptive configurations of the Deinterlacer II IP core.

15.2.3. Motion Adaptive

Motion Adaptive algorithm avoids the weaknesses of bob and weave algorithms by using bob deinterlacing for moving areas of the image and weave deinterlacing for still area.

All color spaces and bits per pixel per color plane are supported, although a YCbCr color space is used internally for high memory bandwidth configurations with video over film cadence detection.

If the motion computed from the current and the previous pixels is higher than the stored motion value, the stored motion value is irrelevant. The function uses the computed motion in the blending algorithm, which then becomes the next stored motion value. However, if the computed motion value is lower than the stored motion value, the following actions occur:

The blending algorithm uses the stored motion value.
The next stored motion value is an average of the computed motion and of the stored motion.

This computed motion means that the motion that the blending algorithm uses climbs up immediately, but takes about four or five frames to stabilize. The motion-adaptive algorithm fills in the rows that are missing in the current field by calculating a function of other pixels in the current field and the three preceding fields as shown in the following sequence:

Pixels are collected from the current field and the three preceding it (the X denotes the location of the desired output pixel).
Figure 54. Pixel Collection for the Motion-Adaptive Algorithm
These pixels are assembled into two 3×3 groups of pixels. Figure 15–3shows the minimum absolute difference of the two groups.
Figure 55. Pixel Assembly for the Motion-Adaptive Algorithm
The minimum absolute difference value is normalized into the same range as the input pixel data. The function compares the motion value with a recorded motion value for the same location in the previous frame. If it is greater, the function keeps the new value; if the new value is less than the stored value, the function uses the motion value that is the mean of the two values. This action reduces unpleasant flickering artifacts.
The function uses a weighted mean of the interpolation pixels to calculate the output pixel and the equivalent to the output pixel in the previous field with the following equation:
$Output Pixel = M \times \frac{Upper Pixel + Lower Pixel}{2} + (1 - M) \times Still Pixel$

15.2.4. Motion Adaptive High Quality (Sobel Edge Interpolation)

Motion Adaptive High Quality (Sobel edge interpolation) is the highest quality algorithm, applying a merged bob and weave based upon the amount of motion detected, and in areas of high motion applying a Sobel-based edge detection algorithm to interpolate between two pixels.

Figure 56. Sobel Edge DetectionThe figure shows the kernel of pixels from which an interpolation decision is made.

For the pixel being generated, P, in a missing line, N, from the current frame being generated, a kernel of 20 pixels is examined from lines N–3, N–1, N+1 and N+3.
These 20 pixels are used to generate 7 smaller kernels over which Sobel transforms are performed (two of these are highlighted in yellow and red in the figure above).
The Sobel transforms produce 7 motion vectors (as indicated by the arrows in the figure above), each comprised of a direction and magnitude.
The deinterlacer uses this information to make the best possible interpolation over a wide kernel of 34 pixels taken from lines N-1 and lines N+1.
Figure 57. Sobel-based Edge Interpolation

15.3. Run-time Control

Enable run-time control if you require access to the register map.

If you do not select run-time control interface, the Deinterlacer II IP core starts deinterlacing as soon as it receives input video.

15.4. Pass-Through Mode for Progressive Frames

All configurations of the Deinterlacer II IP core support progressive passthrough of up to 1080p resolution. Some configurations support 4K progressive passthrough,

When progressive video passes through, there is no loss of precision or color space conversion internally.

15.5. Cadence Detection (Motion Adaptive Deinterlacing Only)

Motion-adaptive configurations of the Deinterlacer II IP core provide the option to detect both 3:2 and 2:2 cadences in the input video sequence, and perform a reverse telecine operation for perfect restoration of the original progressive video.

The video over film feature allows non-cadenced sections of video to be deinterlaced normally, regardless of the cadence. The video over film feature also enables enhanced scene change detection and a comprehensive register map for debugging and tuning the deinterlacer performance.

Note: Intel recommends you enable this feature for broadcast quality deinterlacing applications.

Figure 58. 2:2 Cadence (Progressive Segmented) ContentThe figure below shows an example of four frames from a film; each frame is split into odd and even fields.

Figure 59. 3:2 Cadence

The Deinterlacer II handles such video sequence by detecting the cadence and reconstructing (reverse pulldown) the original film. This is achieved by comparing each field with the preceding field of the same type (3:2 detection) or detecting possible comb artifacts that occur when weaving two consecutive fields (2:2 detection).

Figure 60. 3:2 Detection and 2:2 Detection ComparisonThe figure below shows the comparison between 3:2 and 2:2 detection.

The 3:2 cadence detector tries to detect matches separated by four mismatches. When the 3:2 cadence detector sees this pattern a configurable number of times, it locks. The 3:2 cadence detector unlocks after configurable number of successive mismatches. Locking and unlocking thresholds for the 2:2 cadence detector are similarly configurable.

Refer to the Deinterlacing Control Registers section for more information.

Figure 61. Weave Current and Weave PastWhen the cadence detect component enters a lock state, the deinterlacer continuously assembles a coherent frame from the incoming fields, by either weaving the current incoming field with the previous one (weave current) or by weaving the two past fields together (weave past).

If the incoming video contains any cadenced video, enable the Cadence detection and reverse pulldown option. Then, select the cadence detection algorithm according to the type of content you are expecting. If the incoming video contains both 3:2 and 2:2 cadences, select 3:2 & 2:2 detector.

The cadence detection algorithms are also designed to be robust to false-lock scenarios—for example, features on adjacent fields may trick other detection schemes into detecting a cadence where there is none.

The Deinterlacer II IP core also provides 3:2 & 2:2 detector with video over film option. Select this option to deinterlace correctly the subtitles, credits, or other closed-caption contents that were added over the top of movies or other cadenced contents. Because this feature introduces a field of latency to allow weave operations to be performed either forwards or backwards, also set the Fields buffered prior to output to 1.

15.6. Avalon-MM Interface to Memory

Motion adaptive or weave deinterlacing algorithms require external memory storage, which may be configured as required.

The Deinterlacer II parameter editor calculates the top of the address space based on the configuration chosen.

15.7. Motion Adaptive Mode Bandwidth Requirements

The bandwidth usage for motion adaptive mode is 100% efficient.

For the example of 10bit 4:2:2 YCbCr video 1080i video, the requirements may be calculated as below.

Image data:
For every pair of output lines produced there are two phases:

Phase 1: Read 2 lines = 1920 × 10 bits x 2 (YCbCr) × 2 = 76,800 bits per input line

Phase 2: Write 1 line, Read 1 line, = 1920 × 10 bits × 2 × 2 = 76,800 bits per input line

Phase 1 + phase 2 accesses = 153,600 bits of image data per input line

153600 × 540 lines = 82,944,000 bits per output frame

82944000 × 60 frames per second = 4,976,640,000 = 4.977 GBps of image data read/written
Motion data:
Motion data is always 8 bits per pixel regardless of color space.

Read & Write motion = 1920 × 8 bits × 2 (one read and one write) = 30,720 bits per input line

30,720 × 540 lines = 16,588,800 bits per output frame

16,588,800 × 60 frames per second = 995,328,000 = 0.995 GBps of motion data written/read
Video-over-film data:
Motion data is always 24 bits per pixel regardless of color space.

1920 × 24 bits × 2 (one read and one write per pixel) = 92,160 bits per input line

92,160 × 540 lines = 49,766,400 bits per output frame

49,766,400 × 60 frames per second = 2,985,984,000 = 2.985 GBps of video over film data written/read

Total bandwidth (without video over film cadence detection) = 4.977 + 0.995 = 5.972 GBps

Total bandwidth (with video over film cadence detection) = 5.972 + 2.985 = 8.957 GBps

CAUTION:

Some memory configurations for motion adaptive configurations of the Deinterlacer II (excluding video over film configurations) can result in motion information being incorrectly overwritten by information from subsequent lines, resulting in a reduction in QOR manifesting in weave artifacts. If you observe weave artifacts in motion adaptive configurations, file an Intel Premier Support case on the Intel Support page.

15.8. Avalon-ST Video Support

You can configure the Deinterlacer II to accept interlaced content to a maximum width of 2048 and maximum height of 1080 (1080i).

Note: The Deinterlacer II IP core supports a maximum field width of 2048 pixels to allow the deinterlacing of progressive segmented frames of this width, as specified in SMPTE ST 2048-2:2011 Annex A.

Progressive content of all resolutions, including 4K content, will be passed through unchanged.

The Deinterlacer II always passes through user packets. The Deinterlacer II IP core contains an embedded Avalon-ST Stream Cleaner IP core for motion adaptive configurations, which you may disable. It is included to ensure that there is no possibility of malformed input packets locking up the deinterlacer.

You may disable the embedded stream cleaner but this is only recommended if a stream cleaner already exists in the system upstream of the deinterlacer or if you select one of the simpler algorithms (Vertical interpolation (“Bob”) or Field weaving (“Weave”), which have complete resilience to all possible malformed inputs.

15.9. 4K Video Passthrough Support

The Deinterlacer II IP core offers different approaches for 4K video passthrough based on your deinterlacer configuration..

To determine the best approach for your deinterlacer configuration, refer to the table below.

Table 47. Appropriate Approaches for 4K (SDR and HDR) PassthroughFor Bob and Weave configurations, 4K HDR and SDR passthrough is natively supported. However, if you observe that the required f_MAX (e.g. 300 MHz for 2 pixels in parallel) cannot be reached, then use approach B to switch the deinterlacer to 4 pixels in parallel and half the f_MAX.
Deinterlacing Algorithm	Cadence Detection Algorithm	Approach
Bob	Not applicable	None — 4K passthrough supported natively
Weave	Not applicable	None — 4K passthrough supported natively
Motion adaptive	Any settings	Approach B — 4K passthrough not supported natively; use two Switch II IP instances
Motion adaptive high quality	3:2 and 2:2 detector with video over film	Approach A — 4K passthrough supported natively but conversion for f_MAX required
Motion adaptive high quality	Other settings	Approach B — 4K passthrough not supported natively; use two Switch II IP instances

15.9.1. Approach A

Approach A is for motion adaptive with 3:2 and 2:2 detector with video over film configurations where 4K HDR and SDR passthrough is natively supported but require f_MAX conversion.

Figure 62. 4K Video Passthrough (10-bit 4:4:4 YCbCr)The figure below shows the recommended approach to convert the data coming in and out of the deinterlacer.

The following sequences describe the conversion:

The Color Plane Sequencer II IP core converts between 2 and 4 pixels in parallel.
Dual-clock FIFOs (Avalon-ST Dual Clock FIFO IP core) transfer data with a 50% duty cycle on the Avalon-ST valid signal in a 300MHz clock domain to 100% duty cycle in a 150MHz clock domain.
The deinterlacer accepts pixels in parallel data and converts any interlaced content to 1 pixel in parallel for deinterlacing.
Progressive content (and user packets) is maintained at the configured number of pixels in parallel and is unaffected by passing through the deinterlacer.

15.9.2. Approach B

Approach B is for motion adaptive configurations where 4K HDR and SDR passthrough is not natively supported.

For these configurations, you can use a pair of switches together with appropriate software control to ensure that any 4K content bypasses the deinterlacer.

Figure 63. Using Pair of Switches with Appropriate Software ControlThe Color Plane Sequencer II, Dual-clock FIFO, and Color Space Converter II instances are necessary to ensure the deinterlacer reaches the f_MAX target.

The software control should operate in the following sequence:

Upon start up, or on Clocked Video Input loss of lock, the software places the mixer and the first switch into consume mode.
Figure 64. Mixer and First Switch in Consume Mode

In this mode, the frame buffer continues to operate, absorbing any residual frames from the deinterlacer subsystem, and repeating the last valid output frame consumed by the mixer. The mixer produces its background layer. The first switch consumes any frames or fields, or parts of frames or fields from the Clocked Video Input II instance. Therefore, the deinterlacer does not observe any new frames or fields.
The software continues to monitor the status of the Clocked Video Input II instance. If the Clocked Video Input instance continues to detect and produce frames with consistent resolution, the software removes the IP cores from consume mode. The software then sets the two switches according to the resolution reported by the Clocked Video Input instance or for 4K progressive video.
Figure 65. Without Bypassing the Deinterlacer

Figure 66. Bypassing the Deinterlacer for 4K Progressive Video

All Deinterlacer II variants support 1080p passthrough, so progressive resolutions of 1080p or less can either bypass the deinterlacer or not.

Approach B depends on the software control making the necessary switch CSR writes before any video packets reach the switch. The Clocked Video Input instance updates the resolution status registers at the end of the previous frame, so the software loop has a good portion of the vertical blanking interval to configure the video pipe. If little or no blanking is present, the software could insert a 1-line FIFO at the start of the video pipe to allow for the necessary time required. Intel recommends that you place an Avalon-ST Video Stream Cleaner instance after the Clocked Video Input instance.

You can use a software control code similar to the example below for Approach B. In this example, a down-scaler follows the second switch; in lines 66-67 and 91-92 the scaler's status bit detects when all video has been flushed from the pipe. If a scaler is not present, the software may use the status bit for whichever component that follows the second switch for this purpose.

In this code, the following const declarations are assumed:

VIP_HDMI_CVI_BASE - Address of the Clocked Video Input II
VIP_CLEANER_BASE - Address of the Avalon-ST Video Stream Cleaner
VIP_SWI_0_BASE - Address of the first (upstream) Switch II
VIP_DIL_BASE - Address of the Deinterlacer II
VIP_SWI_1_BASE - Address of the downstream Switch II
VIP_SCALE_DOWN_BASE - Address of the down Scaler II
VIP_VFB_BASE - Address of the triple-buffering Frame Buffer II
VIP_MIXER_BASE - Address of the Mixer II

Software Control Code

void start_cvi (unsigned int base) {
  unsigned int CVI_STAT=0;
  
  IOWR(base, 0, 1); // Starts the IP Core
  CVI_STAT=IORD(base, 1); // Read current status
  if ((CVI_STAT & 0x00000200)!=0) { // check the overflow
    IOWR(base, 1 , 0x00000200); // Reset the Overflow
  }
}

// read the current status of the video input
unsigned int current_cvi_status (unsigned int base) {
  unsigned int current_status;
  current_status = IORD(base, 1);
  return current_status;

  struct image_config {
   int x;
   int y;
   bool interlace;
   color_info image_color;
}

//-- Read the current image_config of the video input
image_config current_cvi_image_config (unsigned int cvi_base_addr) {

  image_config  current_image_config;
  unsigned int  current_status;
  unsigned int  interlaced_input;
  unsigned int  height_f1;
  int           color_pattern_reg;

  current_image_config.y = IORD(cvi_base_addr, 5); //-- Using f0 height only
  current_image_config.x = IORD(cvi_base_addr, 4);

  current_status   = current_cvi_status(cvi_base_addr);
  interlaced_input = ((current_status & 0x80)>>7);

  if (interlaced_input) {
    height_f1              = IORD(cvi_base_addr, 6);
    current_image_config.y = current_image_config.y + height_f1; // height is f0 + f1
  }

  current_image_config.interlace = interlaced_input;

  color_pattern_reg = IORD(cvi_base_addr, 14);
  current_image_config.image_color.space =  color_pattern_reg & 0x00FF;
  current_image_config.image_color.depth = (color_pattern_reg & 0xFF00)>>8;

  if ( current_image_config.image_color.space==Y420_COLOR_SPACE ) {
     current_image_config.x = current_image_config.x << 1; // width is halved for 4:2:0
  }

  return current_image_config;
}

void start_vip_core (unsigned int base) {
  IOWR(base,  0, 1);
}

void configure_dil_out_of_pipe() {
  unsigned int status;
  status = IORD(VIP_SWI_0_BASE, 1);
  status = IORD(VIP_SWI_1_BASE, 1);
  status = IORD(VIP_DIL_BASE, 1);

  IOWR(VIP_SWI_1_BASE,  0, 0); //Stop SWI0
  IOWR(VIP_SWI_0_BASE,  0, 0); //Stop SWI1

  //Wait until STATUS reflects STOPPED in the scaler:
  status = 1;
  while (status == 1) {
     status = IORD(VIP_SCALE_DOWN_BASE, 1);
  }

  IOWR(VIP_SWI_0_BASE,  4, 1); //SWI0 Output 0 control - ON
  IOWR(VIP_SWI_0_BASE,  5, 0); //SWI0 Output 1 control - OFF
  IOWR(VIP_SWI_1_BASE,  4, 1); //SWI1 Output 0 control - outputs from input 0 (passthru) 
  IOWR(VIP_SWI_1_BASE,  0, 1); //Start SWI1
  IOWR(VIP_SWI_0_BASE,  0, 1); //Start SWI0 
  IOWR(VIP_DIL_BASE,    0, 0); //Stop dil   

}

void configure_dil_into_pipe() {
  unsigned int status;  

  IOWR(VIP_SWI_1_BASE,  0, 0); //Stop SWI0
  IOWR(VIP_SWI_0_BASE,  0, 0); //Stop SWI1

  //Wait until STATUS reflects STOPPED:
  status = 1;
  while (status == 1) {
     status = IORD(VIP_SCALE_DOWN_BASE, 1);
  }

  IOWR(VIP_SWI_0_BASE,  4, 0); //SWI0 Output 0 control - OFF
  IOWR(VIP_SWI_0_BASE,  5, 1); //SWI0 Output 1 control - ON
  IOWR(VIP_SWI_1_BASE,  4, 2); //SWI1 Output 0 control - outputs from input 1 (dil) 
  IOWR(VIP_SWI_1_BASE,  0, 1); //Start SWI1
  IOWR(VIP_DIL_BASE,    0, 1); //Start dil      
  IOWR(VIP_SWI_0_BASE,  0, 1); //Start SWI0

}

int main() {
  
  const int NUM_VFB_FRAMES_TO_ABSORB = 10;  //Very safe,could be less
  int       vip_pipe_active          = 0;

  image_config prev_cvi_image_config;
  image_config cvi_image_config;
  image_config output_image_config;
  image_config requested_output_image_config;

  start_vip_core(VIP_HDMI_CVI_BASE);
  start_vip_core(VIP_CLEANER_BASE);  
  start_vip_core(VIP_DIL_BASE);
  start_vip_core(VIP_VFB_BASE);
  start_vip_core(VIP_SCALE_DOWN_BASE);

  configure_dil_out_of_pipe(); 

  int vfb_frame_counter_prev = 0;
  int vfb_frame_counter      = 0;
  int vfb_frame_counter_orig = 0;
  int dil_configured         = 0;
  int cleaned_frames_passed  = 0;

  while(1) {

  // Always keep a frame count:
  vfb_frame_counter_prev = vfb_frame_counter;
  vfb_frame_counter = IORD(VIP_VFB_BASE, 3);
    
  prev_cvi_image_config = cvi_image_config;
  cvi_status       = current_cvi_status(VIP_HDMI_CVI_BASE);
  cvi_image_config = current_cvi_image_config(VIP_HDMI_CVI_BASE);

  if (   ((cvi_status & 0x00000800)==0)
   || (cvi_image_config.x != prev_cvi_image_config.x)
   || (cvi_image_config.y != prev_cvi_image_config.y)
   || (cvi_image_config.x >  UHD_DIM.x)
   || (cvi_image_config.y >  UHD_DIM.y)
   || (cvi_image_config.image_color.space != prev_cvi_image_config.image_color.space)
   ) {
    // The CVI is unlocked or the incoming resolution has changed or resolution 
    // is invalid.  Put mixer's VIP pipe input in to consume so that output keeps
    // running and display is clean.
    if (vip_pipe_active) {
      IOWR(VIP_MIXER_BASE, 10,   2); // input 0 consumed

    // Need to put the first SWI into consume mode too, because we cannot allow any 
    // resolutions >1080p into the dil:
      IOWR(VIP_SWI_0_BASE, 16,   1);        
        
    vfb_frame_counter_orig = IORD(VIP_VFB_BASE, 3);
      }

    vip_pipe_active       = 0;
    cleaned_frames_passed = 0;
    dil_configured        = 0;

    } else if (((cvi_status & 0x00000C00) >> 10) == 3) {// CVI locked with valid resolution 

    if (vip_pipe_active == 0) {

    // Before pipe is started, configure switches:
    if (!dil_configured) {

    cvi_image_config = current_cvi_image_config(VIP_HDMI_CVI_BASE);
    if (cvi_image_config.interlace) {
        configure_dil_into_pipe();
        } else {
        configure_dil_out_of_pipe();
        }

     // It is safe to take the switch out of consume mode now:
        IOWR(VIP_SWI_0_BASE, 16,   0);        

        dil_configured = 1;
        }

    input_is_4k = (cvi_image_config.x==3840)?1:0;
    vip_pipe_active = 1;
          
      }

    if (cleaned_frames_passed == 0) {
    if (vfb_frame_counter != vfb_frame_counter_prev) {
    if ((vfb_frame_counter - vfb_frame_counter_orig) >= NUM_VFB_FRAMES_TO_ABSORB) {
    // Mixer path re-enabled after NUM_VFB_FRAMES_TO_ABSORB frames, as cleaned 
    // frames should now be flushed through
          IOWR(VIP_MIXER_BASE, 10,   1);                  
          cleaned_frames_passed = 1;
          } 
        }
      }

    }

    prev_cvi_image_config = cvi_image_config;

  } //-- while(1)

  return 0;
}

15.10. Behavior When Unexpected Fields are Received

The behavior of the deinterlacer assumes an uninterrupted sequence of pairs of interlaced fields (F0, F1, F0, …) each having the same height.

You can guarantee this behavior through the use of the integrated Avalon-ST Video Stream Cleaner or an external stream cleaner. Some video streams may not follow this rule, and the behavior of the deinterlacer is undefined in such cases.

Note: When the bob algorithm is used, and synchronization is done on a specific field (input frame rate = output frame rate), the IP core always discards the field that is constantly unused. The IP core uses the other field to build a progressive frame.

15.11. Handling of Avalon-ST Video Control Packets

In all parameterizations, the Deinterlacer II IP core generates a new and updated control packet just before the processed image data packet.

This packet contains the correct frame height and the proper interlace flag so that the following image data packet is interpreted correctly by the following IP cores.

Note: The Deinterlacer II IP core uses 0010 and 0011 to encode interlacing values into the generated Avalon-ST Video packets. These flags mark the output as being progressive and record information about the deinterlacing process, which can be used by other IP cores. For example, the Interlacer IP core uses this information to precisely extract the original interlaced fields from the deinterlaced frame. The interlacing is encoded as 0000 when the Deinterlacer II IP core passes a progressive frame through.

15.12. Deinterlacer II Parameter Settings

Table 48. Deinterlacer II Parameter Settings
Parameter	Value	Description
Maximum width of interlaced content	32–2048, Default = 1920	Specify the maximum frame width of any interlaced fields. The maximum frame width is the default width at start-up.
Maximum height of the generated progressive output	32–1080, Default = 1080	Specify the maximum progressive frame height in pixels. The maximum frame height is the default progressive height at start-up.
Disable embedded Avalon-ST Video stream cleaner	On or Off	Turn on this option only if your system can guarantee to always supply well-formed control and video packets of the correct length.
Number of pixels in parallel	1, 2, 4, or 8	Select the number of pixels to be transmitted every clock cycle.
Bits per color sample	4–20	Select the number of bits per pixel (per color plane).
Number of color planes	2 or 3	Select the number of color planes per pixel.
Color planes transmitted in parallel	On or Off	Select if the Avalon-ST symbols are being transmitted in parallel.
YCbCr	On or Off	Turn on if you are using YCbCr 4:2:2 data format.
4:2:2	On or Off	Turn on if you are using 4:2:2 data format. Note: 4:2:2 mode does not support odd frame widths and heights.
Deinterlacing algorithm	Vertical interpolation ("Bob") Field weaving ("Weave") Motion Adaptive Motion Adaptive High Quality (Sobel edge interpolation)	Select the deinterlacing algorithm you want to use.
Vertical interpolation ("Bob") deinterlacing behavior	Produce one frame every F0 field Produce one frame every F1 field Produce one frame every field	Determines the rate at which frames are produced and which incoming fields are used to produce them. Note: Only relevant if you set the deinterlacing algorithm to Vertical interpolation ("Bob").
Run-time control	On or Off	Turn on to enable run-time control of the deinterlacer. When you turn on this parameter, the `Go` bit gets deasserted by default. When you turn off this parameter, the `Go` is asserted by default. Note: Intel strongly recommends run-time control when in motion adaptive modes with 3:2 & 2:2 detector with video over film.
Cadence detection algorithm and reverse pulldown	3:2 detector 2:2 detector 3:2 & 2:2 detector 3:2 & 2:2 detector with video over film	Select the cadence detection algorithm you want to use.
Fields buffered prior to output	0 or 1, Default = 1	Either 0 or 1 field is buffered prior to output. You must select 1 field of buffering for video over film cadence detection modes. Other modes incur no fields of latency delay.
Cadence detection and reverse pulldown	On or Off	Turn on to enable automatic cadence detection and reverse pulldown. Note: Cadenced content originates from movies or TV shows. Enable Cadence detection and reverse pulldown only if this content type is processed, otherwise disable this feature to save resources.
Avalon-MM master(s) local ports width	16 32 64 128 256 512	Specify the width of the Avalon-MM ports used to access external memory. It is recommended to match this width to the Avalon-MM width of your EMIF controller.
Use separate clock for the Avalon-MM master interface(s)	On or Off	Turn on to add a separate clock signal for the Avalon-MM master interface(s) so that they can run at a different speed to the Avalon-ST processing. The separation decouples the memory speed from the speed of the data path. Intel expects most applications to use separate Avalon-MM and Avalon-ST clock rates, so make sure this parameter is turned on.
Base address of storage space in memory	0–0×7FFFFFFF, Default = 0×00000000	Select a hexadecimal address of the frame buffers in external memory.
Top of address space	0×00ca8000	For your information only. Top of the deinterlacer address space. Memory above this address is available for other components.
FIFO depth Write Master	8–512, Default = 64	Select the FIFO depth of the Avalon-MM write master interface.
Av-MM burst target Write Master	2–256, Default = 32	Select the burst target for the Avalon-MM write master interface.
FIFO depth EDI Read Master	8–512, Default = 64	Select the FIFO depth of the edge-dependent interpolation (EDI) Avalon-MM read master interface.
Av-MM burst target EDI Read Master	2–256, Default = 32	Select the burst target for EDI Avalon-MM read master interface.
FIFO depth MA Read Master	8–512, Default = 64	Select the FIFO depth of the motion-adaptive (MA) Avalon-MM read master interface.
Av-MM burst target MA Read Master	2–256, Default = 32	Select the burst target for MA Avalon-MM read master interface.
FIFO depth Motion Write Master	8–512, Default = 64	Select the FIFO depth of the motion Avalon-MM write master interface.
Av-MM burst target Motion Write Master	2–256, Default = 32	Select the burst target for the motion Avalon-MM write master interface.
FIFO depth Motion Read Master	8–512, Default = 64	Select the FIFO depth of the motion Avalon-MM read master interface.
Av-MM burst target Motion Read Master	2–256, Default = 32	Select the burst target for motion Avalon-MM read master interface.

15.13. Deinterlacing Control Registers

Deinterlacer II Control Register Maps

The tables below describe the Deinterlacer II IP core control register map for run-time control. The Deinterlacer II reads the control data once at the start of each frame and buffers the data inside the IP core. The registers may safely update during the processing of a frame. Use these registers in software to obtain the best deinterlacing quality.

Table 49. Deinterlacer II Control Register Map for Vertical Interpolation (Bob) and Field Weaving (Weave) Deinterlacing Algorithms
Address	Register	RO/RW	Description
0	`Control`	RW	Bit 0 of this register is the `Go` bit, all other bits are unused. Setting this bit to 0 causes the Deinterlacer II IP core to stop the next time that control information is read. When you enable run-time control, the `Go` bit gets deasserted by default. If you do not enable run-time control, the `Go` is asserted by default. Power on value: 0
1	`Status`	RO	Bit 0 of this register is the `Status` bit, all other bits are unused. The Deinterlacer II IP core sets this address to 0 between frames when the `Go` bit is set to 0. The Deinterlacer II IP core sets this address to 1 while the core is processing data and cannot be stopped. Power on value: 0
2	Reserved	–	This register is reserved for future use.

For motion-adaptive configurations, Intel recommends that you initially retain all values at their reset value, with the exception of the following:

The Motion Shift and Motion Scale registers may require adjustments to correct for weave artifacts caused by insufficient motion sensitivity. Refer to Tuning Motion Shift and Motion Scale Registers for more information.
The cadence-related registers in Set A (primarily registers 8 and 9) may require adjustments to correct undetected cadences.

The tables below detail the run-time control registers for the motion adaptive configurations of the Deinterlacer II IP core. Configurations with video-over-film mode enabled use Set B registers, all other configurations use Set A registers.

Note: For Set A registers, if you have not configured the Cadence detection and reverse pulldown parameter, the cadence-related registers have no effect.

Table 50. Deinterlacer II Control Register Map for Motion-Adaptive Parameterizations Set A
Address	Register	RO/RW/WO	Width	Description
0	`Go`	RW	32	Setting this bit to 0 causes the Deinterlacer II IP core to stop the next time that control information is read. When you enable run-time control, the `Go` bit gets deasserted by default. If you do not enable run-time control, the `Go` is asserted by default. Power on value: 0
1	`Status`	RO	32	The Deinterlacer II IP core sets this address to 0 between frames when the `Go` bit is set to 0. The Deinterlacer II IP core sets this address to 1 while the core is processing data and cannot be stopped. Power on value: 0
2	Reserved	–	–	This register is reserved for future use.
3	Unused	–	–	Unused
4	`3:2 Cadence Diff Count`	WO	8	If the IP core could not detect 3:2 cadences, poll this register to facilitate the tuning of 3:2 detection. For example: while(1){ pollCount++; if (pollCount%350 == 0) { diffCount = IORD(VIP_DIL_BASE, 4); printf ("Dil diffCount = %0d\n", diffCount);} The register returns the number of lines in the current field for which a difference is detected compared to the preceding field. A successfully detected 3:2 cadence generates output such as the following: Dil diffCount = 253 Dil diffCount = 253 Dil diffCount = 0 Dil diffCount = 248 Dil diffCount = 249 Dil diffCount = 244 Dil diffCount = 245 Dil diffCount = 0 Dil diffCount = 240 Dil diffCount = 241 Dil diffCount = 237 Dil diffCount = 239 Dil diffCount = 0 Dil diffCount = 236 Dil diffCount = 235 Dil diffCount = 233 You are required to experiment with the polling rate to poll at a rate of approximately once per field.
5	`3:2 Cadence Lock Threshold`	WO	8	The higher the threshold value, the more stringent the requirements for the deinterlacer to start performing reverse telecine deinterlacing. You may set lower threshold values for greater sensitivity to cadenced sequences. The default value is 6.
6	`3:2 Cadence Unlock Threshold`	WO	8	Set this register lower than the `3:2 Cadence Lock Threshold` register. The closer the value is to the `3:2 Cadence Lock Threshold` register, the less stringent the requirements for the deinterlacer to lose lock and stop performing reverse telecine deinterlacing. The default value is 4.
7	`3:2 Cadence Diff Threshold`	WO	8	The `3:2 Cadence Diff Count` register increments when the “score” from the cadence detector for a line is equal or more than the `3:2 Cadence Diff Threshold` register. The default value is 16.
8	`3:2 Cadence Diff Line Ratio`	WO	8	This register affects the proportion of lines within a field that are required to exhibit a cadence before a 3:2 cadence lock can be achieved. Shift the number of lines using this value: `Lines >> 3:2 Cadence Diff Threshold` The default value is 4.
9	`3:2 Cadence Diff Noise Suppresion`	WO	8	Use this register to shift the result of difference calculation logic which is used to determine whether a line is being repeated: `Diff result >> 3:2 Cadence Diff Noice Suppresion` The default value is 5 to allow resilience against noise in the LSBs masking a cadence relationship.
10	`2:2 Cadence Lock Threshold`	WO	4	The higher the threshold value, the more stringent the requirements for the deinterlacer to start performing reverse telecine deinterlacing.
11	`2:2 Cadence Unlock Threshold`	WO	4	Set this register lower than the `2:2 Cadence Lock Threshold` register. The closer the value is to the `2:2 Cadence Lock Threshold` register, the less stringent the requirements for the deinterlacer to lose lock and stop performing reverse telecine deinterlacing. The default value is 4.
12	`2:2 Cadence Comb Threshold`	WO	8	A "comb count" for a field increments when the "score" from the 2:2 cadence detector for a line is ≥ `2:2 Cadence Comb Threshold`. Its reset value is 16. Decrease this value to increase the deinterlacer's sensitivity to 22 cadences.
13	Unused	–	–	Unused
14	`Motion Shift`	WO	8	Specifies the amount of raw motion (SAD) data that is right-shifted. Shifting is used to reduce sensitivity to noise when calculating motion (SAD) data for both "bob" and "weave" decisions and cadence detection. Note: It is very important to set this register correctly for good deinterlacing performance. Tune this register in conjunction with the motion visualization feature. Higher values decrease sensitivity to noise when calculating motion, but may start to introduce weave artifacts if the value used is too high.
15	Unused	–	–	Unused
16	`Visualize Motion Values`	WO	3	Set bit 0 (`Visualize Motion Values`) to color pixels where motion is detected with pink for debugging purposes. The greater the luminance of the color pink, the more motion is detected. Set bit 1 (`Bob-Only Mode`) to temporarily force the deinterlacer to do a pure "bob" deinterlace regardless of motion. Set bit 2 (`Weave-Only Mode`) to temporarily force the deinterlacer to do a pure "weave" deinterlace regardless of motion.

Table 51. Deinterlacer II Control Register Map for Motion-Adaptive with Video Over Film Parameterizations Set B

Address

RO/RW/WO

Width

Description

Control

Bit 0 of this register is the Go bit, all other bits are unused.

Setting this bit to 0 causes the Deinterlacer II IP core to stop the next time that control information is read.

When you enable run-time control, the Go bit gets deasserted by default. If you do not enable run-time control, the Go is asserted by default.

Power on value: 0

Status

Bit 0 of this register is the Status bit, all other bits are unused.

The Deinterlacer II IP core sets this address to 0 between frames when the Go bit is set to 0.
The Deinterlacer II IP core sets this address to 1 while the core is processing data and cannot be stopped.

Power on value: 0

Reserved

–

This register is reserved for future use.

Cadence Detected

When polled, the least significant bit (LSB) to 1, indicates the Deinterlacer II IP core has detected a 3:3 or 2:2 cadence and is performing reverse telecine.
Bit 0 indicates otherwise.

Range: 0–1

Power on value: 0

Extra Status

Bits [1:0]: 0 = Interlaced, 1 = Weave_current, 2 = Weave_future, 3 = Unused
Bit 2: VOF Lock Delay Up. The IP core sets this bit when the VOF lock delay period is over.
Bit 3: Adequate VOF region is detected.
Bit 4: Cadence 22 mode. The IP cores sets this bit when in 2:2 mode.

3:2 Cadence Film Pixels locked

Number of pixels displaying film content in a given field.

Range: 0–(2³²–1)

Power on value: 0

Motion in field

Total motion detected in the current field, computed from the sum of absolute differences (SAD) in Luma to the previous field of the same type, plus the Luma SAD of the previous field, and the next field, divided by 16.

Range: 0–(2³²–1)

Power on value: 0

3:2 Cadence VOF Histogram Total Phase 1

Histogram of locked pixels, that is used for debugging purposes before the VOF lock. Indicates the number of pixels showing the presence of a potential cadence for this phase. If one phasing shows more pixels with a cadence present compared to other phasing by a factor 4 or more, all pixels in the field will be locked. Reverse telecine on per-pixel basis will commence VOF Lock Delay fields after the lock.

Range: 0–(2³²–1)

Power on value: 0

3:2 Cadence VOF Histogram Total Phase 2

3:2 Cadence VOF Histogram Total Phase 3

3:2 Cadence VOF Histogram Total Phase 4

3:2 Cadence VOF Histogram Total Phase 5

Cadence Detect and advanced tuning registers On

This register enables the cadence detection feature and (if configured) the video over film feature together with all the motion and cadence/VOF tuning registers.

Setting the LSB of this register to 1 enables cadence detection and tuning registers.
Setting the LSB of this register to 0 disables cadence detection and tuning registers.
Cadence detection is disabled on reset.

Range: 0–1

Power on value: 0

Video Threshold

The most important register to tune the video over film features. Set lower values for more emphasis on video and higher values for more emphasis on film. Set dynamically in software when the input content changes for best results.

Range: 0–255

Power on value: 255

Film Lock Threshold

Bits 2:0 - Lock threshold for 3:2 cadence detection
Bits 10:8 - Lock threshold for 2:2 cadence detection
Bits 23:16 - Comb threshold for 2:2 cadence detection

Other bits are unused.

Range:

Lock thresholds = 3–7
Comb threshold = 4–255

The higher the threshold values, the more stringent the requirements for the deinterlacer:

to mark a pixel as locked and
to start performing reverse telecine deinterlacing

You may set lower threshold values for greater sensitivity to cadenced sequences. Intel recommends that you leave all values at their reset value, unless a change to sensitivity is required.

Power on value: 0x0010_0707

Lock thresholds = 7
Comb threshold = 16

Film Unlock Threshold

Bits 2:0 - Unlock threshold for 3:2 cadence detection
Bits 10:8 - Unlock threshold for 2:2 cadence detection
Bits 23:16 - Delta threshold for 2:2 cadence detection

Other bits are unused.

Range:

Unlock thresholds = 0–5 (must be set to a value lower than the equivalent lock threshold)
Delta threshold = 4–255

The greater the difference between the lock and unlock threshold values, the more stringent the requirements for the deinterlacer:

to mark a pixel as unlocked and
to stop performing inverse telecine deinterlacing

You may set a small difference in the threshold values for greater sensitivity to changes in cadenced sequences. Intel recommends that you leave all values to their reset value, unless a change to sensitivity is required.

Power on value: 0x0005_0

Unlock threshold for 3:2 cadence detection = 2
Unlock threshold for 2:2 cadence detection = 4
Delta threshold = 5

VOF Lock Delay

Specifies the number of fields elapsed after the core detects a cadence, but before reverse telecine begins. The delay allows for any video to drop out. If you set a value less than five, the core locks to cadence quicker but costs potential film artifacts.

Range: 0–31

Power on value: 5

Minimum Pixels Locked

Specifies the least number of pixels showing a cadence for lock to occur. Increase the value of this register if inverse telecine is being erroneously applied to scenes where telecine should not be present.

Range: 0–(2³²–1)

Power on value: 40000

Note: Use a higher value for 1080i compared to PAL or NSTC video.

Minimum Valid SAD Value

When considering whether pixels should remain locked, the SAD values less than this range are ignored. Set this value high to prevent film pixels from decaying over time if they do not show a strong 3:2 cadence.

Range: 0–255

Power on value: 255

Scene Change Motion Multiplier

The Deinterlacer II IP core's scene change detection algorithm detects any scene changes or edits regardless of whether any current cadence continues or is interrupted. Scene changes cause immediate loss and reacquisition of cadence lock, which allows for very smooth deinterlacing of even rapid scene changes.

The algorithm detects scene changes based on a set of motion deltas between adjacent fields. The algorithm uses a multiplier in this calculation. This register sets the value of this multiplier, with a default value of 5 corresponding to a 4× motion delta between adjacent scenes. You may set other values as shown in Scene Change Motion Multiplier Value.

Range: 0–9

Power on value: 5

Minimum Film to Closed Caption Ratio

The Deinterlacer II IP core determines cadence for each pixel based on its immediate surroundings. For some standard definition content, film pixels may drop into video deinterlacing mode due to insufficient cadence signal. When the pixels go into video deinterlacing mode, you may set a minimum film to closed caption ratio.

The deinterlacer compares a count of pixels identified as film content in a reference area, with a count of those identified as film content in likely closed caption area. The deinterlacer only enters full video over film mode if the ratio of film content in the reference area to the closed caption area exceeds the threshold value.

This register sets the following threshold values:

Minimum Film to Closed Caption Register	Minimum Ratio to Switch into Video Over Film Mode
0	1 (no effect)
1	4
2	16
3	64
4	256
5	1024

Range: 0–5

Power on value: 0

Minimum Pixel Kernel SAD for Field Repeats

Once a video achieves cadence lock, every pixel in the frame will either maintain or lose lock independently from then on. If the SAD value is less than the value for this register, then its lock count will be incremented. If it is higher than this value, its lock count will either remain unchanged or be decremented (if less than min valid SAD value).

Range: 0–255

Power on value: 200

History Minimum Value

The cadence bias for a given pixel.

Setting a lower value biases the pixels toward film.
Setting a higher value biases the pixels toward video.

The pixel SAD values are scaled according to the recent history that gives the frames an affinity for their historical state.

Range: 0–3

Power on value: 0

History Maximum Value

The cadence bias for a given pixel.

Setting a lower value biases the pixels toward film.
Setting a higher value biases the pixels toward video.
The value for this register must be higher than the value for the History Minimum Value register.

Range: 3–7

Power on value: 7

SAD Mask

When detecting cadences, the SAD values are AND’ed with this value. This value allows the LSBs to be masked off to provide protection from noise.

For example, use binary 11_1111_0000 to ignore the lower 4 bits of the SAD data when detecting cadences. This register works orthogonally from the Motion Shift register (Offset 25), which affects both motion calculation in general AND cadence detection.

Range: 512–1023

Power on value: 1008 (binary 1111110000)

Motion Shift

Specifies the amount of raw motion (SAD) data that is right-shifted. Shifting is used to reduce sensitivity to noise when calculating motion (SAD) data for both bob and weave decisions and cadence detection.

Note: It is very important to set this register correctly for good deinterlacing performance.

Tune this register in conjunction with the motion visualization feature. Higher values decrease sensitivity to noise when calculating motion, but may start to introduce weave artifacts if the value used is too high.

To improve video-over-film mode quality, consider using software to check the 3:2 Cadence State (VOF State) register, and to add one or two to the motion shift register's value when deinterlacing cadenced content.

Range: 0–7

Power on value: 3

Refer to Tuning Motion Shift and Motion Scale Registers for more information.

Visualize Film Pixels

Specifies the film pixels in the current field to be colored green for debugging purposes. Use this register in conjunction with the various VOF tuning registers.

Range: 0–1

Power on value: 0

Visualize Motion Values

Specifies the motion values for pixels represented with pink for debugging purposes. The greater the luminance of pink, the more motion is detected.

Range: 0–1

Power on value: 0

Reserved

–

This register is reserved for future use.

Reserved

–

This register is reserved for future use.

Motion Scale

An 8-bit quantity that is used to scale the effect of the detected motion. Refer to Tuning Motion Shift and Motion Scale Registers for more information.

The register scales the motion according to the following equation: $S c a l e d M o t i o n = M o t i o n . \frac{M o t i o n S c a l e}{32}$

A value of 32 does not produce any scaling effect.
A value of 1 produces a scaling of 1/32.
A value of 255 produces a scaling of ×7.97

The lower the scaled motion value, the more weave the IP core performs. Therefore, if any weave artifacts are visible, increase this register value.

Power on value: 125 (corresponds to ×3.9)

15.13.1. Scene Change Motion Multiplier Value

Table 52. Scene Change Motion Multiplier Value
Scene Change Motion Multiplier Register	Motion in Field Multiplier
0	×1
1	×1.06
2	×1.14
3 (suggested setting for 480i or 576i)	×1.33
4	×2
5 (default and suggested setting for 1080i)	×4
6	×8
7	×16
8	×32
9	×64

15.13.2. Tuning Motion Shift and Motion Scale Registers

To tune the motion shift register, follow these steps:

Enable motion visualization; set Visualize Motion Values register to 1.
Enable cadence detection and tuning registers by setting register 12 to 1.
Feed the Deinterlacer II IP core with the sequence of interest, ideally one with static areas and areas in motion, such as a waving flag sequence. Areas in the image where motion is detected will appear in pink, with the luminance in proportion to the amount of motion detected.
Adjust the Motion Shift register through software when the Deinterlacer II IP core runs, to observe the effect on the motion detected. Choose a motion shift value that does not cause any motion to be detected in static areas of the image.
When you are satisfied that the correct amount of motion shift is applied, disable motion visualization by resetting the register back to 0.
Look for weave artifacts in moving portions of the image, ideally using a test sequence with fast moving sharp edges or narrow lines. If you do not detect any visible weave artifacts, gradually decrease the Motion Scale register value from the default 125 until the artifacts become visible.
Gradually increase the value of Motion Scale register until all the weave artifacts disappear.

16. Frame Buffer II IP Core

The Frame Buffer II IP core buffers video frames into external RAM.

The Frame Buffer II IP core offers the following features:

Buffers progressive and interlaced video fields.
Supports double and triple buffering with a range of options for frame dropping and repeating
- When frame dropping and frame repeating are not allowed—the IP core provides a double-buffering function that can help solve throughput issues in the data path.
- When frame dropping and/or frame repeating are allowed—the IP core provides a triple-buffering function that can be used to perform simple frame rate conversion.
Supports up to 8 pixels per transmission.
Supports a configurable inter-buffer offset to allow the best interleaving of DDR banks for maximum efficiency
Supports compile-time or run-time controlled variable buffer delay up to 4,095 frames
Supports reader-only or writer-only modes
Configurable user packet behavior

The Frame Buffer II IP core has two basic blocks:

Writer—stores input pixels in memory
Reader—retrieves video frames from the memory and produces them as outputs

Figure 67. Frame Buffer Block Diagram

16.1. Double Buffering

For double-buffering, the Frame Buffer II IP core uses two frame buffers in external RAM.

The writer uses one buffer to store input pixels.
The reader locks the second buffer that reads the output pixels from the memory.
When both writer and reader complete processing a frame, the buffers are exchanged.
The input frame can then be read back from the memory and sent to the output, while the buffer that has just been used to create the output can be overwritten with fresh input.

This feature is used when:

The frame rate is the same both at the input and at the output sides but the pixel rate is highly irregular at one or both sides.
A frame has to be received or sent in a short period of time compared with the overall frame rate. For example, after the Clipper IP core or before one of the foreground layers of the Alpha Blending Mixer IP core.

16.2. Triple Buffering

For triple-buffering, the IP core uses three frame buffers in external RAM.

The writer uses one buffer to store input pixels.
The reader locks the second buffer that reads the output pixels from the memory.
The third buffer is a spare buffer that allows the input and the output sides to swap buffers asynchronously. The spare buffer can be clean or dirty.
- Considered clean if it contains a fresh frame that has not been sent.
- Considered dirty if it contains an old frame that has already been sent by the reader component.

When the writer completes storing a frame in memory, it swaps its buffer with the spare buffer if the spare buffer is dirty.

The buffer locked by the writer becomes the new spare buffer and is clean because it contains a fresh frame.
If the spare buffer is already clean when the writer completes writing the current input frame:
- If dropping frames is allowed—the writer drops the newly received frame and overwrites its buffer with the next incoming frame.
- If dropping frames is not allowed—the writer stalls until the reader completes its frame and replaces the spare buffer with a dirty buffer.

When the reader completes reading and produces a frame from memory, it swaps its buffer with the spare buffer if the spare buffer is clean.

The buffer locked by the reader becomes the new spare buffer; and is dirty because it contains an old frame that has been sent previously.
If the spare buffer is already dirty when the reader completes the current output frame:
- If repeating frames is allowed—the reader immediately repeats the frame that has just been sent.
- If repeating frames is not allowed—the reader stalls until the writer completes its frame and replaces the spare buffer with a clean buffer.

16.3. Locked Frame Rate Conversion

The locked frame rate conversion allows the Frame Buffer II IP to synchronize the input and output frame rates through an Avalon memory-mapped slave interface.

The decision to drop and repeat frames for triple-buffering is based on the status of the spare buffer. Because the input and output sides are not tightly synchronized, the behavior of the Frame Buffer II IP is not completely deterministic and can be affected by the burstiness of the data in the video system. This may cause undesirable glitches or jerky motion in the video output, especially if the data path contains more than one triple buffer.

By controlling the dropping or repeating behavior, the IP keeps the input and output synchronized. To control the dropping or repeating behavior, you must select triple-buffering mode and turn on Support for locked frame rate conversion or Locked rate support parameters.

16.3.1. Converting Frame Rates

You can enable or disable locked frame rate conversion at runtime.

Turn on Locked Rate Support in the GUI, which adds hardware that supports locked frame rate conversion.
Set register 10 with the current input rate and register 11 with the current output rate.
For more detail, refer to Frame Buffer II Control Registers Map table.
For example:
- For a 60Hz frame rate, set a value of 60.
- To convert 60 Hz to 50 Hz, set register 10 to 60 and register 11 to 50.
- To convert from SDI 59.94 to 60, set register 10 to 5994 and register 11 to 6000.
Enable or disable locked frame rate conversion with the Locked Mode Enable register (register 9).

16.4. Handling of Avalon-ST Video Control Packets and User Packets

The Frame Buffer II IP core stores non-image data packets in memory.

Some applications may repeat and drop the user packets together with their associated frame. For example, if the packets contain frame-specific information such as a frame ID.

The behavior of the IP core is not determined by the field dimensions announced in the Avalon-ST Video control packets and relies exclusively on the startofpacket and endofpacket signals to delimit the frame boundaries.

The IP core consequently handles and propagates mislabeled frames. Use this feature in a system where you cannot drop a frame. The latency introduced during the buffering could provide enough time to correct the invalid control packet.
The buffering and propagation of image data packets incompatible with preceding control packets is an undesired behavior in most systems. Dropping invalid frames is often a convenient and acceptable way of dealing with glitches from the video input.

You can parameterize the Frame Buffer II IP core to drop all mislabeled fields or frames at compile time.

To drop and repeat user packets:

Set the user packet affinity bit (bit 1) of the Misc. register.
Turn on Drop invalid frames parameter.

Turn on the Frame repeating parameter to guarantee that reader keeps on repeating the last valid received frame— freezes the output—when the input drops.

16.5. Frame Buffer Parameter Settings

Table 53. Frame Buffer II Parameter Settings
Parameter	Value	Description
Maximum frame width	32–8192, Default = 1920	Specify the maximum frame width in pixels.
Maximum frame height	32–8192, Default = 1080	Specify the maximum progressive frame height in pixels.
Bits per color sample	4–20, Default = 20	Select the number of bits per pixel (per color plane).
Number of color planes	1–4, Default = 2	Select the number of color planes that are sent in sequence.
Color planes transmitted in parallel	On or Off	Turn on to transmit color planes in parallel. Turn off to transmit color planes in series.
Number of pixels in parallel	1, 2, 4, or 8	Specify the number of pixels transmitted or received in parallel.
Interlace support	On or Off	Turn on to support consistent dropping and repeating of fields in an interlaced video stream. Note: Do not turn on this parameter to double-buffer an interlaced input stream on a field-by-field basis.
Use separate clock for the Avalon-MM master interface(s)	On or Off	Turn on to add a separate clock signal for the Avalon-MM master interfaces so that they can run at a different speed to the Avalon-ST processing. This decouples the memory speed from the speed of the data path, and is sometimes necessary to reach performance target.
Avalon-MM master(s) local ports width	16-512, Default = 256	Specify the width of the Avalon-MM ports used to access external memory.
FIFO depth Write	16–1024, Default = 64	Select the FIFO depth of the write-only Avalon-MM interface.
Av-MM burst target Write	2–256, Default = 32	Select the burst target for the write-only Avalon-MM interface.
FIFO depth Read	16–1024, Default = 64	Select the FIFO depth of the read-only Avalon-MM interface.
Av-MM burst target Read	2–256, Default = 32	Select the burst target for the read-only Avalon-MM interface.
Align read/write bursts on read boundaries	On or Off	Turn on to avoid initiating read and write bursts at a position that would cause the crossing of a memory row boundary.
Maximum ancillary packets per frame	Any 32-bit value, Default = 0	Specify the number of non-image, non-control, Avalon-ST Video packets that can be buffered with each frame. Older packets are discarded first in case of an overflow. Note: The Maximum length ancillary packets in symbols parameter is disabled or unused when you specify the number of packets buffered per frame to 0. User packets are no longer delayed through the DDR memory (as with the Frame Buffer I IP core). The packets are instead grouped at the output immediately following the next control packet. Then the video packets swap places with the user packets which arrive before the next control packet.
Maximum length ancillary packets in symbols	10–1024,Default = 10	Select the maximum packet length as a number of symbols. The minimum value is 10 because this is the size of an Avalon-ST control packet (header included). Extra samples are discarded if the packets are larger than allowed.
Frame buffer memory base address	Any 32-bit value, Default = 0x00000000	Select a hexadecimal address of the frame buffers in external memory when buffering is used. The information message displays the number of frame buffers and the total memory required at the specified base address.
Enable use of inter-buffer offset	On or Off	Turn on if you require maximum DDR efficiency, at the cost of increased memory footprint per frame.
Inter-buffer offset	Any 32-bit value, Default = 0x01000000	Specify a value greater than the size of an individual frame buffer.
Module is Frame Reader only	On or Off	Turn on if you want to configure the frame buffer to be a frame reader.. Note: You must select run-time reader control if you select frame reader only.
Module is Frame Writer only	On or Off	Turn on if you want to configure the frame buffer to be a frame writer. Note: You must select run-time writer control if you select frame writer only.
Frame dropping	On or Off	Turn on to allow frame dropping.
Frame repeating	On or Off	Turn on to allow frame repetition.
Delay length (frames)	1–2047, Default = 1	When you turn on the Drop/repeat user packets parameters, the IP core implements a minimum of 3 buffers (triple buffer), which gives a delay through the buffer of 1 frame. You can configure the IP core the implement more frame buffers and create a longer delay, up to a maximum of 2047 frames. This feature enables the pausing of a video stream up to 2048 seconds (input frames per second), by applying the back-pressure to the Avalon-ST video output of the frame buffer for the duration of the pause.
Locked rate support	On or Off	Turn on to add an Avalon-MM slave interface that synchronizes the input and output frame rates. Note: You can only turn on this parameter if you also turn on Frame dropping, Frame repeating, and Run-time writer control parameters.
Drop invalid frames	On or Off	Turn on to drop image data packets that have lengths that are not compatible with the dimensions declared in the last control packet.
Drop/repeat user packets	On or Off	Turn on to drop or repeat user packets when associated frames are dropped or repeated.
Run-time writer control	On or Off	Run-time control for the write interface. The Frame Buffer II has two sides – a reader and a writer. Each side has a register interface, one of which can be configured to be visible to the user. Both control interfaces contain all the necessary registers to control the behavior of the IP core while for the writer, registers 3 and 4 (frame counter and drop/repeat counter) reflect information on dropped frames. Note: When you turn on this parameter, the `Go` bit gets deasserted by default. When you turn off this parameter, the `Go` is asserted by default. Refer to the Frame Buffer II Control Register Map.
Run-time reader control	On or Off	Run-time control for the read interface. The Frame Buffer II has two sides – a reader and a writer. Each side has a register interface, one of which can be configured to be visible to the user. Both control interfaces contain all the necessary registers to control the behavior of the IP core while for the reader, registers 3 and 4 (frame counter and drop/repeat counter) reflect information on repeated frames. Note: When you turn on this parameter, the `Go` bit gets deasserted by default. When you turn off this parameter, the `Go` is asserted by default. Refer to the Frame Buffer II Control Register Map.

16.6. Frame Buffer Application Examples

The example use cases provide some guidance for your designs.

Table 54. Example Use Cases for Various Locked and Frame Dropping/Repeating Configurations
Locked Rate Support	Frame Dropping	Frame Repeating	Application Example
Yes	Yes	Yes	A system with source-synchronous input and outputs (sharing the same clock, or genlocked), with an input frame rate of 60 Hz and an output frame rate of 24 Hz. The frame buffer implements a triple buffer, providing a regular drop/repeat pattern to ensure that the lower output rate is maintained with minimal perceived jitter in the output video. Register 10 (`Input Frame Rate`) should be set to 60 and register 11 (`Output Frame Rate`) to 24, or any other two short int values to represent the 60:24 ratio.
Yes	No	No	Illegal configuration. The frame buffer must be able to drop and repeat frames when input and output rates are locked.
No	Yes	Yes	A system with inputs and outputs which are not source-synchronous (no common clock), with an input frame rate of 60 Hz and an output frame rate of 24 Hz. The frame buffer implements a "triple buffer", providing a variable drop/repeat pattern to accommodate any phase drift seen due to the different clocks. This is the most common configuration used for video standard conversion applications.
No	No	No	A system with source-synchronous input and outputs (sharing the same clock, or genlocked), with an input frame rate of 50 Hz and an output frame rate of 50 Hz. This configuration may be useful where the input and output have different burst characteristics, for example a DisplayPort input and an SDI output. The frame buffer implements a "double buffer", providing very little backpressure to the input, while maintaining the required steady rate at the output.

16.7. Frame Buffer Control Registers

A run-time control can be attached to either the writer component or the reader component of the Frame Buffer II IP core but not to both. The width of each register is 32 bits.

Table 55. Frame Buffer II Control Register MapThe table below describes the register map for the Frame Buffer II IP core when configured as a Frame reader only (Reader column), Frame writer only (Writer column) or as a frame buffer (Buffer column). Y indicates the register is applicable for the feature and N/A means not applicable.
Note: Registers 3 and 4 return differently, depending on whether the register interface is a reader or writer control.
Address	Register	Reader	Writer	Buffer	Type	Description
0	`Control`	Y	Y	Y	RW	Bit 0 of this register is the `Go` bit, Setting this bit to 0 causes the IP core to stop the next time control information is read. When you enable run-time control, the `Go` bit gets deasserted by default. If you do not enable run-time control, the `Go` is asserted by default.
1	`Status`	Y	Y	Y	RO	Bit 0 of this register is the `Status` bit, all other bits are unused.
2	`Interrupt`	Y	Y	Y	RW	The frame writer raises its interrupt line and sets bit 0 of this register when the IP core writes a frame to DDR and the frame is ready to be read. You can clear the interrupt by writing a 1 to this bit. The frame reader raises its interrupt line and sets bit 0 of this register when a complete frame is read from DDR. You can clear the interrupt by writing a 1 to this bit.
3	`Frame Counter`	Y	Y	Y	RO	For a writer control interface, the counter is incremented if the frame is not dropped. For a reader control interface, this counter is incremented if the frame is not repeated.
4	`Drop/Repeat Counter`	Y	Y	Y	RO	For a writer control interface, the counter is incremented if the frame is dropped. For a reader control interface, this counter is incremented if the frame is repeated.
5	`Frame Information`	Y	Y	N/A	RW	Bit 31 of this register is the `Available` bit used only in the frame writer mode. A 0 indicates no frame is available and a 1 indicates the frame has been written and available to read. Note: In Frame Writer only mode, you must acknowledge each available frame before the next frame is available. Refer to the `acknowledge` bit in the `Misc` register. Bit 30 of this register is unused. Bits 29 to 26 contain the interlaced bits of the frame last written by the buffer. Bits 25 to 13 of this register contain the width of the frame last written by the buffer. Bits 12 to 0 of this register contain the height of the frame last written by the buffer.
6	`Frame Start Address`	Y	Y	N/A	RW	This register holds the frame start address for the frame last written to DDR by the writer. If configured as a Reader only, you must write the frame start address to this register. For the frame writer configuration, the frame start address is valid only when the `Available` bit in the `Frame Information` register is set.
7	`Frame Reader`	Y	N/A	N/A	RO	Bit 26 of this register is the `Ready` bit. This bit is set when the reader is ready to accept the details of the next frame to be read. Bits 25 to 13 of this register indicate the maximum width of frames that may be read, as configured in the parameter editor. Bits 12 to 0 of this register indicate the maximum height of frames that may be read, as configured in the parameter editor.
8	`Misc`	Y	Y	Y	RW	Bit 0 of this register is the `acknowledge` bit. Applies only to Frame Writer only mode. Set this bit to 1 to indicate that the available frame has been completely handled (refer to `Available` bit in the `Frame Information` register). Writing a 1 triggers the buffer to be reset and the Frame Writer reuses the buffer. Bit 1 of this register is the `user packet affinity` bit. Set this bit to 1 you want to drop and repeat user packets together with their associated video packet (this is the next video packet received). This mode allows for specific frame information that must be retained with each frame. Set this bit to 0 if all user packets are to be produced as outputs in order, regardless of any dropping or repeating of associated video packets. This mode allows for audio or closed caption information. Bits 15 to 2 of this register are unused. Bits 27 to 16 of this register contain the frame delay. The default delay value is 1, but you may introduce additional delay to the buffer by writing a value from 2 to 4095 to this register.
9	`Locked Mode Enable`	N/A	N/A	Y	RW	Bit 0 of this register is enables locked mode. When you set the locked mode bit, the specified `Input Frame Rate` and `Output Frame Rate` registers tightly control the dropping and repeating of frames. Setting this bit to 0 switches off the controlled rate conversion and returns the triple-buffering algorithm to a free regime where dropping and repeating is only determined by the status of the spare buffer. Other bits are unused.
10	`Input Frame Rate`	N/A	N/A	Y	RW	Bits 15:0 contains a short integer value that corresponds to the input frame rate. Other bits are unused.
11	`Output Frame Rate`	N/A	N/A	Y	RW	Bits 15:0 contains a short integer value that corresponds to the output frame rate. Other bits are unused.

16.7.1. Frame Writer Only Mode

To configure the Frame Buffer II IP core in frame writer mode, select Module is Frame Writer only mode in the parameter editor.

In this mode, the frame buffer starts writing incoming video frames to DDR at the Frame buffer memory base address register and automatically advances the write address with each incoming frame. The address of each newly written frame is made available through the Frame Start Address register when the write has completed. This is indicated by the available bit (31) of the Frame Start Address register. This register also holds the height, width, and interlaced information for the written frame. It is not possible to instruct the frame buffer where to write individual frames.

Frame details persist until cleared through a write to bit 0 of the Misc register. The write indicates to the Frame writer that the frame has been completely handled and the buffer may be reused. This also causes the Frame Buffer II to clear the available bit, unless another frame has been received in the meanwhile. In this case, the bit remains set and the new Frame Information becomes available.

The Frame Buffer II also raises its interrupt line and sets bit o of the Interrupt register when a new frame is available. The interrupt is cleared down by writing a 1 to the bit.

If additional frames are presented at the input when the frame buffer is already full and you have turned on the Frame dropping parameter, the incoming frames will be dropped. If you did not turn on the Frame dropping parameter, the Frame Buffer II stalls the input.

16.7.2. Frame Reader Only Mode

To configure the Frame Buffer II IP core in frame reader mode, select Module is Frame Reader only mode in the parameter editor.

In this mode, when you set the frame dimensions through the Frame Information register and the frame start address through the Frame Start Address register, the IP core starts transmitting video frames read from DDR.

Writing to the Frame Start Address register initiates the video reading. If new frame dimensions are set, you must perform another write to the Frame Start Address register for the new settings to take effect (even if the frame start address is unchanged).

The Frame Buffer II IP core cannot determine if a frame is dirty or clean; the IP core keeps producing a frame from wherever it is currently addressed until a new address is written. Therefore, frame reading applications may use one of the following based on the target application:

A conventional fixed set of 2 or 3 buffers
A dozen buffers dynamically allocated at runtime
A constant test pattern buffer and a set of dynamic buffers

A simple 3-buffer frame reader may operate as follows:

Wait until the ready bit of the Frame Reader register is high, indicating that it is ready to receive details of a new frame to transmit.
Note: The Frame Buffer II IP core allocates sufficient frame information buffering for the number of frames set through the Delay Length parameter.
Write the frame dimensions into the Frame Information register.
Note: The frame dimensions include the 4 interlaced bits (e.g. 0 for progressive, 8 for interlaced F0, and 12 for interlaced F1)
Write the data for the frame into a buffer area N . This area may be at any address range visible to the frame buffer, as long as the Frame Buffer II is not already transmitting from that region of memory.
Write the start address of the buffer to the Frame Start Address register (0x6).
The Frame Buffer II starts transmitting image data from the buffer.
Increment buffer number N, N = (N+1)%3, and repeat from step 1.

The frame reader issues an interrupt immediately after coming out of reset. You can use it to synchronize the system and trigger initiation of the first frame. To use it, you must set up your design to read a new frame after an interrupt indicates the previous one is done.

16.7.3. Memory Map for Frame Reader or Writer Configurations

When creating content for on-screen display using a frame reader, or when processing frame data written to DDR through a frame writer, it is necessary to understand the memory mapping used by the Frame Buffer II IP core.

The frame data is tightly packed into memory and aligned on frame (or field) boundaries to minimize storage usage and maximize memory bandwidth usage.

The figure below shows an example of memory map for a frame buffer with the configuration settings below:

Bits per pixel per color sample = 8 bits
Number of color planes = 2
Pixels in parallel = 1
Avalon-MM master(s) local ports width = 25
Av-MM burst target Write = 32
Av-MM burst target Read = 32
Align read/write bursts on read boundaries = On
Maximum ancillary packets per frame = 10
Frame buffer memory base address = 0x6800 0000
Enable use of inter-buffer offset = On
Inter-buffer offset = 0x0100 0000
Delay length (frames) = 1

The maximum length of ancillary packets is ignored if you turn on Align read/write bursts on read boundaries.

Figure 68. Example of Memory Map for Base Address 0x6800_0000

The ancillary (user) packets are located in memory after the frame storage when you enable Align read/write bursts on read boundaries. Each packet will be offset in memory by (Avalon-MM local ports width * burst target )/8. In this example configuration, the offset is 256*32 / 8 = 1024 (0x400)

Therefore, for the 3 buffers configured, any ancillary packets are written to memory at the following addresses:

Anc buffer 0, anc packet 0 = 0X6B00_0000

Anc buffer 0, anc packet 1 = 0X6B00_0000 + 1*0x400 = 0X6B00_0400

Anc buffer 0, anc packet 2 = 0X6B00_0000 + 2*0x400 = 0X6B00_0800

... Anc buffer 0, anc packet 9 = 0X6B00_0000 + 9*0x400 = 0X6B00_2400

Anc buffer 1, anc packet 0 = 0X6B00_2800

Anc buffer 1, anc packet 1 = 0X6B00_2800 + 1*0x400 = 0X6B00_2800

Anc buffer 1, anc packet 2 = 0X6B00_2800 + 2*0x400 = 0X6B00_2B00...

... Anc buffer 1, anc packet 9 = 0X6B00_2800 + 9*0x400 = 0X6B00_4C00

Anc buffer 2, anc packet 0 = 0X6B00_5000

Anc buffer 2, anc packet 1 = 0X6B00_5000 + 1*0x0400 = 0X6B00_5400

Anc buffer 2, anc packet 2 = 0X6B00_5000 + 2*0x0400 = 0X6B00_5800...

... Anc buffer 2, anc packet 9 = 0X6B00_5000 + 9*0x0400 = 0X6B00_7400

Figure 69. Memory Map for Base Address 0x1000_0000 for Non 8-Bit Pixel ValuesThe figure below illustrates the aliasing that occurs in memory for non 8-bit pixel values that you need to take into account when generating or using pixel addresses in DDR.

The least significant bit (LSB) of the lead pixel is held in the LSB of the first memory word.

17. Gamma Corrector II IP Core

The Gamma Corrector II IP core primarily alters the color values in each pixel in a video stream to correct for the physical properties of the intended display. For example, the brightness displayed by a cathode-ray tube monitor has a nonlinear response to the voltage of a video signal.

The Gamma Corrector II IP core offers the following features:

Configurable look-up table (LUT) that models the nonlinear function to compensate for the non-linearity.
Generic LUT based approach, and user programmable LUT contents that allows the IP core to implement any transform that maps individual color plane value at the input to new values at the output according to a fixed mapping.
Supports up to 4 pixels in parallel.
Supports extra pipelining registers.

The Gamma Corrector II IP core implements one LUT for each color plane in the pixel. The contents of each LUT are independent of the other LUTs, so each color plane may have its own unique transform mapping. You program the contents of each LUT at run time through an Avalon-MM control slave interface. At this time, the IP core does not support any preset values or a fixed operation mode where you may specify the LUT contents at compile time. As a result, the contents of the LUT(s) are initialized to 0 after every reset. You must overwrite the desired values before processing begins.

You can choose up to two data banks for each LUT to allow two separate transforms to be defined at one time for each color plane. A switch in the register map controls which bank is used to transform the data for each frame. The inclusion of the second LUT bank allows for rapid switching between two transforms on a frame-by-frame basis, and one LUT bank to be updated with a new transform while the video is processed by the other bank without any disruption.

17.1. Gamma Corrector Parameter Settings

Table 56. Gamma Corrector II Parameter SettingsYou program the actual gamma corrected intensity values at run time using the Avalon-MM slave interface.
Parameter	Value	Description
Input bits per color sample	4–16, Default = 8	Select the number of bits per color plane sample for input streams.
Output bits per color sample	4–16, Default = 8	Select the number of output bits per color plane sample for output streams.
Number of color planes	1–3, Default = 2	Select the number of color planes per pixel.
Number of pixels in parallel	1, 2, 4, 8, Default = 1	Select the number of pixels transmitted per clock cycle.
Color planes transmitted in parallel	On or Off	Select whether to send the color planes in parallel or in sequence (serially).
Enable 2 banks of LUT coefficients	On or Off	Turn on if you want to enable two data banks for each LUT to allow two separate transforms to be defined at one time for each color plane.
How user packets are handled	No user packets allowed Discard all user packets received Pass all user packets through to the output	If your design does not require the IP core to propagate user packets, then you may select to discard all user packets to reduce ALM usage. If your design guarantees that the input data stream will never have any user packets, then you further reduce ALM usage by selecting No user packets allowed. In this case, the Gamma Corrector II IP core may lock if it encounters a user packet.
Add extra pipelining registers	On or Off	Turn on this parameter to add extra pipeline stage registers to the data path. You must turn on this parameter to achieve: Frequency of 150 MHz for Cyclone V devices Frequencies above 250 MHz for Intel^® Arria^® 10, Intel^® Cyclone^® 10 GX, Arria V, or Stratix V devices
Reduced control register readback	On or Off	If you do not turn on this parameter, the values written to registers 4 and 5 in the control slave interface can be read back. If you turn on this parameter, the values written to registers 3 and upwards cannot be read back through the control slave interface. This option reduces ALM usage. Note: The values of registers 6 and above cannot be read back in any mode.

17.2. Gamma Corrector Control Registers

The Gamma Corrector II IP core requires an Avalon-MM slave interface but the Gamma Corrector IP core can have up to three Avalon-MM slave interfaces.

The Gamma Corrector II IP core requires an Avalon-MM slave interface in all modes to enable run-time updating of the coefficient values. As is the convention with all VIP IP cores, when a control slave interface is included, the IP core resets into a stopped state and must be started by writing a ‘1’ to the Go bit of the control register before any input data is processed.

Table 57. Gamma Corrector II Control Register Map
Address	Register	Description
0	`Control`	Bit 0 of this register is the `Go` bit, all other bits are unused. Setting this bit to 0 causes the IP core to stop at the end of the next frame/field packet.
1	`Status`	Bit 0 of this register is the `Status` bit, all other bits are unused. The IP core sets this address to 0 between frames. The IP core sets this address to 1 when it is processing data and cannot be stopped.
2	`Interrupt`	This bit is not used because the IP core does not generate any interrupts.
3	`Read bank`	Set to 0 to select LUT bank 0 Set to 1 to select LUT bank 1 Ignored if dual bank mode is not enabled.
4	`Write bank`	Set to 0 to enable run-time updating of LUT bank 0 Set to 1 to enable run-time updating of LUT bank 1 Ignored if dual bank mode is not enabled.
5	`Write color plane`	Selects to which color plane (LUT) the writes to the register map will be applied.
6 – 5+2 ^N where N is the number of bits per symbol	`LUT contents`	Each register aliases to one address in the selected write color of the selected write bank. Note: The values written to registers 6 and above cannot be read back in any mode.

18. Configurable Guard Bands IP Core

The Configurable Guard Bands IP core compares each color plane in the input video stream to upper and lower guard bands values.

If the value in any color plane exceeds the upper guard band then the value is replaced by the upper guard band. Likewise, if the value in any color plane falls below the lower guard band then the value is replaced by the lower guard band.

You may specify different guard band values for each color plane. If you enable the run-time control feature, then these values may be altered at run time through the Avalon-MM control slave interface.

You may specify the input as unsigned data or signed 2’s complement data. In this case, the IP core converts the data to an unsigned format (by adding half the maximum range) before guard banding, and the guard bands are specified as unsigned values.
You may specify that the output data should be driven as signed data but, as with signed input data, all guard banding is done on the unsigned data before conversions to signed output. The IP core converts the output data to an unsigned format by subtracting half the maximum range after guard banding.
You may not select both signed input and signed output data.

18.1. Guard Bands Parameter Settings

Table 58. Guard Bands Parameter Settings
Parameter	Value	Description
Bits per color sample	4–16, Default = 8	Select the number of bits per color plane per pixel.
Number of color planes	1–3, Default = 2	Select the number of color planes per pixel.
Number of pixels in parallel	1, 2, 4, 8, Default = 1	Select the number of pixels transmitted per clock cycle.
Color planes transmitted in parallel	On or Off	Select whether to send the color planes in parallel or in sequence (serially).
4:2:2 data	On or Off	Turn on to indicate that the input data is 4:2:2 sampled. Note: 4:2:2 mode does not support odd frame widths and heights.
Signed input data	On or Off	Turn on to indicate that the input data should be treated as signed 2’s complement numbers.
Signed output data	On or Off	Turn on to indicate that the output data should be treated as signed 2’s complement numbers
Run-time control	On or Off	Turn on to enable run-time control of the guard band values. Note: When you turn on this parameter, the `Go` bit gets deasserted by default. When you turn off this parameter, the `Go` is asserted by default.
Lower/Upper guard band for color <0–3>	0 to (1^{Bits per color sample})-1	These parameters to define the guard bands for each color plane (up to 4 colors per pixel—color 0 is in the LSBs of the Avalon-ST Video data bus). If you enable Run-time control, these values are just used as defaults at reset and may be overwritten at run time. These are unsigned values.
How user packets are handled	No user packets allowed Discard all user packets received Pass all user packets through to the output	If your design does not require the IP core to propagate user packets, then you may select to discard all user packets to reduce ALM usage. If your design guarantees that the input data stream will never have any user packets, then you further reduce ALM usage by selecting No user packets allowed. In this case, the IP core may lock if it encounters a user packet.
Add extra pipelining registers	On or Off	Turn on this parameter to add extra pipeline stage registers to the data path. You must turn on this parameter to achieve: Frequency of 150 MHz for Cyclone V devices Frequencies above 250 MHz for Intel^® Arria^® 10, Intel^® Cyclone^® 10 GX, Arria V, or Stratix V devices
Reduced control register readback	On or Off	If you do not turn on this parameter, the values of all the registers in the control slave interface can be read back after they are written. If you turn on this parameter, you cannot read back the guard band values written through the control slave interface. The control, interrupt and status register values may still be read. This option reduces the size of the control slave logic.

18.2. Configurable Guard Bands Control Registers

Table 59. Configurable Guard Bands Register MapYou may choose to enable an Avalon-MM control slave interface for the Configurable Guard Bands IP core to enable run-time updating of the guard band values. As is the convention with all VIP IP cores, when a control slave interface is included, the IP core resets into a stopped state and must be started by writing a ‘1’ to the `Go` bit of the control register before any input data is processed.
Address	Register	Description
0	`Control`	Bit 0 of this register is the `Go` bit. All other bits are unused. Setting this bit to 0 causes the IP core to stop at the end of the next frame/field packet. When you enable run-time control, the `Go` bit gets deasserted by default. If you do not enable run-time control, the `Go` is asserted by default.
1	`Status`	Bit 0 of this register is the `Status` bit, all other bits are unused. The IP core sets this address to 0 between frames. The IP core sets this address to 1 when it is processing data and cannot be stopped.
2	`Interrupt`	This bit is not used because the IP core does not generate any interrupts.
3	`Lower guard band 0`	Value for lower guard band for color 0.
4	`Upper guard band 0`	Value for upper guard band for color 0.
5	`Lower guard band 1`	Value for lower guard band for color 1.
6	`Upper guard band 1`	Value for upper guard band for color 1.
7	`Lower guard band 2`	Value for lower guard band for color 2.
8	`Upper guard band 2`	Value for upper guard band for color 2.
9	`Lower guard band 3`	Value for lower guard band for color 3.
10	`Upper guard band 3`	Value for upper guard band for color 3.

19. Interlacer II IP Core

The Interlacer II IP core converts streams of progressive frames into streams of alternating F0 and F1 fields by discarding either the odd lines (F0) or the even lines (F1). The output field rate is consequently equal to the input frame rate.

You can parameterize the Interlacer II IP core to implement a number of optional features:

Pass through or discard of interlaced fields received at the input.
Start interlaced output streams created from progressive input with either an F0 or F1 field.
Override the default alternating F0/F1 output sequence for progressive input frames preceded by control packets with interlaced nibbles indicating that the progressive frame was created by deinterlacing original interlaced content. When you enable this option, the following interlaced nibbles are detected:
- 0000 and 0100 – progressive frames deinterlaced using F0 as the last field. These are interlaced back into F0 fields.
- 0001 and 0101 – progressive frames deinterlaced using F1 as the last field. These are interlaced back into F1 fields.

You can also enable an Avalon-MM slave interface to control the behavior of the Interlacer II IP Core at run time. When you enable the Avalon-MM slave interface, you can enable or disable the optional features above at run time. Otherwise, their behavior is fixed by your selection in the parameter editor.

Enabling the Avalon-MM slave interface also allows you to enable and disable all interlacing of progressive frames at run time, giving the option of progressive passthrough. When interlacing progressive input, the interlacer automatically resets to a new F0/F1 sequence when a change of resolution is detected in the incoming control packets, starting again with an F0 or F1 fields as defined by your parameterization or run-time control settings. You may also reset the F0/F1 sequence at any point using the Avalon-MM slave interface (see Interlacer Control Registers for details).

19.1. Interlacer Parameter Settings

Table 60. Interlacer II Parameter Settings
Parameter	Value	Description
Maximum image height	32–8192, Default = 1080	Specify the maximum number of lines in the input frame/field.
Bits per color sample	4–16, Default = 8	Select the number of bits per color plane per pixel.