The Ultra Low Latency Ethernet 10G reference design demonstrates Low
Latency 10G Ethernet solution for
Stratix® 10 devices.
This Ethernet solution is developed using Low Latency (LL) Ethernet 10G
(10GbE) Media Access Controller (MAC)
Intel® FPGA IP core and
Stratix® 10 H-tile Transceiver Native PHY with small form-factor
pluggable plus (SFP+)
transceiver module. This reference
design, which uses
10GBASE-R PHY with IEEE 1588v2 mode, is capable to achieve a lower
round-trip latency, 171.0 nanoseconds (ns) compared to 10GBASE-R Ethernet design example for
Stratix® 10 devices (246.5 ns).
The reference design consists of various components.
Figure 1. System Architecture OverviewThis figure shows the high-level block diagram of the design's system
Table 1. Design Components
LL 10GbE MAC
The Low Latency Ethernet 10G MAC
Intel® FPGA IP core with the following configuration:
Datapath options: TX
Enable ECC on memory
blocks: Not selected
Enable 10GBASE-R register
TX and RX datapath Reset/Default
To Enable: Selected
Use legacy XGMII Interface: Not
Use legacy Avalon Memory-Mapped
Use legacy Ethernet 10G MAC
Interfaces: Not selected
The L-Tile/H-Tile Transceiver Native PHY
FPGA IP configured for the 10GBASE-R protocol. The 10GBASE-R 1588 mode preset sets the
PHY's TX FIFO MODE and RX FIFO MODE to
Transceiver Reset Controller
The Transceiver PHY Reset Controller
FPGA IP core. Resets the transceiver.
Decodes the addresses of the components such as traffic controller, LL 10GbE MAC,
Synchronizes the reset of all design components.
Generates a TX serial clock for the
Stratix® 10 10G transceiver.
A dual clock FIFO adapter, which converts 32-bit
interface of MAC to 64 bit.
Avalon® Streaming (
FIFO, which is used to buffer the RX and TX data between the MAC IP core and the
The traffic controller consists of:
Traffic generator: Generates burst packets to the MAC for transmission.
Figure 2. Clocking and Reset SchemeThis figure shows the clocking and reset scheme for this reference
design. In this design’s top level, there are two external clock sources, ref_clk_clk (644.53125 MHz) and csr_clk (125 MHz). There is one active-low and asynchronous master reset signal,
master_reset_n in the top-level of this reference design.
The master reset signal is used to reset all modules in the reference design.
Download the reference design from
Design Store and restore the design
Quartus® Prime software.
Quartus® Prime software
and open the project file (top.qpf).
Before running the design compilation, click Assignments > Settings. Select Always regenerate design files
for IP cores and Generate IP
simulation model when generating IP under IP Settings category.
To compile the design, click Processing > Start Compilation.
Ensure that QUARTUS_ROOTDIR
environment variable is pointing to the installation path of the
Quartus® Prime software.
10.6c and change the directory to <project_directory>/simulation/ed_sim/mentor.
Run the following command to set up the required libraries,
compile the functional simulation model, and exercise the simulation model with
the provided testbench:
Figure 3. Testbench
Table 2. Component Description
Device under test (DUT)
Components of the design that is tested in this simulation. This DUT
consists of MAC and PHY.
Avalon®-ST master bus
functional models (BFMs). This driver forms the TX and RX paths.
This driver also provides access to the
Avalon®-MM interface of the DUT.
Ethernet Packet Monitor
Monitors TX and RX datapaths, and displays the frames in the simulator
Table 3. Testbench File Descriptions. The testbench files below are needed to perform the simulation and these
files are located in <project_directory>/simulation/ed_sim/models.
Avalon® BFM that is
used by the Avalon_driver.sv.
SystemVerilog HDL driver that uses the BFMs to form the transmit and
receive path and access the
SystemVerilog HDL testbench that contains parameters to configure the
BFMs. The configuration is specific to DUT. As such, the content of
this file should not be changed.
SystemVerilog HDL testbench that monitors the
Avalon®-ST transmit and
SystemVerilog HDL package that maps addresses to
SystemVerilog HDL package that defines the test parameters for MAC
configuration and Ethernet packet generation.
SystemVerilog HDL testbench that handles the creation of Ethernet
TCL scripts that starts a simulation session of
the DUT and other logic blocks.
Top-level testbench file that consists of the DUT and other logic
Signal tracing macro script that the
ModelSim® simulator uses to
display testbench signals.
The simulation test case performs the following steps:
Configures the MAC, TX and RX FIFO buffers, and Ethernet packet
Ethernet packet generator generates and transmits 32 packets with 66 bytes and random
payload to the
Avalon®-ST TX path.
Waits until all the packets are loopback and received on
Avalon®-ST RX path.
At the end of the simulation, the transcript window displays the number of
good and bad packets, which monitored by Ethernet packet monitor (refer to Figure 4), and the values
of the MAC statistic counters (refer to Figure 5 and Figure 6). The transcript window also
displays "Simulation PASSED" if all statistics error
counters are zero and the RX MAC statistics counters are equal to the TX MAC statistics
simulation of this design requires approximately 404800 ns to
In the simulation waveform window (refer to Figure 7), the measurement cursors
indicates the roundtrip latency for serial loopback to show that time taken to transmit the
first data from
Avalon®-ST TX datapath to the
Avalon®-ST RX datapath.
Before you run the design, you need to set up the development board as shown in
the following figures:
Figure 8. SFP+ External Loopback Hardware Test Setup
Figure 9. SFP+ External Loopback Hardware Test Setup in Lab
Download the reference design from
Design Store and restore the design using
Quartus® Prime software.
Quartus® Prime software and open
the project file (top.qpf).
Before running the design compilation, go to Assignments > Settings. Under IP Settings category, select
Always regenerate design files for IP cores and
Generate IP simulation model when generating
Click Processing > Start Compilation to compile the design.
Note: You may experience timing violations after running compilation for the
design. To resolve these timing violations, you must launch the Design Space Explorer II
Quartus® Prime software (Tools > Design Space Explorer II) and perform seed sweep to get the best quality of fit.
After the design is compiled successfully, a programming file (top.sof) is generated and located in <project_directory/output_files>.
Set up the
Stratix® 10 GX H-Tile Signal
Integrity (SI) Development Board.
Connect the programming cable to the JTAG connection port (CN1).
Connect the board to the power supply input (J103).
Connect the 10G SFP+ transceiver module along with optical loopback
cable to SFP+ port (J29).
Quartus® Prime software, select
Tools > Programmer to launch the programmer.
Stratix® 10 GX
SI development board using the generated programming file (altera_eth_top.sof).
Reset the Ethernet design by using the push button (USER_PB0).
Note: The design must be reset
whenever you begin a new test.
Quartus® Prime software, click
Tools > System Debugging Tools and launch the system console.
In the System Console command shell, change the directory to project_directory/hwtesting/system_console.
You can now run the predefined hardware tests using the provided test commands listed in
Table 4. Test Command
SFP+ external loopback
The generator generates and sends about 100 000 packets with random bytes
(maximum packet length = 1518 bytes). Wait 1 minutes for it to complete its
The monitor checks the number of good and bad packets received.
This script displays the values of the statistics counters
Avalon®-ST reverse loopback
This command enables the
Avalon®-ST loopback. This test is
used with an external tester such as Spirent tester
The following diagrams show excerpts of the output, which shows that the Ethernet packet
generator configuration, the Ethernet packet monitor status, and the TX and RX statistics
Figure 10. Sample Test Output—Ethernet Packet Generator Configuration
Figure 11. Sample Test Output—Ethernet Packet Monitor Status
Figure 12. Sample Test Output—TX Statistics Counter
Figure 13. Sample Test Output—RX Statistics Counter
Table 5. Clock and Reset Signals
Configuration clock for the
Avalon®-MM interface and core logics. Frequency is 125 MHz.
Reference clock for the ATX PLL and fPLL.
Frequency is 644.53125 MHz.
322.265625 MHz clock for the TX datapath. This
clock is output from tx_clkout of PHY.
322.265625 MHz clock for the RX datapath. This clock is output
from rx_clkout of PHY.
161.1328125 MHz clock for components such as address decoder,
traffic controller, FIFO, and adapter. This clock is generated from fPLL.
The high-speed serial clock generated by the ATX PLL that drives
the Native PHY. Frequency is 5.15625 GHz.
The reference clock source for PHY’s RX CDR PLL. This clock is
sourced from ref_clk_clk.
Assert this asynchronous and active-low signal to reset the
whole design example.
Active-low reset signal for the
Active-low reset signal for the TX datapath.
Active-low reset signal for the RX datapath.
Avalon®-MM Interface Signals
Assert this signal to request a write.
Assert this signal to request a read.
Use this bus to specify the register address you want to read
from or write to.
Carries the data read from the specified register.
Carries the data to be written to the specified
Asserted when IP core is busy and not ready to accept any read
or write request.
Table 7. PHY Interface Signals
RX serial input data to PHY.
TX serial input data from PHY.
Table 8. Status Signals
Asserted when the link synchronization is successful.
Asserted when the RX channel is ready for data
Asserted when the TX channel is ready for data