MACsec Intel® FPGA System Design User Guide

ID 767516
Date 3/31/2024
Document Table of Contents

2.1. System Architecture

The main design components of this reference design are chosen considering the reusability which reduces the time to market of your end product.
Figure 4. Detailed Block Diagram of Reference Design

In this reference design, the same QSFP loopback acts as LAN for both MACsec control and datapath transfers. The MACsec protocol provides two different standards for data and control transfers respectively: IEEE 802.1AE and IEEE 802.1X. Two Ethernet MACs are enabled with 2 different MAC addresses connected and controlled through two host VMs and get data from two different Ethernet MAC ports.

The data path is offloaded to FPGA hardware but controlled through the CSR interface from host VMs. MCDMA enables the CSR path through the PIO interface. The same PIO is driving all the components in the two MACsec paths. The FPGA Platform Designer (Standard) fabric interconnect takes the responsibility of driving to correct VM path as the PF number is also part of the PIO AVMM address. Before any data transfer starts, the MACsecs need to be configured with appropriate keys for each supported security association in a given secure channel along with the SCI info for look up which contains the destination MAC address. All these are programmed into the MACsec IP registers using the MACsec driver in the Linux software stack. The MACsec IP driver acts as an underlying layer for Linux’s default MKA protocol stack (WPA). Different machines on the LAN interact and elect one machine as Key Server based on a priority number. The same machine is responsible for providing keys to all the clients over the same LAN.

In this reference design, one of the 2 VMs acts as a key server and the other acts as a key client to configure the keys for the MACsecs in the datapath using Linux’s default MKA protocol stack (WPA). Once the keys are written to the MKA stack, the underlying MACsec driver writes the keys into the respective registers inside the MACsec IP through the CSR interface.

Once the keys are established, the application running in a VM can start sending the Ethernet data packets from the Packet Generators continuously or in one-hot mode depending on its configuration. The MACsec and Crypto IPs inline in the datapath provide encryption to the packets. The Ethernet MAC and PHY transmits these ciphertext packets by appending the required preamble + CRC over the QSFP. On the other side, these ciphertext packets are received through another Ethernet MAC port and the data is fed to the MACsec core (RX) attached to the particular MAC to decrypt the packet. The encryption and decryption parameters are managed by the MACsec stack running on the host machine.

The MACsec IP in the datapath monitors the 32-bit packet numbers received or calculated internally and informs the Host VM through an interrupt or poll mechanism once it reaches the terminal/limit count. The security association (SA) needs to be changed to the next one and new keys are chosen for the datapath which is called "key rotation". The MCDMA in the host path enables the communication between various VM-clients and MACsec’s uncontrolled ports. It operates on queues set up by the MCDMA driver software to transfer the data between the local FPGA and the host. The elements of each queue are descriptors. The MCDMA’s control logic reads the queue descriptors and executes them. Separate queues are used for reading and writing DMA operations for each channel. Each PF consumes one MCDMA channel in this design. The logic in the FPGA fabric needs to route the packets based on their channel ID sideband signal from the AVST streaming interface. The channel adaptor logic takes care of packet routing in both D2H and H2D directions for 2 PFs.

The host offloads the data transfer overhead to MCMDA for transferring EAPOL packets or any non-SecTAG packets between the host and MACsec’s uncontrolled ports. In MCDMA, once configured with channel specific BDs, as soon as it sees a packet in the packet buffer, it forms a PCIe TLP for the actual length of packet received and sends it to the host. It may consume multiple BDs if the Ethernet packet length is higher than the MPS value configured in the PCIe EP. In addition to that, it also updates the BD status and hands it over to the SW control once served. The MACSec uncontrolled ports get the plaintext as well as ciphertext data. It is your responsibility to filter the packets and decide on which packets go to the host.

The Ethernet MAC instance can be a direct E-Tile/F-Tile Ethernet IP or HSSI-SS IP based on the FPGA device/Devkit chosen. The choice of HSSI-SS is better (provided the subsystem is available for the targeted tile) as it provides a standard AXI streaming interface and is therefore tile-agnostic. In this document, the terms "E-Tile/F-Tile" and "HSSI-SS" are used interchangeably. The MACsec IP also supports the AXI streaming interface so it’s easy to connect. The user logic in RX path, between MAC and MACsec can check the packet status from the specific TUSER bits (for example, if the CRC error bit is asserted) to decide whether to drop the packet or send it to the MACsec IP. There is no need to filter the Ethernet packets based on destination MAC with your logic as the same check happens inside the MACsec IP.

There are multiple protocol bridges and CDC bridges in the design to accommodate the data transfers between AXI streaming and AVST, and between different clock domains. The details are discussed in the later sections.

The reference design is flexible enough to use one of the MACsec paths with external test equipment as the key client/server based on the priority setting at the SW stack level for interoperability testing. At the packet client interface, you can enable the loopback and use a 3rd party tester as a pattern generator.