Article Sections Overview The Mid-Range Network Processing Problem Intel's Vision for the IXP23XX Product Line Key Architectural Decisions Application Examples Summary More Info Author Bio References February 2005 Issue Integrated Data and Control Plane Processing Using Intel® IXP23XX Network Processors IDF Spring 2005 Unleashes the Possibilities of Tomorrow's Platforms Promoting the Benefits of Modular Communications Platforms COM Express: The Next Trend in Embedded Computing Small Form Factors About Home

Integrated Data and Control Plane Processing Using Intel® IXP23XX Network Processors
by Muthaiah Venkatachalam

Overview: Support for Value-Added Network Services
The recent introduction of the Intel® IXP2325 and Intel® IXP2350 network processors (NPUs) brings Intel's fully programmable architecture to new, lower cost and higher performance points for access and edge applications. With their flexible, high-performance architecture, these innovative mid-range network processors deliver processing performance to support value-added network services from T1/E1 to 2-gigabit per second (Gbps) line rates.

The Intel® IXP23XX product line integrates key performance features from the Intel® IXP4XX and Intel® IXP2XXX product lines, along with several new architectural features of its own. The first network processor offering from Intel built on 90-nm technology, the Intel IXP23XX product line brings together network processing, control processing, and versatile integrated input/output (I/O) to support highly efficient network access and edge solutions such as digital subscriber line access multiplexers (DSLAMs), wireless infrastructure systems, enterprise routers and integrated security appliances.

This article outlines the mid-range network processing problem and then describes the key architectural innovations of the IXP23XX product line, and how these innovations help enhance network application throughput. Also provided are several network application examples that use the IXP2325 and/or IXP2350 to illustrate the architectural advantages of these network processors

The Mid-Range Network Processing Problem
Network processors have been a growing market segment in the communications building block space. Virtually every major telecom and networking vendor is currently using or considering network processor technology in their product designs. Network processors are typically classified as "high-end" (10-Gbps line rate) or "mid-range" (1-Gbps to 10-Gbps line rates).

A mid-range network processor, if designed properly, can have wide applicability in a variety of networking applications, ranging from an integrated Node-B transport card dealing with Asynchronous Transfer Mode (ATM) cells or Internet protocol (IP) frames to an application-level gateway platform dealing with virus detection using application payload bits. A designer of mid-range network processors needs to comprehend a broad range of customer requirements, including Bill of Materials (BOM) cost, power dissipation, time to market and performance characteristics.

	Access and edge networking solutions have relatively high volume with each line card involving one or more NPUs. Given the high volume and cost-sensitive nature of these networking solutions, BOM cost is a critical issue. This in turn translates to cost sensitivity for the NPU to be used in the solutions.
	Power dissipation is another key issue facing networking equipment manufacturers, especially in chassis-based and rack-mounted systems like 3rd Generation Partnership Project (3GPP) UTRAN elements: Node-B and radio network controllers (RNCs). Usually in such systems, there will not be enough real estate for thermal coolants.
	Quick time to market is another customer requirement. System vendors typically look for a fast ramp and short completion time on their networking solutions. The NPU, which is an important component of the solution, should help to enable quick time to market. However, in doing so, a well designed NPU should not sacrifice its wider applicability.
	A final, critical requirement for networking applications is performance. Performance for an application dealing with packets is often measured in terms of the packet rate that the NPU can handle. There can be other performance metrics depending on the application, some examples being connections per second, bit rate, latency through the NPU, and so forth.

Intel's Vision for the IXP23XX Product Line
The Intel IXP23XX product line of network processors, consisting of the IXP2325 and the IXP2350, addresses all four key design requirements of the mid-range network processing problem and provides integrated data and control plane processing capabilities.

To provide flexibility for a variety of networking applications, the IXP23XX product line was designed with multiple programmable entities on a single chip: microengines (MEs) for high-speed data plane processing, and the Intel XScale® core for robust control plane processing. The MEs are specialized processors that are extremely flexible, and are meant for a wide variety of fast packet-processing functions from light touch to deep inspection. They are capable of handling very high data rates.

The Intel XScale core is a general purpose embedded core based on the ARMv5TE instruction set that can be used for a number of control processing and signaling functions. Figure 1 shows a block diagram of the IXP2350 network processor [1]. The IXP2325 network processor is very similar to the IXP2350, but is limited to two MEs [2].

Figure 1. Intel® IXP2350 network processor block diagram.

The IXP23XX product line implements a comprehensive, balanced architectural design to ensure robust performance across a wide spectrum of application requirements [4]. The MEs running at various speeds (300, 600, 900 MHz) can support the right packet throughput for mid-range applications ranging from a few hundred megabits per second (Mbps) to a few gigabits per second (Gbps) for minimum-sized Ethernet packets.

The Intel XScale core at various frequencies (600, 900, 1200 MHz) can provide the required control plane and signaling compute performance. The media interfaces have enough bandwidth to support data rates up to 3.6 Gbps. The memory interfaces provide a peak double data rate (DDR) memory bandwidth of 19.2 Gbps, and a total quad data rate (QDR) memory bandwidth of 12.8 Gbps, sufficient for even the most challenging access and edge applications. Finally, the internal system buses that provide the connections between the various elements on chip have a bandwidth of up to 43.2 Gbps on the ME side and up to 19.2 Gbps on the Intel XScale core side, ensuring that there are virtually no performance bottlenecks within the NPU.

The IXP23XX architecture addresses the BOM cost and power dissipation requirements in two ways. First, the IXP23XX architecture supports a variety of combinations of ME and Intel XScale core operational frequencies, helping system vendors meet their cost, performance and power targets. Second, Intel uses 90-nm silicon technology, its most advanced silicon process for miniaturization and cost reduction, to achieve significant integration on chip, thereby reducing the number of peripheral components needed on the platform and hence lowering platform power dissipation. For example, the IXP23XX integrates two Gigabit Ethernet media access controls (MACs) on chip, eliminating the need for external Gigabit MACs in most applications. A powerful Intel XScale core is integrated on chip, eliminating the need for an external control processor in most applications.

To address the fast time to market requirement, the IXP23XX product line includes a development platform [3], several reference designs and software building blocks along with a user-friendly and robust tool chain that can help system vendors kick start their product development. These reference designs provide a solid starting point upon which system vendors can design-in their value-add features.

Several members of the Intel® Communications Alliance and other third-party vendors are available for customizing these reference designs to meet the exact needs of system vendors. These Intel Communications Alliance members can also provide services and support such as writing new software building blocks, developing NPU-based platforms, providing the operating systems to control the NPU platforms, and doing system integration.

Key Architectural Decisions
In designing the IXP23XX product line of network processors, the following key decisions were made by Intel to enhance the processor architecture:

Configurability versus programmability: The most fundamental architectural decision was to choose a flexible usage model for the NPU. A configurable architecture provides quick time to market for a very limited set of network applications versus a fully flexible and programmable architecture that has applicability across a wide range of applications. The decision was made to go with the more programmable and flexible architecture. Thus the IXP23XX design includes MEs, along with a new, enhanced Intel XScale core. The MEs are multithreaded 32-bit RISC (Reduced Instruction Set Computing) engines with an instruction set optimized for packet processing, with eight hardware contexts, dedicated local memory, control store, and an abundance of registers to enable rich packet and cell processing at wire speed.

Maximum integration: Another important architectural decision for the IXP23XX product line, which helps system vendors achieve minimum BOM cost and lower platform power dissipation, was to provide a high level of integration on chip. The IXP2350 integrates the ME subsystem and the Intel XScale core subsystem, which provide flexibility, high packet throughput, and high compute performance. Each of these subsystems has an independent DDR memory interface. The NPU also has an integrated internal 128 KB SRAM called the MSG_SRAM.

This MSG_SRAM is dual ported to allow the MEs and the Intel XScale core simultaneous access to memory without impacting performance on either data or control planes. A scratchpad communications unit is integrated to enhance the packet processing by helping the MEs communicate effectively for all packet processing data plane applications. A flexible hash unit is available for computing hash functions needed for table lookups and packet header/trailer parameters.

Also included are two integrated Gigabit Ethernet MACs, which provide 10/100/1000 Ethernet compatibility, along with the Network Processing Engines (NPEs), which are integrated hardware accelerators with preloaded firmware to perform tasks such as HDLC[8] processing for up to 16E1s and 256 channels, ATM-TC, IMA[9] assist, as well as support for two 10/100 Ethernet MACs. There is also security acceleration on chip to accelerate popular DES[10], 3DES[11], AES[12], SHA-1[13] and MD-5[14] cryptography algorithms in hardware.

Finally, there are a number of media interfaces implemented in the Media and Switch Fabric (MSF) interface on chip, supporting a wide range of Utopia, POS-PHY, T1/E1, MII and PCI variants. An integrated expansion bus provides a "glueless" interface to flash and other devices. DSP connections are supported via HPI. A number of other peripherals are also integrated, including 16 GPIO pins (which can be used to implement I2C, for example), four timers (one watchdog), an interrupt controller, a performance monitor unit, two universal asynchronous receiver-transmitters (UARTs), and reset logic.

Intel XScale core subsystem enhancement: Another key architectural decision was to significantly enhance the performance of the Intel XScale core subsystem in the IXP23XX product line. The IXP23XX now supports core speeds up to 1.2 GHz, enabling developers to eliminate the need for an external control processor in many applications. Figure 2 shows a diagram of the Intel XScale core subsystem.

Figure 2. Intel XScale® core subsystem within the Intel® IXP23XX product line.

Complementing the increased core speed, the following key architectural enhancements have been made to the Intel XScale core subsystem.

1. 512 KB L2 cache. This can significantly enhance the performance for the processing stacks on the Intel XScale core by exploiting locality in the stack data structures. The cache is 8-way set associative.
2. Cache locking. This feature enhances the Intel XScale core performance by locking out the desired cache lines from being evicted, thereby helping to avoid extra memory accesses from the core. Therefore, when the core needs the data from these lines, there is always a cache hit and the memory latency penalty is not incurred. Up to 7/8th of the L2 cache can be locked, thereby helping the Intel XScale core cut down on memory latencies even when there is not much locality in the data structures.
3. Independent Intel XScale core DDR memory controller. This separate DDR controller helps provide performance isolation by minimizing or eliminating accesses to the ME side DDR, preserving data plane as well as control plane processing performance.
4. L2 Cache coherency with the Intel XScale core DDR. The L2 cache can be made completely coherent with the core side DDR memory, so that the programmer is relieved of manual coherency operations using software.
5. Push feature. The MEs have the ability to write to the Intel XScale core side DDR with a certain PUSH bit ON, thereby triggering the coherency protocol between the L2 and the core side DDR and making the data appear in the L2 cache. This way, when the core needs the data, there will be an L2 hit and the memory latency penalty is not incurred. This is an extremely useful feature when there are many packet descriptors being passed from ME to the core in a given time interval, as in the case of a complex signaling stack with a lot of message types.
6. Access to MSG_SRAM. Access is available through a separate QDR port that is different from the port used by the MEs, providing performance and memory latency isolation between the MEs and the Intel XScale core.
7. Access to ME-side memories and peripherals. The Intel XScale core has complete access to the ME-side memories, the scratchpad unit, the hash unit and the PCI interface through the XSI-CPP bridge.

Other architectural design decisions: Other features were implemented in the IXP23XX to support flexible development of high-performance applications. For example, the NPU can be booted from either of the two DDR memories on the IXP23XX, allowing system vendors to populate either one or both of the DDRs and boot from either DDR memory, depending on the application requirements. More importantly, both DDRs are completely transparent to the ME software, so that the same software can be used for different memory configurations.

Application Examples
The previous section examined the key architectural features of the IXP23XX product line of network processors. The following section provides several application examples to illustrate the architectural advantages of these network processors.

Integrated Node-B ATM transport card with HSDPA
Figure 3 shows the typical architecture of a Node-B transport card using the IXP2350 network processor. The Iub interface consists of up to 16E1s in the diagram, but could scale up to channelized 2xOC-3. For the 16E1/T1 scenario, the ATM-TC processing on the T1/E1 is implemented using NPEs. For these relatively low data rates, it is optional to populate the ME-side DDR, as both MEs and the Intel XScale core have the bandwidth they need via the core DDR.

Figure 3. Intel® IXP2350 network processor in a Node-B transport card.

It is also optional to populate the external QDR, as MSG_SRAM and DDR bandwidth should be sufficient for meeting the 16E1/T1 data rates. In the case of a Gigabit Ethernet backplane, there is no need for external MACs due to the integrated MACs (a backplane based on U2 could take advantage of the native IXP2350 interface as well). Finally, encryption and decryption are done using NPE cryptography engines for the control plane traffic. Hence, by using the IXP2350 network processor in a Node-B transport card, system designers can save BOM costs by eliminating an external device for ATM-TC assist, an external MAC, an external encryption engine for control traffic, the ME-side DDR memory and the external QDR memory.

For ATM-RAN applications, the MEs would perform the packet processing. Figure 4 diagrams the ATM-RAN data flow. In the Iub to baseband direction, TDM bytes being transported on E1s (or equivalently on channelized OC-3s) are reassembled into ATM cells by the NPE (or an external framer mapper for 2xOC-3) and further sent to MEs for processing. The MEs perform the IMA function. The MEs then reassemble the ATM cells into variable-sized frames using AAL2 CPS[15], SSAR[16] and SSTED[16] for data/voice traffic, and AAL5[17] for control traffic and encapsulate these frames in UDP/IP/Ethernet and send the frames to the baseband card via the Ethernet MAC on chip. In the opposite direction, MEs receive the payload from the Gigabit MAC, decapsulate UDP/IP/Ethernet, segment the payload into ATM AAL5 cells for control traffic and use AAL2 CPS/SSSAR/SSTED for data traffic, and send it out in a controlled manner using per-VC ATM traffic shaping[18] on the E1/T1 (or channelized OC-3) interfaces.

Figure 4. ATM RAN data flows.

To speed development, processing on the MEs can be modularized using basic software building blocks (called microblocks) using the Intel® IXA Software Development Kit [19]. The Intel XScale core will correspondingly execute the NPF FAPIs (Family Application Programmer Interfaces) to control and configure the software microblocks running on the MEs. This modular approach helps ensure development of efficient, easily maintained code, and fosters software reuse among multiple applications. Since the Intel XScale core has sufficient compute performance, it can also be used for running the control and signaling stacks for the transport card, the NBAP[20] and ALCAP[21] along with SSCOP[22] and SSCF[23] bearer stacks.

For the 16E1/T1 configuration, the MEs running at 300 MHz on the IXP2350 network processor will have over 50 percent headroom^† [4]. The Intel XScale core running at 1200 MHz will have considerable headroom^† as well; the actual number depends on the ALCAP connection rate and the NBAP message throughput.

A similar platform architecture and performance headroom can be achieved for an IP-based transport card that implements a broad number of functions including PPP[24], ML/MC-PPP[25, 26], PPPMux[27], IP header compression[28], routing, IPSec[29] and IP-QoS.

The headroom can be used in a valuable way by integrating HDSPA[30] on the transport card. HSDPA is used for high-speed data access in the 3G wireless network in the downlink airgap. The idea of HSDPA is to move part of the WCDMA MAC, called the MAC-hs to the Node-B. The MAC-hs data path, including HARQ, Iub framing, and Iub flow control can then be integrated onto the free MEs. The MAC-hs radio bandwidth management function can be integrated onto the MEs or the Intel XScale core.

The IXP2350 network processor is an excellent fit for implementing 3G transport cards due to the high level of integration, support for key interfaces, and sufficient headroom to enable the integration of HSDPA.

Figure 5. 802.16e Basestation MAC card conceptual diagram.

Figure 5 provides a conceptual architecture diagram of an 802.16e [6] basestation platform using the IXP2350 network processor. The key integration features that the IXP2350 brings to this application are the on-chip Gigabit Ethernet MACs for backhaul purposes; flexible media interfaces to mate to the PHY device; enough ME compute power to not only implement the MAC layer of 802.16e, but also to optionally implement the transport layer for the backhaul involving IP forwarding and IP QoS; sufficient Intel XScale core compute power to implement MAC signaling and radio schedulers; and an on-chip security accelerator for the data traffic up to 200 Mbps.

The MAC layer implemented on the MEs consists of the standard and optional 802.16e MAC features like packing, fragmentation, convergence sublayer classification, packet header suppression, ARQ, H-ARQ support, encryption/decryption, and so on. The Intel XScale core contains the relevant signaling stacks for CID setup, MAC management stacks and the radio bandwidth schedulers for OFDMA and AAS, which provide burst schedule information to the ME data path every transmission time interval.

Optimized DSLAM line card for xDSL
The IXP23XX product line is also an excellent fit for DSLAM line cards for ADSL/ADSL2/VDSL [7]. With the new breed of IP-DSLAMs in particular, the IXP23XX product line offers the right level of integration and flexibility to support numerous functions, including AAL5 segmentation and reassembly, packet classification, MAC bridging, IP forwarding, packet queuing, and traffic management—with the throughput to handle increased port density and elevated DSL line rates.

Figure 6. Intel® IXP23XX in DSLAM line card.

A DSLAM line card example is shown in Figure 6. The key value-add features of the IXP23XX for this application are as follows:

	Using the 16-bit Utopia2 MPHY127 mode of operation, the IXP23XX product line of network processors can gluelessly connect to the xDSL PHYs, thereby eliminating FPGAs or other glue logic.
	A choice of SKUs with varying ME performance provides a scalable solution to support a range of required features and port densities.
	Sufficient compute power on the Intel XScale core integrates the signaling stack and eliminates an external control processor on the platform.
	Presence of the internal MSG_SRAM, which can be used in place of external QDR, helps to save valuable real estate and BOM cost on the platform.
	On-chip Gigabit Ethernet MACs with optional redundancy support for the WAN eliminates external MACs on the platform.

Summary
With its unique integration of data plane and control plane processing on a single chip, the Intel IXP23XX product line of network processors offers a powerful, highly efficient solution for a wide range of access and edge applications that previously required expensive, high-speed ASICs.

By eliminating the need for a separate control plane processor, QDR memory, and several other peripherals in many situations, the IXP23XX product line helps to reduce BOM costs and save valuable board real estate. At the same time, its enhanced performance capabilities, rich integration, and programmable architecture enable network equipment manufacturers to meet the demands of next-generation access and edge solutions in wired and wireless networks.

Contact the Editor
Tell us what you think about this article.

		Muthaiah Venkatachalam Network Architect Intel Communications Group Intel Corporation

Muthaiah Venkatachalam is a network architect with the Intel Communications Group. He led the software/system architecture definition for the Intel IXP23XX product line of network processors. At Intel, he has also led the architecture development efforts on network processor-based ATM and IP traffic management solutions, radio scheduling algorithms for WiMAX, system architectures for broadband access and wireless access platforms and metropolitan optical networking systems. His interests lie in wireless communications and networking, network processor architectures, QoS algorithms, systems modeling and analysis, and keeping pace with the innovations in today's networking industry. He has a bachelor's degree from the Indian Institute of Technology at Kharagpur and a graduate degree from the University of Texas at Austin.