Adapter installation suggestions
- Install the Network adapter in a slot that matches or exceeds the bus width of the adapter.
- Example 1: if you have a 32 bit PCI adapter put it in a 32 or 64 bit PCI or PCI-X* slot.
- Example 2: if you have a 64 bit PCI-X adapter put it in a 64 bit PCI-X slot.
- Example 3: if you have an x4 PCIe* adapter put it in an x4, x8, or x16 PCIe slot.
For PCI and PCI-X, install the Network Adapter in the fastest available slot.
||Some PCIe* slots are physically wired with fewer channels than the dimensions of the slot would indicate. In that case a slot that matches an x8 dimensions would have the functionality of an x4, x2 or x1 slot. Check with your system manufacture.|
- Example 1: if you have a 64 bit PCI adapter put it in a 66 MHz 64 bit PCI slot.
- Example 2: if you have a 64 bit PCI-X adapter put in a 133 MHz (266 or 533 if available) 64 bit PCI-X slot.
Try to install the adapter in a slot on a bus by itself. If add-in cards share a bus they will compete for bus bandwidth.
||The slowest board on a bus dictates the maximum speed of the bus. i.e. If you install a 66MHz and a 133 MHz add-in card in a 133 MHz bus then all devices on that bus will function at 66 MHz.|
Driver configuration suggestions
Reduce Interrupt Moderation Rate to Low, Minimal, or Off
- Also known as Interrupt Throttle Rate (ITR)
- The default is "Adaptive" for most roles.
- The low latency profile sets the rate to off.
- The storage profiles set the rate to medium.
Enable Jumbo Frames to the largest size supported across the network (4KB, 9KB, or 16KB)
||Decreasing Interrupt Moderation Rate will increase CPU utilization.|
- The default is "Disabled".
Disable Flow Control.
||Enable Jumbo Frames only if devices across the network support them and are configured to use the same frame size.|
- The default is "Generate & Respond".
Increase the Transmit Descriptors buffer size.
||Disabling Flow Control can result in dropped frames.|
- The default is "256". Maximum value is 2048.
Increase the Receive Descriptors buffer size.
||Increasing Transmit Descriptors will increase system memory usage.|
- The default is "256". Maximum value is 2048.
||Increasing Receive Descriptors will increase system memory usage.|
TCP configuration suggestions
- Tune the TCP window size (Applies to Windows* Server editions prior to Windows Server 2008*)
Optimizing your TCP window size can be complex as every network is different. Documents are available on the Internet (search for TCP tuning) that explains the considerations and formulas used to determine window size.
Prior to Windows Server 2008, the network stack used a fixed-size receive-side window. Starting with Windows Server 2008, Windows provides TCP receive window auto-tuning. The registry keywords TcpWindowSize, NumTcbTablePartitions, and MaxHashTableSize are ignored starting with Windows Server 2008.
- Resource: General Introduction
- Resource: Detailed Information
- See Related Topics at the end of this page
Teaming considerations and suggestions
When teaming multiple adapter ports together to maximize bandwidth, the switch needs to be considered. Typically, dynamic or static 802.3ad link aggregation is the preferred teaming mode. However, this teaming mode demands multiple contiguous ports on the switch. Consideration should be given to port groups on the switch. Typically, a switch has multiple ports grouped together which are serviced by one PHY. This one PHY can have a limited shared bandwidth for all the ports which it supports. This limited bandwidth for a group might not be enough to support full utilization of all ports in the group.
Teaming 4 ports on Intel® Gigabit Network Adapters or LANs on motherboard together in an 802.3ad static or dynamic teaming mode, when the switch has shared bandwidth across contiguous ports, the performance gain can be limited to the bandwidth shared. (In the above example, 4 gigabit ports would share a total PHY bandwidth of 2 Gbps.) Whether switch ports are grouped together is dependent on the switch manufacturer and model and varies from switch to switch.
Alternative teaming modes can sometimes mitigate these performance limitations, for instance ALB (Adaptive Load Balancing including Receive Load Balancing). ALB has no demands on the switch and does not need to be connected to contiguous switch ports. If the link partner has port groups, an ALB team can be connected to any port of the switch, distributing the connections across the available port groups on the switch. This action can increase the overall network bandwidth.
Performance testing considerations
- When copying a file from one system to another (1:1) using one TCP session, throughput is significantly lower than doing multiple simultaneous TCP sessions. Low throughput performance on 1:1 networks is because of latency inherent in a single TCP/IP session. A few file transfer applications support multiple simultaneous TCP streams. Some examples are: bbFTP*, gFTP*, and FDT*.
This graph is intended to show (not guarantee) the performance benefit of using multiple TCP streams. These are actual results from an Intel® 10 Gigabit CX4 Dual Port Server Adapter, using default Advanced settings, under Windows 2008* x64.
- Direct testing of your network interface throughput capabilities can be done by using tools like: iperf*, and Microsoft NTTTCP*. These tools can be configured to use one or more streams.
- When copying a file from/to one system to another, the hard drives of each system can be a significant bottle neck. Consider using high RPM, higher throughput hard drives, striped RAIDs, or RAM drives in the systems under test.
- Systems under test should connect through a full-line rate, "non-blocking" switch.
- Theoretical Maximum Bus Throughput:
- PCI Express* (PCIe*) Theoretical Bi-Directional Bus Throughput
|PCI Express Implementation
||Encoded Data Rate
||Unencoded Data Rate|
||4 Gb/sec (0.5 GB/sec)|
||16 Gb/sec (2 GB/sec)|
||32 Gb/sec (4 GB/sec)|
||64 Gb/sec (8 GB/sec)|
- PCI & PCI-X Bus Theoretical Bi-Directional Bus Throughput
|Bus and Frequency
||32-Bit Transfer Rate
||64-Bit Transfer Rate|
||The PCIe* link width can be checked in Windows* through the adapter's properties. Select the Link Speed tab and push the Identify Adapter button. Intel® PROSet for Windows* Device Manager must be loaded for this utility to function.|
User Guide and Release Notes provide configuration details and known issues that may affect performance.
Advanced driver settings for Intel® Ethernet 10 Gigabit Server Adapters
Performance Tuning* for 10 Gigabit Ethernet adapters in Linux*
Maximizing File Transfer Performance Using 10Gb Ethernet and Virtualization (Linux* and VMware VSphere*)
Performance Tuning Guidelines* for Windows Server 2012*
Performance Tuning Guidelines for Windows Server 2008 R2*
Performance Tuning Guidelines for Windows Server 2008*
TCP Features* of Windows 2000 and Windows Server 2003*
How to Improve Windows 2003* File Server Performance*
Improving Measured Latency in Linux* for Intel® 82575/82576 or 82598/82599 Ethernet Controllers
Assigning Interrupts to Processor Cores using an Intel® 82575/82576 or 82598/82599 Ethernet Controller
Parameter Talk: TX and RX descriptors explains how adjusting receive or transmit buffers can affect performance.
Low virtual network I/O performance with Intel 1Gbps NIC using igb driver explains how to improve ESXi performance for certain workloads.
This applies to: