Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Low virtual network I/O performance with Intel 1Gbps NIC using igb driver (2018891)

Details

  • In a virtual environment, for certain workloads and (or) configurations, the network performance achieved on an Intel 1Gbps NIC using the igb driver might be low because the interrupt throttling rate for the igb driver is not optimal for that workload.
  • The network Bandwidth of  Intel 82576 and Intel i340T4/i350  do not go up to the supported 1 Gb/s
  • It performs slower than other 1Gb/s cards

Solution

The InterruptThrottleRate parameter of the igb driver that regulates the rate at which interrupts are generated by the adapter affects network and system performance. The igb driver uses dynamic conservative mode (setting 3) as the default value in which it dynamically adjusts the InterruptThrottleRate based on the incoming traffic. The valid range of the InterruptThrottleRate is the following:
  • 0 for off
  • 1 for dynamic
  • 3 for dynamic conservative (default)
  • 100-100000 for programming the adapter to send out a maximum of that many interrupts per second, even if more packets have come in
Consider setting the InterruptThrottleRate to a dynamic or static value when the default dynamic conservative mode results in inadequate performance. In most environments, setting the InterruptThrottleRate to dynamic should increase the interrupt rate and result in improved performance. However, in certain environments where even the dynamic mode does not result in an optimal interrupt moderation rate, setting the InterruptThrottleRate to a static value might be more desirable, yielding better performance.
 
To set the InterruptThrottleRate to dynamic, run the following command on the ESXi host:

vmkload_mod driver_name InterruptThrottleRate=value,value
 
For example:

vmkload_mod igb InterruptThrottleRate=8000,8000
 
Note: The value 1 sets the rate to dynamic. Each value represents a port. For example, the above command is for two ports. Additional values can added, separated by comas, depending on the number of available ports. For a four-port adapter, the command is:
 
vmkload_mod igb InterruptThrottleRate=8000,8000,8000,8000
 
Alternatively, use esxcfg-module to persist this parameter across ESXi reboots. For example, to set the parameter for an ESXi host with a dual-port adapter and a single-port adapter.

# esxcfg-module -s "InterruptThrottleRate=8000,8000,8000" igb

Verify the output of the command ethtool -c vmnicX .

Note: Make sure rx-usecs is set to 125 which indicates that the interrupt was changed recently.

Note: Setting InterruptThrottleRate to a static value will program the adapter to send out a maximum of that many interrupts per second, even if more packets have come in. Setting the value to a number lower than 3000 reduces the load on the system and can lower CPU utilization, but will increase latency and reduce throughput as the packets are not processed quickly. An InterruptThrottleRate value of 8000 was found to provide good performance for most traffic types, but lacked in small packet performance and latency. Consider using a larger InterruptThrottleRate value for network traffic that consists of small packets and has a requirement for extremely low latency.

For more information, see Linux* Base Driver for Intel(R) Ethernet Network Connection on the Intel Web site.

This Article Replaces

2014252

Update History

8/12/2012 - Updated InterruptThrottleRate in example to 8000

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 7 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 7 Ratings
Actions
KB: