Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

What is a TOE NIC and does ESXi/ESX support any TOE NICs? (1006143)

Purpose

This article provides information about the TCP Offload Engine (TOE).

Resolution

What is a TOE NIC?

TCP Offload Engine (TOE) is a technology used in network interface cards to offload processing of the entire TCP/IP stack to a specialized network hardware device. It is primarily used with high-speed network interfaces, such as gigabit Ethernet and 10 gigabit Ethernet, where processing overhead of the network stack becomes significant.

The term TOE is often used to refer to the NIC itself, although it more accurately refers only to the integrated circuit included on the card which processes the TCP headers.

Does ESXi/ESX support any TOE NICs?

At the time of publication, ESX does not support all TOE NIC features. If you find that some of the NICs listed under ESXi/ESX in the VMWare Compatibility Guide are TOE capable, those NICs are only supported as regular NICs, and not all the TOE features available on those NICs are utilized by ESXi/ESX.

Does ESX allow creating virtual machines with TOE capabilities?

ESX 3.5 provides limited support for TCP Segmentation Offload (TSO). The operating system sends TCP packets to the NIC, which then segments them for transmission over Ethernet. This is actually part of a TCP Offload Engine, which processes both inbound and outbound packets.

Inbound Ethernet frames are reassembled into TCP packets at the NIC, then handed to the operating system assembled. This is called Large Receive Offload (LRO). vSphere 4 supports Large Receive Offload (LRO), a feature that coalesces TCP packets from the same connection to reduce CPU utilization. Using LRO with ESX provides a 40 percent improvement in both throughput and CPU costs. For more information, see the white paper, What's New in VMware vSphere 4: Performance Enhancements.

Note: The Intel VT NICs (igb driver) and Broadcom BCM5708 and BCM5709 9 (bnx2) support TSO and LRO.

If you are using TSO in a guest, and the NICs installed in the host do not support it, then the Ethernet frames are segmented in the VMkernel and sent to the NIC that way.

To identify if TOE and other TCP features are enabled or not, run the command:
  • To check the status of TSO in ESX/ESXi, run the command:

    ethtool -k vmnicX

    To disable TSO within a Linux guest OS, run the command:

    ethtool -K ethX tso off

Additional Information

Tags

toe-nic

See Also

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 17 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 17 Ratings
Actions
KB: