vSphere support for TOE NICs
search cancel

vSphere support for TOE NICs

book

Article ID: 309741

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

This article provides information about the TCP Offload Engine (TOE).

Environment

VMware ESXi 4.1.x Embedded
VMware vSphere ESXi 5.1
VMware ESXi 3.5.x Installable
VMware vSphere ESXi 5.5
VMware ESX Server 3.0.x
VMware ESXi 4.1.x Installable
VMware ESX 4.0.x
VMware ESX Server 3.5.x
VMware ESXi 4.0.x Installable
VMware ESXi 4.0.x Embedded
VMware ESXi 3.5.x Embedded
VMware vSphere ESXi 5.0
VMware vSphere ESXi 6.0
VMware ESX 4.1.x

Resolution

What is a TOE NIC?

TCP Offload Engine (TOE) is a technology used in network interface cards to offload processing of the entire TCP/IP stack to a specialized network hardware device. It is primarily used with high-speed network interfaces, such as gigabit Ethernet and 10 gigabit Ethernet, where processing overhead of the network stack becomes significant.

The term TOE is often used to refer to the NIC itself, although it more accurately refers only to the integrated circuit included on the card which processes the TCP headers.

Does ESXi/ESX support any TOE NICs?

At the time of publication, ESX does not support all TOE NIC features. If you find that some of the NICs listed under ESXi/ESX in the VMWare Compatibility Guide are TOE capable, those NICs are only supported as regular NICs, and not all the TOE features available on those NICs are utilized by ESXi/ESX.

Does ESX allow creating virtual machines with TOE capabilities?

ESX 3.5 provides limited support for TCP Segmentation Offload (TSO). The operating system sends TCP packets to the NIC, which then segments them for transmission over Ethernet. This is actually part of a TCP Offload Engine, which processes both inbound and outbound packets.

Inbound Ethernet frames are reassembled into TCP packets at the NIC, then handed to the operating system assembled. This is called Large Receive Offload (LRO). vSphere 4 supports Large Receive Offload (LRO), a feature that coalesces TCP packets from the same connection to reduce CPU utilization. Using LRO with ESX provides a 40 percent improvement in both throughput and CPU costs. For more information, see the white paper, What's New in VMware vSphere 4: Performance Enhancements.

Note: The Intel VT NICs (igb driver) and Broadcom BCM5708 and BCM5709 9 (bnx2) support TSO and LRO.

If you are using TSO in a guest, and the NICs installed in the host do not support it, then the Ethernet frames are segmented in the VMkernel and sent to the NIC that way.

To identify if TOE and other TCP features are enabled or not, run the command:
  • To check the status of TSO in ESX/ESXi, run the command:

    ethtool -k vmnicX

    To disable TSO within a Linux guest OS, run the command:

    ethtool -K ethX tso off


Additional Information

For related information, see Enabling TSO in:

¿Qué es un TOE NIC? ¿ESXi/ESX son compatibles con cualquier TOE NIC?
TOE NIC および ESXi/ESX の TOE NIC のサポートについて
vSphere 对 TOE 网卡的支持