Packet drops observed on vmxnet3 when packet length greater than vNIC's MTU
Article ID: 317479
Updated On:
Products
VMware vSphere ESXi
Issue/Introduction
Symptoms:
Packet drops increase when vmxnet3 is used on vsphere67u2 or later.
To get vmxnet3 stats, run the following command in an SSH session to the ESXi host managing the VM in question:
Packet drops increase when vmxnet3 is used on vsphere67u2 or later.
To get vmxnet3 stats, run the following command in an SSH session to the ESXi host managing the VM in question:
net-stats -l
From the output of the above command, note the PortNum and SwitchName values that correlate with your VM, and substitute their values in the below command:vsish -e cat /net/portsets/<SwitchName>/ports/<PortNum>/vmxnet3/rxSummary
Example:vsish -e cat /net/portsets/DvsPortset-0/ports/67108902/vmxnet3/rxSummary
stats of a vmxnet3 vNIC rx queue {
...
number of packets dropped due to pkt length exceeds vNic mtu:462727 <=== this counter increases
...
stats of a vmxnet3 vNIC rx queue {
...
number of packets dropped due to pkt length exceeds vNic mtu:462727 <=== this counter increases
...
Environment
VMware vSphere 6.7.x
Cause
This issue occurs because in vSphere 6.7 Update 2, MTU check was enforced on packets on receive path to not allow non-LRO (Large Receive Offload) packets greater than vNIC MTU. This limit was not enforced in previous releases.
Resolution
To resolve this issue, config options are provided in vSphere 6.7 P01 at two different granularities.
Note: The default will still be to enforce a stricter MTU check.
Use one of these options to re-enable the previous behavior (that is, to avoid the MTU check):
Where, "ethernet0" should be replaced by the specific vNIC on which the configuration is to be applied.
Please use the VMware KB article below to follow steps on how to:
Modifying advanced virtual machine settings using vSphere Client (1016098)
https://kb.vmware.com/s/article/1016098
Workaround:
To work around this issue on vSphere 6.7 Update 2 and later versions, change the receiving vNIC's MTU (inside the VM) as per the packet lengths expected.
Example:
If vSwitch MTU is 9000, then packets up to 9000 bytes in length can be delivered as non-LRO packets. In this case, increase the vNIC's MTU to 9000.
Note: The default will still be to enforce a stricter MTU check.
Use one of these options to re-enable the previous behavior (that is, to avoid the MTU check):
- Host-wide config option:
esxcli system settings advanced set -o "/Net/Vmxnet3NonTsoPacketGtMtuAllowed" -i 1
Note: This configuration is applicable over all vmxnet3 vNICs (host wide). This setting will be then applied to each VM that will be powered on after making this change
OR
Note: This configuration is applicable over all vmxnet3 vNICs (host wide). This setting will be then applied to each VM that will be powered on after making this change
OR
- vNIC specific config option:
Use ethernet0.rxAllowPktGtMtu = "1" in the .vmx file:
Where, "ethernet0" should be replaced by the specific vNIC on which the configuration is to be applied.
Please use the VMware KB article below to follow steps on how to:
Modifying advanced virtual machine settings using vSphere Client (1016098)
https://kb.vmware.com/s/article/1016098
Workaround:
To work around this issue on vSphere 6.7 Update 2 and later versions, change the receiving vNIC's MTU (inside the VM) as per the packet lengths expected.
Example:
If vSwitch MTU is 9000, then packets up to 9000 bytes in length can be delivered as non-LRO packets. In this case, increase the vNIC's MTU to 9000.
Feedback
Yes
No
Powered by
