Hosts May Disconnect from vCenter when using a X710 or XL710 or X722 family card when LLDP is enabled.
search cancel

Hosts May Disconnect from vCenter when using a X710 or XL710 or X722 family card when LLDP is enabled.

book

Article ID: 324503

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Why This Happens:
By design, vmk0 copies the physical MAC address of vmnic0. This is useful for some customers as they can prepare DHCP reservations ahead of the hardware arriving as server vendors will give you the MAC address for the onboard adapters once it is assembled. This family of network adapters has an embedded Link Layer Discovery Protocol (LLDP) engine that will keep issuing ARP requests of that MAC to the first port even if VMK0 moves to another port. This can cause the host to disconnect from vCenter if vmk0 is used for management traffic as the CAM table on the switch above gets confused about where to send packets. 
 
Example
If vmnic0 has a mac of 00:00:00:00:00:00 and vmnic1 has a mac of 11:11:11:11:11:11. The vmkernel adapter vmk0 will have mac 00:00:00:00:00:00. If LLDP packets are coming from vmnic0 (physical) over layer 2 to 00:00:00:00:00:00 and vmk0 moves to vmnic1, the physical switch CAM tables won’t know how to get the traffic to the correct ethernet port reliably because vSphere's attempt to issue a  reverse ARP will conflict with LLDP's request.

Troubleshooting: 
If you have syslog configured for your physical switches you may see Mac Flapping (MACFLAP on Cisco) entries in the switch logs. This may also manifest itself in the switch logs as a high volume of CAM table updates as the 2 ports fight over the MAC Address. 

vobd.log will report mac conflicts using the events "VmMacConflictEvent" or "VmStaticMacConflictEvent

 


Environment

VMware vSphere 6.x

Resolution

There is no solution, however using any of the following methods can help to work around the issue. 

  • Create, but don’t use VMK0 for anything. 
  • Put VMK0 on a tagged NON-Native VLAN. This will prevent conflict with the LLDP agent and MAC address if it's on a completely different broadcast domain.
  • Run LACP across the original physical port and another port.
  • Disable the NIC's built-in LLDP engine (see below).

Network driver i40en version 1.5.6 introduces additional parameters that allow you to allow or disallow the flow of LLDP frames to the network stack. This is currently supported with Intel X710/XL710/XXV710 adapters using FW 6.0.x and later and Intel X722 adapters using FW 4.0 or later.

Download: https://customerconnect.vmware.com/web/vmware/details?downloadGroup=DT-ESXI65-INTEL-I40EN-156&productId=614

Note: You may also have to change the "LLDP Agent" setting to "Disabled" in the NIC BIOS for each port.  Please contact your hardware vendor for exact instructions to accomplish this. The hardware vendor will also know the correct commands to run to disable or enable LLDP on the physical NICs.

For 2-port NICs:
To Disable LLDPesxcli system module parameters set -m i40en -p LLDP=0,0
To Enable LLDPesxcli system module parameters set -m i40en -p LLDP=1,1

For 4-port NICs:
To Disable LLDPesxcli system module parameters set -m i40en -p LLDP=0,0,0,0
To Enable LLDPesxcli system module parameters set -m i40en -p LLDP=1,1,1,1


Note: A reboot of the host is required for the changes to take effect.


For more information on resolving MAC address conflicts, see vmk0 management network MAC address is not updated when NIC card is replaced or vmkernel has duplicate MAC address

Additional Information

ESXi host that uses Intel Corporation Ethernet Controller X710 for 10GbE SFP+ NIC with the i40e driver fails with PSOD