LACP NICs down after ESXi host reboot
Article ID: 317997
Updated On:
Products
VMware vSphere ESXi
Issue/Introduction
Symptoms:
- LACP NICs are down after reboot ESXi host running ESXi 6.7 U2
- LACP ports on physical switch are blocked
- Hosts using LAG for management cannot be managed anymore after host reboot.
- Ports used for LACP are blocked on physical switch:
Example of port status of physical switch after host reboot:
1/1 Active Trk55 Up Yes Success 0 616
1/2 Active Trk55 Up Yes Success 0 616
1/3 Active Trk55 Up Yes Success 0 616
1/7 Active Trk54 Up Yes Success 0 615
...
2/11 Active Trk56 Blocked Yes Failure 0 617
2/12 Active Trk56 Blocked Yes Failure 0 617
...
2/1 Active Trk55 Up Yes Success 0 616
2/2 Active Trk55 Up Yes Success 0 616
2/3 Active Trk55 Up Yes Success 0 616
...
Example of NIC status after host reboot:
esxcli network nic list
vmnic0 0000:01:00.0 Down 1000 Full 1500 igbn 0.1.1.0 0x80000ec5 90:1b:0e:ec:be:6b 8086 1521 1734 11ce Intel Corporation I350 Gigabit Network Connection
vmnic1 0000:01:00.1 Down 0 Half 1500 igbn 0.1.1.0 0x80000ec5 90:1b:0e:ec:be:6c 8086 1521 1734 11ce Intel Corporation I350 Gigabit Network Connection
vmnic2 0000:3d:00.0 Down 0 Half 1500 i40en 1.3.1 3.33 0x800010ef 255.65535.255 4c:52:62:03:41:d5 8086 37d1 1734 1230 Intel(R) Ethernet Connection X722 for 1GbE
vmnic3 0000:3d:00.1 Up 1000 Full 1500 i40en 1.3.1 3.33 0x800010ef 255.65535.255 4c:52:62:03:41:d6 8086 37d1 1734 1230 Intel(R) Ethernet Connection X722 for 1GbE
vmnic4 0000:3d:00.2 Up 1000 Full 1500 i40en 1.3.1 3.33 0x800010ef 255.65535.255 4c:52:62:03:41:d7 8086 37d1 1734 1230 Intel(R) Ethernet Connection X722 for 1GbE
vmnic5 0000:3d:00.3 Down 0 Half 1500 i40en 1.3.1 3.33 0x800010ef 255.65535.255 4c:52:62:03:41:d8 8086 37d1 1734 1230 Intel(R) Ethernet Connection X722 for 1GbE
vmnic6 0000:18:00.0 Down 0 Half 1500 igbn 0.1.1.0 0x80000ee3 b4:96:91:32:1b:90 8086 1521 8086 00a1 Intel Corporation Ethernet Server Adapter I350-T4
vmnic7 0000:18:00.1 Up 1000 Full 1500 igbn 0.1.1.0 0x80000ee3 b4:96:91:32:1b:91 8086 1521 8086 00a1 Intel Corporation Ethernet Server Adapter I350-T4
vmnic8 0000:18:00.2 Down 0 Half 1500 igbn 0.1.1.0 0x80000ee3 b4:96:91:32:1b:92 8086 1521 8086 00a1 Intel Corporation Ethernet Server Adapter I350-T4
vmnic9 0000:18:00.3 Up 1000 Full 1500 igbn 0.1.1.0 0x80000ee3 b4:96:91:32:1b:93 8086 1521 8086 00a1 Intel Corporation Ethernet Server Adapter I350-T4
vmnic10 0000:d8:00.0 Down 0 Half 1500 igbn 0.1.1.0 0x80000f06 b4:96:91:37:86:94 8086 1521 8086 00a2 Intel Corporation Ethernet Server Adapter I350-T2
vmnic11 0000:d8:00.1 Down 0 Half 1500 igbn 0.1.1.0 0x80000f06 b4:96:91:37:86:95 8086 1521 8086 00a2 Intel Corporation Ethernet Server Adapter I350-T2
esxcli network vswitch dvs vmware lacp config get
dvSwitchProd lag1 232217653 vmnic2,vmnic1,vmnic8,vmnic6,vmnic5,vmnic0 true Active Src and dst ip
esxcli network vswitch dvs vmware lacp stats get
DVSwitch LAG ID NIC Rx Errors Rx LACPDUs Tx Errors Tx LACPDUs
-----------------------------------------------------------------------------
dvSwitchProd 232217653 vmnic8 0 4 0 5
dvSwitchProd 232217653 vmnic1 0 4 0 5
dvSwitchProd 232217653 vmnic6 0 3 0 5
dvSwitchProd 232217653 vmnic2 0 4 0 5
dvSwitchProd 232217653 vmnic5 0 3 0 5
dvSwitchProd 232217653 vmnic0 0 12 0 13
1/1 Active Trk55 Up Yes Success 0 616
1/2 Active Trk55 Up Yes Success 0 616
1/3 Active Trk55 Up Yes Success 0 616
1/7 Active Trk54 Up Yes Success 0 615
...
2/11 Active Trk56 Blocked Yes Failure 0 617
2/12 Active Trk56 Blocked Yes Failure 0 617
...
2/1 Active Trk55 Up Yes Success 0 616
2/2 Active Trk55 Up Yes Success 0 616
2/3 Active Trk55 Up Yes Success 0 616
...
Example of NIC status after host reboot:
esxcli network nic list
vmnic0 0000:01:00.0 Down 1000 Full 1500 igbn 0.1.1.0 0x80000ec5 90:1b:0e:ec:be:6b 8086 1521 1734 11ce Intel Corporation I350 Gigabit Network Connection
vmnic1 0000:01:00.1 Down 0 Half 1500 igbn 0.1.1.0 0x80000ec5 90:1b:0e:ec:be:6c 8086 1521 1734 11ce Intel Corporation I350 Gigabit Network Connection
vmnic2 0000:3d:00.0 Down 0 Half 1500 i40en 1.3.1 3.33 0x800010ef 255.65535.255 4c:52:62:03:41:d5 8086 37d1 1734 1230 Intel(R) Ethernet Connection X722 for 1GbE
vmnic3 0000:3d:00.1 Up 1000 Full 1500 i40en 1.3.1 3.33 0x800010ef 255.65535.255 4c:52:62:03:41:d6 8086 37d1 1734 1230 Intel(R) Ethernet Connection X722 for 1GbE
vmnic4 0000:3d:00.2 Up 1000 Full 1500 i40en 1.3.1 3.33 0x800010ef 255.65535.255 4c:52:62:03:41:d7 8086 37d1 1734 1230 Intel(R) Ethernet Connection X722 for 1GbE
vmnic5 0000:3d:00.3 Down 0 Half 1500 i40en 1.3.1 3.33 0x800010ef 255.65535.255 4c:52:62:03:41:d8 8086 37d1 1734 1230 Intel(R) Ethernet Connection X722 for 1GbE
vmnic6 0000:18:00.0 Down 0 Half 1500 igbn 0.1.1.0 0x80000ee3 b4:96:91:32:1b:90 8086 1521 8086 00a1 Intel Corporation Ethernet Server Adapter I350-T4
vmnic7 0000:18:00.1 Up 1000 Full 1500 igbn 0.1.1.0 0x80000ee3 b4:96:91:32:1b:91 8086 1521 8086 00a1 Intel Corporation Ethernet Server Adapter I350-T4
vmnic8 0000:18:00.2 Down 0 Half 1500 igbn 0.1.1.0 0x80000ee3 b4:96:91:32:1b:92 8086 1521 8086 00a1 Intel Corporation Ethernet Server Adapter I350-T4
vmnic9 0000:18:00.3 Up 1000 Full 1500 igbn 0.1.1.0 0x80000ee3 b4:96:91:32:1b:93 8086 1521 8086 00a1 Intel Corporation Ethernet Server Adapter I350-T4
vmnic10 0000:d8:00.0 Down 0 Half 1500 igbn 0.1.1.0 0x80000f06 b4:96:91:37:86:94 8086 1521 8086 00a2 Intel Corporation Ethernet Server Adapter I350-T2
vmnic11 0000:d8:00.1 Down 0 Half 1500 igbn 0.1.1.0 0x80000f06 b4:96:91:37:86:95 8086 1521 8086 00a2 Intel Corporation Ethernet Server Adapter I350-T2
esxcli network vswitch dvs vmware lacp config get
dvSwitchProd lag1 232217653 vmnic2,vmnic1,vmnic8,vmnic6,vmnic5,vmnic0 true Active Src and dst ip
esxcli network vswitch dvs vmware lacp stats get
DVSwitch LAG ID NIC Rx Errors Rx LACPDUs Tx Errors Tx LACPDUs
-----------------------------------------------------------------------------
dvSwitchProd 232217653 vmnic8 0 4 0 5
dvSwitchProd 232217653 vmnic1 0 4 0 5
dvSwitchProd 232217653 vmnic6 0 3 0 5
dvSwitchProd 232217653 vmnic2 0 4 0 5
dvSwitchProd 232217653 vmnic5 0 3 0 5
dvSwitchProd 232217653 vmnic0 0 12 0 13
Environment
VMware vSphere ESXi 6.7
Cause
This issue is caused due to LACPDU without SYNC flags.
Resolution
This issue has been fixed in VMware ESXi 6.7 P02 , release name ESXi670-20200400
Workaround:
To workaround this issue re-enable all affected NICs on the ESXi host.
Example:
esxcli network nic down -n vmnic0
esxcli network nic up -n vmnic0
Workaround:
To workaround this issue re-enable all affected NICs on the ESXi host.
Example:
esxcli network nic down -n vmnic0
esxcli network nic up -n vmnic0
Feedback
Yes
No
Powered by
