Symptoms:

• You see unexplained packet drops on VXLAN networks using VXLAN port 4789.
• Some packets on a given datapath will successfully reach their destination whereas others along the same datapath get dropped by the ESXi host.
• You see the dropped packets when you capture packets on the "Drop" capture point despite the fact that DVS teaming and routing is configured correctly. 

Your environment is configured as follows:

• You are using NSX-v 6.2.3 or newer and VXLAN is configured on UDP port 4789.
• You have hardware switches upstream.
• The hardware switches have VXLAN tunnelling configured on them.

VMware NSX for vSphere 6.2.x
VMware NSX for vSphere 6.3.x
VMware NSX for vSphere 6.4.x

This issue occurs because the hardware switches are using port 4789 for VXLAN and NSX-v uses the same port for VXLAN. This means that within the physical infrastructure, you will find nested VXLAN.

This is a known issue affecting VMware NSX for vSphere 6.2.x, 6.3.x and 6.4.x.

Currently, there is no resolution.

Note: Your physical hardware switch may not have support for nested VXLAN with both the inner and outer VXLAN encapsulation using the same port (4789).

Workaround:
To work around this issue, change the VXLAN port either in NSX or in the physical switches so that they are both not using port 4789.

Note: Starting with NSX 6.2.3, the default VXLAN port is 4789, the standard port assigned by IANA. Before NSX 6.2.3, the default VXLAN UDP port number was 8472. 

For more information on instructions to change the port in NSX-v to port 8472, see the Change VXLAN Port section of the NSX Upgrade Guide. If you are using Cross-vCenter NSX, see the Change VXLAN Port in Cross-vCenter NSX section of the NSX Upgrade Guide.