Symptoms:

• Multiple Virtual Machines (VMs) are been deployed at the same time (using automation systems such as VMware vRealize Automation).
• After the VMs are deployed, some VMs are not able to connect to the network and the NSX-T logical switch port for the impacted VMs have an operational state marked as "Down" in the NSX-T UI

Example screenshot show the VM ports as operational state marked as "Down":

• The impacted VM NVDS ports are marked as blocked

#net-dvs -l | grep -E "port |port.block|volatile.status"
port c2602212-cec6-4783-865a-2e9ae8fe6a71:
                com.vmware.common.port.block = true , propType = POLICY
                com.vmware.common.port.volatile.status = inUse linkUp blocked portID=67108871 Port blocked by admin propType = RUNTIME

VMware NSX-T Data Center 2.x
VMware NSX-T Data Center
VMware NSX-T

The issue is due to a software bug in the NSX-T Manager appliance which occurs when multiple VMs are deployed at the same time (batch processing). The issue is due to how the Controller component handles Virtual Network Interface (VIF) and Logical Switch Ports (LSP) association during batch processing. The result is that a VIF may not be connected to a LSP which would cause the VM port to be marked as blocked on the ESXi host and the VM not having network connectivity.

This issue is resolved in NSX-T 2.5.1.

Workaround:
The following workaround can be used to "unblock" the ports on the affected VMs:

1. Identify all the ports that are in a Blocked state, by running the below command
#net-dvs -l | grep -E "port |port.block|volatile.status"

2. Identify the port IDs for all connected interfaces on the ESXi host
#net-stats -l

3. Match the output of 2 and 3 to see what ports correspond to which vNic interfaces.

4. Identify the N-VDS switch name on the host
#esxcfg-vswitch -l

5. Run below command to unblock the ports which are in blocked state
#net-dvs -s com.vmware.common.port.block=false <N-VDS-NameFromStep4> -p <LogicalSwitchPor UUIDfromStep1>
Example:
#net-dvs -s com.vmware.common.port.block=false N-VDS-hostswitch -p c2602212-cec6-4783-865a-2e9ae8fe6a71

6. In a rare event, the VLAN of the port maybe lost as well. To re-add the VLAN back, run:
net-dvs -v <VLAN-ID> -p <LogicalSwitchPortUUIDfromStep1> <N-VDS-NameFromStep4>
Example:
net-dvs -v 2081 -p c2602212-cec6-4783-865a-2e9ae8fe6a71 N-VDS-hostswitch