The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
ESXi/ESX hosts in APD may appear Not Responding in vCenter Server (1030980)
- The ESXi/ESX host appears as
Not Respondingin the vCenter Server inventory.
- Virtual machines utilizing the LUNs in APD may become unresponsive.
- Connecting to the ESXi/ESX host using the vSphere Client, vCLI, or PowerCLI fails.
- Adding a host to vCenter Server fails with error:
Failed to read resource pool tree from host
- Connecting to the ESXi/ESX host using SSH is successful.
vmware-hostdmanagement service is running.
- Connecting to the
vmware-hostdmanagement service using the
- The last line in the
verbose 'FSVolumeProvider'] RefreshVMFSVolumes called
ValidationDetermine whether there are any LUNs in an All-Paths-Down (APD) state on an ESXi/ESX host:
- Open a console to the ESXi/ESX host. For more information, see Unable to connect to an ESX host using Secure Shell (SSH) (1003807) or Using Tech Support Mode in ESXi 4.1 and ESXi 5.0 (1017910).
- Use the
esxcfg-mpathcommand to obtain a list of all device paths, and filter by their State:
# esxcfg-mpath --list-paths --device <device mpx/naa name> | grep state
If you do not know the problem device ID or you have many devices it may be more efficient to use this command to identify the dead paths:
# esxcfg-mpath -b | grep -C 1 dead
- If any path reports the State
dead, but other paths to the same device report the state
Up, perform a rescan to remove the stale device entries. For more information, see Performing a rescan of the storage on an ESXi/ESX host (1003988).
- If every path to a LUN reports a State of
dead, then the LUN is in an All-Paths-Down state.
Preemptive workaroundIf the APD condition is noticed prior to any process opening a file on the affected VMFS datastores, the impending blocking I/O can be fast-failed by setting the advanced configuration option
VMFS3.FailVolumeOpenIfAPD = 1on ESXi/ESX 4.1. For more information, see Configuring advanced options for ESX/ESXi (1038578).
In situations where any dead path or APD is noticed, individual HBAs can be rescanned using the following command:
# esxcfg-rescan -dvmhbaX
Note: Replace vmhbaX with the appropriate HBA, for example vmhba33.
In ESX/ESXi 4.1 and later, all HBAs can be rescanned using the following command:
# esxcfg-rescan -A
Note: If any device is already in an APD condition with active I/O already waiting for the device to return, setting this option does not cause the already-issued I/O to fail. It is necessary to either bring the LUN paths back up, or to wait for the I/O to eventually fail.
For more information, see Virtual machines stop responding when any LUN on the host is in an all-paths-down (APD) condition (1016626).
To avoid the APD state on an ESXi/ESX host, be sure to use the correct method to unpresent the LUNs. For more information on the correct procedure for unpresenting LUNs, see Removing a LUN containing a datastore from VMware ESXi/ESX 4.x (1029786).
- Unable to connect to an ESX host using Secure Shell (SSH)
- Performing a rescan of the storage on an ESX/ESXi host
- Virtual machines stop responding when any LUN on the host is in an all-paths-down (APD) condition
- Using Tech Support Mode in ESXi 4.1 and ESXi 5.x
- Removing a LUN containing a datastore from VMware ESXi/ESX 4.x
- Configuring advanced options for ESX/ESXi
- Unmounting a LUN or Detaching a Datastore/Storage Device from multiple ESXi 5.x hosts
- Permanent Device Loss (PDL) and All-Paths-Down (APD) in vSphere 5.0
Request a Product Feature
To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.