Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Troubleshooting an unresponsive host and multiple Disconnected virtual machines (1019082)

Symptoms

  • A VMware ESX or ESXi host is grayed out.
  • Not Responding is appended to the host name. 
  • All virtual machines that are registered to the ESX or ESXi host are grayed out.
  • Disconnected is appended to the virtual machine name.
  • You are unable to connect to the ESX or ESXi host directly using vSphere Client.
  • The vpxd.log files residing in the vCenter Server may contain events indicating an error when attempting to communicate with the ESXi host. The events always contain the words vmodl.fault.HostCommunication and may appear similar to the following examples:

    [VpxLRO] -- ERROR task-internal-6433833 -- host-24499 -- vim.host.NetworkSystem.queryNetworkHint: vmodl.fault.HostCommunication:
    (vmodl.fault.HostCommunication) {
       dynamicType = <unset>,
       faultCause = (vmodl.MethodFault) null,
       msg = "",
    }

    [VpxdMoHost::CollectRemote] Stats collection cannot proceed because host may no longer be available or reachable: vmodl.fault.HostCommunication.

    The issue may appear on multiple hosts, keep note on the opID that identifes the offending ESX/ESXi host:

    2012-04-09T15:03:51.540-04:00 [29348 verbose 'Default' opID=f6a80d55] [ServerAccess] Attempting to connect to service at vc1.hostname.vmware.net:10443

    Note: For more information about this type of failure, see the vSphere SDK documentation.

  • In the event that this is the result of a communication issue between the ESXi host and the vCenter Server. If the ESX host is still responsive to user interaction you may also observe events similar to the following in the /var/log/vmware/vpxa.log files in the ESX or ESXi host.  Here is a sample of the events that may be reported:

    Failed to bind heartbeat socket (-1). Using any IP.

    Agent can't send heartbeats.msg size: 66, sendto() returned: Network is unreachable.

Purpose

This article provides troubleshooting steps to determine why an ESX host is inaccessible from vCenter Server or vSphere Client.

Resolution

To determine why an ESX/ESXi host is inaccessible:

  1. Verify the current state of the ESX/ESXi host hardware and power. Physically go to the VMware ESX/ESXi host hardware, and make note of any lights on the face of the server hardware that may indicate the power or hardware status. For more information regarding the hardware lights, consult the hardware vendor documentation or support.

    Note: Depending on the configuration of your physical environment, you may consider connecting to the physical host by using a remote hardware interface provided by your hardware vendor. For more information about how this interface interprets the condition of the hardware, consult the hardware vendor documentation or support.

  2. Determine the state of the ESX host's user interface in the physical console.

    Note: Depending on the configuration of your physical environment, you may consider connecting to the physical host by using a remote application such as a Keyboard/Video/Mouse switch or a remote hardware interface provided by your hardware vendor. These interfaces have been known to interfere with keyboard and mouse functionality. VMware recommends verifying the responsiveness at the local physical console prior to taking any action.
  3. Determine if the ESX host responds to ping responses. For more information, see Testing network connectivity with the Ping command (1003486) If you are using ESXi, there are several menu options provided to test the management network. If the ESX host responds to user interaction, but does not respond to pings, you may have a networking issue. For more information, see ESX/ESXi host has intermittent or no network connectivity (1004109).
     
  4. Verify that you can connect to the VMware ESX/ESXi host using vSphere Client:
    1. Open vSphere Client.
    2. Specify the hostname or IP address of the VMware ESX/ESXi host, along with the appropriate credentials for the root user.
    3. Click Login.
  5. Determine if the ESX host has been rebooted.
    1. Physically log in to the console of the VMware ESX host.
    2. Type the command uptime to view the uptime of the VMware ESX host. If the VMware ESX host has been recently rebooted, see Determining why a ESX/ESXi host was powered off or restarted (1019238).

Additional Information

For related information, see Determining why multiple virtual machines are inaccessible (1019000).

High Availability

High Availability (HA) feature uses a different trigger than vCenter Server when ensuring that an ESX or ESXi is operational. The following is a brief explanation of the each criteria:
  • The "Host connection and power state" alarm is triggered as a result of a "HostCommunication fault".  A "HostCommunication fault" occurs if a vCenter Server is unable to communicate to an ESX or ESXi host using the vSphere API.
  • The HA isolation response is triggered as a result of an agent on the ESX or ESXi host that is unable to communicate with agents on other ESX or ESXi hosts (not the vCenter server).  It must also fail to communicate with a designated isolation address (by default, it is the default gateway).  If both of these conditions are met, the host performs the designated HA isolation response.
Both systems are managed by different agents and may communicate with different hosts on the network. Therefore, with respect to the relationship:
  • A host that is Not Responding within vCenter Server does not always trigger a high availability isolation response. It may still be maintaining a network connection with other hosts or its isolation address and thus is not isolated .
  • A host experiencing an HA isolation response is likely to appear as Not Responding within vCenter Server.

Tags

client-connection host-connection host-hung host-hung not-responding root-cause-analysis

See Also

Update History

07/10/2012 - Updated for 5.x

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 19 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 19 Ratings
Actions
KB: