Determining the mapping between hostname and hostId in a VMware HA cluster
search cancel

Determining the mapping between hostname and hostId in a VMware HA cluster

book

Article ID: 342952

calendar_today

Updated On:

Products

VMware vCenter Server VMware vSphere ESXi

Issue/Introduction

This article outlines steps to determine the hostname of an ESXi host from the hostId located in the log files.


Symptoms:
  • An unexpected failover in the vSphere HA (High Availability) cluster occurs.
  • The vpxd logs on vCenter Server contains entries referencing hostIds similar to:

    Note: hostIds are host-1208 and host-1214.

    On the vCenter server (vpxd.log):, located by default under %ALLUSERSPROFILE%\Application Data\VMware\VMware VirtualCenter\Logs\

    T16:48:45.796+01:00 [01724 verbose 'DAS' opID=SWI-d75d1509] [FdmManager::GetHostList] host list version for vim.ClusterComputeResource:domain-c9 (Cluster): 21
    T16:48:45.796+01:00 [01724 verbose 'DAS' opID=SWI-d75d1509] [FdmManager::GetHostList] New host list is: (csi.HostList) {
    dynamicType = <unset>,
    faultDomainId = "44BA0E8A-D6E9-4902-86FE-A0BDE160ACF8-9-d409ae6-vCenter01",
    version = 21,
info 'Default'] [VpxdMoHost::UpdateDasState] VC state for host host-1208 (initialized -> initialized), FDM state (Live -> FDMUnreachable), src of state (host-406 -> host-406)
info 'Default'] [VpxdMoHost::UpdateDasState] VC state for host host-1214 (initialized -> initialized), FDM state (Live -> FDMUnreachable), src of state (host-406 -> host-406)


Note: The vpxd logs are located by default at:
    • Windows 2008: C:\ProgramData\VMware\VMware VirtualCenter\vpxd.cfg
    • Windows 2003: C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\vpxd.cfg
    • vCenter Server Appliance: /etc/vmware-vpx/vpxd.cfg
  • The /var/log/fdm.log on ESXi hosts contains log entries referencing hostIds similar to:

    T14:48:45.563Z [48D03B90 info 'Cluster' opID=SWI-d75d1509] [ClusterManagerImpl::StoreDone] Wrote hostlist version 21 verbose 'Cluster' opID=SWI-2bc801f9] [ClusterSlave::LiveCheck] Heartbeat still pending for slave @ host-1208
    verbose 'Cluster' opID=SWI-2bc801f9] [ClusterSlave::LiveCheck] Heartbeat still pending for slave @ host-1214

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.  


Environment

VMware vCenter Server 7.0.x
VMware vCenter Server 5.5.x
VMware vSphere ESXi 7.0.0
VMware vCenter Server 5.0.x
VMware vSphere ESXi 6.0
VMware vCenter Server 5.1.x
VMware vCenter Server 6.5.x
VMware vSphere ESXi 5.1
VMware vSphere ESXi 5.5
VMware vCenter Server 6.0.x
VMware vCenter Server 6.7.x
VMware vSphere ESXi 5.0
VMware vSphere ESXi 6.7
VMware vSphere ESXi 6.5

Cause

This issue occurs when the vCenter Server vpxd and ESXi host fdm logs reference hostIds rather than hostnames.

Resolution

To resolve this issue, determine the hostId from the vpxd.log file or the fdm.log file, match the hostId in the logs with the ESXi hostnames.
 
To determine the hostname of an ESXi host from the hostId located in the vpxd.log files:
  1. In the %ALLUSERSPROFILE%\Application Data\VMware\VMware VirtualCenter\Logs\vpxd log search for the Synchronizing host and the hostId.

    For example:

    T17:44:31.563Z [7FB34D20A700 info 'vpxdvpxdInvtHostCnx'] [VpxdHostSync] Synchronizing host: host-1208 of the host (server12345.domain.local, XX.XXX.XXX.XX)

    Where host-1208 corresponds to server 12345.
When vCenter Server logs are not accessible, determine the hostname of an ESXi host from the hostId located in the fdm.log files. Run the prettyPrint.sh script with the hostlist parameters on the ESXi host to display FDM configuration.
 
To determine the hostname of an ESXi server from the hostId located in the fdm.log files:
 
Note: Troubleshooting and further analysis of logs is performed when VMware have the hostname. It is essential to gather all the ESXi host log files as soon as possible after a failure.
 
To determine the hostname of an ESXi server from the hostId located in the /var/log/fdm.log:
  1. As user root on any host on the cluster open a Secure Shell (SSH) connection to the host.
  2. Run the command:

    /opt/vmware/fdm/fdm/prettyPrint.sh hostlist

    Note: /etc/opt/vmware/fdm/hostlist is an xml file and contains information on the hostnames, IP Addresses, MAC addresses and heartbeat datastore of hosts participating in the cluster.
    This is an example of the hostlist file, where you can determine the relationship between hostId and hostName, host-1208 from the vpxd / FDM logs corresponds to ESXi hostName server12345.domain.local.

    Warning: The prettyPrint.sh script is only available to display information. Do not modify these files for any reason. The cluster configuration is performed exclusively by the vSphere Client or by PowerCLI.

    <faultDomainId>44BA0E8A-D6E9-4902-86FE-A0BDE160ACF8-9-d409ae6-vcbr04</faultDomainId>
    <version>21</version>
    <hostId>host-1208</hostId>
    <hostName>server12345.domain.local</hostName>

    <sslThumbprint>21:E5:C9:24:9F:BC:CC:AF:84:E7:6A:95:02:4C:ED:22:8A:BD:D4:93</sslThumbprint>
    <ipAddress>xx.xx.xx.xxx</ipAddress>
    <mac>xx:xx:xx:xx:xx:xa</mac>
    <mac>xx:xx:xx:xx:xx:xb</mac>
    <mac>xx:xx:xx:xx:xx:xc</mac>
    <mac>xx:xx:xx:xx:xx:xd</mac>
    <mac>xx:xx:xx:xx:xx:xe</mac>
    <mac>xx:xx:xx:xx:xx:xf</mac>
    <mac>xx:xx:xx:xx:xx:xh</mac>
    <mac>xx:xx:xx:xx:xx:xi</mac>
    <heartbeatDatastore>/vmfs/volumes/5028de7e-36c10eaa-7037-0017a4770000</heartbeatDatastore>
    <heartbeatDatastore>/vmfs/volumes/4fbf41e5-da3ba7c6-993c-0017a4770460</heartbeatDatastore>
    <hostdPort>443</hostdPort>
    <version>5.0.0</version>
    <build>474610</build>
Note: The clusterconfig, compatlist and hostlist output of prettyPrint.sh have their own version mentioned between xml tags <version> </version>.
It is important to match the file version with the messages in the logs allowing for the time stamp in the fdm or vpxd logs.


Additional Information

On the ESXi host the /opt/vmware/fdm/fdm/prettyPrint.sh script displays information on the HA cluster. For example, the command syntax is:

prettyPrint.sh <hostlist|clusterconfig|compatlist|vmmetadata>

Note: Related FDM files are located in the directory /etc/opt/vmware/fdm.

Warning: The prettyPrint.sh script is only available to display information. Do not modify these files for any reason. The cluster configuration is performed exclusively by the vSphere Client or by PowerCLI.
 
For more information on collecting logs, see:
“vm-support” command in ESX/ESXi to collect diagnostic information
Collecting diagnostic information for VMware vCenter Server 4.x, 5.x and 6.x
Determining which virtual machines are restarted during a vSphere HA failover
HA/FDM fails to restart a virtual machine with the error: vim.fault.NoCompatibleHost
Determining hosts compatible for HA protected virtual machines
Collecting diagnostic information for VMware ESX/ESXi

Impact/Risks:
Warning: The prettyPrint.sh script is only available to display information. Do not modify these files for any reason. The cluster configuration is performed exclusively by the vSphere Client or by PowerCLI.