Knowledge Base

Search the Knowledge Base:
Products:
Search In:
 

Virtual machines may unexpectedly reboot when using VMware HA with Virtual Machine Monitoring on ESX 3.5 Update 3

Symptoms

  • Virtual machines may unexpectedly reboot after a VMotion migration to an ESX 3.5 Update 3 (build 123630) host.
  • Virtual machines may unexpectedly reboot after a Power On operation on an ESX 3.5 Update 3 (build 123630) host, when the VMware High Availability (HA) feature with virtual machine monitoring is active.
  • This issue may following the periodic vpxuser password update and service restart, as shown in the logs:

    [2009-06-03 11:18:55.759 'TaskManager' 110087088 info] Task Created : haTask-ha-folder-root-vim.host.LocalAccountManager.updateUser-1870238977
    [2009-06-03 11:19:18.132 'App' 3076440992 info] END SERVICES ....
    [2009-06-03 11:20:45.968 'App' 65657776 verbose] [vpxaVmFailover::NotifyVmFailover] VM 2, time 1244042445, failure type 0, interval 40, uptime 4410909
    [2009-06-03 11:20:45.968 'App' 65657776 verbose] [vpxaVmFailover::NotifyVmFailover] VM 6, time 1244042445, failure type 0, interval 40, uptime 6678154
    [2009-06-03 11:20:45.968 'App' 65657776 verbose] [vpxaVmFailover::NotifyVmFailover] VM 7, time 1244042445, failure type 0, interval 40, uptime 5941025
    [2009-06-03 11:20:45.968 'App' 65657776 verbose] [vpxaVmFailover::NotifyVmFailover] VM 8, time 1244042445, failure type 0, interval 40, uptime 66781 54
    [2009-06-03 11:20:45.968 'App' 65657776 verbose] [vpxaVmFailover::NotifyVmFailover] VM 9, time 1244042445, failure type 0, interval 40, uptime 4737749

Purpose

A virtual machine may reboot itself if:
  • The virtual machine is running on a ESX 3.5 Update 3 host, either by virtue of VMotion or a Power On operation

AND

  • The host has VMware HA enabled with the virtual machine monitoring option active.

Virtual machine monitoring is dependent on VMware tools heartbeats to determine the state of the virtual machines.

With ESX Server 3.5 Update 3 after a VMotion or a Power On operation, the host agent running on the ESX server may delay sending the heartbeat state of the virtual machine to the Host. VMware HA detects this as a failure of the virtual machine and attempts to restart the virtual machine.

Resolution

The issue has been resolved in VirtualCenter 2.5 Update 4.  If you are at an earlier version, perform one of the following workarounds.
 
Workaround 1: Disabling virtual machine monitoring
 
To disable virtual machine monitoring:
  1. Right-click the VMware HA cluster and choose Edit Settings.

    Note: T    his feature can also be enabled for a new cluster on the VMware HA page of the New Cluster wizard.

  2. In the Cluster Settings dialog box, select VMware HA in the left column.
  3. Deselect Enable virtual machine monitoring.
  4. Click OK.
Workaround 2: Setting hostd heartbeat delay to 0
 
Note: Before you begin please refer to Restarting the Management agents on an ESX Server (1003490) for important information on restarting the mgmt-vmware service.
 
To set the hostd heartbeat delay to 0:
  1. Disconnect the host from VirtualCenter. Right-click on host in VMware Infrastructure (VI) Client and choose Disconnect.
  2. Log in as root to the ESX host with an SSH client.
  3. Using a text editor open the file /etc/vmware/hostd/config.xml.
  4. Set the heartbeatDelayInSecs tag under vmsvc to 0 seconds:

    <vmsvc>
    <heartbeatDelayInSecs>0</heartbeatDelayInSecs>
    <enabled>true</enabled>
    </vmsvc>

  5. Save your changes and exit.
  6. Restart the management agents for this change to take effect. For more information, see Restarting the Management agents on an ESX Server (1003490).
  7. Reconnect the host in VirtualCenter. Right-click on the host in VI Client and choose Connect.
Notes:
  • Setting heartbeatDelayInSecs to 0 does not disable virtual machine monitoring.
  • When any ESX patches are applied, changes to the config.xml file are over-written.

Additional Information

If you perform workaround 2, the heartbeat alarm in VirtualCenter (for virtual machines running on the ESX 3.5 Update 3 host) may go red until the VMware Tools inside GOS starts to heartbeat.

Feedback

Rating: 1 - Lowest 2 3 4 5 - Highest (18 Ratings)   

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (2000 or fewer characters)
Submit
Rating: 1 - Lowest 2 3 4 5 - Highest (18 Ratings)   
Actions
  • KB Article: 1007899
  • Updated: Aug 14, 2009
  • Products:
    VMware ESX
    VMware ESXi
    VMware VirtualCenter
  • Product Versions:
    VMware ESX 3.5.x
    VMware VirtualCenter 2.5.x