Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

vMotion fails at 10% with the error: Operation timed out (2007343)

Symptoms

  • When you attempt to vMotion a virtual machine, the task fails at 10%.

  • In virtual machine's vmware.log file, you see these errors:

    vmx| MigrateStatusFailure: Now in new log file.
    vmx| MigrateStatusFailure: Timed out waiting for migration start request.


  • In the source ESXi/ESX host's hostd.log file, you see errors similar to:

    vm:/vmfs/volumes/<Datastore>/<VM>.vmx' 128215984 warning] Failed to find activation record, event user unknown.

    Message on <VM> on <HOST> in ha-datacenter: Cannot open/create log file '/vmfs/volumes/<Datastore>/<VM>/vmware.log': Device or resource busy

    Unable to open log file "/vmfs/volumes/<Datastore>/<VM>/vmware.log". Check your configuration to make sure that the path specified for the log file is valid, and that you have write privileges in this directory.


  • In the destination ESXi/ESX host's VMkernel file, you see these errors multiple times:

    Checking if lock holders are live for lock [type 10c00001 offset 22906880 v 235, hb offset 4097536

    Exceeded retries removing vmware-0.log from f530 28 2 47c7a4e6 d89cf356 1400ffb9 56471c5e 3c0c244 14 0 0 0 0 0


  • When you examine the virtual machine's .vmx file, you see that the vmware.log rotation size is set to a very low value. For example:

    log.rotateSize = "500"

Cause

By default, the virtual machine log file (vmware.log) is rotated as a result of the virtual machine's Power On or Power Off operation. To configure log rotation based on file size, the log.rotateSize line is included in the virtual machine's .vmx file. For example:

log.rotateSize = <maximum size in bytes the file can grow to>

When the log.rotateSize value in the virtual machine's .vmx file is set to a very low value, it causes the vmware.log file to rotate so quickly that by the time the destination host is requesting the vmware.log file's VMFS lock, it has already rotated through and there is a new vmware.log file associated with a new inode in the file system. The destination host is then unable to acquire a proper file lock, and this causes the vMotion failure.

Resolution

To resolve this issue, modify the log.rotateSize value to a larger number to prevent the log file from rotating too quickly:

  1. Log into the ESXi/ESX host or the vCenter Server using the vSphere Client.
  2. Power off the virtual machine.
  3. Click Edit Settings on the virtual machine's Summary page, then click the Options tab.
  4. Under Advanced, click General.
  5. Click Configuration Parameters > Configuration.
  6. Find the log.rotateSize parameter and change the setting to a larger value.

    Note: Do not change the log.keepOld setting to a large number or the datastore may fill up. The default value for log.keepOld is 6.

  7. Click OK.
  8. Power on the virtual machine.
  9. Verify that the issue is resolved by using vMotion to migrate the virtual machine.

Impact/Risks

Be cautious about making major changes to your vmware.log settings, as they could easily cause your datastores to fill up.

Additional Information

See Also

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 1 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 1 Ratings
Actions
KB: