Knowledge Base

The VMware Knowledge Base provides support solutions, error messages and troubleshooting guides
 
Search the VMware Knowledge Base (KB)   View by Article ID
 

Storage vMotion fails with a generic DMotion error (1009113)

Symptoms

  • Storage vMotion fails or is interrupted.
  • Storage vMotion fails at 99% and throws the following message: ERROR: A specified parameter was not correct. spec"
  • A generic DMotion error displays on the console in the Remote Command-Line Interface (RCLI).
  • In VirtualCenter or VMware Infrastructure (VI) Client connected directly to an ESX host, the virtual machine appears running or stopped in the Inventory.
  • If you right-click the virtual machine and select one of the hard disks, it references a DMotion file instead of the VMDK.
  • If you right-click the virtual machine in VirtualCenter, the Power and Snapshot options are not available.
  • If you right-click the virtual machine in VirtualCenter, the Complete Migration option is available, but it does not work.
  • The .vmx and vmware.log files are moved to the destination datastore but have duplicates in the original datastore. The files in the original datastore contain original virtual machine information.
  • The DMotion*-delta.vmdk and the VMName*-flat.vmdk files may remain on the source volumes or may be moved to the destination datastore.

Resolution

If this issue occurs, power-down the virtual machine and try to cold migrate the disks.
 
If cold migration fails, you must restore the original VMDK file.
 
To restore the original VMDK file without data loss, perform the following steps in order. If step 1 does not resolve the issue, proceed to Step 2. If Step 2 does not resolve the issue, proceed to Step 3. Do not skip a step.
 
If the virtual machine has independent persistent disks, you must shut the virtual machine down and change the setting to not independent. When you have gone through the following steps, you may revert the disk configuration back to being independent persistent. For more information, see Storage VMotion Requirements and Limitations in the Basic System Administration guide for your version of the VMware product.
 
Step 1
  1. If the orphaned virtual machine appears in the Inventory, remove it from VirtualCenter.

    Alternatively, try to re-register the virtual machine to the ESX host:

    1. Open VI Client and log in as root directly to the ESX host on which you want to register the virtual machine.
    2. Use the datastore browser and navigate to the .vmx file for the virtual machine.
    3. Right-click on the .vmx file and choose Add to Inventory.
    4. Use the wizard to register the virtual machine on the host. This entry replaces the orphaned entry in VirtualCenter.

      Note: If this procedure fails, then remove the virtual machine from the VirtualCenter Inventory.
       
  2. Verify the DMotion chain by checking the CID and Parent CID of the DMotion disks. Look at the .vmdk files of the virtual machine disks referenced in the Destination datastore .vmx file.

    Note
    : A virtual machine disk may not have a DMotion disk. You are looking for DMotion files that were left behind. 
     
  3. If the DMotion chains are not broken, run these commands on the destination .vmx file.
    1. To confirm that the virtual machine does not have any snapshots, run the command:

      vmware-cmd vm-cfg-path hassnapshot


      The output hassnapshot()= confirms that there are no snapshots.
       
    2. To create a valid snapshot, run the command:

      vmware-cmd vm-cfg-path createsnapshot name description quiesce memory
       
    3. To delete all snapshots, run the command:

      vmware-cmd vm-cfg-path removesnapshots


      This command commits the DMotion snapshot and the snapshot you created in step 3b.
Step 2
  1. From VI Client pointed at the ESX host directly, go into the snapshot manager and delete all snapshots.

    If the virtual machine is stuck in Virtual Center, you may be unable to remove the virtual machine from the inventory and may have to remove the ESX host that the virtual machine is registered to.

     
  2. Backup the . vmx file in the destination datastore.
     
  3. Edit the .vmx file to remove all references to DMotion parents:

    scsiX:X.DMotionParent = ""

    The destination .vmx file now contains accurate information.

     
  4. Delete the VMSD Snapshot Database file in the destination datastore.
     
  5. Remove the virtual machine from the ESX host with the command:

    vmware-cmd -s unregister vm-cfg-path

     
  6. If you removed the ESX host from Virtual Center, add it back.
     
  7. Add the virtual machine back to VirtualCenter using the datastore browser.
Note: If there are duplicate files (such as vmx, vmsd, logs, vmxf, nvram) in the source directory, they can now be deleted.
 
Step 3
  1. Use the vmkfstools -i command on the original disk to copy the hard drive.
     
  2. Remove the new VMDK files from the new datastore by removing the reference to them in the .vmx file.
     
  3. Open an SSH session to the host and run a command similar to:

    vmkfstools -i /src/original.vmdk /dest/copyof.vmdk 
     
     
  4. When the clone of the VMDK is completed, edit virtual machine settings to add the new copy of the original disk in the destination folder.
     
  5. Power on the virtual machine.

Additional Information

Tags

storage-vmotion storage-vmotion-fails

See Also

This Article Replaces

1005844, 1003985, 1006886.

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 35 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (4000 or fewer characters)
  • 35 Ratings
Actions
KB: