Search the VMware Knowledge Base (KB)
View by Article ID

Migrating virtual machines with Raw Device Mappings (RDMs) (1005241)

  • 110 Ratings
Language Editions

Purpose

This article discusses some of the common questions that arise when migrating virtual machines that use Raw Device Mappings (RDMs).

Migrating virtual machines with RDMs can be performed in three ways:
  • Warm migration (vMotion), with the virtual machine powered on.
  • Cold migration, with the virtual machine powered off.
  • Storage migration (Storage vMotion), with the virtual machine powered on.

Resolution



Warning:
When an RDM is removed from the virtual machine configuration, the device associated with this RDM becomes visible in the list of available devices during an attempt to increase the size of a VMFS datastore. This device can be selected during the process, which can overwrite the existing data present on that device.

To prevent such accidental lost, use one of these options:

  • Inform all necessary parties involved to ensure these devices does not get selected when performing VMFS datastore size increase.
  • Temporarily detach the device that has issue from all the ESXi hosts on the cluster.

    Note: An HBA rescan can potentially reattach these devices.

vMotion

  • Files for a virtual machine are not relocated when it is migrated using vMotion.
  • The virtual machine is re-registered to the destination host.
  • Any RDMs remain as RDMs when the virtual machine is registered to another host. That is, no changes to the virtual machine itself are made.

Cold migration

With file relocation:
  • Any non-RDM virtual disks are physically moved to the destination.
  • The virtual machine configuration files are physically moved to the destination.
  • Raw LUNs themselves cannot be moved, as they are raw disks presented from the SAN. However, the pointer files (RDMs) can be relocated if required.
  • When performing a cold migration of a virtual machine with RDMs attached to it, the contents of the raw LUN mapped by the RDM are copied into a new .vmdk file at the destination, effectively converting or cloning a raw LUN into a virtual disk. This also applies when the virtual machine is not moving between ESX hosts. In this process, your original raw LUN is left intact. However, the virtual machine no longer reads or writes to it. Instead, the newly-created virtual disk is used.
  • If you want to cold migrate a virtual machine without cloning or converting its RDMs, remove them from the configuration of the virtual machine before migrating. You can delete the RDM from the disk when removing it (the raw LUN contents are not changed). Re-add them to the configuration when completed.
  • For ESXi 5.x and later: During the migration you can use the Advanced section of the migration wizard and select if you want to maintain the same format of the files at the destination or convert it to a thick/thin disk.
  • The virtual machine registration changes, but the files are left untouched.

Cloning

If you want to clone a virtual machine without its RDMs, remove them from the configuration of the virtual machine before migrating. You can delete the RDM from the disk when removing it (the raw LUN contents are not changed, only the RDM mapping file is deleted). Re-add them to the configuration when completed.

Storage vMotion

  • When you perform Storage vMotion, the virtual machine files are physically relocated to a destination datastore. The same host retains ownership or registration of the virtual machine after Storage vMotion completes.
  • For Virtual Infrastructure 3.5, virtual disks and virtual and physical mode RDM pointer files can be relocated to the destination datastore but cannot be converted to thick-provisioned or thin-provisioned disks during migration. The RDM pointer files of the virtual machine remain as RDM pointer files when the process completes.
  • For vSphere 4.0 and later:

    • Virtual disks and virtual mode RDM pointer files can be relocated to the destination datastore.
    • Virtual disks can be converted to thick-provisioned or thin-provisioned disks during migration as long as the destination is not an NFS datastore.
    • Physical mode RDM pointer files can be relocated to the destination datastore, but physical mode RDM data cannot be migrated to a VMDK disk using Storage vMotion.
    • Virtual mode RDM data can be migrated to a VMDK disk with Storage vMotion by selecting the Disk Format from Advance View, provided sufficient licensing is present.
Notes:
  • In ESXi 5.x and later, during the Storage vMotion operation, virtual mode RDM data can be migrated to new VMDK disks by selecting either thick or thin provisioning for the disk type under the Advanced section of the migration wizard.
  • When migrating Physical mode RDM pointer files that are greater than 2TB using Storage vMotion, you must use the vSphere Web Client.

  • Physical mode RDM data can be migrated to VMDK disks only via cold migration. For more information, see the Migrating RDMs, and a question for RDM Users blog post.

  • If you do not change the destination format to thick or thin provisioned for an RDM in the advanced section of the migration wizard, only the RDM mapping file will be migrated to the destination datastore.
  • If using N-Port ID Virtualization (NPIV), Storage vMotion is not supported. For more information, see the NPIV Capabilities and Limitations section in the vSphere Storage Guide.
  • If you are attempting a Storage vMotion of a virtual mode RDM using the Advanced method, and the RDM pointer mapping file is already present in the target datastore, the Storage vMotion finishes quickly, but without moving the data. This is because the Storage vMotion detects that the source and target datastores for the mapping file are the same, and therefore concludes that no movement is needed.

    To work around this issue:

    1. Remove the RDM from the virtual machine, and delete it from disk to ensure that the mapping file is deleted. This will not delete the data in the RDM.
    2. Add the RDM again, and put the mapping file in a different datastore than the ultimate target of the Storage vMotion.
    3. After the virtual machine reconfiguration is complete, attempt the Storage vMotion again.

      Note: To avoid downtime, you can migrate the virtual mode RDM pointer file to an alternate datastore (that is, any other datastore than the actual destination datastore), then Storage vMotion the same pointer file back using either Thick Zeroed or Thick Eager Zeroed disk format. Using the du -ah command, you can confirm if the pointer exists in the directory on the alternate datastore.

For more information on the various steps involved during the migration, see Migrating virtual machines (1017769).

For instructions on using Storage vMotion, see Moving virtual machines with Storage vMotion (1005544).

This table summarizes the available options and requirements:

Storage vMotion (SvMotion) VirtualCenter 2.5 vCenter Server 4.x/5.x RDM virtual compatibility mode RDM physical compatibility mode The virtual machine can change hosts Virtual machine Snapshots
ESXi/ESX 3.5 Supported using the remote CLI script:
svmotion.pl
Yes Yes Yes No Virtual machine must not have snapshots while performing SvMotion.
ESXi/ESX 4.x Not supported: VirtualCenter 2.5 cannot manage ESXi/ESX 4.x hosts. Yes Yes Yes Yes Virtual machine must not have snapshots while performing SvMotion.
ESXi 5.0 and later

ESXi 6.0.x
Not supported: VirtualCenter 2.5 cannot manage ESXi 5.0 and later hosts as well as ESXi 6.0.x
Only vCenter Server 5.x
Only vCenter Server 6.x
Yes Yes Yes Yes

Notes:
For information about the Storage vMotion CLI script, see:
For Storage vMotion requirements and limitations, see the Storage vMotion Requirements and Limitations section of the Basic System Administration Guide for your version of ESXi/ESX.

Additional Information

Snapshots

Ensure that snapshots are committed before performing storage migrations, especially if you are removing mappings or disks from your virtual machine configuration and later re-adding them. During re-addition of the respective disk, the VMware Infrastructure or vSphere Client does not allow you to re-select a specific (the previous) snapshot level or .vmdk file to add back to the virtual machine configuration.

Block size and capacity limitations of your destination datastore

Raw Device Mappings occasionally represent large raw LUNs. If you clone a virtual machine with an RDM still attached, the contents of the raw LUN mapped by the RDM are copied into a new virtual disk (.vmdk) file at the destination. Converting an RDM into a virtual disk (.vmdk) file does not succeed if your destination or target datastore does not have the capability of storing single contiguous files as large as the raw LUN. Large virtual disks have the same requirements or constraints.

For example, if your destination datastore is VMFS-3 created with a 2 MB block size, and the RDM or virtual disk being converted or migrated is 640 GB, the process fails. A VMFS-3 block size of 4 MB or higher is necessary to store files larger than 512 GB. For more information, see Block size limitations of a VMFS datastore (1003565).

Tags

storage vmotion rdm,RDM to VMDK conversion failed,Move vm from one host to another with RDM disk

See Also

Update History

04/08/2010 - Added the sections: Snapshots, Physical and virtual compatibility mode Raw Device Mappings, and Block size and capacity limitations of your destination datastore. 05/03/2010 - Added table and links to more information. 11/04/2010 - Removed second section from the Additional Information area regarding Physical RDMs; it was incorrect or misleading, per PR 620232. 06/23/2011 - Added link to 1006599 in the See Also section 11/1/2012 - Added link to 1017769 under storage vMOtion and also updated the table with VC 5.x. Updated the cold migration and Storage vMotion section. 05/18/2012 - Added information that RDM pointer files must reside on VMFS datastores and added link to KB 1001856 11/26/2012 - Added ESXi and vCenter Server 5.1 7/24/2014 - Added Warning under the Resolution section.

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 110 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.

What can we do to improve this information? (4000 or fewer characters)




Please enter the Captcha code before clicking Submit.
  • 110 Ratings
Actions
KB: