Migrating virtual machines with Raw Device Mappings (RDMs) (1005241)
Migrating virtual machines with RDMs can be performed in three ways:
- Warm migration (vMotion), with the virtual machine powered on.
- Cold migration, with the virtual machine powered off.
- Storage migration (Storage vMotion), with the virtual machine powered on.
When an RDM is removed from the virtual machine configuration, the device associated with this RDM becomes visible in the list of available devices during an attempt to increase the size of a VMFS datastore. This device can be selected during the process, which can overwrite the existing data present on that device.
To prevent such accidental lost, use one of these options:
- Inform all necessary parties involved to ensure these devices does not get selected when performing VMFS datastore size increase.
- Temporarily detach the device that has issue from all the ESXi hosts on the cluster.
Note: An HBA rescan can potentially reattach these devices.
- Files for a virtual machine are not relocated when it is migrated using vMotion.
- The virtual machine is re-registered to the destination host.
- Any RDMs remain as RDMs when the virtual machine is registered to another host. That is, no changes to the virtual machine itself are made.
Cold migrationWith file relocation:
- Any non-RDM virtual disks are physically moved to the destination.
- The virtual machine configuration files are physically moved to the destination.
- Raw LUNs themselves cannot be moved, as they are raw disks presented from the SAN. However, the pointer files (RDMs) can be relocated if required.
- When performing a cold migration of a virtual machine with RDMs attached to it, the contents of the raw LUN mapped by the RDM are copied into a new
.vmdkfile at the destination, effectively converting or cloning a raw LUN into a virtual disk. This also applies when the virtual machine is not moving between ESX hosts. In this process, your original raw LUN is left intact. However, the virtual machine no longer reads or writes to it. Instead, the newly-created virtual disk is used.
- If you want to cold migrate a virtual machine without cloning or converting its RDMs, remove them from the configuration of the virtual machine before migrating. You can delete the RDM from the disk when removing it (the raw LUN contents are not changed). Re-add them to the configuration when completed.
- For ESXi 5.x and later: During the migration you can use the Advanced section of the migration wizard and select if you want to maintain the same format of the files at the destination or convert it to a thick/thin disk.
- The virtual machine registration changes, but the files are left untouched.
CloningIf you want to clone a virtual machine without its RDMs, remove them from the configuration of the virtual machine before migrating. You can delete the RDM from the disk when removing it (the raw LUN contents are not changed, only the RDM mapping file is deleted). Re-add them to the configuration when completed.
- When you perform Storage vMotion, the virtual machine files are physically relocated to a destination datastore. The same host retains ownership or registration of the virtual machine after Storage vMotion completes.
- For Virtual Infrastructure 3.5, virtual disks and virtual and physical mode RDM pointer files can be relocated to the destination datastore but cannot be converted to thick-provisioned or thin-provisioned disks during migration. The RDM pointer files of the virtual machine remain as RDM pointer files when the process completes.
- For vSphere 4.0 and later:
- Virtual disks and virtual mode RDM pointer files can be relocated to the destination datastore.
- Virtual disks can be converted to thick-provisioned or thin-provisioned disks during migration as long as the destination is not an NFS datastore.
- Physical mode RDM pointer files can be relocated to the destination datastore, but physical mode RDM data cannot be migrated to a VMDK disk using Storage vMotion.
- Virtual mode RDM data can be migrated to a VMDK disk with Storage vMotion by selecting the Disk Format from Advance View, provided sufficient licensing is present.
- In ESXi 5.x and later, during the Storage vMotion operation, virtual mode RDM data can be migrated to new VMDK disks by selecting either thick or thin provisioning for the disk type under the Advanced section of the migration wizard.
When migrating Physical mode RDM pointer files that are greater than 2TB using Storage vMotion, you must use the vSphere Web Client.
Physical mode RDM data can be migrated to VMDK disks only via cold migration. For more information, see the Migrating RDMs, and a question for RDM Users blog post.
- If you do not change the destination format to thick or thin provisioned for an RDM in the advanced section of the migration wizard, only the RDM mapping file will be migrated to the destination datastore.
- If using N-Port ID Virtualization (NPIV), Storage vMotion is not supported. For more information, see the NPIV Capabilities and Limitations section in the vSphere Storage Guide.
- If you are attempting a Storage vMotion of a virtual mode RDM using the Advanced method, and the RDM pointer mapping file is already present in the target datastore, the Storage vMotion finishes quickly, but without moving the data. This is because the Storage vMotion detects that the source and target datastores for the mapping file are the same, and therefore concludes that no movement is needed.
To work around this issue:
- Remove the RDM from the virtual machine, and delete it from disk to ensure that the mapping file is deleted. This will not delete the data in the RDM.
- Add the RDM again, and put the mapping file in a different datastore than the ultimate target of the Storage vMotion.
- After the virtual machine reconfiguration is complete, attempt the Storage vMotion again.
Note: To avoid downtime, you can migrate the virtual mode RDM pointer file to an alternate datastore (that is, any other datastore than the actual destination datastore), then Storage vMotion the same pointer file back using either Thick Zeroed or Thick Eager Zeroed disk format. Using the
du -ahcommand, you can confirm if the pointer exists in the directory on the alternate datastore.
For more information on the various steps involved during the migration, see Migrating virtual machines (1017769).
For instructions on using Storage vMotion, see Moving virtual machines with Storage vMotion (1005544).
This table summarizes the available options and requirements:
|Storage vMotion (SvMotion)||VirtualCenter 2.5||vCenter Server 4.x/5.x||RDM virtual compatibility mode||RDM physical compatibility mode||The virtual machine can change hosts||Virtual machine Snapshots|
|ESXi/ESX 3.5||Supported using the remote CLI script:
||Yes||Yes||Yes||No||Virtual machine must not have snapshots while performing SvMotion.|
|ESXi/ESX 4.x||Not supported: VirtualCenter 2.5 cannot manage ESXi/ESX 4.x hosts.||Yes||Yes||Yes||Yes||Virtual machine must not have snapshots while performing SvMotion.|
|ESXi 5.0 and later
|Not supported: VirtualCenter 2.5 cannot manage ESXi 5.0 and later hosts as well as ESXi 6.0.x
||Only vCenter Server 5.x
Only vCenter Server 6.x
- Storage vMotion of virtual machines using virtual compatibility mode RDMs is supported. However, clustered applications or clustered virtual machine configurations do not support Storage vMotion.
.vmdkpointer files must reside on VMFS datastores. The NFS protocol does not support SCSI commands. For more information, see Unable to add a physical mode disk mapping (RDM) to a virtual machine stored on an NFS datastore (1001856).
- Storage vMotion Remote Command-Line Syntax section in the Basic System Administration Guide for your version of VMware ESXi/ESX.
- The vSphere Command-Line Interface Documentation.
Ensure that snapshots are committed before performing storage migrations, especially if you are removing mappings or disks from your virtual machine configuration and later re-adding them. During re-addition of the respective disk, the VMware Infrastructure or vSphere Client does not allow you to re-select a specific (the previous) snapshot level or
.vmdkfile to add back to the virtual machine configuration.
Block size and capacity limitations of your destination datastore
Raw Device Mappings occasionally represent large raw LUNs. If you clone a virtual machine with an RDM still attached, the contents of the raw LUN mapped by the RDM are copied into a new virtual disk (
.vmdk) file at the destination. Converting an RDM into a virtual disk (
.vmdk) file does not succeed if your destination or target datastore does not have the capability of storing single contiguous files as large as the raw LUN. Large virtual disks have the same requirements or constraints.
For example, if your destination datastore is VMFS-3 created with a 2 MB block size, and the RDM or virtual disk being converted or migrated is 640 GB, the process fails. A VMFS-3 block size of 4 MB or higher is necessary to store files larger than 512 GB. For more information, see Block size limitations of a VMFS datastore (1003565).
- Unable to add a physical mode disk mapping (RDM) to a virtual machine stored on an NFS datastore (1001856)