VMotion CPU Compatibility - Migrations Prevented Due to CPU Mismatch - How to Override Masks

Background

To determine if the source and target CPUs meet VMotion requirements, VirtualCenter and vCenter Server compare the target CPU to a default bit mask definition to exclude unimportant features from the comparison. As new features are introduced by processor vendors, the bit masks used by VirtualCenter and vCenter Server are updated. For example, in ESX Server 2.x, the NX (AMD) or XD (Intel) feature is not used to compute VMotion compatibility, but in ESX 3.x, it is.

To disable VMotion’s CPU-compatibility checking for a specific feature, you can modify the default bit mask. The bit mask-modification process is sometimes referred to in some VMware documents as “relaxing” a particular constraint.

A full discussion of the underlying components and how VirtualCenter Server and ESX Server host systems handle them is beyond the scope of this article. For more information, see "CPU Compatibility Masks" in Basic System Administration. Another useful resource is the technical paper on VMware VMotion and CPU Compatibility.

For specific information about Intel and AMD CPUs and VMotion compatibility, see:

• VMotion CPU Compatibility Requirements for Intel Processors (1991)
• VMotion CPU Compatibility Requirements for AMD Processors (1992)

CPU Identification Tools

Scope of Modifications

Per-Virtual-Machine Masking with ESX/ESXi 3.x and Later

1. Launch VI Client or vSphere Client and connect to the VirtualCenter Server or vCenter Server as an administrator.
2. Select the virtual machine that you want to migrate in the inventory.
3. Click Edit Settings on the Summary tab. The Properties window for the virtual machine appears.
4. Click the Options tab in the Properties window.
5. Select the CPUID Mask option to display the CPU Identification Mask information and settings.
6. Click Advanced to display several settings-related boxes, including CPU Identification Mask, in the right-hand pane.

Modifying Default NX/XD Mask

1. Navigate to the CPUID Mask option on the virtual machine Options tab (see steps above, if necessary).
2. Select Hide the NX flag from guest to disable the CPU compatibility check for the selected virtual machine, or Expose the NX flag to guest to enable this CPU compatibility check for the selected virtual machine.
3. Click OK to save the change.

Modifying Default Mask for Intel CPUs

Power off the virtual machine before modifying its bit mask.

To modify other CPU masks:

1. Navigate to the virtual machine Options tab (see steps above, if necessary).
2. Click Advanced to open the CPU Identification Mask properties dialog box.
   
   Note: The CPU Identification Mask dialog box has two tabs—Virtual Machine Default, and AMD Override. Most modifications for Intel CPU features are made on the Virtual Machine Default tab. Modifications for AMD CPU features are made on the AMD Overrides tab, and are discussed in the next section of this article.
   
   
3. Click the Virtual Machine Default tab to activate the dialog box, if necessary.

To modify the mask for a specific feature, enter the series of dashes and 0s as shown in the table below.

Feature	Level	Row	Mask
SSE3	1	ecx	---- ---- ---- ---- ---- ---- ---0 -0-0
SSSE3	1	ecx	---- ---- ---- -0-- ---- --0- ---0 -0--
	1	edx	---- ---- ---- --0- ---- ---- ---- ----
SSE4.1	1	ecx	---- ---- ---- 0--- ---- ---- ---- ----
SSE4.2	1	ecx	---- ---- ---0 ---- ---- ---- ---- ----

When all modifications are complete, click OK and exit the CPU Identification Mask dialog box.

Modifying Masks for AMD CPUs

Features that are specific to AMD processors are listed and can be changed on the AMD Override tab. Support terms for these feature masks are listed in KB 1992. In general, VMware neither supports nor recommends modifying the VMotion constraints for CPU features used by applications, such as SSE3, because of the risk of application or guest operating system failure after migration.

Power off the virtual machine before modifying its bit mask.

To modify other CPU masks for AMD Processors:

1. Navigate to the virtual machine Options tab (see steps above, if necessary).
2. Click Advanced to open the CPU Identification Mask properties dialog box.
3. Click the AMD Override tab to activate the dialog box. The AMD Override page displays.

To modify the mask for a specific feature, enter the series of dashes and 0s as shown in the table below.

Feature	Level	Row	Mask
SSE3	1	ecx	---- ---- ---- ---- ---- ---- ---- ---0
RDTSCP	80000001	edx	---- 0--- ---- ---- ---- ---- ---- ----
	80000001	ecx	---- ---- ---- ---- ---- ---- ---- 0---
CMPXCH16B	1	ecx	---- ---- ---- ---- --0- ---- ---- ----
FFXSR	80000001	edx	---- --0- ---- ---- ---- ---- ---- ----

When all modifications are complete click OK and exit the CPU Identification Mask dialog box.

Combining Mask Modifications

You can combine modifications to the default bit mask to allow migration with VMotion between groups that are incompatible based on more than one CPU feature. For example, to filter-out the compatibility check for both SSE3 and SSSE3 combine the following:

---- ---- ---- ---- ---- ---- ---0 -0-0

and:

---- ---- ---- -0-- ---- --0- ---0 -0--

to yield:

---- ---- ---- -0-- ---- --0- ---0 -0-0

for the ecx register mask.

Global Masking with ESX Server 2.x and Later

For ESX/ESXi hosts managed by VirtualCenter or vCenter Server, you can modify the bit mask globally by manually editing the VirtualCenter or vCenter Server configuration file (vpxd.cfg ). The configuration file, vpxd.cfg , contains XML tags defining various elements and settings for the VirtualCenter Server or vCenter Server system. Bit-masks can be modified by adding the appropriate XML tags to define the mask in the context of a guest operating system configuration option to the configuration file .

Changes to the global bit mask do not take effect for individual virtual machines until they have been powered off and powered on again.

Note: The vpxd.cfg file is located by default in C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter

Although a full discussion of building these elements is beyond the scope of this article, the About the Tags section provides a brief overview. See the following section for specific steps related to Editing the Configuration File and XML tags for Common Mask Patterns like, SSE3 , SSSE3 , and SSE3 and SSSE3 Combined .

Warning: VMware neither supports nor recommends modifying the VMotion constraints for CPU features used by applications, such as SSE3, because of the risk of application or guest operating system failure after migration.

About the Tags

The tags identify the register to which the mask applies, and for which hosts and which versions the mask applies. The tags are direct descendants of the <vpxd> tag, in the vpxd.cfg file. Tags are listed in the order in which they must be nested:

• <guestOSDescriptor> — Direct descendant (child) tag of <vpxd> element. The vpxd.cfg can contain a single guestOSDescriptor element.
• <host-product-and-version> — Between the opening and closing <guestOSDescriptor> tags, you can nest multiple host-product-and-version-tags that specify the version and host to which the masks you define apply. The <host-product-and-version> tags can be specified generally, as in <esx-2-x-x> for an ESX Server 2.x host, <esx-3-x-x> for an ESX/ESXi 3.x host or can be specified precisely, as in <esx-3-5-0> . For example, the tag <esx-2-5-x> identifies various maintenance release levels of the ESX Server to which the subsequently nested mask applies.
• <virtual-machine-configuration-hardware-version> — Allows for the configuration to be restricted by virtual machine hardware version. For example, <all-versions> , <vmx-03> , or <vmx-04> . In the example below <all-versions> is specified, meaning that the mask applies to all virtual machine hardware versions.
• <virtual-machine guest version> — Allows for a guest version to be specified. For example, <all-guests> , <winXPPro64Guest> , <winVistaGuest> , and so on. In the example shown below, <all-guests> is specified, and therefore the mask applies to all guest versions.
• <cpuFeatureMask> — This tag precedes the actual mask. The mask is then defined for <default-vendor> , <amd> , or <intel> , depending on your needs. Tag elements that are used to define the mask, identify the CPU mask. The element details include vendor, CPU ID level, and the CPU register. The valid choices for these are:

CPU vendor	<default-vendor>, <amd>, <intel>
CPU ID level	<level-0>, <level-1>, <level-80000000>, <level-80000001>
CPU register	<eax>, <ebx>, <ecx>, <edx>

Tags must be embedded in the order shown in the table. A CPU-vendor tag is followed by a CPU-ID-level tag, followed by a CPU-register tag. Immediately following the CPU-register tag is the sequence of 32 dashes and x-s that represent the actual 32-bit register mask appropriate for the feature being modified. The mask is then followed by the CPU-register-, CPU-ID-level-, and CPU-vendor-closing tags.

Note: A mask defined using <default-vendor> tag is used by the system only when no vendor-specific mask has been specified for the same CPU ID level.

• When you have finished defining masks, close with </guestOSDescriptor> .

Below is an example of the tag order, shown in the context of the vpxd.cfg file. The <config> and <vpxd> opening and closing tags are already in the vpxd.cfg file. The <guestOSDescriptor> opening tag can be placed directly after the <vpxd> opening tag, embed the appropriate tags to suit your needs, followed by the closing </guestOSDescriptor> tag.

<config>
<vpxd>
<guestOSDescriptor>
<esx-2-x-x>
<all-versions>
<all-guests>
<cpuFeatureMask>
[Elements and mask definition go in here. Common Mask Patterns can be copy-and-pasted from the below example patterns.]

</cpuFeatureMask>
</all-guests>
</all-versions>
</esx-2-x-x>
</guestOSDescriptor>
...

</vpxd>

</config>

In this example, any mask embedded in the [Elements and mask...] placeholder section applies to all ESX Server 2.x hosts being managed by the VirtualCenter Server, and to all guest operating systems.

Editing the VirtualCenter Configuration File

To edit the VirtualCenter Configuration File:

1. Navigate to the location of the vpxd.cfg file. By default, the vpxd.cfg file is located in the C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter directory.
2. Use a text editor (such as WordPad) to open the vpxd.cfg file. The vpxd.cfg file is an XML-formatted text file that contains various optional settings for the VirtualCenter Server or vCenter Server system.
3. Add opening and closing tags to identify the hosts, versions, and other specifics to which the mask applies.
4. Copy and paste from the examples in Common Mask Patterns, or apply other masks as needed for your deployment.
5. When the appropriate changes are made, restart the VMware VirtualCenter Server Service to enable the new masking.

Common Mask Patterns

The following are example masks for several common CPU feature masks. Copy, paste, and modify them as required for the environment. VMware does not support or recommend using the masks shown below, because of the risk of application or guest operating system failure after migration.

SSE3

<cpuFeatureMask>
<default-vendor>
<level-1>
<ecx>----:----:----:----:----:----:---x:-x-x</ecx>
</level-1>
</default-vendor>
<amd>
<level-1>
<ecx>----:----:----:----:----:----:----:---x</ecx>
</level-1>
</amd>
</cpuFeatureMask>

SSSE3

<cpuFeatureMask>
<default-vendor>
<level-1>
<ecx>----:----:----:-x--:----:--x-:---x:-x--</ecx>
<edx>----:----:----:--x-:----:----:----:----</edx>
</level-1>
</default-vendor>
</cpuFeatureMask>

SSE3 and SSSE3 Combined

<cpuFeatureMask>
<default-vendor>
<level-1>
<ecx>----:----:----:-x--:----:--x-:---x:-x-x</ecx>
<edx>----:----:----:--x-:----:----:----:----</edx>
</level-1>
</default-vendor>
<amd>
<level-1>
<ecx>----:----:----:----:----:----:----:---x</ecx>
</level-1>
</amd>
</cpuFeatureMask>

Reverting to the Default CPU Compatibility Bit Mask

Revert to the default VMotion compatibility constraints by reverting any changes made to the default mask. This can be accomplished by following the appropriate steps for the type of masking that was applied:

• When using per-virtual-machine masks, reset any changed rows to the defaults by clicking Reset All to Default on the CPU Identification Mask dialog box.
• When using global masks: Remove any added tags entered in the vpxd.cfg file and restart the VirtualCenter service.