VMotion CPU Compatibility - Migrations Prevented Due to CPU Mismatch - How to Override Masks

• Background
  
  
  • Scope of Modifications
    
    
  • CPU Identification Tools
    
    
• VirtualCenter 1.x
  
  
  • Modifying the Default Mask
    
    
• VirtualCenter Server 2.x
  
  
  • Per Virtual Machine Masking with ESX Server 3.x
    
    
    • Modifying the NX/XD Mask
      
      
    • Modifying the Default Mask
      
      
    • Modifying the AMD Mask
      
      
  • Global Masking with ESX Server 2.x and ESX Server 3.x
    
    
    • About the Tags
      
      
    • Editing the Configuration File
      
      
    • Common Mask Patterns
      
      
• Reverting to the Default CPU Compatibility Bit Mask
  

Background

Compatible CPUs for VMotion purposes means that source and target CPUs must have the same manufacturer (Intel or AMD) and be members of the same basic processor family, Pentium 4 or Core, for example. Within a given processor family, the source and target CPUs must also have certain common and extended features implemented (or not implemented, depending on the specific feature) for VMotion to succeed.

To determine if the source and target CPUs meet VMotion requirements, VirtualCenter compares the target CPU to a default bit mask definition. The default bit mask has evolved with each version of VirtualCenter to support (or preclude) VMotion migration given a specific set of CPU features. For example, the VirtualCenter 1.x bit mask does not flag the NX (Intel) or XD (AMD) bits, but the VirtualCenter 2.x bit mask does.

At the same time, manufacturers (Intel and AMD, specifically) continually improve their CPUs, releasing new features that may or may not be compatible with VMotion.

To circumvent VMotion’s CPU-compatibility checking for a specific feature or extended feature, you can modify the default bit mask. The bit mask-modification process (sometimes referred to in various VMware documents as “relaxing” a particular constraint) varies according to VirtualCenter Server version and ESX version.

A full discussion of the underlying components and how VirtualCenter Server and ESX Server host systems handle them is beyond the scope of this article. For more information, see CPU Compatibility Masks and Configuring Virtual Machines in the Basic System Administration Guide. Another useful resource is the technical paper on VMware VMotion and CPU Compatibility.

For specific information about Intel and AMD CPUs and VMotion compatibility, see:

• VMotion CPU Compatibility Requirements for Intel Processors (1991)
• VMotion CPU Compatibility Requirements for AMD Processors (1992)

Scope of Modifications

The process of modifying VMotion compatibility settings depends on the VirtualCenter version and the ESX Server host system running the virtual machines. Also varying by VirtualCenter Server version and ESX Server version, is the scope of modifications to the default bit mask, as follows:

• VirtualCenter 1.x—Modifications to the default bit mask apply to all ESX Server 2.x hosts being managed by VirtualCenter.
• VirtualCenter 2.x with ESX Server 3.x—Modifications must be made on a per-virtual-machine basis using the VI Client.
• VirtualCenter 2.0.1 Patch 2 (and later) with ESX Server 2.x or ESX Server 3.x—Modifications are made at the VirtualCenter Server level but can be defined by processor manufacturer, by version number, and other criteria.
• VirtualCenter 2.5.x (and later)—Modifications to the CPU mask must be made while the virtual machine is powered off due to a risk of application or guest operating system instability.
• VirtualCenter 2.5.0 Update 2 (and later)—Adds support for Enhanced VMotion Compatibility(EVC). EVC masks are applied before per-virtual-machine masks and global VirtualCenter server masks.

CPU Identification Tools

To obtain information about a host system’s CPU, you can use the appropriate vendor’s tool or VMware’s CPUID tool, depending on the system:

• VMware’s ESX Server 3.x CD-ROM includes an ISO image file (\images\cpuid.iso.gz ) that can be uncompressed and used to create a bootable CD-ROM that provides CPU information about a given host prior to operating system (or ESX Server) installation. If you have questions about obtaining or using this tool, contact VMware Technical Support.
• Intel’s processor identification utility can be obtained from Intel at:
  
  http://www.intel.com/support/processors/tools/piu/
  
  
• AMD’s processor identification utility can be obtained from AMD at:
  
  http://www.amd.com/us-en/Processors/TechnicalResources/0,,30_182_871_9706,00.html
  

VirtualCenter Server 1.x

VirtualCenter Server 1.x bit mask applies to all virtual machines on the ESX Server 2.x host system. VirtualCenter Server 1.x uses a locally stored text file, config.ini , for configuration information.

Modifying the Default Bit Mask

VirtualCenter Server 2.x and 2.5.x 

VirtualCenter Server 2.x provides two different ways to modify VMotion CPU constraints, depending on the version of ESX Server being used:

Per-Virtual-Machine Masking with ESX Server 3.x

In Virtual Center 2.x, the default bit mask for VMotion constraints among ESX Server 3.x systems can be applied on a per-virtual-machine basis using the VI Client. Regardless of CPU type or mask modifications, accessing the necessary dialog boxes is generally the same, as follows:

1. Launch VI Client and connect to the VirtualCenter Server as an administrator.
2. Click the Virtual Machine that you want to migrate from Inventory.
3. Click Edit Settings on the Summary tab. The Properties window for the virtual machine displays.
4. Click the Options tab in the Properties window.
5. Click the CPUID Mask option to display the CPU Identification Mask information and settings.
6. Click Advanced to display several settings-related boxes, including CPU Identification Mask, in the right-hand pane.

Modifying Default NX/XD Mask

1. Navigate to the virtual machine Options tab (see steps above, if necessary).
2. Select Hide the NX flag from guest or the Expose the NX flag to guest to disable or enable this CPU compatibility check for the selected virtual machine.
3. Click OK to save the change.
   
   Note: The NX/XD mask can only be modified when the virtual machine is powered off.

Modifying Default Mask

1. Navigate to the virtual machine Options tab (see steps above, if necessary).
2. Click Advanced to open the CPU Identification Mask properties dialog box.
   
   Note: The CPU Identification Mask dialog box has two tabs—Virtual Machine Default, and AMD Override. Most modifications for Intel CPU features are made on the Virtual Machine Default page. Modifications for AMD CPU features are made on the AMD Overrides page and are discussed in the next section of this article.
   
   
3. Click the Virtual Machine Default tab to activate the dialog box, if necessary.

To modify the mask for a specific feature, enter the series of dashes and 0s as shown in the table below.

Modifying AMD-Specific Masks

1. Navigate to the virtual machine Options tab (see steps above, if necessary).
2. Click Advanced to open the CPU Identification Mask properties dialog box.
3. Click the AMD Override tab to activate the dialog box. The AMD Override page displays.

To modify the mask for a specific feature, enter the series of dashes and 0s as shown in the table below.

Combining Mask Modifications

Global Masking with ESX Server 2.x and ESX Server 3.x

About the Tags

• <guestOSDescriptor> —Direct descendant (child) tag of <vpxd> element. The vpxd.cfg can contain a single guestOSDescriptor element.
• <host-product-and-version> —Between the opening and closing <guestOSDescriptor> tags, you can nest multiple host-product-and-version-tags that specify the version and host to which the masks you define apply. For ESX Server hosts, the <host-product-and-version> tags can be specified generally, as in <esx-2-x-x> for an ESX Server 2.x host, <esx-3-x-x> for an ESX Server 3.x host or can be specified precisely—<esx-3-5-0> , for example. The tag <esx-2-5-x> identifies various maintenance release levels of the ESX Server to which the subsequently nested mask applies.
• <virtual-machine-configuration-hardware-version> —Allows for the configuration to be restricted by virtual machine hardware version. For example, <all-versions> , <vmx-03> , or <vmx-04> . In the example below <all-versions> is specified, meaning that the mask applies to all virtual machine hardware versions.
• <virtual-machine guest version> —Allows for a guest version to be specified. For example, <all-guests> , <winXPPro64Guest> , <winVistaGuest> , and so on. In the example shown below, <all-guests> is specified, and therefore the mask applies to all guest versions.
• <cpuFeatureMask> —This tag precedes the actual mask. The mask is then defined for <default-vendor> , <amd> , or <intel> , depending on your needs. Tag elements that are used to define the mask, identify the CPU mask. The element details include vendor, CPU ID level, and the CPU register. The valid choices for these are:

CPU vendor	<default-vendor>, <amd>, <intel>
CPU ID level	<level-0>, <level-1>, <level-80000000>, <level-80000001>
CPU register	<eax>, <ebx>, <ecx>, <edx>

Tags must be embedded in the order shown in the table. A CPU-vendor tag is followed by a CPU-ID-level tag, followed by a CPU-register tag. Immediately following the CPU-register tag is the sequence of 32 dashes and x-s that represent the actual 32-bit register mask appropriate for the feature being modified. The mask is then followed by the CPU-register-, CPU-ID-level-, and CPU-vendor-closing tags.

Note: A mask defined using <default-vendor> tag is used by the system only when no vendor-specific mask has been specified for the same CPU ID level.

• When you have finished defining masks, close with </guestOSDescriptor> .

Editing the VirtualCenter Configuration File

1. Navigate to the location of the vpxd.cfg file. By default, the vpxd.cfg file is located in the C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter directory.
2. Use a text editor (such as WordPad) to open the vpxd.cfg file. The vpxd.cfg file is an XML-formatted text file that contains various optional settings for the VirtualCenter Server.
3. Add opening and closing tags to identify the hosts, versions, and other specifics to which the mask applies.
4. Copy and paste from the examples in Common Mask Patterns, or apply other masks as needed for your deployment.
5. When the appropriate changes are made, restart the VMware VirtualCenter Server Service to enable the new masking.

Common Mask Patterns

The following are example masks for several common CPU feature masks. Copy, paste, and modify them as required for the environment. VMware does not support or recommend modifying the VMotion constraints for CPU extended features because of the risk of application or guest operating system failure after migration.

SSE3

SSSE3

Reverting to the Default CPU Compatibility Bit Mask

• For VirtualCenter 1.x, remove any of the lines added to the config.ini file, and restarting the VirtualCenter Server.
• For VirtualCenter 2.x and ESX Server 3.x using per-virtual-machine masks, reset any changed rows to the defaults by clicking Reset All to Default on the CPU Identification Mask dialog box.
• For VirtualCenter 2.x and ESX Server 2.x or ESX Server 3.x using global masks: Remove any added tags entered in the vpxd.cfg file and restart the VirtualCenter service.