VMware
 

Knowledge Base

Search the Knowledge Base:

Products:
Search In:
 

Scheduler Cell Size on Six-Core Processors

Details

What is a scheduler cell?

The VMware ESX scheduler logically partitions a host's physical processor cores into cells. In ESX 3.5, four physical cores are grouped into a cell.

By grouping processors into cells, ESX can make virtual machine scheduling decisions locally for each cell without affecting other cells, thus making the ESX host more scalable. A virtual machine can be scheduled only within a cell, not across cells, leading to a cell size equal to the widest vSMP virtual machine supported by that release of ESX (thus the cell size of four for ESX 3.5).

The default cell size of four has provided optimal performance with current and past versions of processors released by both Intel and AMD. These include:

- single-core processors, in which case a cell consists of four sockets;

- dual-core processors, in which case a cell consists of two sockets;

- and quad-core processors, in which case a cell consists of a single socket.

Default Cell Size and Intel 7400-Series (Dunnington) Architecture

 
The Intel® Xeon® 7400-series processors (codename Dunnington), with six cores per package, are an exception to the trend of cores per socket being a power of two (1, 2, 4, 8...). On a Dunnington processor the ESX default cell size of four causes some cells to be entirely within one socket (using four of that socket's six cores) and some cells to span sockets (using two cores from one socket and two from a different socket). This asymmetry in cell construction could result in variations in the performance of virtual machines due to on-chip cache access behavior, inter-vCPU communication, etc. The performance variation could be two virtual machines performing differently (if they are scheduled to different cell types) or the performance of a single virtual machine varying over time (as it is scheduled first to one cell type and then to the other). The impact would depend on your virtual machine configuration and workloads. Note that occasional rebalancing happens across cells so that no cells are unfairly overloaded.

Solution

If you think your virtual machines on a Dunnington host may have performance issues associated with the default cell size, you can increase the cell size from four to six. This way a cell would correspond to a socket, removing asymmetry between different cells. Changing the cell size can be done as follows:
 

Using the VI Client:

  1. Select the Configuration tab in the VI client.
  2. Select Advanced Settings.
  3. Select VMkernel.
  4. In the right pane, locate VMkernel.Boot.cpuCellSize .
  5. Change the value to 6 .
  6. This will take effect the next time the ESX host is rebooted.
From the command line interface (on classic ESX):
  1. Enter:
    esxcfg-advcfg --set-kernel 6 cpuCellSize
  2. Reboot the ESX host.

From the remote command line interface (on ESXi):

  1. Enter:
    vicfg-advcfg --set-kernel 6 cpuCellSize
  2. Reboot the ESX host.

Feedback

Rating: 1 - Lowest 2 3 4 5 - Highest (12 Ratings)   

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.
What can we do to improve this information? (2000 or fewer characters)
Submit
Rating: 1 - Lowest 2 3 4 5 - Highest (12 Ratings)   
Actions