Intel Cluster-on-Die (COD) Technology, and VMware vSphere 5.5 U3b and 6.x

Products

VMware

Issue/Introduction

This article explains the Intel processor feature, Cluster-on-Die (CoD), and how it affects the role of processor topology detection by VMware vSphere 5.5 U3b, 6.0 and later versions.

When an x86-based server boots, the BIOS (firmware) creates tables in memory that describe how CPU cores, processors, and memory (DRAM) are clustered into logical domains referred to as NUMA nodes (or domains). The components within a NUMA node generally have lower access latency to each other compared with access to components of another NUMA node. An operating system (such as VMware vSphere) uses the NUMA node assignments in these BIOS tables to correctly detect processor topology so that it can properly schedule virtual machines across the multiple cores, physical processors, and memory regions of a complex server. This information is also essential for per-socket licensing model of VMware vSphere.

Cause

Within a physical processor (the processor package that inserts into a processor socket), the NUMA node assignments (logical clustering) reflect the physical connection of CPU cores to integrated memory controllers and the associated DRAM. A processor socket is often represented as just one NUMA node. However it can also be represented as multiple NUMA nodes depending on the physical layout of CPU cores and memory controllers. For many processors, it is not possible to change the physical connection between these components in a way that changes access latency.

However, starting with the Intel Xeon E5-2600 v3 processor family (code named Haswell) , the Cluster-on-Die (CoD) feature was offered by Intel to make possible boot-time configuration of NUMA nodes within a processor. As implemented by Intel, this feature can logically partition the processor into either one NUMA node, or multiple NUMA nodes where each node is some number of CPU cores with an integrated memory controller on the same processor socket. The feature changes internal registers within the components of the processor socket to better optimize access latency for the selected mode. Customers select the appropriate mode in BIOS that best suits their application needs.

Resolution

ESXi 6.0 and later versions support systems with the CoD capability:

ESXi 6.0, released on 3/12/15, supports the COD feature in Intel Haswell EP/EN/EX (CPUID family 6, model 3F) processors.
- You can download:
  - VMware vSphere ESXi 6.0, available at VMware Downloads. For more information, see the vSphere 6.0 Release Notes.
ESXi 5.5 U3b (released on 12/8/15), ESXi 6.0 U1 (released on 9/10/15), and later versions support CoD for Intel Haswell EP/EN/EX (CPUID family 6, model 3F) and future processors.
- You can download:
  - VMware vSphere ESXi 6.0 U1, available at VMware Downloads. For more information, see the vSphere 6.0 U1 Release Notes.
  - VMware vSphere ESXi 5.5 U3b, available at VMware Downloads. For more information, see the vSphere 5.5 U3b Release Notes

This feature was initially supported for the Xeon E5-2600 v3 processor in vSphere 6.0. In addition, the CoD feature on Intel processor and future processors is supported in vSphere ESXi 5.5 U3b, and vSphere ESXi 6.0U1 and later.

Additional Information

Intel Cluster-on-Die (COD) 技术与 VMware vSphere 5.5 U3b 和 6.x
Intel Cluster-on-Die (COD) 技術と VMware vSphere 5.5 U3b および 6.x