Search the VMware Knowledge Base (KB)
Products:
View by Article ID

Understanding vSAN 6.x memory consumption (2113954)

  • 3 Ratings

Purpose

This article provides  information about vSAN 6.x memory consumption and a variety of example scenarios.

Resolution

To calculate vSAN 6.x memory consumption, use this equation:

BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + (SSDMemOverheadPerGB x SSDSize)))
 
Where:
  • BaseConsumption: This  is the fixed amount of memory consumed by vSAN per ESXi host. This is currently 3 GB. This memory is mostly used to house the vSAN directory, per host metadata, and memory caches.
  • NumDiskGroups: This is the number of disk groups in the host, should range from 1 to 5.
  • DiskGroupBaseConsumption: This is the fixed amount of memory consumed by each individual disk group in the host. This is currently 500 MB. This is mainly used to allocate resources used to support inflight operations on a per disk group level.
  • SSDMemOverheadPerGB: This is the fixed amount of memory we allocate for each GB of SSD capacity. This is currently 2 MB in hybrid systems and is 7 MB for all flash systems. Most of this memory is used for keeping track of blocks in the SSD used for write buffer and read cache.
  • SSDSize: The size of the SSD in GB. 

Scenario 1

In this scenario, we cover working examples of both hybrid and all-flash configurations where the hosts have more than 32GB of memory per host, the number of hosts in the cluster is less than 16, and the SSD size is 400 GB.
 
For example:
  • One disk group per host, hybrid configuration
BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + ( SSDMemOverheadPerGB  x SSDSize)))

3GB   +    (1   x     (500MB   +    (2MB    x    400)))
3GB   +    (500MB + 800MB)
3GB   +   1.3GB
                 ~= 4.3 GB 

  • Three disk groups per host, hybrid configuration
BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + ( SSDMemOverheadPerGB  x SSDSize)))

3GB   +    (3   x     (500MB   +    (2MB    x    400)))
3GB   +    (3   x     (500MB + 800MB)
3GB   +    (3   x     (1.3GB)
3GB   +   3.9GB
                 ~= 6.9 GB 

  • One disk group per host, all flash configuration
BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + ( SSDMemOverheadPerGB  x SSDSize)))

3GB   +    (1   x     (500MB   +    (7MB    x    400)))
3GB   +    (500MB + 2800MB)
3GB   +   3.3GB
                 ~= 6.3 GB 

  • Three disk groups per host, all flash configuration
BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + ( SSDMemOverheadPerGB  x SSDSize)))

3GB   +    (3   x     (500MB   +    (7MB    x    400)))
3GB   +    (3   x     (500MB + 2800MB)
3GB   +    (3   x     (3.3GB)
3GB   +   9.9GB
                 ~= 12.9 GB 

Scenario 2

In this scenario, we cover working examples where the hosts have more than 32 GB of memory per host, the number of hosts in the cluster is more than 32, and the SSD size is 600 GB. When there are more than 32 nodes in a vSAN cluster (implying that the goto11 option is used), the BaseConsumption increases by 300 MB to a total of 3.3 GB.
 
For example:

  • One disk group per host, hybrid configuration
BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + ( SSDMemOverheadPerGB  x SSDSize)))

3.3GB   +    (1   x     (500MB   +    (2MB    x    600)))
3.3GB   +    (500MB + 1200MB)
3.3GB   +   1.7GB
                 ~= 5 GB 

  • Three disk groups per host, hybrid configuration
BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + ( SSDMemOverheadPerGB  x SSDSize)))

3.3GB   +    (3   x     (500MB   +    (2MB    x    600)))
3.3GB   +    (3   x     (500MB + 1200MB)
3.3GB   +    (3   x     (1.7GB)
3.3GB   +   5.1GB
                 ~= 8.4 GB 
  • One disk group per host, all flash configuration
BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + ( SSDMemOverheadPerGB  x SSDSize)))

3.3GB   +    (1   x     (500MB   +    (7MB    x    600)))
3.3GB   +    (500MB + 4200MB)
3.3GB   +   4.7GB
                 ~= 8 GB 

  • Three disk groups per host, all flash configuration
BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + ( SSDMemOverheadPerGB  x SSDSize)))

3.3GB   +    (3   x     (500MB   +    (7MB    x    600)))
3.3GB   +    (3   x     (500MB x 4200MB)
3.3GB   +    (3   x     (4.7GB)
3.3GB   +   14.1GB
                 ~= 17.4 GB 

Scenario 3

In this final scenario, we have some examples of host with less than 32 GB of memory. In systems with less than 32 GB of RAM, the amount of memory used will be scaled down linearly according to the formula (SystemMemory / 32) where SystemMemory is the amount of memory in the system in GB. Thus, if the system has 16 GB of RAM, the amount of memory consumed will be 1/2 of the output given the formula use to compute memory consumption. If the system has 8 GB it will be scaled down by 1/4. Lets assume that the host has 16 GB of memory, the number of hosts in the cluster is less than 16, and that the SSD size is 400 GB.
 
For example:

  • One disk group per host, hybrid configuration

    (BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + ( SSDMemOverheadPerGB x SSDSize)))) * SystemMemory / 32 
(3GB   +    (1   x     (500MB   +    (2MB    x    400))) * 0.5)
(3GB   +    (500MB + 800MB)) * 0.5)
(3GB   +   1.3GB) * 0.5)
                 ~= 4.3 GB * 0.5
~= 2.15 GB
  • Three disk groups per host, hybrid configuration
(BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + ( SSDMemOverheadPerGB x SSDSize)))) * SystemMemory / 32

(3GB   +    (3   x     (500MB   +    (2MB    x    400)))) * 0.5
(3GB   +    (3   x     (500MB + 800MB))) * 0.5
(3GB   +    (3   x     1.3GB)) * 0.5
(3GB   +   3.9GB) * 0.5
                 ~= 6.9 GB  * 0.5
~= 3.45GB
  • One disk group per host, all flash configuration
(BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + ( SSDMemOverheadPerGB x SSDSize)))) * SystemMemory / 32

(3GB   +    (1   x     (500MB   +    (7MB    x    400)))) * 0.5
(3GB   +    (500MB + 2800MB)) * 0.5
(3GB   +   3.3GB) * 0.5
                 ~= 6.3 GB  * 0.5
~= 3.15GB


  • Three disk groups per host, all flash configuration
(BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + ( SSDMemOverheadPerGB x SSDSize)))) * SystemMemory / 32

(3GB   +    (3   x     (500MB   +    (7MB    x    400)))) * 0.5
(3GB   +    (3   x     (500MB + 2800MB))) * 0.5
(3GB   +    (3   x     3.3GB)) * 0.5
(3GB   +   9.9GB) * 0.5
                 ~= 12.9 GB  * 0.5
~= 6.45GB

Additional Information

Considerations when calculating memory requirements for hosts participating in a vSAN cluster :
  • vSAN scales back on its memory usage when hosts have less than 32GB of memory.
  • vSAN consumes additional memory when the number of nodes in the cluster is greater than 32.
  • All Flash vSAN configurations consume additional memory resources when compared to hybrid configurations.
For more information about sizing and designing a vSAN cluster, see:

See Also

Request a Product Feature

To request a new product feature or to provide feedback on a VMware product, please visit the Request a Product Feature page.

Feedback

  • 3 Ratings

Did this article help you?
This article resolved my issue.
This article did not resolve my issue.
This article helped but additional information was required to resolve my issue.

What can we do to improve this information? (4000 or fewer characters)




Please enter the Captcha code before clicking Submit.
  • 3 Ratings
Actions
KB: