ESX/ESXi 3.x/4.x hosts do not support 2-terabyte LUNs (3371739)
Note: As of ESXi 5.0, support for LUNs larger then 2TB is available with both VMFS5 datastores and Physical-mode RDMs. However, virtual disks used in virtual machines and Virtual-mode RDMs are still limited to 2TB - 512 bytes. For more information, see vSphere 5 FAQ: VMFS-5 (2003813).
The largest LUN size which can be presented to an ESX/ESXi host is 2TB - 512 bytes. This limitation is outlined for multiple versions of ESX and ESXi in the Configuration Maximums.
Understanding the Limitation
VMware ESX and ESXi uses the SCSI-2 standard. As per the SCSI-2 standard, ESX uses a
READ_CAPACITY (10-byte) call which returns a 32-bit number to represent the size of the LUN.
Although the call is capable of returning a 32-bit number, only the
0xfffffffe values can be used. The value
0xffffffff is reserved.
Based on the standard, the largest valid LUN capacity support is
(2^32)-1 sectors. This is the math involved in the calculation:
(2^32) = 4294967296
4294967296 - 1 = 4294967295
If you take the above number and multiply it by 512 bytes per sector, you get the size limitation in bytes:
4294967295 x 512 = 2199023255040
The limit of 2TB translates to:
- 4294967295 sectors
- 2199023255040 bytes
Raw Device Mapping Limit
In the case of physical mode RDMs, the SCSI traffic sent by the guest operating system to do reads or writes is passed through by the VMware ESX storage layer to the underlying storage. This allows some guests to break the 2-terabyte limit, but this is an unsupported configuration because the storage stack of the ESX is not designed to handle such capacity.
Note: When migrating and powering on virtual machines, ESX/ESXi 4.0 enforces the 2TB - 512 bytes limit. ESX 3.5 does not enforce this limit.
Although the LUN size is limited to 2TB - 512 bytes, the VMFS and datastore volume size limit is much greater. The maximum volume size is 64TB - 16KB. This can be accomplished by leveraging the maximum extents (32) per volume.
If you require a single datastore that is larger than 2TB - 512 bytes in size, you may want to add extents so that the datastore resides on multiple LUNs of 2 TB - 512 bytes or less.
For information on extents, see the ESX Configuration Guide.
Keep these considerations in mind when implementing the workaround:
- Once you add an extent to a datastore, you cannot remove the extent.
- When checking the LUN size fdisk, fdisk as well as GNU fdisk will report disk size in GiB. This is due to a misinterpretation (blocks vs sectors and N sectors vs Last sector) this will be a smaller size if this value is divided by 1024.
- The maximum file size which can reside on a datastore is based on the block size of the filesystem. The maximum block size for VMFS version 3 (used by ESX3 and ESX4) is 8MB, which limits the maximum file size to 2TB - 512 bytes in size. For more information about the maximum file size, see the Configuration Maximums for the appropriate version of ESX.
If you install ESX 3.5 Update 4 on a host that has a local disk larger than 2TB, you may havew trouble booting. For more information, see Mounting root fails after installing ESX 3.5 when local storage exceeds the 2 terabyte limit (1012757).
For more information about the maximum configuration of ESX, see the Configuration Maximums for the appropriate version of ESX.