An ESX host fails to boot after increasing the Service Console or system memory (1016239)
- An ESX fails to boot
- You may see a purple diagnostics screen that contains an error similar to:
- With 256GB of physical memory installed in a server, the ESX VMkernel reserves 1289MB of Low Memory.
- With 128GB of physical memory installed in a server, the ESX VMkernel reserves 644MB of Low Memory.
The relationship between Physical Low and High Memory is as follows:
- Maximum Console OS memory = Total Low Memory - VMkernel Reserved Low Memory - ~100MB Drivers
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 0000000000098400 (usable)
BIOS-e820: 0000000000098400 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000007ff42980 (usable)
BIOS-e820: 000000007ff42980 - 000000007ff76300 (ACPI data)
BIOS-e820: 000000007ff76300 - 0000000080000000 (reserved)
BIOS-e820: 00000000c0000000 - 00000000d0000000 (reserved)
BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000004040000000 (usable)
Note: Memory ranges from 000000007ff76300 (2047MB) to 0000000100000000 (4096MB) are reserved by the BIOS and are not usable by the OS.
The following lines from /var/log/messages record the VMkernel interpretations of the RAM map:
user-defined physical RAM map:user: 0000000000000000 - 0000000000098400 (usable)
user: 0000000000098400 - 00000000000a0000 (reserved)
user: 00000000000e0000 - 0000000000100000 (reserved)
user: 0000000000100000 - 0000000011000000 (usable)
VMNIX: linux MA=[0x0,0x11000000)
VMNIX: vmkernel MPN=[0x11000,0x7ff41]
VMNIX: vmkernel MPN=[0x100000,0x403ffff]
Note: The line containing linux MA reflect the Machine Address memory ranges allocated (in bytes) to the service console. The Service Console is using the lower 272MB. The lines containing vmkernel MPN reflect the Machine Page Numbers (4k blocks) allocated for the VMkernel. The VMkernel is using memory ranges 272MB - 2047MB and 4GB - 256GB.
The table below contains several combinations of Physical Memory and Service Console Memory assignment, along with their impact on the amount Unreserved Low Memory available when 2GB of low memory is reserved by the BIOS. The combination of 256GB of Physical Memory and 800MB of Service Console Memory with only 2GB of Low Memory leaves the server with only 7MB of Unreserved Low Memory, insufficient for booting. In this case the ESX VMkernel startup stops with the purple diagnostics screen.
|Physical Memory||Physical Low Memory||Reserved Low Memory||Service Console Memory||Unreserved Low Memory|
|256 GB||4096 MB||1289 MB||272 MB||2535 MB|
|256 GB||4096 MB||1289 MB||800 MB||2007 MB|
|256 GB||2048 MB||1289 MB||272 MB||487 MB|
|256 GB||2048 MB||1289 MB||800 MB||7 MB|
|128 GB||4096 MB||644 MB||272 MB||3180 MB|
|128 GB||4096 MB||644 MB||800 MB||2652 MB|
|128 GB||2048 MB||644 MB||272 MB||1132 MB|
|128 GB||2048 MB||644 MB||800 MB||604 MB|
- Reduce the amount of Physical Memory in the server.
- Reduce the amount of Physical Memory reserved by the BIOS.
Note: The BIOS memory reservation may or may not be configurable in a particular server. Consult your hardware vendor for more information.
- Reduce the amount of Service Console Memory assigned by using the following steps:
- Restart the ESX host.
- When presented the boot menu, ensure you have the default boot option highlighted.
- Press the e key to edit the default boot option. A new list appears.
- Use the arrow keys to highlight the line which starts with kernel /vmlinuz.
- Press the e key to edit the line in the boot configuration. A prompt with the entire line will appear.
- Use the arrow key to move over the portion of the line which has a parameter similar to the following:
- Change the value to something between 500 and 650 megabytes. The format is mem=<value in megabytes>M.
Note: The exact threshold may vary depending on the hardware configuration. To determine the exact value, engage your hardware vendor or test different values.
- Press enter to accept the changes.
- Press the b key to boot the host using the configuration.
Note: The configuration change is not permanent. To make the change permanent, proceed to step 10.
- Wait for the host to boot.
- When the host has booted, open the vSphere Client and connect to the host.
- Under the configuration tab, change the memory properties of the ESX host to ensure that the RAM assignment has been reduced. For more information on adjusting the RAM assignment using the vSphere Client, see Increasing the amount of RAM assigned to the ESX Server Service Console (1003501).