• ESXi hosts randomly disconnect from the Active Directory domain or sometime disconnects from the vCenter Server.

• On the vmkernel.log file, you see entries similar to:

2019-11-12T22:52:10.902Z cpu5:2924874)MemSched: 14642: uw.2924860 (14074759) extraMin/extraFromParent: 256/256, likewise (790) childEmin/eMinLimit: 99871/100096
2019-11-12T22:52:10.902Z cpu5:2924874)MemSched: 14635: Admission failure in path: likewise/lwsmd.2924860/uw.2924860
2019-11-12T22:52:10.903Z cpu5:2924874)MemSched: 14642: uw.2924860 (14074759) extraMin/extraFromParent: 256/256, likewise (790) childEmin/eMinLimit: 99871/100096
2019-11-12T22:52:10.903Z cpu5:2924874)MemSched: 14635: Admission failure in path: likewise/lwsmd.2924860/uw.2924860

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

The issue occurs due to exhaustion of Likewise memory because of existing memory leaks in Active Directory operations and related libraries.

When smart card authentication is enabled and configured on the ESXi hosts, Likewise memory leaks are also observed.

This issue is partially resolved in VMware vSphere ESXi 6.5 Patch ESXi650-202007001 and  vSphere ESXi 6.7 Patch ESXi670-202008001  and vSphere  ESXi 7.0 U1. To download the patch, go to  Customer Connect Patch Downloads page.

To download vSphere ESXi 7.0 U1 go to Customer Connect Downloads
 

To work around this issue non-persistently:

1. Download the lwis-mem-check-2.zip file attached to this Knowledge Base article. 
2. Unzip lwis-mem-check-2.zip and copy the lwis-mem-check-2.sh script in /tmp directory. 
3. Verify if the script is executable. If the script is not executable, run  the command:

chmod +x /tmp/lwis-mem-check-2.sh

4. Run the script using the command:

setsid /tmp/lwis-mem-check-2.sh >/dev/null 2>&1 < /dev/null &

The script monitors the likewise memory usage generating lwis-oom-stats file in /tmp directory with Likewise memory stats when Likewise runs out of memory and remediates the issue.

Note: The above workaround is only temporary and will not persist if the ESXi host is rebooted.


 

To work around this issue persistently:

Important: Please ensure you have a ESXi host configuration backup 

You can always contact VMware support to assist with the below steps, if you are not familiar in making the below changes.

Note: You will need a shared datastore path where you can place the lwis-mem-check.sh script that can be retrieved during startup.

1. SSH to the ESXi host
2. Backup the file: 

cp /etc/rc.local.d/local.sh /root/local.sh

3. Edit the file: 

vi /etc/rc.local.d/local.sh

4. Add the following lines before “exit 0”:

cp /vmfs/volumes/<Datastore>/<script>/lwis-mem-check-2.sh /tmp/
chmod +x /tmp/lwis-mem-check-2.sh
setsid /tmp/lwis-mem-check-2.sh >/dev/null 2>&1 < /dev/null &

Note: <Datastore>/<script> should be replaced with the correct datastore directory path where the script is located.

5. Save the changes
6. To ensure the local.sh file will save during a reboot operation run: 

auto-backup.sh

Note: This is only a temporary workaround and not a permanent fix. The above steps should be reverted once the issue has been identified and resolved fully.

If smart card authentication is enabled, disabling this will help stop any Likewise memory leaks created by the smart card authentication.