Host has lost time synchronization error after upgrading to ESX 7.0.3 build 18644231

Products

VMware vSphere ESXi

Issue/Introduction

To synchronizing NTP when the root dispersion has a large value

Symptoms:

Host has lost time synchronization error after upgrading to ESX 7.0.3 build 18644231 error seen on host after upgrading to ESX 7.0.3 build 18644231

In var/log/syslog.log we see similar to below

syslog.1:9829:2021-10-11T11:51:16.043Z ntpd[2157160]: no peer for too long, server running free now

In var/log/vmkwarning.log we see similar to below

vmkwarning.log:2229:2021-10-07T22:30:16.245Z cpu28:2157160)WARNING: NTPClock: 1764: system clock synchronized to upstream time servers
vmkwarning.log:2230:2021-10-07T23:27:10.223Z cpu34:2157160)WARNING: NTPClock: 1457: system clock stepped to 1633649230.381665000, no longer synchronized to upstream time servers
vmkwarning.log:2231:2021-10-07T23:27:10.381Z cpu34:2157160)WARNING: NTPClock: 1764: system clock synchronized to upstream time servers
vmkwarning.log:2239:2021-10-08T00:26:26.380Z cpu28:2157160)WARNING: NTPClock: 1457: system clock stepped to 1633652786.527106000, no longer synchronized to upstream time servers
vmkwarning.log:2240:2021-10-08T00:26:26.527Z cpu28:2157160)WARNING: NTPClock: 1764: system clock synchronized to upstream time servers
vmkwarning.log:2254:2021-10-08T01:23:32.548Z cpu39:2157160)WARNING: NTPClock: 1457: system clock stepped to 1633656212.693259000, no longer synchronized to upstream time servers
vmkwarning.log:2255:2021-10-08T01:23:32.693Z cpu39:2157160)WARNING: NTPClock: 1764: system clock synchronized to upstream time servers

If NTP configuration is correct, but the NTP daemon does not synchronize to the listed NTP server.
For example
[root@esxi-70-1: ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
dc01.domain .LOCL. 1 u 35 64 377 0.696 -83.934 3.527

The list of NTP associations
[root@esxi-70-1:/tmp] ntpq -c associations

ind assid status conf reach auth condition last_event cnt
===========================================================
1 7023 9014 yes yes none reject reachable 1

Next, look at NTP variables associated with this host:

[root@esxi-70-1:/tmp] ntpq -c "rv 7023"
associd=7023 status=9014 conf, reach, sel_reject, 1 event, reachable,
srcadr=dc01.domain.lab, srcport=123, dstadr=10.21.23.139,
dstport=123, leap=00, stratum=1, precision=-6, rootdelay=0.000,
rootdisp=10682.327, refid=LOCL,
reftime=e520d4c8.50147e3c Mon, Oct 25 2021 7:01:28.312,
rec=e521b5cc.076edf15 Mon, Oct 25 2021 23:01:32.029, reach=377,
unreach=0, hmode=3, pmode=4, hpoll=6, ppoll=6, headway=59,
flash=400 peer_dist, keyid=0, offset=-90.556, delay=1.333,
dispersion=16.515, jitter=5.877, xleave=0.031,
filtdelay= 1.33 0.63 0.60 0.70 0.64 0.55 0.64 0.56,
filtoffset= -90.56 -84.03 -84.60 -83.93 -83.91 -83.85 -89.81 -85.07,
filtdisp= 15.63 16.62 17.61 18.58 19.54 20.50 21.48 22.47

The output indicates that the NTP server has been rejected, although it's reachable

Note:The preceding log excerpts are only examples.Date,time and environmental variables may vary depending on your environment.

Environment

VMware vSphere ESXi 7.0.3

Cause

This occurs when the root dispersion value is high due to time servers not syncing

Resolution

This issue is resolved in VMware ESXi 7.0 Update 3c (build number 19193900)

ESXi NTP can be forced to accept time from AD time sources root dispersion has a large value
This is described here https://kb.vmware.com/s/article/1035833

Additional Information

Impact/Risks:
None