I have 3 servers running CentOS 7.9.2009 in a data center (VMware ESXi). These servers report that the time is not synchronized. I have a similar test environment running on an inhouse VMware ESXi server where the servers sync. the time Ok. The production environment was originally set up in exactly the same way – but obviously updated with package updates over time. So they "should" be identical – but I cannot guarantee that any more. The ESXi servers are both version 6.
The servers were originally configured using "ntpd" – but when troubleshooting this issue over the last couple of days I have found that "Chrony" seems to be a better choice on CentOS 7. I have therefore reconfigured the servers to use Chrony – but still have the problem.
Edit: Steps used to change to Chrony
- yum install chrony
- systemctl stop ntpd
- systemctl disable ntpd
- systemctl start chronyd
- systemctl enable chronyd
So when I use timedatectl
I get this output:
Local time: Mon 2021-08-02 09:14:43 CEST
Universal time: Mon 2021-08-02 07:14:43 UTC
RTC time: Mon 2021-08-02 07:16:34
Time zone: Europe/Copenhagen (CEST, +0200)
NTP enabled: yes
NTP synchronized: no
RTC in local TZ: no
DST active: yes
Last DST change: DST began at
Sun 2021-03-28 01:59:59 CET
Sun 2021-03-28 03:00:00 CEST
Next DST change: DST ends (the clock jumps one hour backwards) at
Sun 2021-10-31 02:59:59 CEST
Sun 2021-10-31 02:00:00 CET
If I restart Chrony using systemctl restart chronyd
then after a couple of seconds timedatectl
reports:
Local time: Mon 2021-08-02 09:26:06 CEST
Universal time: Mon 2021-08-02 07:26:06 UTC
RTC time: Mon 2021-08-02 07:26:08
Time zone: Europe/Copenhagen (CEST, +0200)
NTP enabled: yes
NTP synchronized: yes
RTC in local TZ: no
DST active: yes
Last DST change: DST began at
Sun 2021-03-28 01:59:59 CET
Sun 2021-03-28 03:00:00 CEST
Next DST change: DST ends (the clock jumps one hour backwards) at
Sun 2021-10-31 02:59:59 CEST
Sun 2021-10-31 02:00:00 CET
After some time (minutes) it is back to NTP synchronized: no
.
When I run ntpstat
I get:
synchronised to NTP server (217.198.219.102) at stratum 2
time correct to within 124123 ms
polling server every 64 s
or
unsynchronised
poll interval unknown
In the last case then after some time it will show the first output again. But the "within … ms" seems pretty high???
As I can get it synchronized by restarting Chrony then I guess that firewall/network is Ok. I use the default Chrony config (as I did with ntpd before).
VMwaretools service is installed and started (open-vm-tools, http://github.com/vmware/open-vm-tools).
I would appreciate any suggestions for troubleshooting this further – and eventually fix it 😉
Thanks in advance!
/John
Best Answer
I think I have solved it now.
Basically, chrony thought that the time varied too much. So following the link by Peter Rosenberg (and the resources it linked to) I got on the track....
I've put this information here in case someone else searches for it.
First steps on the process was the status from the chronyd service:
It clearly showed that something was wrong. So the next step was:
Notice the
time too variable
for all of the servers....And
chronyc tracking
also shows that the time is not aligned at all:After some more reading in the references to the articles mention I tried to adjust the
makestep
in the/etc/chrony.conf
file to force an update. I had already changed the NTP pool servers to be "nearer" the application servers, so the config file now looks like this:It has now been running for a little time and it seems to be keeping the time synchronized ;-)
EDIT:
As Paul Gear pointed out, I had not solved the issue... The time still drifted.
Using
/usr/bin/vmware-toolbox-cmd timesync status
I found that on the production servers synchronisation of the time with the ESXi host was ENABLED (!!!). I have no idea how this has happened? The VM I originally configured and uploaded to the data center guys did not have it enabled. Any way, obviously, it should not sync. time with the host.It is fairly easy to disable by using:
/usr/bin/vmware-toolbox-cmd timesync disable
And now we have more realistic data from
chronyc sources -v
:as well as
chronyc tracking
:It has now been running smoothly for half an hour so I'm confident this is the solution. Thanks for the input!!!