It's a known issue with VMware that it performs what is called a "stun" during certain operations, such as vmotion, and snapshot create/delete. During this stun, the guest OS is frozen, and so when it comes back, the system clock is behind. Now the stuns are usually pretty quick in human terms (sub-second), but in machine terms they're pretty long, several hundred milliseconds. And the times get worse on VMs with bigger disks, or more memory, which are often the VMs that are more critical. With systems that communicate with each other, these time differences can cause problems.
But in any case, the issue I am trying to address is the clock. The ultimate requirement is to immediately get the system clock back in sync after a stun happens. "Immediate" might a vague term, so lets say within 1 second the clock should be back in sync.
We do use ntp for clock synchronization, but ntp takes several minutes (or longer) to get the system back in sync as it doesn't understand what just happened. It takes a while to verify the time is stable again (that drift rate hasn't spiked), and then slowly correct things. So it's not fast enough.
The best idea I've got is to immediately run ntpdate when a stun happens, but I do not know of any way for the guest OS to discover that a stun has happened.
The systems in question are Linux (CentOS/7).
Best Answer
Do the VM hosts also use the same NTP sources as the guests?
It is possible to set a smaller NTP step threshold, but databases especially don't deal well with time going backwards.