Are you looking for the specific command to start/stop NTP on Linux and Windows? Those are different on different Linux distros. It will be something like "/etc/init.d/ntpd stop" on some, different on others, and I'm not sure on Windows.
There are two ways to do this, and one caveat.
Method 1: Run NTP normally but stop it for the benchmark.
Method 2: Don't run NTP but run NTP in "one-shot" mode right before the benchmark runs.
(On linux that would be "ntpdate SERVERNAME" on Unix/Linux or [i think] "ntp -q" on windows)
Method 1 has some advantages in that the clock will be correct all those times that the you aren't running benchmarks. The clock will also be more accurately set for the benchmark. That is, NTP maintains the clock very accurately even between NTP packets. The NTP packets on the network are simply fine tuning. Between packets NTP uses past experience to nudge the system clock faster or slower. The disadvantage is that when you start NTP again, it will refuse to start if the clock is too far off. This is not a bug, it does this on purpose to prevent syncing to a server that has gone bad. To prevent this, do an "ntpdate" or "ntp -q" right before you restart NTP.
Method 2 has the advantage that it is more simple to implement. However, the clock will not be very accurate. "one shot" mode doesn't set the clock as accurately as running NTP. It is mostly for fixing a far-off clock before you start NTP (see above). However, if your benchmark only has to be accurate to about 1/10th second you'll be fine.
The caveat is that the benchmarks might not be benchmarking what you think they are benchmarking if you turn off NTP. The computer's clock will drift without NTP running constantly. That's ok if you are benchmarking how much drift the computer has but not much else.
(And I should note... the drift will be different not just brand-to-brand, but between particular machines.)
HTH!
Best Answer
I own a hosting company and we do exactly this. Here is how we accomplish this.
To start with, you need a NTP master source. So one of your Linux servers will become the master. I would create a DNS A record called time.example.com (assuming example.com is the domain). This way, if your master moves you need not update the other 19 servers.
On the master server you need to have an appropriately configured ntp.conf file.
Here is what one of our master /etc/ntp.conf files looks like. Note, this is a data center with a private address space (RFC1918) using 172.17.x.x so you'll need to adjust accordingly. If you want more than one master, create more than one DNS A record each with different IP to get a bit of fault tolerance if so desired.
Now on each client, we have an /etc/ntp.conf file that looks like this:
Use the ntpq command to see the servers with which you are synchronized. It provided you with a list of configured time servers and the delay, offset and jitter that your server is experiencing with them. For correct synchronization, the delay and offset values should be non-zero and the jitter value should be under 100.
Also on our client nodes, we have a rc script (/etc/rc.d/rc.local) that synchronizes the clock before starting the NTPD daemon. Here are the important parts... They are order dependent.
Synchronize the client's clock with the master time source /usr/sbin/ntpdate -b time.example.com
Start the NTPD daemon allowing for large time adjustments during start-up. /usr/sbin/ntpd -g -x
Finally, depending on your set up, you'll need to punch a firewall rule to allow your time.example.com master to reach the Public Internet over UDP port. Here is a typical and appropriately placed IPTables rule
iptables -t nat -A POSTROUTING -o $PUB_IF -p udp --dport 123 -j MASQUERADE
Where PUB_IF is the public interface (eth0, eth1, whatever)