To keep things simple and do one thing at a time, you should not change your domain's NS records during this migration. If you want to migrate the web site and also migrate the authoritative DNS server, do it in 2 separate steps. To migrate the web site, change only the A and AAAA records.
A conventional "low value" TTL to set on the A and AAAA records is 300 (5 minutes).
If you can operate both servers at the same time for a short period, then you can do even better than a DNS-based migration. You can set the old server to proxy requests to the new server, so that clients which still contact the old server get their requests forwarded to the new server. This is easy to achieve with Apache with ProxyPass.
Later, if you want to move the DNS hosting, change the NS records and change the delegation in the parent zone (i.e. make the change at your domain's registrar). It is ideal if you can deploy the zone in the new servers ahead of time and leave both sets of nameservers serving the domain for a period of time.
EDIT: Details about ProxyPass
This Apache config snippet, placed in the <VirtualHost> section for your server, or directly in server config if there is no <VirtualHost>, will allow the old server to forward all requests to the new server:
<Location />
ProxyPass http://www.example.org/
</Location>
You normally should have ProxyPassReverse with ProxyPass but in this case it can be omitted since it would have no effect because the old site and new site have identical ServerName.
You probably want to remove any other <Location>, Alias, RewriteRule, or other URL-mapping directives from the old server config at the same time to be certain that the proxy rule and only the proxy rule gets processed. (Keep a backup around, of course, in case you need to back out of your migration!)
Make sure the old server knows the correct new IP address for the web site's domain, otherwise it will proxy the request back to itself!
They're not supposed to, but some DNS services may treat this as more of a suggestion than a hard rule. They may honor the setting down to some minimum, or they may ignore your TTL completely and always use their own setting (I've heard that 2 days is, or at least was, common). You need to be aware there is nothing you can do that will make those providers update any faster, and therefore some requests will end up going to the old address for some time after you make the change.
Ideally in this case, you want to cut over to a new IP address while you still have some control of the old address, such that your server can be set to handle requests via both addresses for a small interim period.
Additionally, some DNS services charge you per request (or per million requests). Moving from 3 hours to 1 minute will increase your DNS requests by a factor 180... you'll get 180 times as many requests as before. It's not likely to break the bank, but just make sure you're prepared for that.
As an example, I have DNS service for a rather small web site where I spend about $20 per year for them to service 5 million requests per month. I admit that I'm not actually sure whether they'll just bill me or stop handling requests if I ever exceed that, though I expect it's the former. Right now I tend to only get about 1/2 million requests per month, but I wonder what would happen if I changed my TTL setting to get 180 times as many more and left it that way for too long.
Still, most DNS services will honor your 1 minute setting. This will help smooth the changeover to the new address, and it's not likely to hurt you at all as long you're careful. Just remember to do this at least 3 hours (the old TTL) ahead of the change. There's no point doing it much earlier; any provider that would need to see the change sooner is not honoring the setting anyway. And, of course, don't forget to put it back when you're done.
You may also want to reference this question:
Migrating DNS Providers
It's a bit different than yours, but some of the issues involved are similar.
Best Answer
You can use a Reverse Proxy on the old web server. It might be a bit of work to set up, but just as long as ITS DNS is up to date you will be OK.
What will happen is:
If you're running Apache, look into mod_proxy. If you're running IIS, look into ISAPI Rewrite to get this sort of functionality.
(note that the DNS on the old web server needs to be up to date if you want to proxy using the domain name. Otherwise, proxy it directly to the IP address and make sure that the host is listening on the IP without a hostname)