How to enable `Keep Alive` in a shared hosting environment

Turn KeepAlive Off, do a graceful restart, and keep it off for some time, make the configuration change, and do another graceful restart. This is similar to the practice of reducing the TTL for a DNS entry before a change.

The benefit from having keep-alive on is that a client will be able to request more than one entity from your server without having to create another TCP connection (3-way handshake with its round-trips included). The problem with this is that if, say, you have a connection limit in apache set to 300, if there's 300 active connections all the others will have to wait until the first 300 clients are done and/or the timeout expires.

Disabling keep-alive will force the clients to create 1 connection per request. When the socket is properly closed on both ends, it goes into the TIME_WAIT status, as you've noticed. This happens to ensure that the port used in that connection does not receive data from a previous connection for a while before being available (see this, but there's much more out there). In my linux system, /proc/sys/net/ipv4/tcp_fin_timeout is set to 60 seconds. You can try reducing that, but don't go too far down. How far? It depends on how many connections per second you are getting. For 100-200req/s, don't bother changing the default.