Apache: Force HTTP 1.1 or Persistent/KeepAlive connections for HTTP 1.0 requests

apache-2.2httphttp-headerskeepalive

I want to force Keep Alive or Persistent connections for all HTTP requests on my Apache 2.2.3 server running on RHEL 5.8. A lot of web crawlers are using HTTP 1.0 for some reason, and I would like to either force persistent connections, or somehow force those connections to use HTTP 1.1 so that the Keep Alive On setting in the Apache config will cause persistent HTTP. This is because I want to reduce the number of TCP connections being opened. How can I accomplish this?

Best Answer

HTTP 1.0 doesn't support keep-alive (A.K.A persistent connections) and there is no system to ask them to use HTTP 1.1 you could return them a 400x but they would have still already created a connection and there is no standard way to tell them to try HTTP 1.1 instead. Even if all your clients do work out they need HTTP1.1 there is nothing you can do on the service to stop them sending the "close" in keep-alive header or dropping the connection and still do a single connection per request.

There are some Linux kernel options (specifically DEFER_ACCEPT andTIME_WAIT) that can help with reducing the TCP connection table but I wouldn't recommend going down that route unless you know what you are doing as its really easy to make things worse rather than better.

But thinking more broadly I think your trying to solve the problem in the wrong way. The internet is dark and full of terrors, If random bots are causing you TCP connection limit issues then you are hopeless vulnerable to SYN floods or other DDOS type events and bots are just a signal that your webserver is either underscaled, under attack or misconfigured. Either way something id wrong.

If you really want to control what kind of requests arrive at apache you need a WAF(web application firewall) or Load balancer infront of apache to filter out bad requests before they consume expensive apache connections. In many cases you can also offload the CPU intensive work of doing SSL/TLS encryption.

I know this isn't a quick fix or the answer your looking for but whatever the issue is your very unlikekly to solve it going down this path.

Related Solutions

Apache IP Connection – How to Limit New Connections Per Second/Hour/Day

I wouldn't do it in apache.. I'd do it at network layer with iptables.

iptables -A INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --set
iptables -A INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --update --seconds 86400 --hitcount 100 -j REJECT

Change 86400 to the number of seconds you want to keep the block for (86400 is 1 day), and 100, is the hit count, how many you're prepared to allow per IP.

You can also change -j REJECT to -j DROP, which defines the packet behaviour when the condition is met. DROP seamlessly drops packets, and REJECT returns a "port unreachable" or similar error.

That said, there was a mod_throttle that would do something similar, but I can't seem to find much information about it. I think it feels neater to do this kind of thing at the network/kernel level, rather than in Apache. Apache is good at serving requests. Let it do what it does best, and don't burden it with having to track connections too.

Linux – What alternative is there to Nginx that supports http keep-alive between backends

I think they're working on putting that in haproxy.

Keep in mind tcp setup on a LAN (=low latency) is usually not a problem, all modern operating systems have this worked out well. Sure it would be nicer just to have open backend connections, but that makes the code of the frontend (nginx or haproxy in this case) a lot more complex in surprising ways.

Best Answer

Related Solutions

Apache IP Connection – How to Limit New Connections Per Second/Hour/Day

Linux – What alternative is there to Nginx that supports http keep-alive between backends

Related Topic