Apache returns 502 Bad Gateway with mod_proxy and large file

Just to add some specific settings, I had a similar setup (with Apache 2.0.63 reverse proxying onto Tomcat 5.0.27).

For certain URLs the Tomcat server could take perhaps 20 minutes to return a page.

I ended up modifying the following settings in the Apache configuration file to prevent it from timing out with its proxy operation (with a large over-spill factor in case Tomcat took longer to return a page):

Timeout 5400
ProxyTimeout 5400

Some backgound

ProxyTimeout alone wasn't enough. Looking at the documentation for Timeout I'm guessing (I'm not sure) that this is because while Apache is waiting for a response from Tomcat, there is no traffic flowing between Apache and the Browser (or whatever http client) - and so Apache closes down the connection to the browser.

I found that if I left the Timeout setting at its default (300 seconds), then if the proxied request to Tomcat took longer than 300 seconds to get a response the browser would display a "502 Proxy Error" page. I believe this message is generated by Apache, in the knowledge that it's acting as a reverse proxy, before it closes down the connection to the browser (this is my current understanding - it may be flawed).

The proxy error page says:

Proxy Error

The proxy server received an invalid response from an upstream server. The proxy server could not handle the request GET.

Reason: Error reading from remote server

...which suggests that it's the ProxyTimeout setting that's too short, while investigation shows that Apache's Timeout setting (timeout between Apache and the client) that also influences this.

Apache has a known and unresolved issue with Expect headers, see bug 46709 and bug 47087.

The issue is that some clients set the Expect header and only send the request headers before a PUT or POST of data. This allows the server to respond to errors/redirects/security violations prior to the client sending the request body (PUT or POST data). This is a laudable goal, but apparently, the client does not wait until it gets a response and just pushes out the body of the request, which results in the 417 error.

If you have control over a .NET client you can use ServicePointManager.Expect100Continue Property set to false, to override this behavior.

If you only have control over the server, it looks like you can either force HTTP 1.0 for those clients (perhaps based on user agent string) or force unset the Expect header using mod_header early on in the request.

To remove the Expect header from the request early using mod_headers use this config directive:

<IfModule mod_headers.c>
RequestHeader unset Expect early
</IfModule>

This works because the client is not actually waiting for the "100 Continue" response and acting as if the Expect header were not set.