Apache as proxy generates failure.

apache-2.2mod-proxy

We are using apache proxy to enable our application servers to reach specific web sites over the internet. The setup is as follows:

application servers --> apache proxy --> Internet website

Some of the requests fail with the below error in the application server log:

<head>
<title>502 Proxy Error</title>
</head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
The proxy server could not handle the request <em><a href="/link">POST
nbsp;/link</a></em>.<p>
Reason: <strong>Error reading from remote server</strong></p></p>
</body>

and the below debug log in the apache error.log file:

[Mon May 20 09:57:54 2013] [debug] mod_proxy_http.c(56): proxy: HTTP: canonicalising URL //myURL.com
[Mon May 20 09:57:54 2013] [debug] proxy_util.c(1506): [client 172.20.101.71] proxy: https: found worker `https://myurl.com/` for `https://myurl.com/link`
[Mon May 20 09:57:54 2013] [debug] mod_proxy.c(1015): Running scheme https handler (attempt 0)
[Mon May 20 09:57:54 2013] [debug] mod_proxy_http.c(1973): proxy: HTTP: serving URL https://myurl.com/link
[Mon May 20 09:57:54 2013] [debug] proxy_util.c(2011): proxy: HTTPS: has acquired connection for (mu=y url)
[Mon May 20 09:57:54 2013] [debug] ssl_engine_io.c(1908): OpenSSL: I/O error, 5 bytes expected to read on BIO#136e90 [mem: 190593]
[Mon May 20 09:57:54 2013] [debug] proxy_util.c(2067): proxy: connecting https://myurl.com/link to myurl.com:443
[Mon May 20 09:57:54 2013] [debug] proxy_util.c(2193): proxy: connected /link to myurl.com:443
[Mon May 20 09:57:54 2013] [debug] ssl_engine_io.c(1908): OpenSSL: I/O error, 5 bytes expected to read on BIO#136e90 [mem: 190593]
[Mon May 20 09:57:54 2013] [info] [client ip] (131)Connection reset by peer: SSL input filter read failed.
[Mon May 20 09:57:54 2013] [error] [client ip2] (131)Connection reset by peer: proxy: error reading status line from remote server myurl.com:443
[Mon May 20 09:57:54 2013] [debug] mod_proxy_http.c(1466): [client ip2] proxy: NOT Closing connection to client although reading from backend server "myurl.com:443 failed.
[Mon May 20 09:57:54 2013] [error] [client ip2] proxy: Error reading from remote server returned by /link
[Mon May 20 09:57:54 2013] [debug] proxy_util.c(2029): proxy: HTTPS: has released connection for (myurl.com)

Any ideas how I can solve this issue, knowing that the majority of the requests are being successfully sent and the response is being received normally. All the requests are the same length and generated automatically so the problem couldn't be in the request itself.

Best Answer

Looks like there are some issue described in this Apache Bugzilla. The solution was adding these line in httpd.conf inside <Proxy> section

SetEnv force-proxy-request-1.0 1
SetEnv proxy-nokeepalive 1

Update

After deploying above solution, I faced another problem as the client was sending the request using HTTP/1.1 and the proxy was forced to use HTTP/1.0 as per the previous setEnv parameters, this caused HTTP error HTTP/1.1 417 Expectation Failed. This thread on SO mention that error could be solved from the client side or from the proxy side.

In the end, I implemented solution in proxy side based on this page. Now I've three parameters inside <Proxy> Section

SetEnv force-proxy-request-1.0 1
SetEnv proxy-nokeepalive 1
RequestHeader unset Expect early

I have been monitoring the solution since yesterday and it is working perfect until now. Also I performed a test yesterday with 500 transactions and they were all successful.

Update 2

It has been added to Apache docs since then.

Related Topic