Varnish throwing 503 errors when there is a healthy backend

The error message means (line number references refer to varnish 2.1.3):

While fetching a header [bin/varnishd/cache_fetch.c:399], either:

a) an overflow occurred [bin/varnishd/cache_httpconn.c:170]

or

b) a error occurred when calling read() [bin/varnishd/cache_httpconn.c:175]

The number at the end is the value of errno, so since it's 0 (no error) I would assume that option a) occurred since read() shouldn't return a negative number without setting errno.

The overflow is detected with the following code [bin/varnishd/cache_httpconn.c:167] returning a negative result:

i = (htc->ws->r - htc->rxbuf.e) - 1;    /* space for NUL */

htc->ws is a struct ws [bin/varnishd/cache.h:126] which is a "workspace structure" The r member is the reserved length of that workspace. htc->rxbuf refers to a struct txt [bin/varnishd/cache.h:109] but there's no comment describing what the members (b & e) refer to. Beginning and end perhaps?

I don't know how workspaces are resized (or even if they are) but - and I'm really in guessing territory here - I would assume that some possible causes of the problem are:

• A very large number of headers
• Very long headers
• A bug in how Varnish resizes workspaces (if it does)

It might be useful to try and find the point at which the error can be forced to occur by searching through the space of:

• Header lengths
• Numbers of headers

and see if you can reliably reproduce the problem.

You may be able to work around the problem by increasing the http_headers runtime option. (If you're running < 2.1, I think it's a compile-time or configure-time option)

This bug report seems to suggest a workaround to your problem; you should give it a try.

https://www.varnish-cache.org/trac/ticket/849