Varnish – How to Fix 502 Errors on Large Files

varnish

Whenever I am serving large files with Varnish, is there some way to make Varnish just pipe files that are larger than a certain size? Varnish is giving me some 502 errors on large files (For example, 845472505 bytes (6.3G)) which I assume is because I only have 256m of malloc cache.

Current configuration:

DAEMON_OPTS="-a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -p send_timeout=604800 -p http_resp_hdr_len=8192 -s malloc,256m"

Note that I cannot just 'remove' the file – I just dont want to cache files over a certain size.

EDIT:
Since I cant answer my own question until 8 hours have passed…

So it seems that my backend timeout was not set high enough to transfer the file. I increased it, and it seems to have stopped giving me 502 errors on large files. I also converted to using the file cache (this was on a SSD VPS) so that I could store the whole file.

Best Answer

If you wish to prevent Varnish from caching a certain file, you can just send a no-cache header from your backend server. For example:

Cache-Control: private, no-cache, no-store, max-age=0

Alternatively you could examine the Content-Length header in VCL and take action based on that.

sub vcl_fetch {

  if (beresp.http.Content-Length !~ "[0-9]{10,12}") {
    return(deliver);
  }
}
Related Topic