I have managed to set up Varnish for magento 2 using nginx to handle SSL. However if I try to use the vcl file provided by magento varnish stops working with the usual
"Error 503 Backend fetch failed
Backend fetch failed"
error. There is nothing in the logs nothing shows up anywhere. There is something dodgy in the configuration file. Anyone else had the same issue?
Update log file:
- ReqUnset Accept-Encoding: gzip, deflate, br
- ReqHeader Accept-Encoding: gzip
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 6 fetch
- Timestamp Fetch: 1575437126.769898 0.000150 0.000150
- RespProtocol HTTP/1.1
- RespStatus 503
- RespReason Backend fetch failed
- RespHeader Date: Wed, 04 Dec 2019 05:25:26 GMT
- RespHeader Server: Varnish
- RespHeader Content-Type: text/html; charset=utf-8
- RespHeader Retry-After: 5
- RespHeader X-Varnish: 5
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/5.2)
- VCL_call DELIVER
- RespUnset Age: 0
- RespHeader Pragma: no-cache
- RespHeader Expires: -1
- RespHeader Cache-Control: no-store, no-cache, must-revalidate, max-age=0
- RespUnset Server: Varnish
- RespUnset X-Varnish: 5
- RespUnset Via: 1.1 varnish (Varnish/5.2)
- VCL_return deliver
- Timestamp Process: 1575437126.769912 0.000163 0.000014
- RespHeader Content-Length: 278
- RespHeader Connection: close
- Timestamp Resp: 1575437126.769948 0.000200 0.000036
- ReqAcct 1158 0 1158 264 278 542
- End
Other part of the log file:
- BereqHeader grace: none
- BereqHeader Accept-Encoding: gzip
- BereqProtocol HTTP/1.1
- BereqHeader X-Varnish: 9
- VCL_call BACKEND_FETCH
- VCL_return fetch
- FetchError no backend connection
- Timestamp Beresp: 1575437127.586058 0.000018 0.000018
- Timestamp Error: 1575437127.586061 0.000021 0.000003
- BerespProtocol HTTP/1.1
- BerespStatus 503
- BerespReason Service Unavailable
- BerespReason Backend fetch failed
- BerespHeader Date: Wed, 04 Dec 2019 05:25:27 GMT
- BerespHeader Server: Varnish
- VCL_call BACKEND_ERROR
- BerespHeader Content-Type: text/html; charset=utf-8
- BerespHeader Retry-After: 5
- VCL_return deliver
- Storage malloc Transient
- ObjProtocol HTTP/1.1
- ObjStatus 503
- ObjReason Backend fetch failed
- ObjHeader Date: Wed, 04 Dec 2019 05:25:27 GMT
- ObjHeader Server: Varnish
- ObjHeader Content-Type: text/html; charset=utf-8
- ObjHeader Retry-After: 5
- Length 278
- BereqAcct 0 0 0 0 0 0
- End
I managed to narrow it down to a 404 error in the apache access.log. This seems to be the issue:
127.0.0.1 - - [04/Dec/2019:06:28:59 +0000] "GET /pub/health_check.php HTTP/1.1" 404 475 "-" "-"
127.0.0.1 - - [04/Dec/2019:06:29:09 +0000] "GET /pub/health_check.php HTTP/1.1" 404 475 "-" "-"
127.0.0.1 - - [04/Dec/2019:06:29:19 +0000] "GET /pub/health_check.php HTTP/1.1" 404 475 "-" "-"
127.0.0.1 - - [04/Dec/2019:06:29:29 +0000] "GET /pub/health_check.php HTTP/1.1" 404 475 "-" "-"
127.0.0.1 - - [04/Dec/2019:06:29:39 +0000] "GET /pub/health_check.php HTTP/1.1" 404 475 "-" "-"
127.0.0.1 - - [04/Dec/2019:06:31:15 +0000] "GET /pub/health_check.php HTTP/1.1" 404 475 "-" "-"
127.0.0.1 - - [04/Dec/2019:06:31:25 +0000] "GET /pub/health_check.php HTTP/1.1" 404 475 "-" "-"
varnishlog -g raw -i Backend_health shows
0 Backend_health - boot.default Still sick 4--X-R- 4 5 10 0.000516 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 4 5 10 0.000516 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 4 5 10 0.000798 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 4 5 10 0.000393 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 4 5 10 0.000796 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 3 5 10 0.000514 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 2 5 10 0.000511 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 1 5 10 0.000609 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 0 5 10 0.000646 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 0 5 10 0.000650 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 0 5 10 0.000580 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 0 5 10 0.000661 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 0 5 10 0.000842 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 0 5 10 0.000931 0.000000 HTTP/1.1 404 Not Found
0 Backend_health - boot.default Still sick 4--X-R- 0 5 10 0.000669 0.000000 HTTP/1.1 404 Not Found
the requested file is there and it is accessible. Unsure what causes the problem.
Best Answer
Based on the logs provided, a backend health check fails in Varnish for
/pub/health_check.php.If you have configured NGINX with
root /path/to/magento/pub(which you should, as it's more secure), then you need to edit the health check VCL code in order to removepub, e.g. here:Change your generated VCL code (earlier link is just to point location in VCL code):
to
Then restart Varnish.
Why: since NGINX's web root is pointed to
pubdirectory, all the web-accessible files in that directory are accessible from webroot, e.g.http://example.com/fooand nothttp://example.com/pub/bar.This is rather a bug with Magento 2 default VCL generation. Ideally, it should detect
pubvsno-pubsetup and generate the proper VCL.If anything, they should have defaulted to
pub as webrootsetup as it is most correct. (makes most of the Magento source files non-accessible from the web browser, thus secure).