If you need to leave the key=SOMEALPHANUM intact to the backend in case not delivered from cache, then it might be better to regsub within the vcl_hash function instead, as this won't really alter the url but instead it just alters the hash of the key.
sub vcl_hash {
if(req.http.host ~ "the-site-in-question.com" & req.url ~ "^/api/") {
set req.http.X-Sanitized-URL = req.url;
set req.http.X-Sanitized-URL = regsub(req.http.X-Sanitized-URL, "&key=[A-Za-z0-9]+", "");
set req.hash += req.http.X-Sanitized-URL;
} else {
set req.hash += req.url;
}
set req.hash += req.http.host;
hash;
}
I am fairly new to caching with Varnish, but here is what I've learned so far: There are several factors to consider when using Varnish for caching against an application.
In your case, know what cookies are being set and for what purpose. If varnish sees a cookie with your request, you will be passed to the backend, resulting in a cache miss.
Google Analytics Cookies
If you are using Google Analytics cookies, you can safely unset them in Varnish; don't worry, you will still the data in your GA reports. Use something like this in your vcl_recv
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|__utma_a2a)=[^;]*", "");
You can try a couple more cleanup lines, also in the vcl_recv
Remove ";" prefixes from Cookies
set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
Unset empty cookies
if (req.http.Cookie ~ "^\s*$") {
unset req.http.Cookie;
}
Application Specific Cookies
If your application sets a cookie when a user logs in to perform a function, those requests should not be cached and sent to the backend directly. Otherwise, you could cached pages viewed by logged in users (bad).
Use something like this:
if (req.http.Authorization || req.http.Cookie) {
return (pass);
}
HTH & good luck.
Edit
Use this to see what cookies varnish is seeing come through:
varnishtop -i RxHeader -I Cookie
If you're regex misses any, catch 'em here!
Best Answer
Yes - you need to "return( miss );" on the requests in the backend. vcl_fetch is only triggered on 'fetching' an item from the backend - if an item is already stored in cache, varnish will not run the fetch code.