I am considering a scheme for passing a timestamp value in a cached response, and then sending it back to the service to perform cache invalidation. I'm wondering if that's a valid approach. Is that a common approach?
For example, if I have a service that returns a "Foo" record, which is a combination of DB lookups and calculations. Let's say Foo is identified with a key. To speed things up, I want to keep a local cache of results.
The problem is, across a bank of servers, the local caches will get out of sync, because the calculated results will change over time.
So my thought is:
- for a request, calculate a Foo
- store the Foo in local cache with an "as of" timestamp
- return the Foo along with the "as of" timestamp
- when the client wants to fetch a Foo, it passes the key as well as the "as of" timestamp.
- if the server finds the key in the local cache, and the "as of" timestamp in cache is newer than the "as of" parameter, return the cached Foo with the Foo, key and "as of" timestamp
- otherwise recalculate the Foo, cache it, and return the (new) Foo, key and (new) "as of" timestamp
The client would need to keep track of both the key and the "as of" timestamp, but then it would essentially control when service instances refresh their local cache, without the service instances talking to each other or some back-end storage for coordination.
If the client wanted to force a cache refresh, it could pass "as of" of max value, which would force a refresh on any service instance it hits.
Certainly I could put a caching layer in front of my service, and I could also save results in a distributed cache. Those solutions have their own data replication and expiration issues too. But for the purpose of this question, I really want to consider local caching & consider the architectural questions separately.
Q: Does that work? Is it valid or good to use a timestamp parameter as a mechanism for cache invalidation?
Best Answer
Local caching is simpler to implement. The issues to watch out for are:
With .NET I handled the cache miss problem by having the cache store the Task<> asynchronous wrapper, and using locking to ensure that only one Task<> to obtain the cached value was created. This worked well.
When the process to obtain the cache value fails, what does your application do? Does it cache the failure returning the error until the cache expires? Or do you create a new attempt to obtain the value?
When you have a multiple servers with independent caches and the underlying values changing regularly the answer will depend entirely on which server handles that request. This may not be an issue in your application, but think long and hard about what this means for your user and their confidence in your competence. If you keep getting conflicting data it can undermine trust.
Implementing a distributed cache with riak and key expiration solves many of these issues when you have multiple application servers. However it does not solve the cache population issue, when multiple threads on multiple servers all decide that the cache needs re-populating.