Do application-level caches belong inside web servers

cachingweb services

A coworker and I have a disagreement about where the cache for a webserver should go. We currently have it implemented as a global (static) in-process cache, which I don't agree with. I think the cache should be offloaded into another server using something like redis. I've always been under the impression that global state is bad, and particularly bad in web servers. State belongs in databases. However, my coworker brings up a fair point which is that offloading it introduces the overhead of network transfer for every cache hit. I can't argue with that, but in-process cache just seems bad. I know we'll eventually run into concurrency issues with the global cache.

Do application-level caches belong inside of web servers, and under what circumstances is this a good choice?

Edit: Some information about what is being cached: In this case we have a microservice that transforms some data from another service before it's served to clients. The reason for caching is two-fold: 1) to reduce the load on the source service, and 2) to cache the results of the data transformation.

Best Answer

As always, one must differentiate: what information is to be stored in the cache?

I always go with these simple rules:

  • Information every webserver instance can calculate for itself should go into an in-RAM cache as these need to be accessible ASAP on request but have no need to be shared (never change or do not contain information relevant to other servers).
    • results of complex calculations that every server can perform independently
    • immutable, shared state (e.g. DB results that do not change: top-10 users of the previous week, ...)
    • Code-Metadata caches (e.g. Annotations or calculations resulting thereof)
  • Shared state that must be accessible faster than your DB can do should go into a separate cache server [cluster] (e.g. Redis, Memcached, ...). Note that many of these things can be tackled by an intelligent load balancer.
    • IP and User ban information
    • Session State (load balancers can handle this using affinity cookies if the application architecture allows it)
    • queues, domain events
  • Shared state that needn't be accessible faster than your DB can do must stay in the DB. Using caches for things that needn't necessarily be cached is a useless source of errors and failures.

As to the question edit: IMHO this data fall in the first category: Every server can calculate the transformation for itself. However, if we are talking about gigabytes of data here, i'd suggest that you do the transformation once, store the result in your DB and only cache the parts requested most frequently. You could also put another 16GB of RAM into your servers... Probably cheaper than implementing several cache layers.

Related Topic