HTTP Caching with Authorization

cachinghttphttp-caching

Given a response from a web server that contains an Authorization header as per the OAuth spec does HTTP caching fail to be useful?

Request1 Authorization : AUTHTOKEN
Request2 Authorization : ANOTHERAUTOTOKEN

In this case given HTTP caching the second request would return the cached response for the first user. This isn't a problem for content that is generic across users, but this feels wrong for a shared cache to be providing responses for other users.

Likewise if we were to use a Vary header and vary by Authorization, this means our cache would store a cached copy per token which surely defeats the purpose of HTTP caching. The browsers local cache (private) would work fine, but this would still mean an origin request from each user at least once per session.

Edit

The service in question requires Authorization for all requests, however based on what I've read, serving responses from a Shared cache that include Authorization headers shouldn't be done unless must-revalidate, public, and s-maxage are present.

My question therefore is, given an API that has both generic (responses the same across all users) and user specific responses, is caching even possible? Having s-maxage and public headers but an authorization header would mean that the cache would resolve UserA's response to UserB, UserC and so on if I'm following the RFC correctly.

Best Answer

See http://greenbytes.de/tech/webdav/rfc7234.html#response.cacheability:

"A cache MUST NOT store a response to any request, unless: The request method is understood by the cache and defined as being cacheable, and ... the Authorization header field (see Section 4.2 of [RFC7235]) does not appear in the request, if the cache is shared, unless the response explicitly allows it, ..."

Introduction

The correct minimum set of headers that works across all mentioned clients (and proxies):

Cache-Control: no-cache, no-store, must-revalidate
Pragma: no-cache
Expires: 0

The Cache-Control is per the HTTP 1.1 spec for clients and proxies (and implicitly required by some clients next to Expires). The Pragma is per the HTTP 1.0 spec for prehistoric clients. The Expires is per the HTTP 1.0 and 1.1 specs for clients and proxies. In HTTP 1.1, the Cache-Control takes precedence over Expires, so it's after all for HTTP 1.0 proxies only.

If you don't care about IE6 and its broken caching when serving pages over HTTPS with only no-store, then you could omit Cache-Control: no-cache.

Cache-Control: no-store, must-revalidate
Pragma: no-cache
Expires: 0

If you don't care about IE6 nor HTTP 1.0 clients (HTTP 1.1 was introduced in 1997), then you could omit Pragma.

Cache-Control: no-store, must-revalidate
Expires: 0

If you don't care about HTTP 1.0 proxies either, then you could omit Expires.

Cache-Control: no-store, must-revalidate

On the other hand, if the server auto-includes a valid Date header, then you could theoretically omit Cache-Control too and rely on Expires only.

Date: Wed, 24 Aug 2016 18:32:02 GMT
Expires: 0

But that may fail if e.g. the end-user manipulates the operating system date and the client software is relying on it.

Other Cache-Control parameters such as max-age are irrelevant if the abovementioned Cache-Control parameters are specified. The Last-Modified header as included in most other answers here is only interesting if you actually want to cache the request, so you don't need to specify it at all.

How to set it?

Using PHP:

header("Cache-Control: no-cache, no-store, must-revalidate"); // HTTP 1.1.
header("Pragma: no-cache"); // HTTP 1.0.
header("Expires: 0"); // Proxies.

Using Java Servlet, or Node.js:

response.setHeader("Cache-Control", "no-cache, no-store, must-revalidate"); // HTTP 1.1.
response.setHeader("Pragma", "no-cache"); // HTTP 1.0.
response.setHeader("Expires", "0"); // Proxies.

Using ASP.NET-MVC

Response.Cache.SetCacheability(HttpCacheability.NoCache);  // HTTP 1.1.
Response.Cache.AppendCacheExtension("no-store, must-revalidate");
Response.AppendHeader("Pragma", "no-cache"); // HTTP 1.0.
Response.AppendHeader("Expires", "0"); // Proxies.

Using ASP.NET Web API:

// `response` is an instance of System.Net.Http.HttpResponseMessage
response.Headers.CacheControl = new CacheControlHeaderValue
{
    NoCache = true,
    NoStore = true,
    MustRevalidate = true
};
response.Headers.Pragma.ParseAdd("no-cache");
// We can't use `response.Content.Headers.Expires` directly
// since it allows only `DateTimeOffset?` values.
response.Content?.Headers.TryAddWithoutValidation("Expires", 0.ToString());

Using ASP.NET:

Response.AppendHeader("Cache-Control", "no-cache, no-store, must-revalidate"); // HTTP 1.1.
Response.AppendHeader("Pragma", "no-cache"); // HTTP 1.0.
Response.AppendHeader("Expires", "0"); // Proxies.

Using ASP.NET Core v3

// using Microsoft.Net.Http.Headers
Response.Headers[HeaderNames.CacheControl] = "no-cache, no-store, must-revalidate";
Response.Headers[HeaderNames.Expires] = "0";
Response.Headers[HeaderNames.Pragma] = "no-cache";

Using ASP:

Response.addHeader "Cache-Control", "no-cache, no-store, must-revalidate" ' HTTP 1.1.
Response.addHeader "Pragma", "no-cache" ' HTTP 1.0.
Response.addHeader "Expires", "0" ' Proxies.

Using Ruby on Rails:

headers["Cache-Control"] = "no-cache, no-store, must-revalidate" # HTTP 1.1.
headers["Pragma"] = "no-cache" # HTTP 1.0.
headers["Expires"] = "0" # Proxies.

Using Python/Flask:

response = make_response(render_template(...))
response.headers["Cache-Control"] = "no-cache, no-store, must-revalidate" # HTTP 1.1.
response.headers["Pragma"] = "no-cache" # HTTP 1.0.
response.headers["Expires"] = "0" # Proxies.

Using Python/Django:

response["Cache-Control"] = "no-cache, no-store, must-revalidate" # HTTP 1.1.
response["Pragma"] = "no-cache" # HTTP 1.0.
response["Expires"] = "0" # Proxies.

Using Python/Pyramid:

request.response.headerlist.extend(
    (
        ('Cache-Control', 'no-cache, no-store, must-revalidate'),
        ('Pragma', 'no-cache'),
        ('Expires', '0')
    )
)

Using Go:

responseWriter.Header().Set("Cache-Control", "no-cache, no-store, must-revalidate") // HTTP 1.1.
responseWriter.Header().Set("Pragma", "no-cache") // HTTP 1.0.
responseWriter.Header().Set("Expires", "0") // Proxies.

Using Clojure (require Ring utils):

(require '[ring.util.response :as r])
(-> response
  (r/header "Cache-Control" "no-cache, no-store, must-revalidate")
  (r/header "Pragma" "no-cache")
  (r/header "Expires" 0))

Using Apache .htaccess file:

<IfModule mod_headers.c>
    Header set Cache-Control "no-cache, no-store, must-revalidate"
    Header set Pragma "no-cache"
    Header set Expires 0
</IfModule>

Using HTML:

<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate">
<meta http-equiv="Pragma" content="no-cache">
<meta http-equiv="Expires" content="0">

HTML meta tags vs HTTP response headers

Important to know is that when an HTML page is served over an HTTP connection, and a header is present in both the HTTP response headers and the HTML <meta http-equiv> tags, then the one specified in the HTTP response header will get precedence over the HTML meta tag. The HTML meta tag will only be used when the page is viewed from a local disk file system via a file:// URL. See also W3 HTML spec chapter 5.2.2. Take care with this when you don't specify them programmatically because the webserver can namely include some default values.

Generally, you'd better just not specify the HTML meta tags to avoid confusion by starters and rely on hard HTTP response headers. Moreover, specifically those <meta http-equiv> tags are invalid in HTML5. Only the http-equiv values listed in HTML5 specification are allowed.

Verifying the actual HTTP response headers

To verify the one and the other, you can see/debug them in the HTTP traffic monitor of the web browser's developer toolset. You can get there by pressing F12 in Chrome/Firefox23+/IE9+, and then opening the "Network" or "Net" tab panel, and then clicking the HTTP request of interest to uncover all detail about the HTTP request and response. The below screenshot is from Chrome:

Chrome developer toolset HTTP traffic monitor showing HTTP response headers on stackoverflow.com

I want to set those headers on file downloads too

First of all, this question and answer are targeted on "web pages" (HTML pages), not "file downloads" (PDF, zip, Excel, etc). You'd better have them cached and make use of some file version identifier somewhere in the URI path or query string to force a redownload on a changed file. When applying those no-cache headers on file downloads anyway, then beware of the IE7/8 bug when serving a file download over HTTPS instead of HTTP. For detail, see IE cannot download foo.jsf. IE was not able to open this internet site. The requested site is either unavailable or cannot be found.

Rest – the difference between POST and PUT in HTTP

Overall:

Both PUT and POST can be used for creating.

You have to ask, "what are you performing the action upon?", to distinguish what you should be using. Let's assume you're designing an API for asking questions. If you want to use POST, then you would do that to a list of questions. If you want to use PUT, then you would do that to a particular question.

Great, both can be used, so which one should I use in my RESTful design:

You do not need to support both PUT and POST.

Which you use is up to you. But just remember to use the right one depending on what object you are referencing in the request.

Some considerations:

Do you name the URL objects you create explicitly, or let the server decide? If you name them then use PUT. If you let the server decide then use POST.
PUT is defined to assume idempotency, so if you PUT an object twice, it should have no additional effect. This is a nice property, so I would use PUT when possible. Just make sure that the PUT-idempotency actually is implemented correctly in the server.
You can update or create a resource with PUT with the same object URL
With POST you can have 2 requests coming in at the same time making modifications to a URL, and they may update different parts of the object.

An example:

I wrote the following as part of another answer on SO regarding this:

POST:

Used to modify and update a resource
POST /questions/<existing_question> HTTP/1.1
Host: www.example.com/
Note that the following is an error:
POST /questions/<new_question> HTTP/1.1
Host: www.example.com/
If the URL is not yet created, you should not be using POST to create it while specifying the name. This should result in a 'resource not found' error because <new_question> does not exist yet. You should PUT the <new_question> resource on the server first.

You could though do something like this to create a resources using POST:
POST /questions HTTP/1.1
Host: www.example.com/
Note that in this case the resource name is not specified, the new objects URL path would be returned to you.

PUT:

Used to create a resource, or overwrite it. While you specify the resources new URL.

For a new resource:
PUT /questions/<new_question> HTTP/1.1
Host: www.example.com/
To overwrite an existing resource:
PUT /questions/<existing_question> HTTP/1.1
Host: www.example.com/

Additionally, and a bit more concisely, RFC 7231 Section 4.3.4 PUT states (emphasis added),

4.3.4. PUT

The PUT method requests that the state of the target resource be created or replaced with the state defined by the representation enclosed in the request message payload.