Nginx – disable POST data caching

cachedisk-space-utilizationhttpnginxreverse-proxy

I noticed that when someone sends big file to Apache website proxied by Nginx, disk usage on Nginx machine goes up. It's especially noticeable when someone uploads file that is big comparing to disk size of Nginx machine. It rises obvious questions – what if someone uploaded lets say 500gb file while Nginx VM has only 10gb drive. It's not that much abstract scenario considering that it's our private cloud that we use for sending VM images (.vmdk or .ova files) which usually have 10+ gigabytes.

I'm already using:

proxy_buffering off;
proxy_no_cache 1;

in http scope. But it doesn't seem to affect uploaded files (only downloaded ones). Is it possible to disable POST caching?

Best Answer

nginx doesn't cache anything by default in it's proxy cache. It even doesn't have it configured. So if you have it configured and if you have configured the caching, you should know how to disable it, and that's not a sarcasm: since you're asking for a way to do this - your disk usage probably grows due to other reasons than stated; not due to caching. Nginx can buffer excessive POST data on the disk, but this gets deleted as soon as the transmission comes to an end.

Another possible scenario - you got a web-server that someone else has configured. Then you should check for proxy_cache <zonename> directives. But I doubt the latter is in effect.

You could easily determine what is consuming the disk by issuing a serie of du -h -d 1 / commands, replacing / on each iteration with more specific path of interest.

Related Solutions

Nginx – How to Set Up as a Caching Reverse Proxy

I don't think that there is a way to explicitly invalidate cached items, but here is an example of how to do the rest. Update: As mentioned by Piotr in another answer, there is a cache purge module that you can use. You can also force a refresh of a cached item using nginx's proxy_cache_bypass - see Cherian's answer for more information.

In this configuration, items that aren't cached will be retrieved from example.net and stored. The cached versions will be served up to future clients until they are no longer valid (60 minutes).

Your Cache-Control and Expires HTTP headers will be honored, so if you want to explicitly set an expiration date, you can do that by setting the correct headers in whatever you are proxying to.

There are lots of parameters that you can tune - see the nginx Proxy module documentation for more information about all of this including details on the meaning of the different settings/parameters: http://nginx.org/r/proxy_cache_path

http {
  proxy_cache_path  /var/www/cache levels=1:2 keys_zone=my-cache:8m max_size=1000m inactive=600m;
  proxy_temp_path /var/www/cache/tmp; 


  server {
    location / {
      proxy_pass http://example.net;
      proxy_cache my-cache;
      proxy_cache_valid  200 302  60m;
      proxy_cache_valid  404      1m;
    }
  }
}

Nginx – Disable caching when serving static files with Nginx (for development)

Since the answer is somehow hidden in the question - here is the solution for nginx in a VirtualBox environment as standalone answer.

In your nginx config (usally /etc/nginx/nginx.conf) or vhost config file change the sendfile parameter to off:

sendfile  off;

While sendfile is at the heart of Nginx's fame (blazing-fast low-level static file serving efficiency) it might be a bane for local development, e.g. Javascripts that change often and need to be reloaded. Nonetheless Nginx sendfile is smart and probably isn't most people's issue; check your browser's "disable cache" options as well!

Best Answer

Related Solutions

Nginx – How to Set Up as a Caching Reverse Proxy

Nginx – Disable caching when serving static files with Nginx (for development)

Related Topic