Nginx – Why is curl requests from terminal always gives X-Cache-Status: MISS

cachecurlnginx

I have set up nginx reverse proxy and added caching to it using nginx caching.

I can verify whether the requested copy is cached or not using added header.

add_header X-Cache-Status $upstream_cache_status;

i have set to cache page if its get accessed more than twice.

proxy_cache_min_uses    2;

Nginx caching is working as it should, i can see the X-Cache-Status: MISS twice and later X-Cache-Status: HIT if i try to access the webpage from any browser.

But if i try to check same with curl ,
if the page is not cached i am getting

X-Cache-Status: MISS

and if the page is cached i am getting

X-Cache-Status: HIT

But my issue is , i cant able to warm up cache using curl. i mean i cant make webserver make cache specific page by accessing page using curl even if i access it 10 times using curl.

why is that ?

So far i have tried .

curl -I https://example.com/page1
curl -v https://example.com/page1
curl --verbose https://example.com/page1
curl -svo /dev/null https://example.com/page1

using latest centos 7×64

latest Nginx

[root]# curl -V
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.19.1 Basic ECC zlib/1.2.7 libidn/1.28 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz

update1:
here is my nginx config file

proxy_cache_path    /usr/local/nginx/cache levels=1:2   keys_zone=mycache:10m       max_size=10g    use_temp_path=off;

server {

access_log  /dev/null;
listen my_ip:80;
server_name example.com ;

location / {
    access_log off;

    proxy_cache             mycache;
    proxy_cache_min_uses    2;
    proxy_cache_valid       200     10m;
    proxy_cache_valid       404     1m;

    proxy_cache_use_stale   error timeout invalid_header updating http_500 http_502 http_503 http_504;  
    proxy_cache_lock        on;

    proxy_pass http://box1.example.com/;

    proxy_set_header    X-Real-IP   $remote_addr;
    proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;

    #hide headers 
    proxy_hide_header Cache-Control;

    #ignore header
    proxy_ignore_headers Cache-Control Expires;

    #add custom headers for users
    add_header X-Cache-Status $upstream_cache_status;
    expires 3h;
    add_header Cache-Control "public, max-age=10800, s-maxage=10800";

    #add no mobile redirect cookie
    add_header Set-Cookie "__cf_mob_redir=0;Domain=.example.com;";
}
}

update 2:
I am using cloudflare on my domain. I have tried disabling cloudflare service altogether and only using it as a DNS server . Tried again with curl and got same response.

It does not matter whether cloudflare is enabled or not curl cant make pages cache.


Update 3 :

Tried using different User-agnts but received same response.

curl -A "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3" -svo /dev/null http://example.com/some-page

Best Answer

0) As I can see you check https:// site but configuration for http:// version. I guess it's not a problem in your case but better to check that you edit right configuration file.
1) You should check what you got from upstream curl -I http://box1.example.com/page1 and I guess that there will be Set-Cookie header. So you should either add Set-Cookie to proxy_ignore_headers or remove cookies from upstream's reply. Or use separate locations: one for pages where you could ignore cookies for caching and another for pages where you can't ignore cookies.