Why, sometimes the full content is returned unchanged, while If-Modified-Since is set

apache-2.2cachedate-modifiedhttp-headerslast-modified

my php-generated thumbnail files and combined javascript and css files (all having their expiration dates anc cache=public set etc) return the full content sometimes, other times they load from proxy cache!

This is the comment that redbot.org gives, sometimes:

An If-Modified-Since conditional request returned the full content unchanged.

*HTTP allows clients to make conditional requests to see if a copy that they hold is still valid. Since this response has a Last-Modified header, clients should be able to use an If-Modified-Since request header for validation. RED has done this and found that the resource sends a full response even though it hadn't changed, indicating that it doesn't support Last-Modified validation.**

the PHP thumbnail generator header

header ("Content-type: image/jpeg");
header ("Last-Modified: " . gmdate("D, d M Y H:i:s", time() - 404800000)." GMT");
header ("Expires: " . gmdate("D, d M Y H:i:s", time() + 2419200) . " GMT");
header ("Cache-Control: public, max-age=2419200");

htaccess

<IfModule mod_headers.c>
ExpiresActive On
ExpiresDefault M172800

##### DEFAULT EXPIRES
<FilesMatch "\\.(ico|jpg|png|gif|svg|swf|css|js|fon|xml|pdf|flv)$">
    ExpiresDefault M1209600
    Header set Cache-Control "max-age=1209600, public"
</FilesMatch>

##### DYNAMIC PAGES
<FilesMatch "\\.(php|cgi|pl)$">
    ExpiresDefault M7200
    Header set Cache-Control "public, max-age=7200"
</FilesMatch>

Header unset Pragma
Header unset ETag
Header unset Last-Modified
FileETag None
</IfModule>

Update: New Info:

google SpeedTest gives 91/100 score on the top it says: and list all the files except the php generated files that have a explicit expire header set ((which still return full content!??) it says:

The following resources are missing a cache validator. Resources that do not specify a cache validator cannot be refreshed efficiently. Specify a Last-Modified or ETag header to enable cache validation for the following resources.

Best Answer

Can you use Firebug and paste the HTTP response headers. I have a feeling that even though in your code you are setting cache control to Public, Apache is overriding that because for Php File type you are setting Cache-Control to private.

One thing you can do it remove the cache settings for Dynamic Pages from Apache configuration. That should fix the problem because a Proxy doesn't cache a response without the correct headers.

EDIT

Hi Sam, Revisiting you question I found the solution to the problem. The following code snippet appears to be a problem. In the output of your Php the Last-Modified header always changes and when a browser sends a 304 If modified request it see a change and therefore re-requests that content.

header ("Last-Modified: " . gmdate("D, d M Y H:i:s", time() - 404800000)." GMT");

Unset Last-Modified and ETags from your content to speed up the website. This site provides some excellent tips too.
http://www.askapache.com/htaccess/apache-speed-last-modified.html

Related Solutions

Linux – Apache gzip configuration

Here's an ideal .htaccess that both compresses and sets sutiable expire headers.

# Insert filter
SetOutputFilter DEFLATE

# Netscape 4.x has some problems...
BrowserMatch ^Mozilla/4 gzip-only-text/html

# Netscape 4.06-4.08 have some more problems
BrowserMatch ^Mozilla/4\.0[678] no-gzip

# MSIE masquerades as Netscape, but it is fine
# BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

# Make sure proxies don't deliver the wrong content
Header append Vary User-Agent env=!dont-vary

<IfModule mod_expires.c>
  ExpiresActive On
  ExpiresDefault "access plus 1 seconds"
  ExpiresByType image/x-icon "access plus 2592000 seconds"
  ExpiresByType image/jpeg "access plus 2592000 seconds"
  ExpiresByType image/png "access plus 2592000 seconds"
  ExpiresByType image/gif "access plus 2592000 seconds"
  ExpiresByType application/x-shockwave-flash "access plus 2592000 seconds"
  ExpiresByType text/css "access plus 604800 seconds"
  ExpiresByType text/javascript "access plus 216000 seconds"
  ExpiresByType application/x-javascript "access plus 216000 seconds"
  ExpiresByType text/html "access plus 600 seconds"
  ExpiresByType application/xhtml+xml "access plus 600 seconds"
</IfModule>

<IfModule mod_headers.c>
  <FilesMatch "\\.(ico|jpe?g|png|gif|swf)$">
    Header set Cache-Control "max-age=2592000, public"
  </FilesMatch>
  <FilesMatch "\\.(css)$">
    Header set Cache-Control "max-age=604800, public"
  </FilesMatch>
  <FilesMatch "\\.(js)$">
    Header set Cache-Control "max-age=216000, private"
  </FilesMatch>
  <FilesMatch "\\.(x?html?|php)$">
    Header set Cache-Control "max-age=600, private, must-revalidate"
  </FilesMatch>
</IfModule>

<IfModule mod_headers.c>
  Header unset ETag
</IfModule>
FileETag None

<IfModule mod_headers.c>
  Header unset Last-Modified
</IfModule>

The following article covers what it does and also talks about compression:

http://www.samaxes.com/2009/01/06/more-on-compressing-and-caching-your-site-with-htaccess/

Hope that helps.

Tomcat – Apache2 – mod_expire and mod_rewrite not working in httpd.conf – serving content from tomcat

Probably the ProxyPass / directive you have in your config routes all requests to the Tomcat backend, bypassing all your mod_rewrite directives. You can try to remove the ProxyPass directive and use RewriteRule with the [P] flag after your other rewrites:

mod_rewrite: P|proxy

For the expire headers the situation is worse — in another similar question a solution was not found.

Option +FollowSymLinks would be needed for using mod_rewrite in .htaccess files; in your case it is not required, because you can put everything in the Apache config. Moving rules from config to .htaccess is pointless and can only make things slower. However, there is an important difference between .htaccess and the Apache config — in .htaccess patterns in RewriteRule are matched against relative filesystem paths (which never start with /), while in the VirtualHost context these patterns are matched against the URL path after the hostname, which always starts with /. Therefore at least one of your rules is incorrect:

RewriteRule ^content/(js|css)/([a-z]+)-([0-9]+)\.(js|css)$ /content/$1/$2.$4

should be

RewriteRule ^/content/(js|css)/([a-z]+)-([0-9]+)\.(js|css)$ /content/$1/$2.$4

(note the additional slash in the beginning).