Magento – Magento Full Page Cahe expire after catalog modification

ee-1.13enterprise-1.13full-page-cachemagento-enterprise

After using EE 1.12 for a while I migrate to EE 1.13.1, using the default new backend Mage_Cache_Backend_File.

After a new order or a catalog modification, the whole FPC expire completely (all categories and products).

I understand that after an order or catalog modification the related product and categories's cache should be refresh or invalidated, but not the entire FPC.

Any idea what i'm doing wrong?

Thanks

UPDATE

Looking in Enterprise/PageCache/etc/config.xml

The adminhtml event "catalogrule_after_apply" trigger "cleanCache()" which delete my FPC. By removing this event my FPC is not deleted after I update a product from the admin. Only the corresponding cache are correctly deleted.

Still have the issue, for new order that invalidate my whole FPC.

Best Answer

I assume the clearing happens when an order is placed because the inventroy is updated.

If you look at the config.xml for enterprise_pagecache you will see the following.

<cataloginventory_stock_full_reindex>
    <observers>
        <enterprise_pagecache>
            <class>enterprise_pagecache/observer_index</class>
            <method>invalidateCacheAfterFullReindex</method>
        </enterprise_pagecache>
    </observers>
</cataloginventory_stock_full_reindex>

It would appear that when the stock is updated after an order the the over ridding of the indexer for enterprise takes over.

<index>
    <indexer>
        <cataloginventory_stock>
            <model>enterprise_cataloginventory/index_dummy</model>
            <action_model>
                <changelog>enterprise_cataloginventory/index_action_refresh_changelog</changelog>
                <all>enterprise_cataloginventory/index_action_refresh</all>
            </action_model>
            <index_table>cataloginventory_stock_status</index_table>
            <sort_order>30</sort_order>
        </cataloginventory_stock>
    </indexer>
</index>

It looks like the event and method that actually call the reindex processes and this the cache clearing is sales_model_service_quote_submit_success -> reindexQuoteInventory

You don't have nearly enough RAM

We have about 240k products
Available ram: 6GB
Threads: 32

You do not have nearly enough RAM for the amount of products you have. As a rule of thumb, we recommend at least 2-4GB RAM per logical core.

If you map out your possible memory usage:

64 PHP Threads with a max_memory of ~768MB = 24GB
240,000 Products will likely mean about 15GB of InnoDB table space
64 PHP Threads will warrant around of 128 MySQL connections, typically this comes at a cost of about 200MB per connection minimum
Backend storage for 240,000 products in Redis AND lzf compressed - will still consume about 6GB of RAM

So the total so far is 70GB of committed RAM - we've not even mentioned the OS etc.

Your hardware is dreadfully underspecified. I would suggest reading over this Magento server set up article for a bit of a feeler on how to progress.

Memcached doesn't support cache tags

If you're using Memcached (not a problem, its very high performance), then you are either storing cache tags or not. If you don't have a slow_backend defined - then you're not storing tags, which basically means your cache cannot discriminate between any of the different cache types - so you won't be able to flush them independently.

Have a read over this, http://www.sonassi.com/knowledge-base/magento-kb/what-is-memcache-actually-caching-in-magento/

We would strongly suggest switching over to Redis. It has its quirks and does need significant fine-tuning for larger stores. But as a whole will perform slightly better than Memcached with the real benefit of cache-tag support.

404's and FPC

FPC has a real problem, in-fact, all caching engines have a problem with 404s. The reason being, any old URLs still being crawled or linked to will land on a page that has to iterate through the entire core_url_rewrite table, try and find a match against all defined routers and namespaces before finally giving up and loading a 404.

Then caching a resource which has no value and will consume space in your cache storage. You'll probably find a huge proportion of your Memcached storage is actually being eaten by 404 content.

With large catalogues (240k products), you're certainly going to have your fair share of product turnover, and thus, changes in URLs and subsequently, many 404's.

FPC Invalidate vs. Clean

At the moment - and by default - FPC's behaviour is to clean the cache on changes, rather than merely invalidate the cache entry. We wrote an extension to alter this behaviour for an EE store to do exactly what you required.

Here's a quick patch to give you an idea of how to solve your issue.

app/code/core/Enterprise/PageCache/etc/config.xml
index 6a56a80..85ebc92 100644
--- app/code/core/Enterprise/PageCache/etc/config.xml
+++ app/code/core/Enterprise/PageCache/etc/config.xml
@@ -139,7 +139,7 @@
             <observers>
                 <enterprise_pagecache>
                     <class>enterprise_pagecache/observer</class>
-                    <method>cleanCache</method>
+                    <method>invalidateCache</method>
                 </enterprise_pagecache>
             </observers>
         </catalogrule_after_apply>

Don't run a crawler

If you've got decent enough footfall - we don't advise running the crawl tool, it generates unnecessary load. People/bots/crawlers browsing the site should keep the cache primed.

But to answer your question, if you look in the configuration file mentioned above - you see the cron schedule that has been defined for the crawl browsing window.

If you can afford stale content

And ultimately, if you have enough RAM. You could well benefit from increasing the TTL of content stored in FPC - to keep your cached data alive for longer.

In the <full_page_cache> tag in your ./app/etc/local.xml just define

<lifetimelimit>86400</lifetimelimit>

The lifetime is defined in seconds. You need to strike a balance between content freshness, performance and the amount of storage space you actually have available.

Why are you using a 3rd party caching extension with EE

You're paying a premium for FPC - which pains me to say, is very good. So why are you running 3rd party alternatives over the top. Remove it.

Put it this way. If your car was running badly - would you just add another engine in the boot to compensate; or just fix the engine already in there?

Magento – Magento Full Page Cache Expires in 2 hours

It can depend on how your site/server was setup. That being said, in my app/etc/local.xml of a EE 1.13 site, using the built in Redis, I have the following:

<full_page_cache>
<backend>Mage_Cache_Backend_Redis</backend>
<backend_options>
    <server>127.0.0.1</server> <!-- or absolute path to unix socket -->
<port>6380</port>
<persistent></persistent> <!-- Specify a unique string like "cache-db0" to enable persistent connections. -->
<database>1</database> <!-- Separate database 1 to keep FPC separately -->
<password></password>
<force_standalone>0</force_standalone>  <!-- 0 for phpredis, 1 for standalone PHP -->
<connect_retries>1</connect_retries>    <!-- Reduces errors due to random connection failures -->
<lifetimelimit>57600</lifetimelimit>    <!-- 16 hours of lifetime for cache record -->
<compress_data>0</compress_data>        <!-- DISABLE compression for EE FPC since it already uses compression -->
</backend_options>
</full_page_cache>

Realistically, more information will need to be known because it will depend on what kind of server you have, who your host is (and whether it is a dedicated or shared box), what caching strategies are available on the server, etc.