Magento – Tweaking magento for performance

magentoperformance

i'm looking on performance (server load time) of magento site and i'm trying to tune search result pages. I realized that when I disabled all heavy things like top navigation, lev layered navigation and product listing and I cleared all cache then after this magento core does like 60 SQL queries agains a database. Does anyone have any procedure how to rid of them or how to reduce them to some acceptable amount?

Also can I somehow reduce a time spent during creating of blocks?

Thank you very much,
Jaro.

Best Answer

Magento is a extremely flexible ecommerce framework, but that flexibility comes with a price: performance. This answer is a collection of pointers and some details on caching (especially for blocks).

One thing to consider is the Magento environment, e.g. tuning the php, the web server (favor nginx over Apache), and MySQL. Also, set up a good caching backend for Magento. All these are covered e.g. in the Magento Performance Whitepaper that applies also to the CE.

After the environment is set up, the other side of things is the code.
Reducing the number of queries is possible for some pages by enabling the flat table catalog (System > Configuration > Catalog > Frontend), but you will always have a high number of queries.

You also can't really reduce the time spent creating the blocks except by tuning the environment (APC, memory, CPU). So as the other commenters said, your best choice is utilizing the caching functionality that Magento has built in.

Magento Block Caching

Because you specifically mentioned blocks in the question, I'll elaborate a bit more on block caching. Block caching is governed by three properties:

cache_lifetime
cache_key
cache_tags

All these properties can be set in the _construct() method of a block using setData() or magic setters, or by implementing the associated getter methods (getCacheLifetime(), getCacheKey(), getCacheTags()).

The cache_lifetime is specified in (integer) seconds. If it is set to false(boolean), the block will be cached for ever (no expiry). If it is set to nullthe block will not be cached (this is the default in Mage_Core_Block_Abstract).

The cache_key is the unique string that is used to identify the cache record in the cache pool. By default it is constructed from the array returned by the method getCacheKeyInfo().

// Mage_Core_Block_Abstract
public function getCacheKeyInfo()
{
    return array(
        $this->getNameInLayout()
    );
}

public function getCacheKey()
{
    if ($this->hasData('cache_key')) {
        return $this->getData('cache_key');
    }
    /**
     * don't prevent recalculation by saving generated cache key
     * because of ability to render single block instance with different data
     */
    $key = $this->getCacheKeyInfo();
    //ksort($key);  // ignore order
    $key = array_values($key);  // ignore array keys
    $key = implode('|', $key);
    $key = sha1($key);
    return $key;
}

The best way to customize the cache key in custom blocks is to override the getCacheKeyInfo() method and add the data that you need to uniquely identify the cached block as needed.

For example, in order to cache a different version of a block depending on the customer group you could do:

public function getCacheKeyInfo()
{
    $info = parent::getCacheKeyInfo();
    $info[] = Mage::getSingleton('customer/session')->getCustomerGroupId()
    return $info;
}

The cache_tags are an array that enable cache segmentation. You can delete sections of the cache matching one or more tags only.
In the admin interface under System > Cache Management you can see a couple of the default cache tags that are available (e.g. BLOCK_HTML, CONFIG, ...). You can use custom cache tags, too, simply by specifying them.
This is part of the Zend_Cache implementation, and needs to be customized far less frequently compared to the cache_lifetime and the cache_key.

Other Caching

Besides blocks Magento caches many other things (collection data, configuration, ...).
You can cache your own data using Mage::app()->saveCache(), Mage::app()->loadCache(), Mage::app()->cleanCache() and Mage::app()->removeCache(). Please look in Mage_Core_Model_App for details on these methods, they are rather straight forward.

You will also want to use a full page cache module. If you are using the Magento EE, you already have one. Otherwise search Magento Connect - there are many options (commercial).
Some of those modules also tune various parts of Magento for you beyond the full page caching aspect, e.g. Nitrogento (commercial).

Using a reverse proxy like Varnish is also very beneficial.

There are quite a number of blog posts on this subject. Here is one post by the publishers of the Nitrogento extension.
If you are running Magento on a more low-scale environment, check out my post on the optimization of the file cache backend on magebase.com.

Related Solutions

Javascript – Which “href” value should I use for JavaScript links, “#” or “javascript:void(0)”

I use javascript:void(0).

Three reasons. Encouraging the use of # amongst a team of developers inevitably leads to some using the return value of the function called like this:

function doSomething() {
    //Some code
    return false;
}

But then they forget to use return doSomething() in the onclick and just use doSomething().

A second reason for avoiding # is that the final return false; will not execute if the called function throws an error. Hence the developers have to also remember to handle any error appropriately in the called function.

A third reason is that there are cases where the onclick event property is assigned dynamically. I prefer to be able to call a function or assign it dynamically without having to code the function specifically for one method of attachment or another. Hence my onclick (or on anything) in HTML markup look like this:

onclick="someFunc.call(this)"

onclick="someFunc.apply(this, arguments)"

Using javascript:void(0) avoids all of the above headaches, and I haven't found any examples of a downside.

So if you're a lone developer then you can clearly make your own choice, but if you work as a team you have to either state:

Use href="#", make sure onclick always contains return false; at the end, that any called function does not throw an error and if you attach a function dynamically to the onclick property make sure that as well as not throwing an error it returns false.

Use href="javascript:void(0)"

The second is clearly much easier to communicate.

Sqlite – Improve INSERT-per-second performance of SQLite

Several tips:

Put inserts/updates in a transaction.
For older versions of SQLite - Consider a less paranoid journal mode (pragma journal_mode). There is NORMAL, and then there is OFF, which can significantly increase insert speed if you're not too worried about the database possibly getting corrupted if the OS crashes. If your application crashes the data should be fine. Note that in newer versions, the OFF/MEMORY settings are not safe for application level crashes.
Playing with page sizes makes a difference as well (PRAGMA page_size). Having larger page sizes can make reads and writes go a bit faster as larger pages are held in memory. Note that more memory will be used for your database.
If you have indices, consider calling CREATE INDEX after doing all your inserts. This is significantly faster than creating the index and then doing your inserts.
You have to be quite careful if you have concurrent access to SQLite, as the whole database is locked when writes are done, and although multiple readers are possible, writes will be locked out. This has been improved somewhat with the addition of a WAL in newer SQLite versions.
Take advantage of saving space...smaller databases go faster. For instance, if you have key value pairs, try making the key an INTEGER PRIMARY KEY if possible, which will replace the implied unique row number column in the table.
If you are using multiple threads, you can try using the shared page cache, which will allow loaded pages to be shared between threads, which can avoid expensive I/O calls.
Don't use !feof(file)!

I've also asked similar questions here and here.