Magento – loading shipping methods is very slow

Looking at Using Javascript to validate response in the text box pointed me in the right direction.

I still have to clean up the code and integrate it correctly in firecheckout, but the following code basically does exactly what I want:

/** 
 * Check if customer email address already exists 
 */ 

public function checkEmailexistsAction()
{       

    if ($this->_expireAjax()) {
        echo 'Ajax Expired'; 
        return;
    }
    if (!$this->getRequest()->isPost()) {
        echo '!isPost'; 
        return;
    }

    $websiteId = Mage::app()->getWebsite()->getId();
    $data = $this->getRequest()->getPost('billing', array());
    $email = $data['email'];

    $customer = Mage::getModel('customer/customer');

    if ($websiteId) {
        $customer->setWebsiteId($websiteId);
        }

    $customer->loadByEmail($email);

    if ($customer->getId()) {
        // This is the part of the code that runs when a customer exists ... 
        echo 'Yup - this one exists!, email address = ' . $email . ' and ID = ' . $customer->getId(); 
        }
    else {
        // This is the part of the code that runs when a customer does not yet exist ...
        echo 'Nope.. This one is not there...'; 
    }
}

Posting 'billing[email]=someemailaddress@somehost.org' to /firecheckout/index/checkEmailexists now reliably returns a yes or no.

It seems my main problem is session concurrency happening for whatever reason. I tried to use session caching with Redis as proposed in the documentation.

'session' =>
array (
  'save' => 'redis',
  'redis' =>
  array (
    'host' => '127.0.0.1',
    'port' => '6379',
    'password' => '',
    'timeout' => '2.5',
    'persistent_identifier' => '',
    'database' => '2',
    'compression_threshold' => '2048',
    'compression_library' => 'gzip',
    'log_level' => '1',
    'max_concurrency' => '6',
    'break_after_frontend' => '5',
    'break_after_adminhtml' => '30',
    'first_lifetime' => '600',
    'bot_first_lifetime' => '60',
    'bot_lifetime' => '7200',
    'disable_locking' => '0',
    'min_lifetime' => '60',
    'max_lifetime' => '2592000'
  )
),

But as I have the session locking problem I think this comes into account:

break_after_frontend

Number of seconds to wait before trying to break the lock for frontend (that is, storefront) session.

Meaning there is always a delay of max. 5 seconds. When I switched to memcached session it got slightly better because the wait was lower. It defaults to these settings:

memcached.sess_lock_wait_min => 1000
memcached.sess_lock_wait_max => 2000
memcached.sess_lock_retries => 5

These are the defaults from some older php-memcached version. The newest version proposes these defaults:

memcached.sess_lock_wait_min => 150
memcached.sess_lock_wait_max => 150
memcached.sess_lock_retries => 200

After reading these two: https://github.com/php-memcached-dev/php-memcached/pull/350 https://github.com/php-memcached-dev/php-memcached/issues/269

I tried with:

ini_set('memcached.sess_lock_wait_min', 150);
ini_set('memcached.sess_lock_wait_max', 150);
ini_set('memcached.sess_lock_retries', ini_get('max_execution_time') * 1000 / 150);

in Magento's index.php and this again speeds up the process. I now get a load of the checkout in roughly 4 seconds. The cart is a bit slower with ~5 secs because of the loading of totals-information. (all first try no browser caching)

I suppose fixing the session locking problem would again speed up the loading times as looking at the profiles reveals there is still a wait for session_start() from 300 to 800 ms for several calls.