Magento – Varnish and Turpentine

I am finding that whenever I restart Varnish on my server, I am loosing my sessions for my users.

This is is turn making my customers loose their shopping carts.

Is this normal behaviour for Varnish or is my VCL to blame? It would seem that its not

Further Info.

On further investigation, it appears that this problem is related to Issue #725 on GitHub.

My Magento installation is version 1.9.1.0. It should also be said that my entire frontend is being run under https. I am using Pound in front of Varnish to terminate SSL.

It appears that the default Magento behaviour in this version is to create a secondary frontend cookie, typically called 'frontend_cid', In an attempt to test against MITM attacks.

It looks like the generated VCL file by Turpentine is not passing on this cookie, which is causing invalid sessions.

Can anyone explain how the VCL file passes on the cookies that Magento makes on to the Client?

I have narrowed this down to Varnish not generating the required cookies.

As of Magento 1.9.1.0, A 'frontend_cid' cookie was introduced to block MITM attacks.

This can be found in the Mage_Core_Model_Session_Abstract_Varien class, at line 135

if (Mage::app()->getFrontController()->getRequest()->isSecure() && empty($cookieParams['secure'])) {
    // secure cookie check to prevent MITM attack
    $secureCookieName = $sessionName . '_cid';
    if (isset($_SESSION[self::SECURE_COOKIE_CHECK_KEY])
        && $_SESSION[self::SECURE_COOKIE_CHECK_KEY] !== md5($cookie->get($secureCookieName))
    ) {
        session_regenerate_id(false);
        $sessionHosts = $this->getSessionHosts();
        $currentCookieDomain = $cookie->getDomain();
        foreach (array_keys($sessionHosts) as $host) {
            // Delete cookies with the same name for parent domains
            if (strpos($currentCookieDomain, $host) > 0) {
                $cookie->delete($this->getSessionName(), null, $host);
            }
        }
        $_SESSION = array();
    }
    if (!isset($_SESSION[self::SECURE_COOKIE_CHECK_KEY])) {
        $checkId = Mage::helper('core')->getRandomString(16);
        $cookie->set($secureCookieName, $checkId, null, null, null, true);
        $_SESSION[self::SECURE_COOKIE_CHECK_KEY] = md5($checkId);
    }
}

In order to provide secure connections for clients, Varnish has to genereate a 'frontend' cookie, which Magento will later use to identify that particular customer. So far, it appears to do this just fine. However, it looks like as of Magento 1.9.1.0, it now also needs to generate the 'frontend_cid' cookie.

Varnish has to do this because, by caching the response, it also caches the response header, which contains the 'frontend' cookies.

Therefore, by default, Varnish blasts away any cookies that the backend responds with when its handling 'lookup' or 'pass' conditions. It does this to stop multiple users being issued with the same cached frontend cookie (that would compromise peoples sessions).

Any time varnish handles the request with 'pipe', Magento is able to create the required cookies and attach them to the users browser. This result in the system failing the initial validation, but then providing a new session to the user. This symptom manifests as a loss of cart or the inability to add products to the shopping cart.

The Turpentine VCL will 'pipe' any request that IS NOT of method type GET or HEAD as seen by this code in the vcl_recv function:

// We only deal with GET and HEAD by default
// we test this here instead of inside the url base regex section
// so we can disable caching for the entire site if needed
if (!true || req.http.Authorization ||
    req.request !~ "^(GET|HEAD)$" ||
    req.http.Cookie ~ "varnish_bypass=1") {
    return (pipe);
}

Therefore, the symptom is most noticable when user attempts to add an item to their cart, or attempts to checkout for the first time.

How to fix?

I believe the solution to this issue is to have the Turpentine VCL also create a 'frontend_cid' cookie for incoming visitors, and then have the turpentine module add that cookie to the current session like it does now for the 'frontend' cookie.

So… how do we implement this?

Caveat: I could be wrong, I am very new to Varnish, but I have spent a lot of hours on this now and this is what I am seeing, anyone support right now would be greatly appreciated.

FINAL UPDATE AND MY CHOSEN FIX – 2015 10 30

It is impossible to create a 'frontend_cid' cookie in varnish as the cookie is created at random on the server by Magento and stored as a MD5 hash in the customers session. This stops you from creating in externally outside of the customers session.

The best solution I came up with on this issue is to instead overwrite the way Magento handles customer sessions.

Currently Magento handles invalid sessions like this:

IF
    The requested session by the customer is flagged as invalid
THEN
    Stop processing request
    Redirect to the appropriate page

My new logic goes as follows:

IF
    The requested session by the customer is flagged as invalid
THEN
    Create a new session
    Complete the requested task
    Redirect to the appropriate page

My new approach allows varnished to handle the customers response on even the very first visit. Which is not how the latest implementation of turpentine works.

My Issue, Issue #829 – /nexcess/magento-turpentine/issues/829 on GitHub. A copy of my VCL can be found here.

My issue on GitHub has been closed as it is a duplicate of a much older problem found here:

Issue #345