Magento – How to implement master/slave database configuration in Magento 2 community edition

We've got a fairly vast experience of MySQL clusters - and Percona have worked with us on a number of occasions when pushing the boundaries of complex configurations.

Can Magento natively handle read-only slaves

Magento is natively capable of splitting off reads/writes to different database servers (with the exception of a few broken releases, eg. EE 1.11) - allowing you to offset select load to an additional (or more) server(s); and forwarding all the update/write queries to a single master.

When should I do it

This is a more appropriate question. With dedicated Magento operating systems like MageStack - it is becoming more common for in-built server side advanced caching techniques to be available and easily used (such as Varnish front end caching and Redis back end caching).

Historically, Magento has never been bound by MySQL - but rather PHP. But as Varnish and Full Page Caching (FPC) are used more frequently, the burden of repeated tasks (category/product loads, frequent searches) is suddenly absorbed and PHP becomes less of a burden. In fact, it only really comes into play to generate the content initially, or complete non-cachable scenarios (add to cart, order completion etc.); for the purpose of explanation we're deliberately ignoring administrative load.

We have always stood by the fact that MySQL isn't an areas of concern for most retailers, as seen both here and here. But if your in the region of processing hundreds of orders per hour, not single or double digits - it will soon become an areas for optimisation.

Ultimately for smaller stores (<25k daily unique visitors)

Your efforts would be far better focused on simply finding an appropriate host who can suggest the right hardware to be on from the offset and that has configured the machine in the most optimal fashion for your store. Don't waste your time pursuing Master/Slave or Master/Master configurations - which will yield no performance benefit and will ultimately require continual attention and advanced MySQL knowledge.

Ultimately hardware sizing and selection will have a bigger part to play than MySQL optimisation.

But for larger stores

As your store starts to grow, converting or transactional load becomes more of a burden with the repeated task of completing complex inserts and updates. The addition of each new order will trigger the decrement of catalogue stock, callbacks from payment gateways and updates from EPOS/ERP systems. Combine this with the associated cache purge of the respective products/categories and you'll soon see MySQL load disproportionately increase.

Multi-master is never a solution we recommend or consider as a viable option, but Master/Slave can yield benefits (we stress, on Enterprise-size stores) by offsetting read load to secondary/tertiary nodes.

But I still want to do it

First configure your slaves. We're big advocates of the Percona utilities and MySQL branches - they have an ideal tool for taking hot backups of your existing DB - innobackupex. There is a good write up here.

On the master

Replace $TIMESTAMP or tab complete.

mysql
> GRANT REPLICATION SLAVE ON *.*  TO 'repl'@'$slaveip' IDENTIFIED BY '$slavepass';
> quit;
innobackupex --user=username --password=password /path/to/backupdir
innobackupex --user=username --password=password /
       --apply-log /path/to/backupdir/$TIMESTAMP/

rsync -avprP -e ssh /path/to/backupdir/$TIMESTAMP TheSlave:/path/to/mysql/
scp /etc/mysql/my.cnf TheSlave:/etc/mysql/my.cnf

On the slave

/etc/init.d/mysql stop
mv /path/to/mysql/datadir /path/to/mysql/datadir_bak
mv /path/to/mysql/$TIMESTAMP /path/to/mysql/datadir
chown -R mysql:mysql /path/to/mysql/datadir
sed -i 's#server-id=1#server-id=2#g' /etc/mysql/my.cnf
/etc/init.d/mysql start
cat /var/lib/mysql/xtrabackup_binlog_info
> TheMaster-bin.000001     481

mysql
> CHANGE MASTER TO MASTER_HOST='$masterip', MASTER_USER='repl', MASTER_PASSWORD='$slavepass', MASTER_LOG_FILE='TheMaster-bin.000001', MASTER_LOG_POS=481;
> START SLAVE;

Then once your slave is operational, in practice, it only takes a few additional lines of code to achieve.

In ./app/etc/local.xml

<default_read>
  <connection>
    <use/>
    <host><![CDATA[host]]></host>
    <username><![CDATA[username]]></username>
    <password><![CDATA[password]]></password>
    <dbname><![CDATA[dbname]]></dbname>
    <type>pdo_mysql</type>
    <model>mysql4</model>
    <initStatements>SET NAMES utf8</initStatements>
    <active>1</active>
  </connection>
</default_read>

Sources

You're getting into a broad, broad world of optimisation here and there certainly isn't a one size fits all approach.

Define performance

Do you mean the page load time for a single user, or the overall capacity/total concurrency? The two are very distinctly different - and not strictly related. It is entirely possible to have a fast store with limited capacity; or a slow store with lots of capacity.

So when addressing either type of performance:

1. Single user perceived page load time
2. Total capacity/concurrency

You have to tackle each independently with their own solutions - especially since each have their own bottlenecks.

Lets make the assumption you are with a competent host that has already configured the other aspects of your server optimally for your store.

Single user perceived page load time

Is MySQL the bottleneck
No. Not directly. Its all about latency, in the majority of cases when testing page load time - only the caches will be hit. So the key here is to minimise latency.

• Tune MySQL cache sizes appropriately (there is no right answer, we tune settings entirely differently, monthly, per store)
• Reduce network latency. For 64 byte frames; 51.2µs for 10Mbps, 5.12µs for 100Mbps and 4.096µs for 1Gbps. This gives a improvement of 20% just by transitioning from 100Mbps to 1Gbps. ^s1
• Increase network capacity. You would be surprised at the many megabytes per second being exchanged between a Web and DB server, usually in excess of 10MB/s - so a minimum of 100Mb/s is required ^s1. Or, just move the DB server locally.
• Using SOLR. External engines are sometimes better suited, SOLR certainly is faster for LARGER catalogues (and I'd stress, larger catalogues). Even un-tuned SOLR will produce layered navigation and search results faster than MySQL can.

But these changes will have such a fractional impact on page load time - where the bottleneck is really elsewhere.

• Tune the application. Magento has some fairly big bugs with the way it builds collections and caches them; we've come across a number of big core code issues that can cripple performance. In a few cases, simply removing the product count display on the layered navigation results shaved 2 seconds of loading a big collection.
• Review MySQL slow logs. Check slow queries and add indexes as necessary. The difference between running a complex query with multiple joins with and without appropriate indexes can be tens of seconds.

The application is the bottleneck. Not the software. So merely improving core-code or making your template less heavy will have a far more dramatic effect on performance than ANY MySQL configuration change.

What wouldn't we bother with

• Changing the storage engine. MariaDB and Percona share the same InnoDB engine - Percona XtraDB. They can be treated as one and the same. In terms of single query execution time - performance will exactly mirror a vanilla MySQL build. This comes into play under load/concurrency.
• Running a MySQL slave. This won't improve performance unless the slave is located physically closer (from a network perspective), or that the slave has better hardware than the master. This comes into play under load/concurrency.
• Running an external DB server. This is by far the worst advice we see repeatedly handed out by many hosts/agencies. Until you have hit a ceiling on hardware/resources or you've got multiple web servers (read: high-availability), MySQL on the local machine for a Magento store is A Good Idea. It cuts out all the network overhead and latency. Even a 100Gb/s network (yes, one hundred gigabits per second) will not compare with a local unix socket for raw volume, throughput and latency.

^s1 For separate database servers only. Doesn't apply to local DB servers.

Total capacity/concurrency

Is MySQL the bottleneck
Maybe. But only once you've nailed your PHP performance and capacity to the point where MySQL is slowing things down. If you've got Varnish and FPC properly configured (don't get us started on how many failed attempts we've seen with either) - then MySQL does become a bottleneck.

So in addition to the above modifications.

• Change MySQL engine. XtraDB can excel under load and does show genuine benefits over a stock MySQL distribution.
• Stay up to date with MySQL. 5.5 performs better than 5.0 under load.
• Change PHP MySQL driver. PHP 5.3 and newer has a native MySQL driver, but in some circumstances, we've found PHP 5.2 with the separate driver to outperform MySQLND for Magento. Test it for yourself
• Change search engine. Moving the search out to SOLR/Sphinx (or even some 3rd party external services) can really alleviate the burden of non-transactional load (ie. people not placing orders)
• Change layered navigation engine. Again, SOLR is a brilliant engine for layered navigation and due to its non-locking nature is far faster than MySQL.
• Add a MySQL slave. This does help browsing (non-transactional) load, but won't help you process more orders per hour - as it is reliant on the Master to process and replicate this data

What wouldn't we bother with

• Master/Master. Due to the pretty high tipping point of hardware saturation of a Master/Slave set up (in excess of 1000 orders per hour) - we've never found it a requirement to use Master/Master in production. We have performed extensive testing, but never found it to be advantageous from a performance perspective and with the inherent risks and problems of Master/Master, it simply isn't worth it.

Read vs Write Scalability

The last paragraph really leads on to a key area of read and write scalability. Read scaling can be performed fairly infinitely without too much complication with the addition of more and more slaves.

Given Magento's ratio of Reads to Writes is about 0.1% - considering writes shouldn't be much of a concern. That's why I've not bothered mentioning MySQL Cluster and its clever features like auto-sharding (splitting tables off to separate machines).

Hardware, hardware, hardware

Hardware is easily the quickest answer when it comes to improvement, so I've deliberately not mentioned it above in both scenarios.

But all the software changes in the world aren't going to make any difference if your underlying hardware is insufficient. That could mean...

• Low-quality switches with limited buffers
• Overly saturated network links
• Geographically distant servers
• Poor network QoS/CoS
• Limited total amount of RAM
• Low memory bandwidth RAM
• Low IOPs HDD subsystem
• Software RAID controller
• Low clock speed CPU
• Low bandwidth chipset
• Hardware virtualisation (almost all types apart from Kernel Level Virtualisation)

Nowadays, there's a really high ceiling on how high you can actually scale on hardware. Lets ignore the myth of infinite scaling "in the cloud" as cloud hardware tends to be fairly mediocre. For example Amazon's flagship models only being 12 Cores @ 3.3GHz. But outside of this, there are some very powerful servers available - our top server has 160 cores and 2TB (yes, Terabytes) of RAM. We've not seen anyone exceed the capabilities of that just yet.

So you've got a massive scope for vertical scaling, before you even need to consider horizontal scaling.

The ever moving target

Its worth mentioning that in the pursuit of performance, the bottleneck will always keep moving.

For a stock Magento store.

Turn the caches on. PHP is the bottleneck
Add a backend cache. PHP is the bottleneck
Add a application-level full page cache. PHP is the bottleneck
Add a server-level front-end cache (eg. Varnish). MySQL is the bottleneck
Add an alternative search/layered navigation engine (eg. SOLR/Sphinx). PHP is the bottleneck
Add more application servers. MySQL is the bottleneck
Add a MySQL slave. Front-end cache is the bottleneck
Add more front-end cache servers. PHP is the bottleneck
Add more application servers. SOLR/Sphinx is the bottleneck

Etcetera.

It pretty much becomes a case of rinse-wash repeat. But what is clear to understand is that MySQL certainly isn't the first port of call for optimisation - and really only comes into play when MySQL is consuming more CPU proportionally to PHP - and this ONLY ever happens when both FPC and Varnish are in use and the server(s) are purely processing orders and nothing much else (because everything else is in caches).

Don't make changes blindly

Simply adding a MySQL slave because you read us say above that it will help, can cost you performance and reliability on a huge level. A congested network, low spec slave server or even improper settings can cause replication problems that can render your store slower than it was to begin with - or cause synchronisation issues between the Master and Slave.

To put things into perspective - some real world examples.

Example 1 - 300 orders per hour

We've used the following hardware to serve 300 orders per hour; and only at that tipping point - we then felt the need to add a dedicated MySQL server and a local MySQL slave.

1 Server
CPU: 2x Intel E5-4620
RAM: 64GB HDD: 4x 80k IOP/s SSDs
RAID: Hardware RAID 10
Magento Version: Magento EE
OS: MageStack

During the entire time, load averages remained under 3.00.

Example 2 - 180 orders per hour

Just two days ago, a new client of ours easily soaked up a big traffic spike. Processing 180 orders per hour with a single-server and Magento Community Edition.

1 Server
CPU: 2x Intel E5-4620
RAM: 64GB HDD: 4x 80k IOP/s SSDs
RAID: Hardware RAID 10
Magento Version: Magento CE
OS: MageStack
Website: Wellgosh.com

During the entire time, load averages remained under 6.00. The load was higher in this scenario and that was down to a couple of factors.

1. No store-side tuning was performed like in Example 1
2. The lack of an application-level FPC

And given the recency of this, we've still got the detailed statistics to give some feedback by means of graphs. These tell an excellent story of how load is distributed amongst the key components of a separated Magento architecture (load balancer, web server, db server etc. - using MageStack).

So from front to back ... the date you want to look at is at 12:00am on 22nd February.

Firewall Packets

Varnish Traffic

Nginx Traffic

MySQL Load

CPU Load

And most importantly, distribution of load

This image really tells it all. And it is that MySQL is certainly not a burden - not yet at least. So our advice, focus your performance concerns elsewhere, unless you are processing more than a few thousand orders per hour.

And in summary

Making performance changes isn't "one size fits all". It is a case of analysing your current bottlenecks and making subtle changes/adjustments to suit your store and infrastructure.

But performance aside, there are other benefits to using Percona

We do use Percona XtraDB, almost exclusively. We run custom-compiled builds of MySQL that we developed specifically for Magento and had consulted Percona during the process. But it wasn't just performance that influenced this choice.

• The Percona Toolkit
  • pt-query-digest
  • xtrabackup
  • etc.
• Percona release frequency
• Percona consultative support
• Warm cache restarts with InnoDB pool preservation

And much more. So using it over MySQL had other advantages than just performance. In fact - MySQL is and has always been the least of our concerns in the pursuit of performance and stability.

Attributions: wellgosh.com, sonassi.com, iebmedia.com.