You're getting into a broad, broad world of optimisation here and there certainly isn't a one size fits all approach.
Define performance
Do you mean the page load time for a single user, or the overall capacity/total concurrency? The two are very distinctly different - and not strictly related. It is entirely possible to have a fast store with limited capacity; or a slow store with lots of capacity.
So when addressing either type of performance:
- Single user perceived page load time
- Total capacity/concurrency
You have to tackle each independently with their own solutions - especially since each have their own bottlenecks.
Lets make the assumption you are with a competent host that has already configured the other aspects of your server optimally for your store.
Single user perceived page load time
Is MySQL the bottleneck
No. Not directly. Its all about latency, in the majority of cases when testing page load time - only the caches will be hit. So the key here is to minimise latency.
- Tune MySQL cache sizes appropriately (there is no right answer, we tune settings entirely differently, monthly, per store)
- Reduce network latency. For 64 byte frames; 51.2µs for 10Mbps, 5.12µs for 100Mbps and 4.096µs for 1Gbps. This gives a improvement of 20% just by transitioning from 100Mbps to 1Gbps. s1
- Increase network capacity. You would be surprised at the many megabytes per second being exchanged between a Web and DB server, usually in excess of 10MB/s - so a minimum of 100Mb/s is required s1. Or, just move the DB server locally.
- Using SOLR. External engines are sometimes better suited, SOLR certainly is faster for LARGER catalogues (and I'd stress, larger catalogues). Even un-tuned SOLR will produce layered navigation and search results faster than MySQL can.
But these changes will have such a fractional impact on page load time - where the bottleneck is really elsewhere.
- Tune the application. Magento has some fairly big bugs with the way it builds collections and caches them; we've come across a number of big core code issues that can cripple performance. In a few cases, simply removing the product count display on the layered navigation results shaved 2 seconds of loading a big collection.
- Review MySQL slow logs. Check slow queries and add indexes as necessary. The difference between running a complex query with multiple joins with and without appropriate indexes can be tens of seconds.
The application is the bottleneck. Not the software. So merely improving core-code or making your template less heavy will have a far more dramatic effect on performance than ANY MySQL configuration change.
What wouldn't we bother with
- Changing the storage engine. MariaDB and Percona share the same InnoDB engine - Percona XtraDB. They can be treated as one and the same. In terms of single query execution time - performance will exactly mirror a vanilla MySQL build. This comes into play under load/concurrency.
- Running a MySQL slave. This won't improve performance unless the slave is located physically closer (from a network perspective), or that the slave has better hardware than the master. This comes into play under load/concurrency.
- Running an external DB server. This is by far the worst advice we see repeatedly handed out by many hosts/agencies. Until you have hit a ceiling on hardware/resources or you've got multiple web servers (read: high-availability), MySQL on the local machine for a Magento store is A Good Idea. It cuts out all the network overhead and latency. Even a 100Gb/s network (yes, one hundred gigabits per second) will not compare with a local unix socket for raw volume, throughput and latency.
s1 For separate database servers only. Doesn't apply to local DB servers.
Total capacity/concurrency
Is MySQL the bottleneck
Maybe. But only once you've nailed your PHP performance and capacity to the point where MySQL is slowing things down. If you've got Varnish and FPC properly configured (don't get us started on how many failed attempts we've seen with either) - then MySQL does become a bottleneck.
So in addition to the above modifications.
- Change MySQL engine. XtraDB can excel under load and does show genuine benefits over a stock MySQL distribution.
- Stay up to date with MySQL. 5.5 performs better than 5.0 under load.
- Change PHP MySQL driver. PHP 5.3 and newer has a native MySQL driver, but in some circumstances, we've found PHP 5.2 with the separate driver to outperform MySQLND for Magento. Test it for yourself
- Change search engine. Moving the search out to SOLR/Sphinx (or even some 3rd party external services) can really alleviate the burden of non-transactional load (ie. people not placing orders)
- Change layered navigation engine. Again, SOLR is a brilliant engine for layered navigation and due to its non-locking nature is far faster than MySQL.
- Add a MySQL slave. This does help browsing (non-transactional) load, but won't help you process more orders per hour - as it is reliant on the Master to process and replicate this data
What wouldn't we bother with
- Master/Master. Due to the pretty high tipping point of hardware saturation of a Master/Slave set up (in excess of 1000 orders per hour) - we've never found it a requirement to use Master/Master in production. We have performed extensive testing, but never found it to be advantageous from a performance perspective and with the inherent risks and problems of Master/Master, it simply isn't worth it.
Read vs Write Scalability
The last paragraph really leads on to a key area of read and write scalability. Read scaling can be performed fairly infinitely without too much complication with the addition of more and more slaves.
Given Magento's ratio of Reads to Writes is about 0.1% - considering writes shouldn't be much of a concern. That's why I've not bothered mentioning MySQL Cluster and its clever features like auto-sharding (splitting tables off to separate machines).
Hardware, hardware, hardware
Hardware is easily the quickest answer when it comes to improvement, so I've deliberately not mentioned it above in both scenarios.
But all the software changes in the world aren't going to make any difference if your underlying hardware is insufficient. That could mean...
- Low-quality switches with limited buffers
- Overly saturated network links
- Geographically distant servers
- Poor network QoS/CoS
- Limited total amount of RAM
- Low memory bandwidth RAM
- Low IOPs HDD subsystem
- Software RAID controller
- Low clock speed CPU
- Low bandwidth chipset
- Hardware virtualisation (almost all types apart from Kernel Level Virtualisation)
Nowadays, there's a really high ceiling on how high you can actually scale on hardware. Lets ignore the myth of infinite scaling "in the cloud" as cloud hardware tends to be fairly mediocre. For example Amazon's flagship models only being 12 Cores @ 3.3GHz. But outside of this, there are some very powerful servers available - our top server has 160 cores and 2TB (yes, Terabytes) of RAM. We've not seen anyone exceed the capabilities of that just yet.
So you've got a massive scope for vertical scaling, before you even need to consider horizontal scaling.
The ever moving target
Its worth mentioning that in the pursuit of performance, the bottleneck will always keep moving.
For a stock Magento store.
Turn the caches on. PHP is the bottleneck
Add a backend cache. PHP is the bottleneck
Add a application-level full page cache. PHP is the bottleneck
Add a server-level front-end cache (eg. Varnish). MySQL is the bottleneck
Add an alternative search/layered navigation engine (eg. SOLR/Sphinx). PHP is the bottleneck
Add more application servers. MySQL is the bottleneck
Add a MySQL slave. Front-end cache is the bottleneck
Add more front-end cache servers. PHP is the bottleneck
Add more application servers. SOLR/Sphinx is the bottleneck
Etcetera.
It pretty much becomes a case of rinse-wash repeat. But what is clear to understand is that MySQL certainly isn't the first port of call for optimisation - and really only comes into play when MySQL is consuming more CPU proportionally to PHP - and this ONLY ever happens when both FPC and Varnish are in use and the server(s) are purely processing orders and nothing much else (because everything else is in caches).
Don't make changes blindly
Simply adding a MySQL slave because you read us say above that it will help, can cost you performance and reliability on a huge level. A congested network, low spec slave server or even improper settings can cause replication problems that can render your store slower than it was to begin with - or cause synchronisation issues between the Master and Slave.
To put things into perspective - some real world examples.
Example 1 - 300 orders per hour
We've used the following hardware to serve 300 orders per hour; and only at that tipping point - we then felt the need to add a dedicated MySQL server and a local MySQL slave.
1 Server
CPU: 2x Intel E5-4620
RAM: 64GB
HDD: 4x 80k IOP/s SSDs
RAID: Hardware RAID 10
Magento Version: Magento EE
OS: MageStack
During the entire time, load averages remained under 3.00.
Example 2 - 180 orders per hour
Just two days ago, a new client of ours easily soaked up a big traffic spike. Processing 180 orders per hour with a single-server and Magento Community Edition.
1 Server
CPU: 2x Intel E5-4620
RAM: 64GB
HDD: 4x 80k IOP/s SSDs
RAID: Hardware RAID 10
Magento Version: Magento CE
OS: MageStack
Website: Wellgosh.com
During the entire time, load averages remained under 6.00. The load was higher in this scenario and that was down to a couple of factors.
- No store-side tuning was performed like in Example 1
- The lack of an application-level FPC
And given the recency of this, we've still got the detailed statistics to give some feedback by means of graphs. These tell an excellent story of how load is distributed amongst the key components of a separated Magento architecture (load balancer, web server, db server etc. - using MageStack).
So from front to back ... the date you want to look at is at 12:00am on 22nd February.
Firewall Packets
Varnish Traffic
Nginx Traffic
MySQL Load
CPU Load
And most importantly, distribution of load
This image really tells it all. And it is that MySQL is certainly not a burden - not yet at least. So our advice, focus your performance concerns elsewhere, unless you are processing more than a few thousand orders per hour.
And in summary
Making performance changes isn't "one size fits all". It is a case of analysing your current bottlenecks and making subtle changes/adjustments to suit your store and infrastructure.
But performance aside, there are other benefits to using Percona
We do use Percona XtraDB, almost exclusively. We run custom-compiled builds of MySQL that we developed specifically for Magento and had consulted Percona during the process. But it wasn't just performance that influenced this choice.
- The Percona Toolkit
- pt-query-digest
- xtrabackup
- etc.
- Percona release frequency
- Percona consultative support
- Warm cache restarts with InnoDB pool preservation
And much more. So using it over MySQL had other advantages than just performance. In fact - MySQL is and has always been the least of our concerns in the pursuit of performance and stability.
Attributions: wellgosh.com, sonassi.com, iebmedia.com.
Best Answer
Lets see, there are multiple places you need to improve to endure such a "attack".
MySql gone away probably means, your webserver is able to handle a lot more requests then your MySql, or if you have both on the same Server, the number of requests affects to much the MySql, like stealing all the CPU or IO time.
Reducing the number of accepted php connections can help a bit, even if this means, users get earlier rejected.
You need to improve your generel performance on the website. There are more complete lists what to do for this, but a few points:
disable visitor logging
Thats one of the most costly parts of a normal website request, as it writes into the databse even on a simple product view. You can disable this by disabling specific observers. I collected once some, so look what each observer I disable here ( https://github.com/Cotya/magento_performance_fixes/blob/master/src/app/code/community/Cotya/PerformanceFixes/etc/config.xml ) does and disable them in your project, if you think you dont need this feature.
An own observer I added there, is for the following
remove category/product ID specific Layout Handles
This Increases the cache sharing a lot, which helps if you get requests on a lot of different URLs. And most people dont write ID specific rules into their layout.xml files. If you did, products and categories allow specifi own layout xml via fields in the admin area.
Increase Product List caching
One of my favorite optimizations, you cache every element(/product) on the list as own entry. Thats a lot of additional cache entries, but means, if the same product gets shown on another category, or people change sort/filter, the single product is still cached and you need less additional querys per product. An example for this is here: https://gist.github.com/Flyingmana/510cee6c7c2668e279ae/revisions
deactivate some functionality
You may have to think if you really need the compare functionality, it can be very performance heavy, and if not much people make use of it at your site, maybe just deactivate it. You absolutely should deactivate the sendfriend functionality, as it allows to send emails to others.
Its not easy to block them, if you cant define a clear pattern how they do requests on your site. But when you can, there is the fail2ban tool, which can automatically block their IPs (even for limited time) if a defined behaviour is detected.