Linux – order 4 allocation failure disaster

Some environment details first:

Hardware:
Intel Server Board S2600GZ
2 x Intel Xeon CPU E5-2620
64GB DDR3 RAM
Intel RAID Controller RS2BL (LSI SAS2108) with 4TB LVM Volume made of SAS disks

Software:
Ubuntu 12.04.4 LTS / Linux 3.11.0-24-generic x86_64 (with latest updates)
qemu/KVM (libvirt) with 6 VM's (running with no problems in spite of the situation)
glusterfs server 3.4.5 (seems to work normally too)
some other lightweght soft (e.g. bind9, keepalived, openvpn etc.)
NO custom / experimental / homegrown software!

For already a long time we have a very strange problem with one of our Ubuntu servers: periodically it floods the syslog with "allocation failure" messages like these:

Aug 28 07:00:18 srvname kernel: [4210234.157335] irqbalance: page allocation failure: order:4, mode:0x1040d0
Aug 28 07:00:19 srvname kernel: [4210234.711173] zabbix_agentd: page allocation failure: order:4, mode:0x1040d0
Aug 28 07:00:20 srvname kernel: [4210235.938599] zabbix_agentd: page allocation failure: order:4, mode:0x1040d0
Aug 28 07:00:34 srvname kernel: [4210250.307283] zabbix_agentd: page allocation failure: order:4, mode:0x1040d0
Aug 28 07:00:51 srvname kernel: [4210267.170359] irqbalance: page allocation failure: order:4, mode:0x1040d0
Aug 28 07:01:02 srvname kernel: [4210278.625530] zabbix_agentd: page allocation failure: order:4, mode:0x1040d0
Aug 28 07:01:19 srvname kernel: [4210295.671569] zabbix_agentd: page allocation failure: order:4, mode:0x1040d0

The messages are logged approximately every 30 secs and they do reflect the real picture: the processes shown in this log snippet are really failing (e.g. zabbix agent fails to transmit data to zabbix server). But it's just the tip of the iceberg. While this memory exhaustion is going on any process that needs to read the /proc directory (e.g. ps, top, mpstat etc.) crashes right after start because it fails to read it (/proc also can't be listed manually with ls) and this event is immediately logged to syslog with the same order 4 allocation failure error.

With all this, there's enough of free RAM (1/4 of the total size), but if we check it out by blocks – the blocks of order 4 are really exhausted. BUT, what I really can't understand is WHY these processes actually do request such large blocks? We have another, almost identical (by hardware and software) server – its blocks of order 4 are exhausted too – and it feels fine, no order 4 allocation failures!
Moreover, this identical server is under MUCH heavier load.

I've searched the web deeply many times for "(high order) allocation failures" symptoms, but nothing seems to be relevant. We've tried to play with various sysctl parameters (e.g. vm.min_free_kbytes, vm.vfs_cache_pressure etc., as suggested by some articles), but nothing helps. Eventually we're rolled back all these changes and now most of sysctl settings are system default. We've also tried echoing to /proc/sys/vm/compact_memory and /proc/sys/vm/drop_caches without any obvious (or prolonged) effect. After some long period of exhaustion, all of a sudden and by itself, everything becomes normal (it seems like memory gets defragmented and order 4 blocks become available, /proc becomes available too), but not for long – after some short period all starts over again. A reboot helps for a longer period (due to the totally unfragmented memory), but eventually everything ends up the same…

In general, the only real troubles (that we're aware of) caused by the described behavior is the inablity to monitor and manage server resources, neither remotely (zabbix), nor locally (ps, top, mpstat etc.).

As far as I understand, the lack of order 4 blocks is a normal regular state of memory under Linux. It's just that the processes shouldn't generally request such blocks (especially processes that DON'T do it on other servers). If anyone has any idea about what could be the cause of such a behavior, what can we check or where to dig – we would really appreciate it! We're ready to supply any additional info on demand.