After upgrading to Magento 2.4, I've been unable to run a stable production environment. HTTPD spawns the maximum number of processes and uses all of my systems memory and swap (it takes 5-20 minutes to reach this point).
I'm constantly seeing the following errors in httpd error_log:
[Mon Aug 17 19:40:48.077047 2020] [php7:error] [pid 16939:tid 140670127806208] [client 127.0.0.1:40644] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/magento/module-csp/Model/BlockCache.php on line 108
[Mon Aug 17 19:40:48.125051 2020] [php7:error] [pid 16939:tid 140670127806208] [client 127.0.0.1:40644] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 40960 bytes) in Unknown on line 0
[Mon Aug 17 19:40:50.052935 2020] [php7:error] [pid 16932:tid 140669993522944] [client 127.0.0.1:40654] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 48238592 bytes) in /var/www/html/vendor/magento/framework/Serialize/Serializer/Json.php on line 24
[Mon Aug 17 19:40:51.592030 2020] [php7:error] [pid 16933:tid 140670169769728] [client 127.0.0.1:40666] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/magento/module-csp/Model/BlockCache.php on line 108
[Mon Aug 17 19:41:12.706529 2020] [php7:error] [pid 17146:tid 140670127806208] [client 127.0.0.1:40702] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/magento/module-csp/Model/BlockCache.php on line 108
[Mon Aug 17 19:41:12.740424 2020] [php7:error] [pid 17146:tid 140670127806208] [client 127.0.0.1:40702] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 122880 bytes) in Unknown on line 0
[Mon Aug 17 19:41:24.887257 2020] [php7:error] [pid 16939:tid 140670111020800] [client 127.0.0.1:40798] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 4096 bytes) in /var/www/html/vendor/magento/module-csp/Model/BlockCache.php on line 78
[Mon Aug 17 19:41:24.913106 2020] [php7:error] [pid 16939:tid 140670111020800] [client 127.0.0.1:40798] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 65536 bytes) in Unknown on line 0
[Mon Aug 17 19:41:39.997127 2020] [php7:error] [pid 16933:tid 140670152984320] [client 127.0.0.1:40904] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/magento/module-csp/Model/BlockCache.php on line 108, referer: http://ip-of-my-ec2-instance:80/
[Mon Aug 17 19:41:40.030659 2020] [php7:error] [pid 16933:tid 140670152984320] [client 127.0.0.1:40904] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 40960 bytes) in Unknown on line 0, referer: http://ip-of-my-ec2-instance:80/
[Mon Aug 17 19:42:08.849538 2020] [php7:error] [pid 17146:tid 140670111020800] [client 127.0.0.1:41016] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/magento/framework/Serialize/Serializer/Json.php on line 42
[Mon Aug 17 19:42:08.865389 2020] [php7:error] [pid 17146:tid 140670111020800] [client 127.0.0.1:41016] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 32768 bytes) in Unknown on line 0
[Mon Aug 17 19:44:23.245570 2020] [php7:error] [pid 17146:tid 140670027093760] [client 127.0.0.1:41798] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/magento/framework/Serialize/Serializer/Json.php on line 42, referer: https://www.google.com/
[Mon Aug 17 19:44:23.312177 2020] [php7:error] [pid 17146:tid 140670027093760] [client 127.0.0.1:41798] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 45056 bytes) in Unknown on line 0, referer: https://www.google.com/
[Mon Aug 17 19:44:29.224653 2020] [php7:error] [pid 16939:tid 140670052271872] [client 127.0.0.1:41742] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/magento/module-csp/Model/BlockCache.php on line 108
[Mon Aug 17 19:44:29.264725 2020] [php7:error] [pid 16939:tid 140670052271872] [client 127.0.0.1:41742] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 90112 bytes) in Unknown on line 0
[Mon Aug 17 19:44:34.254009 2020] [php7:error] [pid 17146:tid 140670043879168] [client 127.0.0.1:41688] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/magento/module-csp/Model/BlockCache.php on line 108
[Mon Aug 17 19:44:34.294439 2020] [php7:error] [pid 17146:tid 140670043879168] [client 127.0.0.1:41688] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 36864 bytes) in Unknown on line 0
[Mon Aug 17 19:44:40.650547 2020] [php7:error] [pid 17146:tid 140670010308352] [client 127.0.0.1:41882] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/magento/framework/Serialize/Serializer/Json.php on line 42
[Mon Aug 17 19:44:40.690603 2020] [php7:error] [pid 17146:tid 140670010308352] [client 127.0.0.1:41882] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 49152 bytes) in Unknown on line 0
[Mon Aug 17 19:44:49.990225 2020] [php7:error] [pid 16939:tid 140670144591616] [client 127.0.0.1:42006] PHP Fatal error: Allowed memory size of 792723456 bytes exhausted (tried to allocate 33554440 bytes) in /var/www/html/vendor/magento/module-csp/Model/Collector/DynamicCollector.php on line 31
Before upgrading to 2.4, I switched to Elasticsearch 6.7 and upgraded to php7.3 (7.4 isn't available on Amazon Linux AMI as far as I can tell).
My system.log and exception.log don't show any relevant errors. What can I do to debug this?
Best Answer
@hunter We ran into the same thing (next to some others). Upraded to 2.4 and the VPS would kill processes via OOM killer because the RAM Magento was consuming was unstoppable.
Nothing worked until I noticed that the structure and order of the error messages were always the same. Serialize error + collector error + csp error. So we dived into the CSP module and found that around the error line first there is
$data = $this->serializer->unserialize($loaded);Then$this->dynamicCollector->add(And then in line 78 it would work with the result and add dynamic policies in the function.So we found the formula that was triggering the error messages and for testing we just disabled the CSP module and tested this in production mode.
After this all was working like a Japanese bullet train.
I will try to bugfix the module tonight but disabling CSP might keep your shop alive for now.