Increased memory usage for uWSGI workers / masters after moving from Debian to Ubuntu

debian-squeezememory usageubuntu-13.04uwsgi

After moving our web servers from Debian to Ubuntu we have noticed a large increase in the amount of memory (RSS) used by uWSGI master and worker processes. The version of uWSGI (1.9.13) remains the same.

I've used pmap to examine the memory usage for each process. Under Debian a master process looked like this:

Address             RSS     Dirty   Mode    Mapping 
----------------    ------  ------  ------          
kB                  15976   11316           
0000000001a880000   7308    7308    rw---   [anon
400000              1608    0       r-x--   uwsgi   
00007ff16c5a70000   772     772     rw---   [anon
00007ff176c3a0000   668     0       r-x--   libcrypto.so.0.9.8  
00007ff175f010000   548     0       r-x--   libc-2.11.3.so  
00007ff177fc70000   512     512     rw---   [anon
00007ff17807b0000   512     512     rw---   [anon
00007ff1759e50000   356     0       r-x--   libstdc++.so.6.0.13 
80                  268     184     rw---   uwsgi

I'm only showing the most relevant memory blocks and the numbers under RSS are in kilobytes.

While now under Ubuntu uWSGI has allocated itself much more memory. 

Address             RSS     Dirty   Mode    Mapping     
----------------    ------  ------  ------              
kB                  24216   19196               
2401000             15508   15508   rw---   [   anon    ]
00007f3094e86000    1648    0       r-x--   libpython2.7.so.1.0     
00007f308c6ce000    772     772     rw---   [   anon    ]
00007f3094abe000    680     0       r-x--   libc-2.17.so        
00007f309595d000    612     0       r-x--   libcrypto.so.1.0.0      
00007f3096de4000    512     512     rw---   [   anon    ]
00007f3096e67000    512     512     rw---   [   anon    ]
400000              480     0       r-x--   uwsgi       
00007f30945b3000    352     0       r-x--   libstdc++.so.6.0.17     
00007f309532e000    340     248     rw---   libpython2.7.so.1.0

The configuration and type of application loaded into these master procesess is otherwise identical. Kernel versions are 2.6.32-5-xen-amd64 for Debian and 3.8.0-19-generic for Ubuntu.

Does anyone have any idea as to why each master would be allocating itself another 8megs of memory?

Best Answer

Discovered the problem, the system version of Python 2.7 that Ubuntu comes with causes uWSGI processes to use more memory. Compiling a new version of Python and using that to launch the uWSGI emperor solved the problem.

Compiling Python and installing uWSGI can be achieved with the below commands on Ubuntu 13.04.

wget http://python.org/ftp/python/2.7.5/Python-2.7.5.tar.bz2
tar jxf Python-2.7.5.tar.bz2
cd Python-2.7.5/
./configure --prefix=/opt/python2.7.5 && make && make install
cd /opt/python2.7.5/
curl -O http://python-distribute.org/distribute_setup.py
bin/python2.7 distribute_setup.py
bin/easy_install-2.7 pip
bin/pip-2.7 install uwsgi
Related Topic