Python – How to use the uWSGI Emperor to limit resources to a given amount for all apps (instead of per each)

djangomemorypythonuwsgi

I have a Django application for which I want to limit the resources used by uWSGI to 12288MB and processes (workers) to 6.

With 3 separate apps, I can limit each to 2 workers, and each worker to 2048MB of memory (limit-as), but I'd rather allow all 3 apps to share the same amount of resources (they're all related, and 1 is not more important than the other – they're only separate apps because they each use Python configurations). I don't want one app to get pounded, while the other 2 are sipping Mai Tais on the beach with 1.5GB of RAM sitting around and processor to take up.

My uWSGI conf looks like this:

--emperor /etc/uwsgi/apps-enabled
--disable-logging
--die-on-term
--master
--uid www-data
--gid www-data
--logto /var/log/uwsgi/emperor.log

Each app is akin to:

[uwsgi]
socket = /tmp/app-1.uwsgi.sock
workers = 2
threads = 40
limit-as = 2048
harakiri = 20
max-requests = 1600
plugins = python
module = myapp.wsgi
callable = application
venv = /var/www/myapp/deps/current/venv
chdir = /var/www/myapp/deps/current/repo/src
touch-reload = /var/www/myapp/uwsgi/reload
auto-procname = true
procname-prefix-spaced = myapp
vacuum = true
reload-mercy = 8

BTW (side-question), are all of those settings interchangeable? Like, can I add settings from an app to the main uWSGI conf (ones that are the same for all 3 apps), and likewise add things like log-to to specific apps – I wasn't sure if every setting was interchangeable like that.

Best Answer

I strongly suggest you to use cgroups (linux-only) to manage that kind of isolation.

remove limit-as

and add

cgroup = /cgroup/app1

cgroup-opt = memory.limit_in_bytes=1067459584

More infos are here:

http://projects.unbit.it/uwsgi/wiki/UseCgroups

the only "downside" is that you need to run the emperor as root, and set uid and gid in each vassal