I am using Django, wsgi with apache, but the server is really slow when I try to insert data into mysql server. It takes 1 second to insert 200 rows in a table. The strange thing is that only 3% of CPU and memory is used by apache and mysql. I am sure there is some setting wrong, but I cannot find it.
Here is my apache config file, wsgi setting and 'top' command result.
I really appreciate if someone could help me a little bit. Thanks!
My apache config file:
WSGIScriptAlias / "/home/jiechao/EZ_Green/apache/ems_django.wsgi"
ServerName www.ezgreenbuildings.com
WSGIDaemonProcess ezgreenbuildings.com processes=2 maximum-requests=1000 threads=25
WSGIProcessGroup ezgreenbuildings.com
wsgi file:
os.environ['DJANGO_SETTINGS_MODULE'] = 'ems.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
sys.path.append("/home/jiechao/EZ_Green/django")
sys.path.append("/home/jiechao/EZ_Green/django/ems")
top command result:
top - 15:56:02 up 25 days, 19:30, 1 user, load average: 0.74, 0.24, 0.11
Tasks: 112 total, 1 running, 111 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.8%us, 0.7%sy, 0.0%ni, 58.5%id, 40.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 4124236k total, 3242020k used, 882216k free, 157960k buffers
Swap: 4192252k total, 440k used, 4191812k free, 2750284k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6487 www-data 20 0 101m 63m 7288 S 1 1.6 0:03.02 apache2
30558 mysql 20 0 313m 44m 7020 S 1 1.1 0:54.98 mysqld
258 root 20 0 0 0 0 D 1 0.0 1:09.21 jbd2/sda1-8
10243 root 20 0 0 0 0 S 0 0.0 0:00.04 kworker/1:1
14410 jiechao 20 0 208m 8268 3296 S 0 0.2 4:39.92 python
1 root 20 0 3504 1876 1276 S 0 0.0 0:01.58 init
2 root 20 0 0 0 0 S 0 0.0 0:00.13 kthreadd
3 root 20 0 0 0 0 S 0 0.0 0:45.39 ksoftirqd/0
5 root 20 0 0 0 0 S 0 0.0 0:00.40 kworker/u:0
6 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/0
7 root RT 0 0 0 0 S 0 0.0 0:03.85 watchdog/0
After Edited.
I added 'WSGIProcessGroup' now, so it should be invoking daemon mode now.
I am using Prefork for MPM, but I don't understand how to see the settings of MPM.
For site visits, I think 1000 visits per second is what I would expect.
Best Answer
You have a few problems.
You are missing WSGIProcessGroup directive, meaning that WSGIDaemonProcess is redundant and not even being used. You are there running in embedded mode and not daemon mode. Embedded mode is not a good idea unless you know how to set up Apache properly. Read:
If you had managed to configure it so daemon mode was used properly, your processes/threads settings are insanely high and would likely just chew up lots of memory on your system, potentially causing it to start swapping and slow your system down.
Based on missing WSGIProcessGroup, you are at least not invoking that yet.
How big a problem you have created for yourself by running in embedded mode really depends on what Apache MPM you are using and what the MPM settings are.
So, before can debug further, you need to work out whether you are using Apache prefork or worker MPM and also what MPM settings you are using. Get that information and edit your question, adding the additional information.
Also give a realistic estimate of how many requests/sec your site is actually handling. That way can evaluate how wrong the MPM and daemon process process/threads settings are.
For additional reading, see:
What is in there will help you work out which MPM you are using and whether embedded mode or daemon mode is being used. Plus whether mod_python is also being loaded which can cause another world of pain as well.
UPDATE 1
The Apache MPM settings are in the Apache configuration files. Go read the Apache documentation about the MPM settings so you know what they are. Especially with prefork MPM, if you don't understand what they are and how to set them up properly to tune your site, you will fail.
As for the claim of 1000 visits per second, if that is true then you are going to slow things down immensely by using maximum-requests of 1000 as your web application processes will be restarting every couple of seconds and the delays in doing that would be slowing everything down. If one can avoid it, you should never use maximum-requests on a production system.
Frankly though I am dubious about claim that you are handling 1000 visits per second. A Django application using a database is highly unlikely to be able to handle anywhere near that on one server. Is that actual traffic or is is just your dreams of what you want to be able to handle? What actual performance monitoring are you running to support the claim you are handling that number of requests per second?