Nginx – How much RAM do I need to run a forum using Nginx, Gunicorn, Django

amazon ec2gunicornmemorynginxswap

I started an ec2 instance (micro) to test the amount of memory I have available to run a website but I was expecting having a bit more. So my question would be: How much RAM do I need to run a site using Django, Nginx and Gunicorn. This is what I found:

Starting a Ubuntu 12.04 instance consumes:

             total       used       free     shared    buffers     cached
Mem:           590        235        354          0          7        182
-/+ buffers/cache:         45        544
Swap:            0          0          0

After running Gunicorn:

             total       used       free     shared    buffers     cached
Mem:           590        195        394          0          9         73
-/+ buffers/cache:        112        477
Swap:            0          0          0

So I have 394MB available although I forgot to create a swap volume. Is that enough to run a forum receiving 3000-4000 visits per month? I would also appreciate some comments on the proper way to configure swap.

Thanks a lot

Best Answer

The used memory is 45MB, not 235MB. to know this you need to go here to get the detail to check how you can know the Actual memory usage using this command and where the memory went.

http://www.linuxatemyram.com/

Immediately create a swap partition, atleast of 2GB or more, as in any case of the server goes out of RAM, then SWAP will be used as memory usage, which will slow down server but atleast wont crash it.
To know the actual amount of usage, based on the number of users you are having, you need to use come load testing tools, which will test your website with 1, 10,20,50,1000-4000 users, and based on what type of requests you give to each user. For example, based on that software, you can record a user loggin in to site , surfing some pages and loggin off, and the software will run the same scenario for number of users you provide. So for each number of users, you can check the amount of RAM used, which will give you an idea that how much RAM your site needs. Some of the load testing tools are :

http://loadimpact.com/

http://www.webresourcesdepot.com/15-free-functionality-and-load-testing-tools-for-web-applications/

http://www.webperformance.com/

https://stackoverflow.com/questions/507683/open-source-tool-for-stress-testing-load-testing-and-performance-testing

http://jcrawler.sourceforge.net/

DONT RUN the tests on Micro instance, as its CPU is throttled not to be used to its actual usage, and you can get false results due to CPU throttling. Atleast use Amazon SMALL server when you are going to perform tests.

Related Solutions

Swap Space – How to Reclaim Increasing Swap Size in Linux

If info is swapped out to disc and later read back into memory, it will often be left allocated in the swap area until swap space runs low. That means that if the same info needs to be swapped out again later and hasn't changed, the OS can just drop the pages from allocated RAM without needing to write anything to disc saving time.

Swap allocated to stuff that has been read back into memory will be freed either

when the relevant pages are no longer needed at all (i.e. are freed by the application)
when the relevant pages are changed (so the copy on disc is no longer up to date)
the machine runs low on swap space so clears some things that are already in RAM to make room

Look in /proc/meminfo for a line called "SwapCached". This entry counts pages that are found both in RAM and in swap partitions. For instance, picking a small VM at random, the /proc/meminfo virtual file one of my VMs shows:

SwapTotal:        698816 kB
SwapFree:         624520 kB
SwapCached:        17232 kB

indicating that 74268K of swap space is allocated, but that 17232K worth of those pages are also currently mapped into RAM too (so could be deallocated from swap at a moment's notice if the space is needed by something else).

Also there will no doubt be pages sat there that was swapped out ages ago and have never been used again since. The kernel will not reload pages from swap just because there is some free RAM to read it back into as that free RAM might be better used for cache or buffers - pages written to swap are generally only reread when they are next needed.

If you want to clear out what is in swap, as long as you have enough free and/or freeable (i.e. free+cache+buffers (less those parts of the c+b counts that are not freeable RightThisInstant)), just turn it off and back on again with swapoff -a && swapon -a.

Of course you could also have a memory leak somewhere too, but that is not the only explanation for the behaviour you are seeing.

Linux – Running out of LowMem with Ubuntu PAE Kernel and 32GB of RAM

The problem is that a lot of the kernel data structures such as the page descriptors (one struct for every 4KB page in the system) need to be in low memory. So as the total memory in the machine goes up, more and more low memory is also needed, and eventually low memory becomes a very scarce resource.

IIRC the usual rule of thumb is that 16 GB total is about the upper sane limit for a 32-bit kernel. There's not very much you can do about it.

You can try to boot with less memory (mem= command line parameter to the kernel). But the real solution is to switch to a 64-bit kernel.

Best Answer

Related Solutions

Swap Space – How to Reclaim Increasing Swap Size in Linux

Linux – Running out of LowMem with Ubuntu PAE Kernel and 32GB of RAM

Related Topic