I administer a Windows 2008 server (well, on Amazon EC2) running IIS and a .NET4 Web app. I got a memory alert the other day and went and looked, and sure enough the process memory had grown over time via some kind of slow leak. It didn't grow by much, just like 60M to 200M, but enough else was going on with the box that it went over our pretty low threshold (75%) to set off the monitor.
I recycled the app's pool and the memory freed up, and I noticed upon reviewing stats that swap space was being used significantly and that more than 1 GB of it freed up with that recycle.
Maybe this is a basic question, but I'm a UNIX guy and I'm used to swap not getting used until you're out of memory. This box has never gone above 75% memory usage. Is this a Windows thing or a .NET thing or an Amazon thing? I suspect that there's a much larger memory leak in this app than suspected – it's not leaking from 60M to 200M, it's leaking from 60M to 1.2GB, but much of that is somehow going "cold" and being pushed out to swap?
I have memory recycling set on the application pool, but it triggers off box full memory, so this app could get really, really big before it recycles automatically.
I could set up regular "timed" recycling, but that's a workaround, I'll get the dev to fix the app but need to understand what's going on here with the swap usage to make sure I am understanding this right.
Edit with more info:
instance memory: 1.7 GB
swap: 4.5 GB
I see the w3wp.exe process in taskmgr showing that Memory: 211,000k. But when I restarted it (it's in its own app pool, and it's the only app on the box), its memory usage went down to its normal starting point of 60M and like 1 GB+ of swap also freed up. In taskmgr I just had the usual Memory (Private Working Set) stat up, but saw the swap change via my other monitoring (Cloudkick). Going back and looking at it today, memory is back up to 195M on the process (1.2 GB total) and swap has crept from 1.0 GB to 1.1 GB, buyt not all the way back up where it was (graphing over time, it's a slow creep).
I'm less concerned about this specific app and more concerned about just understanding when Windows swaps and how it uses that, and what to be concerned about given Windows memory and swap usage in general.
Best Answer
Windows and linux have two different page/swap strategies.
Linux
Linux wants to avoid using swap space at all, and waits until the last possible moment. If you see a large amount of swap in linux, your system likely is or was in trouble. This strategy is good for minimizing overall disk i/o, which is the slowest part of your system, but weaker for systems with alternating periods of light and heavy load (and honestly, that's most of us). The times when your load is already heavy will now be burdened by the "extra" disk i/o, or, put another way, you need to design your server builds with an eye to having enough ram you don't swap even during the highest expected load times.
Windows
Windows wants to treat memory as a mere cache of the page file. Your real memory is always on disk, but it will read/write from the "cache" first if it can. This strategy is good for evening out your load over time; when the system gets busy and needs to swap pages, the current page is already on disk and half the work is already done. This approach made huge sense back when Windows was young, 32MB (forget GB) was still a lot of RAM, and the frequent need to use swap space was a given. Even today this is good for work-loads that alternate between light and busy loads, as it helps spread the disk i/o out more evenly over time.
Modern Windows versions have additional optimizations — such as SuperFetch — to pre-load and prepare memory pages on disk and in RAM when the load is otherwise light, to help avoid the need for extra disk writes when loading a program for the first time. All of this means you can design your system to only need enough RAM for something less than the highest expected load, so you can still have at least acceptable performance all the time, with reduced costs.
Convergence
This concept of measuring or predicting load in a test environment first and then allocating production resources when the load is known is a relatively recent development in system building, made possible, or at least practical, in part with the advent of virtual and then cloud servers. Depending on your load, you may even design the system such that it never needs to swap at all. In these cases, Windows does allow you to turn off paging and behave more like a linux system. However, you have to be careful; if your system design requires more memory than expected you can get yourself into trouble this way.
On the other hand, modern linux kernels are more willing to swap to disk opportunistically than they once were. So the difference in memory management strategies between the two systems is still present, but now less distinct than it used to be. Both systems have their merits, and each watches the other to see which advances they can copy.