IIS6: determine which application pool is using the most resources

Apps under Windows x86 are limited to 2gb/ea. You can change this by adding the /3gb flag to boot.ini, but this can cause unexpected application behavior and should be used with caution. MS officially does not support this (http://technet.microsoft.com/en-us/library/bb124810.aspx)

IIS's memory limits can be set under the Recycling tab in the relevant app pool.

How are you determining that the application is never going over 500MB? If you're using Task Manager, keep in mind that it is rarely (if ever) an accurate representation of memory usage as Windows understands it. Use Process Explorer: http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx

There are a number of other relevant questions:

• Are there errors logged in your Event Logs?
• Is it a particular line of code throwing the exceptions? If so, is this OutOfMemory exception actually an IIS problem, or a code problem?
• Are you limiting your application pools at all with the Recyling setting?
• Are there any other services running on this machine?

Most likely, IIS, ASP.NET, and all their associated baggage are running up against the 2GB limit even though Windows Task Manager doesn't show it. Setting Recycling limits or upgrading to x64 will probably make a huge difference.

The script iisapp.vbs will list this information for you, it is located in the system32 folder, but you should be able to run it straight from the command line.