I have a WCF service that might grow in memory. I put a memory limit on the private bytes to recycle the app pool after 500m of memory usage. I noticed that my w3wp.exe process can get to 600megs when it decides to recycle and any currently executing client requests get a communication error. Is there a way for iis to wait for requests to complete before recycling the process?
Iis memory limit app pool recycle kills executing requests
iiswcf
Related Solutions
The app pool advanced settings allows you to turn on extra logging for the app pool. You can turn on all of the app pool recycle logging. It shouldn't be chatty in a healthy situation and you don't mind if it's chatty while you're troubleshooting.
Then in Event Viewer you should get an event logged every time your app pool recycles.
Recycling
Recycling is usually* where IIS starts up a new process as a container for your application, and then gives the old one up to ShutdownTimeLimit
to go away of its own volition before it's killed.
*- usually: see DisallowOverlappingRotation
/ "Disable overlapped recycle" setting
It is destructive, in that the original process and all its state information are discarded. Using out-of-process session state (eg, State Server or a database, or even a cookie if your state is tiny) can allow you to work around this.
But it is by default overlapped - meaning the duration of an outage is minimized because the new process starts and is hooked up to the request queue, before the old one is told "you have [ShutdownTimeLimit
] seconds to go away. Please comply."
Settings
To your question: all the settings on that page control recycling in some way. "Shutdown" might be described as "proactive recycling" - where the process itself decides it's time to go, and exits in an orderly manner.
Reactive recycling is where WAS detects a problem and shoots the process (after establishing a suitable replacement W3WP).
Now, here's some stuff that can cause recycling of one form or another:
- an ISAPI deciding it's unhealthy
- any module crashing
- idle timeout
- cpu limiting
- adjusting App Pool properties
- as your mum may have screamed at one point: "Stop picking at it, or it'll never get better!"
- "ping" failure * not actually pinging per se, cos it uses a named pipe - more "life detection"
- all of the settings in the screenshot above
What To Do:
Generally:
Disable Idle timeouts. 20 minutes of inactivity = boom! Old process gone! New process on the next incoming request. Set that to zero.
Disable Regular time interval - the 29 hour default has been described as "insane", "annoying" and "clever" by various parties. Actually, only two of those are true.
Optionally Turn on DisallowRotationOnConfigChange (above, Disable Reycling for configuration changes) if you just can't stop playing with it - this allows you to change any app pool setting without it instantly signaling to the worker processes that it needs to be killed. You need to manually recycle the App Pool to get the settings to take effect, which lets you pre-set settings and then use a change window to apply them via your recycle process.
As a general principle, leave pinging enabled. That's your safety net. I've seen people turn it off, and then the site hangs indefinitely sometimes, leading to panic... so if the settings are too aggressive for your apparently-very-very-slow-to-respond app, back them off a bit and see what you get, rather than turning it off. (Unless you've got auto-crash-mode dumping set up for hung W3WPs through your own monitoring process)
That's enough to cause a well-behaved process to live forever. If it dies, sure, it'll be replaced. If it hangs, pinging should pick that up and a new one should start within 2 minutes (by default; worst-case calc should be: up to ping frequency + ping timeout + startup time limit before requests start working again).
CPU limiting isn't normally interesting, because by default it's turned off, and it's also configured to do nothing anyway; if it were configured to kill the process, sure, that'd be a recycling trigger. Leave it off. Note for IIS 8.x, CPU Throttling becomes an option too.
An (IIS) AppPool isn't a (.Net) AppDomain (but may contain one/some)
But... then we get into .Net land, and AppDomain recycling, which can also cause a loss of state. (See: https://blogs.msdn.microsoft.com/tess/2006/08/02/asp-net-case-study-lost-session-variables-and-appdomain-recycles/)
Short version, you do that by touching a web.config file in your content folder (again with the picking!), or by creating a folder in that folder, or an ASPX file, or.. other things... and that's about as destructive as an App Pool recycle, minus the native-code startup costs (it's purely a managed code (.Net) concept, so only managed code initialization stuff happens here).
Antivirus can also trigger this as it scans web.config files, causing a change notification, causing....
Related Topic
- Iis – Does IIS Sometimes Allocate More Worker Processes Than Configured
- Iis – Logging a list of Worker Process Requests when Application Recycle happens
- IIS App Pool High CPU usage despite no requests
- Iis – Does recycling an application pool clear the request queue
- Iis – How to prevent IIS from recycling Application Pool when a worker process is unresponsive
Best Answer
Recycling is killing an process when it reaches certain conditions, replacing it with another one.
In terrible-analogy-land, it's a bit like ensuring that the size of a balloon can only ever be one foot square by having pins positioned in a one-foot sphere - when the balloon hits the limit, it pops and you replace it with a new one.
Processes being recycled get up to {Shutdown Time Limit} to shut themselves down, from the point at which they're told to recycle.
If they can't finish their work in that time {90 second default}, that's it.
If you're using a WCF host with a persistent connection, then I think the answer's no; but the answer is also "pick larger limits". What you've described as your policy is: "I want my app to die when it hits 500MB." But the app itself doesn't know about that limit; it's just trying to grow as it needs to.
Load test your app without a limit; see what it gets to and what might indicate a leak; set limits at the point a leak is indicated.