The short answer to this question is that none of these values are a reliable indicator of how much memory an executable is actually using, and none of them are really appropriate for debugging a memory leak.
Private Bytes refer to the amount of memory that the process executable has asked for - not necessarily the amount it is actually using. They are "private" because they (usually) exclude memory-mapped files (i.e. shared DLLs). But - here's the catch - they don't necessarily exclude memory allocated by those files. There is no way to tell whether a change in private bytes was due to the executable itself, or due to a linked library. Private bytes are also not exclusively physical memory; they can be paged to disk or in the standby page list (i.e. no longer in use, but not paged yet either).
Working Set refers to the total physical memory (RAM) used by the process. However, unlike private bytes, this also includes memory-mapped files and various other resources, so it's an even less accurate measurement than the private bytes. This is the same value that gets reported in Task Manager's "Mem Usage" and has been the source of endless amounts of confusion in recent years. Memory in the Working Set is "physical" in the sense that it can be addressed without a page fault; however, the standby page list is also still physically in memory but not reported in the Working Set, and this is why you might see the "Mem Usage" suddenly drop when you minimize an application.
Virtual Bytes are the total virtual address space occupied by the entire process. This is like the working set, in the sense that it includes memory-mapped files (shared DLLs), but it also includes data in the standby list and data that has already been paged out and is sitting in a pagefile on disk somewhere. The total virtual bytes used by every process on a system under heavy load will add up to significantly more memory than the machine actually has.
So the relationships are:
- Private Bytes are what your app has actually allocated, but include pagefile usage;
- Working Set is the non-paged Private Bytes plus memory-mapped files;
- Virtual Bytes are the Working Set plus paged Private Bytes and standby list.
There's another problem here; just as shared libraries can allocate memory inside your application module, leading to potential false positives reported in your app's Private Bytes, your application may also end up allocating memory inside the shared modules, leading to false negatives. That means it's actually possible for your application to have a memory leak that never manifests itself in the Private Bytes at all. Unlikely, but possible.
Private Bytes are a reasonable approximation of the amount of memory your executable is using and can be used to help narrow down a list of potential candidates for a memory leak; if you see the number growing and growing constantly and endlessly, you would want to check that process for a leak. This cannot, however, prove that there is or is not a leak.
One of the most effective tools for detecting/correcting memory leaks in Windows is actually Visual Studio (link goes to page on using VS for memory leaks, not the product page). Rational Purify is another possibility. Microsoft also has a more general best practices document on this subject. There are more tools listed in this previous question.
I hope this clears a few things up! Tracking down memory leaks is one of the most difficult things to do in debugging. Good luck.
Here's a good way to create a true memory leak (objects inaccessible by running code but still stored in memory) in pure Java:
- The application creates a long-running thread (or use a thread pool to leak even faster).
- The thread loads a class via an (optionally custom)
ClassLoader
.
- The class allocates a large chunk of memory (e.g.
new byte[1000000]
), stores a strong reference to it in a static field, and then stores a reference to itself in a ThreadLocal
. Allocating the extra memory is optional (leaking the class instance is enough), but it will make the leak work that much faster.
- The application clears all references to the custom class or the
ClassLoader
it was loaded from.
- Repeat.
Due to the way ThreadLocal
is implemented in Oracle's JDK, this creates a memory leak:
- Each
Thread
has a private field threadLocals
, which actually stores the thread-local values.
- Each key in this map is a weak reference to a
ThreadLocal
object, so after that ThreadLocal
object is garbage-collected, its entry is removed from the map.
- But each value is a strong reference, so when a value (directly or indirectly) points to the
ThreadLocal
object that is its key, that object will neither be garbage-collected nor removed from the map as long as the thread lives.
In this example, the chain of strong references looks like this:
Thread
object → threadLocals
map → instance of example class → example class → static ThreadLocal
field → ThreadLocal
object.
(The ClassLoader
doesn't really play a role in creating the leak, it just makes the leak worse because of this additional reference chain: example class → ClassLoader
→ all the classes it has loaded. It was even worse in many JVM implementations, especially prior to Java 7, because classes and ClassLoader
s were allocated straight into permgen and were never garbage-collected at all.)
A variation on this pattern is why application containers (like Tomcat) can leak memory like a sieve if you frequently redeploy applications which happen to use ThreadLocal
s that in some way point back to themselves. This can happen for a number of subtle reasons and is often hard to debug and/or fix.
Update: Since lots of people keep asking for it, here's some example code that shows this behavior in action.
Best Answer
To detect a memory leak using Performance Monitor, monitor these counters:
Source
In my experience this is accurate.
I'd also refer you to this Microsoft Advanced Debugging blog by Tess, a Microsoft employee. Who suggests the following counters. I have found the above to be more than enough to indicate a memory leak is present but I trust that Tess's instructions could provide a more indepth insight into the issue.
Debugging Demos - Memory Review