Node.js – possible EventEmitter memory leak detected

Here's a good way to create a true memory leak (objects inaccessible by running code but still stored in memory) in pure Java:

1. The application creates a long-running thread (or use a thread pool to leak even faster).
2. The thread loads a class via an (optionally custom) ClassLoader.
3. The class allocates a large chunk of memory (e.g. new byte[1000000]), stores a strong reference to it in a static field, and then stores a reference to itself in a ThreadLocal. Allocating the extra memory is optional (leaking the class instance is enough), but it will make the leak work that much faster.
4. The application clears all references to the custom class or the ClassLoader it was loaded from.
5. Repeat.

Due to the way ThreadLocal is implemented in Oracle's JDK, this creates a memory leak:

• Each Thread has a private field threadLocals, which actually stores the thread-local values.
• Each key in this map is a weak reference to a ThreadLocal object, so after that ThreadLocal object is garbage-collected, its entry is removed from the map.
• But each value is a strong reference, so when a value (directly or indirectly) points to the ThreadLocal object that is its key, that object will neither be garbage-collected nor removed from the map as long as the thread lives.

In this example, the chain of strong references looks like this:

Thread object → threadLocals map → instance of example class → example class → static ThreadLocal field → ThreadLocal object.

(The ClassLoader doesn't really play a role in creating the leak, it just makes the leak worse because of this additional reference chain: example class → ClassLoader → all the classes it has loaded. It was even worse in many JVM implementations, especially prior to Java 7, because classes and ClassLoaders were allocated straight into permgen and were never garbage-collected at all.)

A variation on this pattern is why application containers (like Tomcat) can leak memory like a sieve if you frequently redeploy applications which happen to use ThreadLocals that in some way point back to themselves. This can happen for a number of subtle reasons and is often hard to debug and/or fix.

Update: Since lots of people keep asking for it, here's some example code that shows this behavior in action.

Solution

The compiler is warning about this for a reason. It's very rare that this warning should simply be ignored, and it's easy to work around. Here's how:

if (!_controller) { return; }
SEL selector = NSSelectorFromString(@"someMethod");
IMP imp = [_controller methodForSelector:selector];
void (*func)(id, SEL) = (void *)imp;
func(_controller, selector);

Or more tersely (though hard to read & without the guard):

SEL selector = NSSelectorFromString(@"someMethod");
((void (*)(id, SEL))[_controller methodForSelector:selector])(_controller, selector);

Explanation

What's going on here is you're asking the controller for the C function pointer for the method corresponding to the controller. All NSObjects respond to methodForSelector:, but you can also use class_getMethodImplementation in the Objective-C runtime (useful if you only have a protocol reference, like id<SomeProto>). These function pointers are called IMPs, and are simple typedefed function pointers (id (*IMP)(id, SEL, ...))¹. This may be close to the actual method signature of the method, but will not always match exactly.

Once you have the IMP, you need to cast it to a function pointer that includes all of the details that ARC needs (including the two implicit hidden arguments self and _cmd of every Objective-C method call). This is handled in the third line (the (void *) on the right hand side simply tells the compiler that you know what you're doing and not to generate a warning since the pointer types don't match).

Finally, you call the function pointer².

Complex Example

When the selector takes arguments or returns a value, you'll have to change things a bit:

SEL selector = NSSelectorFromString(@"processRegion:ofView:");
IMP imp = [_controller methodForSelector:selector];
CGRect (*func)(id, SEL, CGRect, UIView *) = (void *)imp;
CGRect result = _controller ?
  func(_controller, selector, someRect, someView) : CGRectZero;

Reasoning for Warning

The reason for this warning is that with ARC, the runtime needs to know what to do with the result of the method you're calling. The result could be anything: void, int, char, NSString *, id, etc. ARC normally gets this information from the header of the object type you're working with.³

There are really only 4 things that ARC would consider for the return value:⁴

1. Ignore non-object types (void, int, etc)
2. Retain object value, then release when it is no longer used (standard assumption)
3. Release new object values when no longer used (methods in the init/ copy family or attributed with ns_returns_retained)
4. Do nothing & assume returned object value will be valid in local scope (until inner most release pool is drained, attributed with ns_returns_autoreleased)

The call to methodForSelector: assumes that the return value of the method it's calling is an object, but does not retain/release it. So you could end up creating a leak if your object is supposed to be released as in #3 above (that is, the method you're calling returns a new object).

For selectors you're trying to call that return void or other non-objects, you could enable compiler features to ignore the warning, but it may be dangerous. I've seen Clang go through a few iterations of how it handles return values that aren't assigned to local variables. There's no reason that with ARC enabled that it can't retain and release the object value that's returned from methodForSelector: even though you don't want to use it. From the compiler's perspective, it is an object after all. That means that if the method you're calling, someMethod, is returning a non object (including void), you could end up with a garbage pointer value being retained/released and crash.

Additional Arguments

One consideration is that this is the same warning will occur with performSelector:withObject: and you could run into similar problems with not declaring how that method consumes parameters. ARC allows for declaring consumed parameters, and if the method consumes the parameter, you'll probably eventually send a message to a zombie and crash. There are ways to work around this with bridged casting, but really it'd be better to simply use the IMP and function pointer methodology above. Since consumed parameters are rarely an issue, this isn't likely to come up.

Static Selectors

Interestingly, the compiler will not complain about selectors declared statically:

[_controller performSelector:@selector(someMethod)];

The reason for this is because the compiler actually is able to record all of the information about the selector and the object during compilation. It doesn't need to make any assumptions about anything. (I checked this a year a so ago by looking at the source, but don't have a reference right now.)

Suppression

In trying to think of a situation where suppression of this warning would be necessary and good code design, I'm coming up blank. Someone please share if they have had an experience where silencing this warning was necessary (and the above doesn't handle things properly).

More

It's possible to build up an NSMethodInvocation to handle this as well, but doing so requires a lot more typing and is also slower, so there's little reason to do it.

History

When the performSelector: family of methods was first added to Objective-C, ARC did not exist. While creating ARC, Apple decided that a warning should be generated for these methods as a way of guiding developers toward using other means to explicitly define how memory should be handled when sending arbitrary messages via a named selector. In Objective-C, developers are able to do this by using C style casts on raw function pointers.

With the introduction of Swift, Apple has documented the performSelector: family of methods as "inherently unsafe" and they are not available to Swift.

Over time, we have seen this progression:

1. Early versions of Objective-C allow performSelector: (manual memory management)
2. Objective-C with ARC warns for use of performSelector:
3. Swift does not have access to performSelector: and documents these methods as "inherently unsafe"

The idea of sending messages based on a named selector is not, however, an "inherently unsafe" feature. This idea has been used successfully for a long time in Objective-C as well as many other programming languages.

¹ All Objective-C methods have two hidden arguments, self and _cmd that are implicitly added when you call a method.

² Calling a NULL function is not safe in C. The guard used to check for the presence of the controller ensures that we have an object. We therefore know we'll get an IMP from methodForSelector: (though it may be _objc_msgForward, entry into the message forwarding system). Basically, with the guard in place, we know we have a function to call.

³ Actually, it's possible for it to get the wrong info if declare you objects as id and you're not importing all headers. You could end up with crashes in code that the compiler thinks is fine. This is very rare, but could happen. Usually you'll just get a warning that it doesn't know which of two method signatures to choose from.

⁴ See the ARC reference on retained return values and unretained return values for more details.