Python – the best/most Pythonic way to mock a private function

mockingpythontesting

Consider a module with a 'public' function which looks something like this:

def func(arg):
    val = _generate_something(arg)
    _do_something(val)

As you can see this is a 'void function'. Its core logic is in _generate_something.

I want to write unit tests for this function. I can think of several approaches to do so. I would like to know what the best one is, both in terms of general programming practices and best practice in Python specifically.

Essentialy this question is about the Pythonic way of mocking a private function (money patching? Injecting the function? Injecting an object? Etc.)

Option A

In the test monkeypatch _do_something (which is a 'private' function of the module), so I can check it receives a good value.

Option B

Inject _do_something as an argument to func, like so:

def func(arg, do_something=None):
    do_something = do_something or _do_something  # in production use default
    val = _generate_something(arg)
    do_something(val)

Then in my do_something mock I can check I received the correct value.

Option C

Have func be a method of a class, and have that class take do_something as an argument to __init__:

class Something(object):
    def __init__(self, do_something=None):
        self._do_something = do_something or _production_default

    def func(self, arg):
        val = self._generate_something(arg)
        self.do_something(val)

    # ... omitted for brevity ...

Option D

Like Option B or C, but do_something is injected as a 'full' object, not a bare function. For example:

def func(arg, something_doer=None):
    something_doer = something_doer or _something_doer # in production use default
    val = _generate_something(arg)
    something_doer.do_something(val)

As I said, I would like to know what the best option is, also with regard to 'Pythonicness'. For example, is it 'acceptable' in Python to mock bare functions (rather than objects which contain them)?

Best Answer

I would recommend to use the most simple and shortest approach, which is in Python probably option A, monkey patching. Everything else makes the code more complicated, with no substantial benefit. The only thing I would change here is to make this solution more explicit (see below).

Converting your module to a class (option C) should be motivated by the fact you need a class instead of a module, not by the fact you want to mock a single function. This option is preferable in languages like Java or C#, where there is no distinction between a "module" and a "class" in a way which is comparable to Python, and where a function like func would probably "live" inside a class at first hand.

Replacing the method do_something by an object with a method (option D) should not be done for the same reason - use an object instead of a standalone function when a standalone function is "not enough", for the sheer purpose of mocking it brings no advantage.

So this leaves you with option B or A. However, I would not pollute the parameter list of func for the purpose of mocking, nor would I stuff mocking code into a function func just for the purpose of mocking if there is simpler alternative. Better keep the "official" API as as well the functional code as clean as possible. Imagine what happens if you want to mock more "private" methods instead of just one, and if you want to test 5 public methods of the module - the API will contain 90% parameters only used for testing, which decreases the readability by a magnitude.

If you have concerns about the monkey patching approach because it changes something private, you can add a "public" (or "internal") patching function to the module, which allows the manipulation to be done through an "official" function:

def setup_mock_for_do_something(do_something):
    _do_something = do_something

This makes the possibility of "monkey patching" in this module more explicit, by simultanously avoiding unneccessary structural changes to the whole code.

So why is this all more "Pythonic"? Looking at "The Zen of Python", we see the following principles applied here

Explicit is better than implicit.
Simple is better than complex.
Readability counts.
If the implementation is easy to explain, it may be a good idea.

Use polymorphism and duck-typing before `isinstance()`

You generally define what you want to do with the shapes, then either use polymorphism to adjust how each shape responds to what you want to do, or you use duck typing; test if the shape at hand can do the thing you want to do in the first place. This is the invocation versus introspection trade-off, conventional wisdom states that invocation is preferable over introspection, but in Python, duck-typing is preferred over isinstance testing.

So you need to work out why you need to filter on class in the first place; why do you need to find all the rectangles? Perhaps they are the only type that implements a certain method, in which case you use duck typing to test for that method instead.

Or perhaps you need to process each type in a different way, because each type has to be handled just right for whatever you are doing with them. In that case, you need to give each shape the same method, but implement it differently for each type, placing the responsibility for adaptation in each object.

ABCs

With duck-typing, you may find that you have to test for multiple methods, and thus a isinstance() test may look a better option. In such cases, using a Abstract Base Class (ABC) could also be an option; using an ABC let's you 'paint' several different classes as being the right type for a given operation, for example. Using a ABC let's you focus on the tasks that need to be performed rather than the specific implementations used; you can have a Paintable ABC, a Printable ABC, etc.

Zope interfaces and component architecture

If you find your application is using an awful lot of ABCs or you keep having to add polymorphic methods to your classes to deal with various different situations, the next step is to consider using a full-blown component architecture, such as the Zope Component Architecture (ZCA).

zope.interface interfaces are ABCs on steroids, especially when combined with the ZCA adapters. Interfaces document expected behaviour of a class:

if IRectangleShape.providedBy(yourshape):
    # it's a rectangle

but it also let's you look up adapters; instead of putting all the behaviours for every use of shapes in your shape classes, you implement adapters to provide polymorphic behaviours for specific use-cases. You can adapt your shapes to be printable, or paintable, or exportable to XML:

class RectangleXMLExport(object):
    adapts(IRectangleShape)
    provides(IXMLExport)

    def __init__(self, rectangle):
        self.rectangle = rectangle

    def export(self):
        return u'<rectangle><width>{0}</width><height>{0}</height></rectangle>'.format(
            self.rectangle.width, self.rectangle.height)

and your code merely has to look up adapters for each shape:

for shape in shapebag:
    self.result.append(IXMLExport(shape).export())

Unit-testing – the most appropriate testing method in this scenario

Is there any important logic in your "entry point" method? It sounds like its only job is to instantiate some other classes and wire them together. If that's right then I'd say this is trivial code, it's not business critical, you'd notice immediately if it was broken (generally if you get this sort of 'composition' code wrong it won't even compile), so - I know this seems blasphemous - it doesn't need unit tests. "The best code is no code" also applies to tests: if a test isn't testing something important, difficult, or liable to break without detection, then that test is not worth the effort of writing or maintaining.

If the main method is more than just trivial code (try...catch blocks are common in this sort of method, as are if statements for configuration) then you should probably consider extracting some logic. For example, if you have lots of if statements in your app maybe you could extract a Configuration class which handles arbitrary configurations and can easily be unit tested or extended.

Best Answer

Related Solutions

Pythonic use of the isinstance function

Use polymorphism and duck-typing before isinstance()

ABCs

Zope interfaces and component architecture

Unit-testing – the most appropriate testing method in this scenario

Related Topic

Use polymorphism and duck-typing before `isinstance()`