Unit-testing – How to write tests that make sense for visualization software

The standard flow for TDD is:

1. Write a failing test. (Red)
2. Make the smallest code change that makes it pass (Green)
3. Refactor (Keeping it green)

The test for your tests in this case is step 1 - making sure that the test fails before you make any code changes.

Another test that I like is whether you can delete some code and re-implement it a different way, and your tests fail after deletion but work with a different algorithm in place.

As with all things, there is no magic bullet. Forgetting to write a required test is just as easy for a developer to do as forgetting to write the code. At least if you're doing both, you have twice as many opportunities to discover your omission.

If you can send in different types of data to your methods, and some of those types have a chance of working incorrectly, then you need to be testing those possibilities. Plain and simple.

However, if that is really the case, I think the actual problem is that your code is not set up cleanly. I smell problems.

What I'm going to do is first talk about how to refactor your code so that it can easily be tested. Then, I'll explain how you might approach testing a system like the one you've described, if you have to (my hope is that you don't).

Code Refatoring (the best solution)

First, lets talk about why your methods allow so many different types to be sent in when they don't work consistently with them. This is an obvious code smell that should probably be addressed. My guess is that you need to refactor your code so that this isn't a problem in the first place. An approach I would take might look like this:

• I'm going to make sure that all objects sent to this method have a common interface
• I'm going to have the method only use this common interface to do what it needs
• The objects themselves will each have their own implementation of the interface that will always work for that type of object.

If you do this, you can now easily test the method and see that it is correctly using the common interface. You now know that this code is working no matter what object you put in. The interface is defined, so the correct action is taken.

You will also test each of the objects that implement the interface, and make sure that they have implemented it correctly for the type of object it is. Once you've tested that, you are done, you know everything is working without a million different tests testing each object with each type.

To illustrate, I've put together some pseudo-code to demonstrate. Lets say your code looks like this:

function output_to_file (shape x) {
    if (x is a triangle) {
        // output to file
    } else if (x is a square) {
        // output to file
    } else if (x is a pentagon) {
       // output to a file
    }
}

Then your tests are going to require you to send every different type of shape to the output_to_file method. If you have a whole bunch of methods like this, and a whole bunch of shapes, you will need a whole bunch of tests. Every time you add a shape, you need to modify your output_to_file code and add tests. No fun at all. Not good code.

Instead, you need to refactor your code to look more like this:

function output_to_file (shape x) {
    text = x.get_text_representation()
    save text to a file
}

Instead of handling the details of each shape, output_to_file just uses a common interface: the method get_text_representation.

Now to test output_to_file all you have to do is check that it is calling get_text_representation and putting the results in a file. If it is doing that, it is working. The output_to_file method now follows the "open/closed" principle. It is open to new types of objects being sent in (as long as they support the interface), but closed to the need for additional changes.

This means of course that our shapes need the get_text_representation function. But that is simple:

class triangle {
    function get_text_representation {
       // return whatever
    }
}

class square {
    function get_text_representation {
       // return whatever
    }
}

//and so on

Now we test the get_text_representation method on each of those objects to make sure that they are working like they should. That method might be used in a variety of places, but we don't really care, as long as it is working right we know that other methods relying on it will work right.

By setting up our objects in a different way, we've eliminated the need to test a whole bunch of different objects/method combinations. We test that the method is using the correct interface, and we test that the objects implement the interface correctly. Nothing more is needed. Yay!

Testing More Cases (the less than best solution)

Now perhaps you've been reading this so far, and you're thinking, "well that is nice and all, but I can't do that, he doesn't understand my situation."

Ok, first, try really hard to refactor your code. I'll be worth it. Did you try? Ok then, I'll give you benefit of the doubt and assume you are in some type of complicated situation that can't be refactored the way it should be (but I still don't believe you).

Here's the deal: you have to test each possible different type of object with each different method. You just have to. If some of those object/method combinations can cause error, you must test. Sorry.

Now, if it seems tedious or repetitive to be testing the same methods with different types of input again and again, you can automate the process somewhat. A lot of people forget that you can code your tests to do repetitive things for you. Let me give an example of how you might set up your test:

test_objects = {new triangle, new square, new pentagon .... }    

function test_output_to_file {
   foreach (test_objects as x) {
      output_to_file(x)
      assert everything is good
   }
}

This is really the best you can do if you can't refactor your code correctly. Is it tedious, yes. Is it clean, no. Refactor your code if you can, this isn't a good alternative.

Conclusion

In conclusion, the problem you are describing -- being able to send many different types of data to many different functions, some of which are not supported or buggy -- is probably an indication that your code isn't clean. Testing isn't the problem, and refactoring your code is the solution.

If for some reason refactoring isn't possible (I find that hard to believe!) then you are going to have to bite the bullet and test every combination. Make some structure that allows you to do that a little more easily.

I highly recommend anyone interested in creating clean, testable code to read the book Clean Code by Robert C Martin.