Java – Using JUnit as an acceptance test framework

acceptance-testingautomated-testsjavajunit

OK, so I work for a company who has openly adopted agile practices for development in recent years. Our unit tests and code quality are improving. One area we still are working on is to find what works best for us in the automated acceptance test arena. We want to take our well formed user stories and use these to drive the code in a test driven manner. This will also give us acceptance level tests for each user story which we can then automate.

To date, we've tried Fit, Fitnesse and Selenium. Each have their advantages, but we've also had real issues with them as well. With Fit and Fitnesse, we can't help but feel they overcomplicate things and we've had many technical issues using them. The business haven't fully bought in these tools and aren't particularly keen on maintaining the scripts all the time (and aren't big fans of the table style). Selenium is really good, but slow and relies on real time data and resources.

One approach we are now considering is the use of the JUnit framework to provide similiar functionality. Rather than testing just a small unit of work using JUnit, why not use it to write a test (using the JUnit framework) to cover an acceptance level swath of the application? I.e. take a new story ("As a user I would like to see basic details of my policy…") and write a test in JUnit which starts executing application code at the point of entry for the policy details link but covers all code and logic down to the stubbed data access layer and back to the point of forwarding to the next page in the application, asserting on what data the user should see on that page.

This seems to me to have the following advantages:

Simplicity (no additional frameworks required)
Zero effort to integrate with our Continuous Integration build server (since it already handles our JUnit tests)
Full skillset already present in the team (its just a JUnit test after all)

And the downsides being:

Less customer involvement (though they are heavily involved in writing the user stories in the first place from which the acceptance tests will be written)
Perhaps more difficult to understand (or make understood) the user story and acceptance criteria in a JUnit class verses a freetext specification ala Fit or Fitnesse

So, my question is really, have you ever tried this method? Ever considered it? What are your thoughts? What do you like and dislike about this approach? Finally, please only mention alternative frameworks if you can say why you like or dislike them more than this approach.

Best Answer

My experience using JUnit for an Acceptance Testing framework.

Our company did a very similar thing where we started with FitNesse and then went to JUnit. We did this mainly because FitNesse was sitting on top of Java already, so switching to JUnit was easy. FitNesse seemed to not fit (no pun intended) our needs.

Here are my pros and cons of using JUnit:

Pros:

JUnit has a lot of flexibility (because it's Java), so it's relatively easy to create an Internal Domain Specific Language[Fowler]. The DSL can be modified so that it's easy for domain experts (not software engineers) to read/write.
Since it's JUnit you can use Eclipse as an IDE, which gives you auto-complete, syntax checking, debugging, etc.
Specific for our application, we have a CORBA interface to the UI. Java can easily use this CORBA interface as well.

Cons:

People hear JUnit and think unit test. This gets confusing when people start referring to the acceptance tests as "JUnit tests".
To provide a simplified testing environment you have to spend some time extending the framework and developing your DSL.

So in summary, JUnit works just fine for Acceptance Tests if you are willing to spend the time to extend it into you own domain specific testing environment. Otherwise I think I would look elsewhere for something more out of the box.

Related Solutions

Java – How to test a class that has private methods, fields or inner classes

Update:

Some 10 years later perhaps the best way to test a private method, or any inaccessible member, is via @Jailbreak from the Manifold framework.
@Jailbreak Foo foo = new Foo();
// Direct, *type-safe* access to *all* foo's members
foo.privateMethod(x, y, z);
foo.privateField = value;
This way your code remains type-safe and readable. No design compromises, no overexposing methods and fields for the sake of tests.

If you have somewhat of a legacy Java application, and you're not allowed to change the visibility of your methods, the best way to test private methods is to use reflection.

Internally we're using helpers to get/set private and private static variables as well as invoke private and private static methods. The following patterns will let you do pretty much anything related to the private methods and fields. Of course, you can't change private static final variables through reflection.

Method method = TargetClass.getDeclaredMethod(methodName, argClasses);
method.setAccessible(true);
return method.invoke(targetObject, argObjects);

And for fields:

Field field = TargetClass.getDeclaredField(fieldName);
field.setAccessible(true);
field.set(object, value);

Notes:
1. TargetClass.getDeclaredMethod(methodName, argClasses) lets you look into private methods. The same thing applies for getDeclaredField.
2. The setAccessible(true) is required to play around with privates.

Java – How to assert that a certain exception is thrown in JUnit 4 tests

It depends on the JUnit version and what assert libraries you use.

For JUnit5 and 4.13 see answer https://stackoverflow.com/a/2935935/2986984
If you use assertJ or google-truth, see answer https://stackoverflow.com/a/41019785/2986984

The original answer for JUnit <= 4.12 was:

@Test(expected = IndexOutOfBoundsException.class)
public void testIndexOutOfBoundsException() {

    ArrayList emptyList = new ArrayList();
    Object o = emptyList.get(0);

}

Though answer https://stackoverflow.com/a/31826781/2986984 has more options for JUnit <= 4.12.

Reference :

JUnit Test-FAQ

Best Answer

Related Solutions

Java – How to test a class that has private methods, fields or inner classes

Java – How to assert that a certain exception is thrown in JUnit 4 tests

Related Topic