Testing for Exceptions

The old way

There are a few ways of testing for exceptions. The old way was to just catch the exception and then do any asserts that you need. See below:

Advantages:

• you can do anything you like

Disadvantages:

• boilerplate takes up a large part
• errorprone, for example forgetting to add the fail, means the testcase will pass if there's no exception.

TestNG

TestNG provides an extention to the @Test annotation that checks for exceptions. In the example below, it is even possible to check that the appropriate exception message is returned (using a regular expression).

Advantages:

• it is immediately clear that the test tests an error case

Disadvantages:

• not possible to do any checks or asserts after the exception is thrown

I did have a discussion with a colleague of mine, regarding the fact that I wished to verify data as well as exceptions.

His argument is, if your test is both verifying data as well as exceptions, you could pull these two apart into two separate tests. One test that tests for the exception, and one test that verifies the data. I countered that, as in his case, both tests verify the exact same behaviour in the SUT (System Under Test), it should be one test. Let me know what your opinions are.

JUnit

JUnit, in the new version (since 4.7), has an additional solution based on mocking stuff:

Advantages:

• asserts after the exception is thrown become possible
• you can do anything you like

Disadvantages:

• a little boilerplate
• not immediately clear that it's an exceptional testcase
• I don't much like mocking

References

StackOverflow - JUnit Testing Exceptions

http://stackoverflow.com/questions/15216438/junit-testing-exceptions

StackOverflow - JUnit test analysing expected exceptions

http://stackoverflow.com/questions/4489801/junit-test-analysing-expected-exceptions

JDK7 EJB3.1 and Netbeans Project (Part III) - Testing

Part I - Introduction, Part II - Hibernate and Transactions, Part III - Testing

Netbeans + TestNG

I have been trying out TestNG and JMockit in the new Netbeans. ^[1]

TestNG has become a standard part of Netbeans, that is, if you download the Nightly builds^[2] of Netbeans. There is no longer a need to install the plugin from contrib. I used it, and I was suitably impressed.

In Bugzilla you can file bugs under "java/TestNG".

JMockit

I do like adding JMockit to my testing, because I've gotten used to mocking all the classes that I am not interested in and being able to provide their behavior in the tests. Especially handy to prevent having to use a database.

In this case, this was easy as pie. Just download^[3] and add the jmockit.jar to the testing libraries, and away I went.

Results

An example of what the results look like in Netbeans can be viewed at [2]. I like the test reports generated, though, they provide a deal more information on what is going wrong.

References

[1] Netbeans - TestNG

http://wiki.netbeans.org/TestNG

[2] Netbeans - Nightly Builds

http://bits.netbeans.org/download/trunk/nightly/latest/

[3] JMockit

http://code.google.com/p/jmockit/

The JMockit Testing Toolkit Tutorial

http://jmockit.googlecode.com/svn/trunk/www/tutorial.html

Unit Testing With TestNG and JMockit

http://java.dzone.com/articles/unit-testing-with-testng-and-j