Sunday, November 27, 2005

Unit Testing in Perl with Test::Unit

I've come to prefer Test::Unit over Test::More. The main reason is that in Test::Unit you define each test as a separate subroutine, so the variables used for one test cannot pollute another test.

Test::More is a widely used module for Unit Tests in Perl. With Test::More your tests look like this:
# Test method_a()
my $expected_1 = 23 * $CONSTANT; # some value we expect;
is( $expected_1, $object->method_a(23),
"method_a(23) returned expected value.");

my $expected_2 = 17 * $CONSTANT; # A new expected value
is( $expected_2, $object->method_a(17),
"method_a(17) returned expected value"

# Test another_method()
my $expected_3 = _get_favorite_color();
is( $expected_3, $object->another_method('favorite'),
"another_method('favorite') returned proper color.");

my $expected_4 = _get_recent_color();
is( $expected_4 $object->another_method('recent'),
"another_method('recent') returned proper color.");

See how we need to come up with many versions of the $expected variable?

We could also just reuse it, but that can cause trouble if there are many tests and you forget to reset it.

It is better practice to use a different variable name for each test, but in many situations there are a couple dozen tests in that one file where it makes sense to use the same basic variable name for many different tests. That could lead to variable names like $expected_100. Also, there is always the risk that some temporary variables used for one test end up "polluting" a later test where the programmer didn't realize a variable was already defined earlier in the file.

The best solution is to isolate the code for each test, or small group of related tests into their own block, so that temporary variables for one test cannot interfere with another test or tests.

In Test::Unit each test is a subroutine, which can contain any number of assertions - so all the temporary variables and other setup for a particular group of tests are kept isolated from all the other tests. In addition, if you provide a subroutine called set_up() it is automatically called before running each test subroutine, and likewise, tear_down() is called after each test, this is useful for populating objects and test data structures, so that each test gets a clean set of test data to work with.

In Test::Unit the tests above might look like this:
sub test_method_a {
my $self = shift;

my $expected_1 = 23 * $CONSTANT; # some value we expect;
$self->assert_equals( $expected_1, $object->method_a(23));

my $expected_2 = 17 * $CONSTANT;
$self->assert_equals( $expected_2, $object->method_a(17));

sub test_another_method {
my $self = shift;

my $expected_1 = _get_favorite_color();
$self->assert_equals($expected_1, $object->another_method('favorite'));

my $expected_2 = _get_recent_color();
$self->assert_equals($expected_2, $object->another_method('recent'));
Test::Unit provides a bunch of assertion methods:
assert_equals / assert_not_equals

Tries to guess what kind of comparison to make. Usually works fine. If the first argument (the 'expected') is an object it checks if the == operator has been overloaded and will use it if so.


Force numeric/string comparison.

assert_matches(qr/PATTERN/, STRING, [, MESSAGE]), assert_not_matches

Assert that the regular expression matches.

assert_deep_equals(expected_ref, ref_to_test [, MESSAGE ])

Used to compare complex data strucrtures, like hashes of arrays of hashes. Assert that the the data pointed to by ref_to_test matches the data structure pointed to by expected_ref

See also:

Tags: ,,


Chris Dolan said...

I'm afraid your variable polluting argument is invalid. You can easily call Test::More functions from inside subroutines, just as you did in you Test::Unit example.

Matisse Enzer said...

I agree that one can work around the variable polluting problem - for example where I can't use Test::Unit I create a set of subroutines and the call each one using ok() or is(), etc. Something like:

is(test_read_arguments(), 'ok', 'Checking if read_arguments() works.');

But, I like the way Test::Unit makes it easy, and I like the fact that it follow the xUnit approach - now that last point is rather problematic, since it makes Test::Unit less Perlish. For example, Test::Unit throws an exception if a test fails, instead of simply reporting 'not ok'. When I started liking Test::Unit I was working at Technorati, where both Perl and Java and heavily used, so having a Perl test system that was similar to jUnit had an extra attraction to me.

I've asked for opinions about Test::Unit on the perl-qa mailing list and am getting some useful answers. I intend to look into Test::Class

Schwern said...

Subroutines just to provide lexical encapsulation in a test is overkill. Put the tests in a block, that's what I do. It even provides a point to name the block of tests.

# Testing foo
my $name = "foo";
is( $name, "foo" );

# Testing bar
my $name = "bar";
is( $name, "bar" );

If you really want subroutine-based testing, use Test::Class.

Matisse Enzer said...

I agree - using subroutines *only* to provide lexical encapsulation is indeed overkill. My original posting should have made that case.

There are additional reasons for using subroutines in tests - for example it's easier to run just one or a few tests.

Also, these days, I do in fact use Test::Class rather than Test::Unit.