There’s a testing pattern I keep bumping into that I’ve not seen discussed before.
It can be very powerful and flexible, producing clean tests that are easy to maintain.
Because we need to give it a name to discuss it, I’m going to call it the Verifier Pattern, but I suspect people will already have different terms for it.
Background
Back in the day, when all we had was Java 1.4 and JUnit 3, code reuse in tests was a pain. We had no static imports, and adding member variables to JUnit 3 classes could easily result in memory leaks. Code reuse was largely done by creating class hierarchies, or by copy paste.
Then along came Java 5 and life got much better. We could define custom assert statements as static methods, and easily pull them into unrelated classes. We learned to do the same for code that created objects, with factory methods and Object Mothers.
But that wasn’t the only code that we wanted to share.
There was also the code that coordinates everything and glues it together. The test code that actually does stuff.
JUnit 4 and 5 eventually made reusing this code possible with Rules and Extensions, but before they came along we had to find others ways. And sometimes they can still be the better fit.
Which brings us to the Verifier pattern.
Verifiers
The simplest possible of implementation of a class we might call a Verifier looks like this.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
class WidgetVerifier {
static void rejectsInvalidWidgets(WidgetProcessor underTest) {
Widget invalid = createInvalidWidget();
underTest.initialise();
underTest.accept(invalid);
// more method calls to make an valid sequence
WidgetResult result = underTest.complete();
assertThat(result.isValid()).isFalse();
}
static Widget createInvalidWidget() {
// construction code, perhaps just a call to an Object Mother.
}
}
It’s contract is “you give me an Object, I’ll verify it has a certain property or behaviour.”
It’s very similar to a parametrised test method, but implemented in plain old Java without framework magic and, as we’ll see later, it can grow into something much more versatile.
A simple Verifier like this might grow more methods.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
class WidgetVerifier {
static void acceptsValidWidgets(WidgetProcessor underTest) {
var widget = createValidWidget();
result = underTest.process(underTest, invalid);
var result = underTest.complete();
assertThat(result.isValid()).isTrue();
}
static void rejectsInvalidWidgets(WidgetProcessor underTest) {
var widget = createInvalidWidget();
result = underTest.process(underTest, invalid);
var result = underTest.complete();
assertThat(result.isValid()).isFalse();
}
static WidgetResult proccess(WidgetProcessor underTest, Widget widget) {
underTest.initialise();
underTest.accept(widget);
// more method calls to make an valid sequence
return underTest.complete();
}
}
This is ugly, but does achieve some useful things. We’ve captured the sequence of method calls required to process a widget in one place. That doesn’t look too compelling here, but if the sequence were complex, or required difficult to construct collaborators, the benefit would be clearer.
Unlike a parametrised test method, we can call this from any class. So, if we have lots of different implementations (or configurations) of WidgetProcessor to test, we can keep reusing this method, so long as the definition of an invalid widget is constant across them.
If our WidgetProcessor contract ever changes, we’ll have just one bit of code to update even if we have tests for hundreds of different WidgetProccesors.
This approach is limited, however. The Verifier pattern only starts to look useful when things get more complex.
Lets imagine our WidgetProcessor needs to collaborate with some other objects that are mildly painful to construct.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
class WidgetVerifier {
static void rejectsInvalidWidgets(WidgetProcessor underTest) {
var widget = createInvalidWidget();
result = underTest.process(underTest, invalid);
var result = underTest.complete();
assertThat(result.isValid()).isFalse();
}
static WidgetResult proccess(WidgetProcessor underTest, Widget widget) {
underTest.initialise(createComplexObjectA(), createComplexObjectB());
underTest.accept(widget);
// more method calls
return underTest.complete()'
}
}
Our code under test now has three inputs, one supplied directly, two supplied during initialisation. If we expect all our WidgetProcessor implementations to behave in the same way when given the same inputs, we could keep adding static methods to out Verifier class, but that approach quickly falls apart if the inputs need to vary.
Lets change our Verifier so it has some state and follows the Builder pattern.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
class WidgetVerifier {
WidgetProcessor underTest;
ComplexObjectA a = createDefaultComplexObjectA();
ComplexObjectB b = createDefaultComplexObjectB();
Widget widget = createValidWidget();
WidgetVerifier(WidgetProcessor underTest) {
this.underTest = underTest;
}
static WidgetVerifier verifyThatFor(WidgetProcessor underTest) {
return new WidgetVerifier(underTest);
}
WidgetResultAssertion forWidget(Widget widget) {
new WidgetResultVerifier(process());
}
void with(ComplexObjectA a) // implementation ommited
void with(ComplexObjectB b) // implementation ommited
}
class WidgetResultAssertion {
private final WidgetResult result;
public WidgetResultVerifier(WidgetResult result) {
this.result = result;
}
public void resultIsInvalid() {
assertThat(result.isValid()).isFalse();
}
public void resultIsCalid() {
assertThat(result.isValid()).isTrue();
}
}
We can use it like this
1
2
3
4
5
6
7
8
9
10
11
@Test
void xxx() {
var underTest = new FancyWidgetProcessor();
var objectA = // make an objectA not like the default
var widget = // make a widget
WidgetVerfier.verifyThatFor(underTest)
.with(objectA)
.forWidget(widget)
.resultIsInvalid();
}
We’ve created a DSL. It concentrates the knowledge of the contract that WidgetProcessors must adhere to in one place, and hides any messy passing of data back and forth.
If we lean on a library such as AssertJ, we can expand the DSL to include rich custom assertions with very little effort.
1
2
3
4
5
6
7
8
9
10
11
@Test
void xxx() {
var underTest = // construct processor
var objectA = // make an objectA not like the default
var widget = // make a widget
WidgetVerfier.verifyThatFor(underTest)
.with(objectA)
.forWidget(widget)
.validationMessage().contains("Oh no!")
}
If we have areas of common behaviour across WidgetProcessors, we could add a method to the verifier which uses soft assertions to confirm them in a single call.
1
2
3
4
5
6
7
@Test
void xxx() {
WidgetVerfier.verifyThatFor(underTest)
.with(objectA)
.forWidget(widget)
.standardWidgetProcessorContractIsMaintained();
}
And for the custom behaviours we can break out into our DSL to specify them.
Conclusion
So, that’s the Verifier test pattern. It packages up the “glue” code required for tests so it can be reused from anywhere within a suite, and provides easy access to default versions of collaborators and custom assertions. It can result in tests that are clean and maintainable. It really shines when we have many implementations of a single interface or contract.
This post was sparked by a change I made recently to the pitest codebase, introducing a Verifier to replace a prehistoric base class, while finally removing a stubborn dependency on XStream.
You can see the verifier here.
If you look through the commit history, you can also see the base class the verifier replaced. The resulting tests are little ugly as they’ve been quickly migrated from one style to another using find and replace, but the after tests still look a nicer than the befores.
I’ve encountered many classes like this in closed source code bases. Their names and styles vary, but they are all fundamentally the same thing. They usually emerge in a scrappy fashion, pulling in code that has been copy and pasted elsewhere.
The choice of Verifier as a name for this pattern came from EqualsVerifier. This lives in a grey world between testing and static analysis, but is a fantastic tool for checking your hashcode equals contracts, and is still recognisably the same kind of beast as the little verifier I showed above.
Thanks for reading. Checkout our industrial quality mutation testing tools for the jvm.