popper
Project home

Summary	Popper checks a unit's behavior over several data points
Categories	design, testing
License	Common Public License
Owner(s)	dsaff

News!

2007 May 01: Popper version 0.5 has been released.

• TheoryContainer.assumeNotNull(Object...) checks a list of inputs to make sure none is null. (This is useful when using Popper with JUnit Factory.)
• TheoryContainer.assumeNoException(Throwable) converts a fatal exception into an indication that the Theory parameters were invalid.
• Zero-parameter methods annotated with @DataPoint are invoked to obtain values for similarly-typed Theory parameters. If invocation causes an exception, the data point is skipped.
• Added Requirements.each, which requires that each element in a collection pass a requirement.

2007 May 01: Popper version 0.5 has been released.

• TheoryContainer.assumeNotNull(Object...) checks a list of inputs to make sure none is null. (This is useful when using Popper with JUnit Factory. (See my upcoming blog entry.))
• Added Requirements.each, which requires that each element in a collection pass a requirement.

2007 April 26: Popper version 0.4 has been released. The only new feature is that requirements that return null descriptions (which sometimes happens during development) are better reported by the theories runner.

2007 April 19: Popper version 0.3 has been released. This is a quick incremental release with impovements to the new assertion mechanism, including:

• not(requirement), for example, not(containsString("a")). This is implemented by calling requirement.opposite(), which can be overridden in custom Requirements to provide better descriptions, examples, or evaluation strategies for the negation of a Requirement.
• isNull(x)
• Better formatting for assertion errors, to indicate possible trailing spaces on compared values.
• AutoNamedRequirement: an anonymous inner subclass of AutoNamedRequirement constructed in a factory method will automatically determine its name from the name of the factory method. For example:

  	public static Requirement> isNull() {
  		return new AutoNamedRequirement>() {
  			@Override public boolean isSatisfiedBy(Object value) {
  				return value == null;
  			}		
  		};
  	}
  

  will return a requirement whose description is automatically set to "is null".

2007 April 17: Popper version 0.2 has been released, and the tutorial has been updated. Popper is now a single download on top of JUnit, with no other requirements. A new assertion package is now included, with many similarities to Hamcrest, and some differences:

• Assertions are expressed in terms of Requirements, which are very similar to Hamcrest's Matchers.
• Requirement.description() returns a String, simplifying from Hamcrest's describeTo API.
• To combine requirements, an and method is included in the Requirement base class. To say that value contains "a" and "b", use assertThat(value, containsString("a").and(containsString("b"))). This reads better than Hamcrest's allOf method, and typechecks better.
• Requirement.exampleSatisfyingValue() can optionally return a value that satisfies the Requirement, which can assist simple stub generators and constraint solvers.
• containsString can be applied to Objects other than Strings, in which case the requirement is evaluated against the Object's toString().
• is(Class) is typed as applying to any Object, which fixes several typing problems.

Popper has no intent of replacing Hamcrest. My Requirement library is rather sparse, so to use a Hamcrest Matcher, you can adapt it to a Requirement using matches: assertThat(x, matches(greaterThan(10))). I regard several of my changes as bug fixes, and I will work with the Hamcrest authors to see if they can be included in the base Hamcrest. The main reason for separation is to experiment with Popper without requiring changes to Hamcrest, or a patched version of the Hamcrest library.

2007 Feb 6: Popper version 0.1 has been released. Please see the new tutorial for instructions on downloading and usage.

Mission

JUnit allows developers to state and verify individual facts about their code. Popper, an extension to JUnit, allows developers to state and partially verify general system properties called "theories", which may hold over potentially infinite sets of values. Theories were first described in David Saff and Marat Boshernitsan's article The Practice of Theories: Adding "For-all" Statements to "There-Exists" Tests, under submission to IEEE Software. Popper is not a theorem prover, which would logically prove that a theory holds, nor does it search for inputs that might violate the theory. Popper simply verifies the general properties stated against a developer-specified set of input values, and provides helpful feedback if verification fails.

For example, here is a Theory about currency multiplication, together with three data values on which the theory should be verified to work:

	public static int TWO = 2;
	public static int FIVE = 5;
	public static int TEN = 10;

	@Theory public void multiplyIsInverseOfDivide(int amount, int m) {
		assumeThat(m, not(0));
		assertThat(new Dollar(amount).times(m).divideBy(m).getAmount(),
				is(amount));
	}

Popper can be extended with ParameterSuppliers. These are indicated by extension-supplied annotations on the Theory method's parameters. For example, an extension could supply all integer parameters in a range:

	@Theory public void multiplyIsInverseOfDivide(
	  @Between(first=0, last=10) int amount, 
	  @Between(first=0, last=5)  int m
	) {
		assumeThat(m, not(0));
		assertThat(new Dollar(amount).times(m).divideBy(m).getAmount(),
				is(amount));
	}

Popper does not provide any code analysis or heuristic features for guessing parameters that might violate the Theory, a process we call "exploration". There are automated tools like Agitator or JUnit Factory that may help with this.

Popper is implemented in Java using Java 5 language features, and is not guaranteed to work on any previous versions of the JVM. Popper no longer depends on the Hamcrest matcher library for specifying parameter assumptions and assertions, although it still easily integrates with Hamcrest. I hope that the project will soon be absorbed into a more general-purpose testing framework. Until then, this is its temporary home.

Related resources

Popper draws inspiration from:

• JUnit, the original "fact-based" unit-testing framework for Java
• Parameterized Unit Tests , which use symbolic execution to generate tests from general specifications.
• QuickCheck, an automatic testing tool for Haskell that verifies properties on random input
• The Parameterized Test Case pattern.
• The Abstract Test Case pattern.