GenCheck – a generalized property-based testing framework

Test.GenCheck is a Haskell library for generalized proposition-based testing. It simultaneously generalizes QuickCheck and SmallCheck.

Its main novel features are:

• introduces a number of testing strategies and strategy combinators
• introduces a variety of test execution methods
• guarantees uniform sampling (at each rank) for the random strategy
• guarantees both uniqueness and coverage of all structures for the exhaustive strategy
• introduces an extreme strategy for testing unbalanced structures
• also introduces a uniform strategy which does uniform sampling along an enumeration
• allows different strategies to be mixed; for example one can exhaustively test all binary trees up to a certain size, filled with random integers.
• complete separation between properties, generators, testing strategies and test execution methods

The package is based on a lot of previous research in combinatorics (combinatorial enumeration of structures and the theory of Species), as well as a number of established concepts in testing (from a software engineering perspective). In other words, further to the features already implemented in this first release, the package contains an extensible, general framework for generators, test case generation and management. It can also be very easily generalized to cover many more combinatorial structures unavailable as Haskell types.

The package also provides interfaces for different levels of usage. In other words, there is a ‘simple’ interface for dealing with straightforward testing, a ‘medium’ interface for those who want to explore different testing strategies, and an ‘advanced’ interface for access to the full power of GenCheck.

See http://hackage.haskell.org/package/gencheck for further details.

In the source repository (https://github.com/JacquesCarette/GenCheck), the file tutorial/reverse/TestReverseList.lhs shows the simplest kinds of tests (standard and deep for structures, or base for unstructured types) and reporting (checking, testing and full report) for the classical list reverse function. The files in tutorial/list_zipper show what can be done with the medium level interface (this tutorial is currently incomplete). The brave user can read the source code of the package for the advanced usage — but we’ll write a tutorial for this too, later.

User beware: this is gencheck-0.1, there are still a few rough edges.  We plan to add a Template Haskell feature to this which should make deriving enumerators automatic for version 0.2.

Jacques and Gordon