Linking unit tests and properties

QuickCheck allows us to verify software against particular properties. A property can be regarded as an abstraction over many unit tests. QuickCheck uses generated random input data to test such properties. If a counterexample is found, it becomes immediately clear what we have tested. This is not the case when all tests pass, since we do not (and shall not) see the actual generated test cases. How can we be sure about what is tested? QuickCheck has the ability to gather statistics about the test cases, which is insightful. But still it does not tell us whether the particular unit test scenarios we have in mind are included. For this reason, we have developed a tool that can answer this question. It checks if a given unit test can be generated by a property, making it easier to judge the property's quality. We have applied our tool to an industrial use case of testing the AUTOSAR basic software modules and shows that it can handle complex models and large unit tests.

[1]  John M. Rushby,et al.  Automated Test Generation and Verified Software , 2005, VSTTE.

[2]  John Hughes,et al.  Testing telecoms software with quviq QuickCheck , 2006, ERLANG '06.

[3]  Meng Wang,et al.  Feat: functional enumeration of algebraic types , 2012, Haskell.

[4]  John Hughes,et al.  QuickCheck Testing for Fun and Profit , 2007, PADL.

[5]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2000, ICFP.

[6]  Ulf Norell,et al.  An Expressive Semantics of Mocking , 2014, FASE.

[7]  Koen Claessen,et al.  Generating Constrained Random Data with Uniform Distribution , 2014, FLOPS.

[8]  Ulf Norell,et al.  Testing AUTOSAR software with QuickCheck , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[9]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.