Automated Random Testing to Detect Specification-Code Inconsistencies

An interface specification language such as JML provides a means to document precisely the behavior of program modules such as Java classes, and it is being adopted by industry. However, few practical tools exist for programmers to assure the correctness of their interface specifications. Nonetheless, the correctness of an interface specification is a prerequisite for the use of the specification, both as a precise API documentation and as a foundation for formal verification of and reasoning about the implementation. We propose automated random testing as a practical tool to assure the correctness of interface specifications. The key idea of our approach is to fully automate dynamic, random testing to detect as many inconsistencies as possible between the specification and its implementation. For this, we use a runtime assertion checker as a test oracle, and the goal of our testing is to generate as many non-duplicate test cases as possible that incur a certain type of runtime assertion violations. Our approach has been implemented for Java/JML in a prototype tool called JET, and a preliminary experiment shows that it has a potential to be a valuable testing tool for Java/JML. Our approach can be adapted for other interface specification languages.