Towards a Framework for Differential Unit Testing of Object-Oriented Programs

Software developers often face the task of determining how the behaviors of one version of a program unit differ from (or are the same as) the behaviors of a (slightly) different version of the same program unit. In such situations, developers would like to generate tests that exhibit the behavioral differences between the two versions, if any differences exist. We call this type of testing differential unit testing. Some examples of differential unit testing include regression testing, N-version testing, and mutation testing. We propose a framework, called Diffut, that enables differential unit testing of object-oriented programs. Diffut enables "simultaneous" execution of the pairs of corresponding methods from the two versions: methods can receive the same inputs (consisting of the object graph reachable from the receiver and method arguments), and Diffut compares their outputs (consisting of the object graph reachable from the receiver and method return values). Given two versions of a Java class, Diffut automatically synthesizes annotations (in the form of preconditions and postconditions) in the Java Modeling Language (JML) and inserts them into the unit under test to allow the simultaneous execution of the corresponding methods.