Experiments with Data Flow and Mutation Testing

This paper presents two experimental comparisons of data ow and mutation testing. These two techniques are widely considered to be e ective for unit-level software testing, but can only be analytically compared to a limited extent. We compare the techniques by evaluating the e ectiveness of test data developed for each. For a number of programs, we develop ten independent sets of test data; ve to satisfy the mutation criterion, and ve to satisfy the all-uses data ow criterion. These test sets are developed using automated tools, in a manner consistent with the way a test engineer would apply these testing techniques. We use these test sets in two separate experiments. First we apply a \cross scoring", by measuring the e ectiveness of the test data that was developed for one technique in terms of the other technique. Second, we investigate the ability of the test sets to nd faults. We place a number of faults into each of our subject programs, and measure the number of faults that are detected by the test sets. Our results indicate that while both techniques are e ective, mutation-adequate test sets are closer to satisfying data ow, and detect more faults.