Analyzing the test process using structural coverage

A large, commercially developed FORTRAN program was modified to produce structural coverage metrics. The modified program was executed on a set of functionally generated acceptance tests and a large sample of operational usage cases. The resulting structural coverage metrics are combined with fault and error data to evaluate structural coverage in the SEL environment. We can show that in this environment the functionally generated tests seem to be a good approximation of operational use. The relative proportions of the exercised statement sub-classes (executable, assignment, CALL, DO, IF, READ, WRITE) changes as the structural coverage of the program increases. We also propose a method for evaluating if two sets of input data exercise a program in a similar manner. We also provide evidence that implies that in this environment, faults revealed in a procedure are independent of the number of times the procedure is executed and that it may be reasonable to use procedure coverage in software models that use statement coverage. Finally, the evidence suggests that it may be possible to use structural coverage to aid in the management of the acceptance test process.