Probable Correctness Theory

Abstract A theory of ‘probable correctness’ is proposed to assess the reliability of software through testing. Current research in testing is not adequate for this assessment. Most testing methods are intended for debugging, to find failures and connect them to program faults for repair. When these methods no longer expose errors, no analysis has been done to find the confidence that may be placed in the software. (Preliminary results here are that this confidence should be low.) Other work applies conventional decision theory to inputs as samples of a program's use. The application is suspect because the necessary independence and distribution assumpions may be violated; in any case, the results are intuitively incorrect. The proposed theory relies on a uniform distribution of test samples, but relates these to textually occurring faults. Preliminary results include an analysis of partition testing, and suggestions for textual sampling. It is crucial that any such confidence theory be plausible, so the foundations of program sampling are examined in detail.