Predicting Software Reliability from Testing Taking into Account Other Knowledge about a Program

Inference from statistical testing is the only sound method available for estimating software reliability. However, if one ignores evidence other than testing (e.g., evidence from the track record of a developer, or from the quality of the development process), the results are going to be so conservative that they are often felt to be useless for decision-making. Bayesian inference is the main mathematical tool for taking into account such knowledge. Evidence from sources other than testing is modelled as prior probabilities (for values of the failure rate of the program) and is updated on the basis of test results to produce posterior probabilities. We explain these methods and demonstrate their use on simple examples. The measure of interest is the probability that a program satisfies a given reliability requirement, given that it has passed a certain number of tests. The procedures of Bayesian inference explicitly show the weights of prior assumptions vs. test results in determining this probability. We also demonstrate how one can model different assumptions about the faultrevealing efficacy of testing. We believe that these methods are a powerful aid for improving the quality of decision-making in matters related to software reliability.