Acceptance Criteria for Critical Software Based on Testability Estimates and Test Results

Testability is defined as the probability that a program will fail a test, conditional on the program containing some fault. In this paper, we show that statements about the testability of a program can be more simply described in terms of assumptions on the probability distribution of the failure intensity of the program. We can thus state general acceptance conditions in clear mathematical terms using Bayesian inference. We develop two scenarios, one for software for which the reliability requirements are that the software must be completely fault-free, and another for requirements stated as an upper bound on the acceptable failure probability.

[1]  J. Voas,et al.  Software Testability: The New Verification , 1995, IEEE Softw..

[2]  J. S. Armstrong,et al.  Review of Daniel Kahnemann, Paul Slovic, and Amos Tversky (eds.), Judgment Under Uncertainty: Heuristics and Biases , 1984 .

[3]  Jeffrey M. Voas,et al.  Faults on its sleeve: amplifying software reliability testing , 1993, ISSTA '93.

[4]  Lorenzo Strigini,et al.  Engineering judgement in reliability and safety and its limits: what can we learn from research in psychology , 1996 .

[5]  Keith W. Miller,et al.  Confidently Assessing a Zero Probability of Software Failure , 1993, SAFECOMP.

[6]  Bev Littlewood,et al.  Validation of ultrahigh dependability for software-based systems , 1993, CACM.

[7]  David Wright,et al.  Stopping rules for the operational testing of safety-critical software , 1995, Twenty-Fifth International Symposium on Fault-Tolerant Computing. Digest of Papers.

[8]  G. B. Finelli,et al.  The infeasibility of experimental quantification of life-critical software reliability , 1991, SIGSOFT '91.

[9]  John D. Musa,et al.  Operational profiles in software-reliability engineering , 1993, IEEE Software.

[10]  Lorenzo Strigini,et al.  On the Use of Testability Measures for Dependability Assessment , 1996, IEEE Trans. Software Eng..

[11]  Ricky W. Butler,et al.  The infeasibility of experimental quantification of life-critical software reliability , 1991 .

[12]  David Lorge Parnas,et al.  Evaluation of safety-critical software , 1990, CACM.

[13]  Lorenzo Strigini,et al.  Predicting Software Reliability from Testing Taking into Account Other Knowledge about a Program , 1996 .

[14]  Jeffrey M. Voas,et al.  Improving the software development process using testability research , 1992, [1992] Proceedings Third International Symposium on Software Reliability Engineering.

[15]  Jeffrey M. Voas,et al.  Estimating the Probability of Failure When Testing Reveals No Failures , 1992, IEEE Trans. Software Eng..

[16]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.