Tools for the Analysis of the Accuracy of Software Reliability Predictions

Different software reliability models can produce very different answers when called upon to predict future reliability in a reliability growth context. Users need to know which, if any, of the competing predictions are trustworthy. Some techniques are presented which form the basis of a partial solution to this problem. In addition, it is shown that this approach can point the way towards more accurate prediction via models which learn from past behaviour.

[1]  Edward N. Adams,et al.  Optimizing Preventive Service of Software Products , 1984, IBM J. Res. Dev..

[2]  Jean-Claude Laprie,et al.  Dependability Evaluation of Software Systems in Operation , 1984, IEEE Transactions on Software Engineering.

[3]  Bev Littlewood,et al.  Stochastic Reliability-Growth: A Model for Fault-Removal in Computer-Programs and Hardware-Designs , 1981, IEEE Transactions on Reliability.

[4]  Bev Littlewood,et al.  Likelihood Function of a Debugging Model for Computer Software Reliability , 1981, IEEE Transactions on Reliability.

[5]  Bev Littlewood.  How to Measure Software Reliability and How Not To , 1979, IEEE Transactions on Reliability.

[6]  A. Dawid.  Calibration-Based Empirical Probability , 1985 .

[7]  Z. Jelinski,et al.  Software reliability Research , 1972, Statistical Computer Performance Evaluation.

[8]  H. Akaike.  Prediction and Entropy , 1985 .

[9]  Bev Littlewood,et al.  A Bayesian modification to the Jelinski-Moranda software reliability growth model , 1987 .

[10]  J. T. Duane.  Learning Curve Approach to Reliability Monitoring , 1964, IEEE Transactions on Aerospace.

[11]  Walter Freiberger,et al.  Statistical Computer Performance Evaluation , 1972 .

[12]  Amrit L. Goel,et al.  Time-Dependent Error-Detection Rate Model for Software Reliability and Other Performance Measures , 1979, IEEE Transactions on Reliability.

[13]  N. Reid,et al.  Estimating the Number of Faults in a System , 1985 .

[14]  David R. Cox,et al.  The statistical analysis of series of events , 1966 .

[15]  L. H. Crow.  Confidence Interval Procedures for Reliability Growth Analysis , 1977 .

[16]  M. Kendall,et al.  The advanced theory of statistics , 1945 .

[17]  Nozer D. Singpurwalla,et al.  An Empirical Stopping Rule for Debugging and Testing Computer Software , 1977 .

[18]  Douglas R. Miller.  Exponential order statistic models of software reliability growth , 1986, IEEE Transactions on Software Engineering.

[19]  Bev Littlewood,et al.  On the Quality of Software Reliability Prediction , 1983 .

[20]  M. Rosenblatt.  Remarks on a Multivariate Transformation , 1952 .

[21]  A. P. Dawid,et al.  Present position and potential developments: some personal views , 1984 .

[22]  A. Dawid.  The Well-Calibrated Bayesian , 1982 .

[23]  John Aitchison,et al.  Statistical Prediction Analysis , 1975 .

[24]  John D. Musa,et al.  A theory of software reliability and its application , 1975, IEEE Transactions on Software Engineering.

[25]  Henry Braun,et al.  A Comparative Study of Models for Reliability Growth , 1977 .

[26]  Bev Littlewood,et al.  A Bayesian Reliability Growth Model for Computer Software , 1973 .