Evaluating the performance of software-reliability models

A method of quantifying the performance of software reliability models that was proposed by B. Littlewood (1980) is investigated. The method is essentially a Kolmogorov-Smirnov goodness-of-fit test. It is demonstrated that although the Littlewood method is rather unconventional, the standard Kolmogrov-Smirnov tables (for the simple hypothesis) can be directly applied. On this basis, the kind of performance that can be anticipated from an ideal software reliability model is shown. A performance which overcomes a weakness in the Littlewood method is presented. >

[1]  N. L. Johnson,et al.  The probability integral transformation when parameters are estimated from the sample. , 1948, Biometrika.

[2]  George J. Schick,et al.  An Analysis of Competing Software Reliability Models , 1978, IEEE Transactions on Software Engineering.

[3]  J. Durbin.  Distribution theory for tests based on the sample distribution function , 1973 .

[4]  H. Lilliefors.  On the Kolmogorov-Smirnov Test for Normality with Mean and Variance Unknown , 1967 .

[5]  Bev Littlewood,et al.  Evaluation of competing software reliability predictions , 1986, IEEE Transactions on Software Engineering.

[6]  Alan N. Sukert.  Empirical Validation of Three Software Error Prediction Models , 1979, IEEE Transactions on Reliability.

[7]  Z. Jelinski,et al.  Software reliability Research , 1972, Statistical Computer Performance Evaluation.

[8]  Bev Littlewood,et al.  Theories of Software Reliability: How Good Are They and How Can They Be Improved? , 1980, IEEE Transactions on Software Engineering.

[9]  Thomas Downs.  Extensions to an approach to the modeling of software testing with some performance comparisons , 1986, IEEE Transactions on Software Engineering.

[10]  N. Singpurwalla,et al.  Methods for Statistical Analysis of Reliability and Life Data. , 1975 .

[11]  Z. Birnbaum.  Numerical Tabulation of the Distribution of Kolmogorov's Statistic for Finite Sample Size , 1952 .

[12]  P. Garrone,et al.  Some new models of software testing with performance comparisons , 1991 .

[13]  J. Durbin.  Kolmogorov-Smirnov tests when parameters are estimated with applications to tests of exponentiality and tests on spacings , 1975 .

[14]  Bev Littlewood.  The Littlewood-Verrall model for software reliability compared with some rivals , 1980, J. Syst. Softw..

[15]  H. Lilliefors.  On the Kolmogorov-Smirnov Test for the Exponential Distribution with Mean Unknown , 1969 .

[16]  Bev Littlewood,et al.  Criteria for Software Reliability Model Comparisons , 1984, IEEE Transactions on Software Engineering.