Do Software Reliability Prediction Models Meet Industrial Perceptions?

Reliability is one of the key characteristics inherent in the concept "software quality". Understanding software reliability has been vital for both the software managers and software engineers, especially while developing critical systems. In the past several decades, an extensive body of knowledge has been developed and to a certain extent practical knowledge has been gathered by applying them to large scale mission- or safety-critical projects. However, software reliability models are falling short of gaining confidence in industrial community. From the software developer's perspective, software reliability has been primarily associated with defects, which represent a major cost factor in software development. This is called developer oriented approach. As software orientation has become ubiquitous, and software intensive systems have been proliferating, software quality assurance has been embracing more customer oriented perspectives. In this context, we will need richer, realistic and more accurate measures for software reliability. A recent empirical study by the authors highlighted many factors, which the practitioners perceived to be influential in the context of reliability prediction. This paper focuses on different reliability prediction models proposed in the literature, specifically examining whether the influential factors perceived to be contributing to the reliability are addressed or not in these models. We evaluated the assumptions made in each of the considered reliability models and checked their applicability in software product development. This paper concludes with a comparative presentation of a large set of well-known reliability models with respect to the set of factors identified by the industry as influential in software reliability.

[1]  Z. Jelinski,et al.  Software reliability Research , 1972, Statistical Computer Performance Evaluation.

[2]  Aditya P. Mathur,et al.  On the estimation of reliability of a software system using reliabilities of its components , 1997, Proceedings The Eighth International Symposium on Software Reliability Engineering.

[3]  Youngki Hong,et al.  Prediction of defect distribution based on project characteristics for proactive project management , 2010, PROMISE '10.

[4]  Bev Littlewood,et al.  Conceptual Modeling of Coincident Failures in Multiversion Software , 1989, IEEE Trans. Software Eng..

[5]  Brendan Murphy,et al.  Post-release reliability growth in software products , 2008, TSEM.

[6]  John D. Musa,et al.  A theory of software reliability and its application , 1975, IEEE Transactions on Software Engineering.

[7]  Domenico Cotroneo,et al.  RELAI Testing: A Technique to Assess and Improve Software Reliability , 2016, IEEE Transactions on Software Engineering.

[8]  Shigeru Yamada,et al.  Software Reliability Growth Models with Testing-Effort , 1986, IEEE Transactions on Reliability.

[9]  Witold Pedrycz,et al.  Incremental Effort Prediction Models in Agile Development using Radial Basis Functions , 2007, SEKE.

[10]  Bev Littlewood,et al.  Likelihood Function of a Debugging Model for Computer Software Reliability , 1981, IEEE Transactions on Reliability.

[11]  Dave E. Eckhardt,et al.  A Theoretical Basis for the Analysis of Multiversion Software Subject to Coincident Errors , 1985, IEEE Transactions on Software Engineering.

[12]  S. M. K. Quadri,et al.  Comparison of Predictive Capability of Software Reliability Growth Models with Exponentiated Weibull Distribution , 2011 .

[13]  Bev Littlewood,et al.  Software reliability and dependability: a roadmap , 2000, ICSE '00.

[14]  Bojan Cukic,et al.  A Bayesian approach to reliability prediction and assessment of component based systems , 2001, Proceedings 12th International Symposium on Software Reliability Engineering.

[15]  Peter Kubat,et al.  Assessing reliability of modular software , 1989 .

[16]  Zhibo Wu,et al.  A study on software reliability prediction based on triple exponential smoothing method (WIP) , 2014, SummerSim.

[17]  Ram Chillarege,et al.  Identifying risk using ODC based growth models , 1994, Proceedings of 1994 IEEE International Symposium on Software Reliability Engineering.

[18]  Bora Caglayan,et al.  Usage of multiple prediction models based on defect categories , 2010, PROMISE '10.

[19]  Rajib Mall,et al.  An approach for early prediction of software reliability , 2010, SOEN.

[20]  Poong Hyun Seong Reliability and Risk Issues in Large Scale Safety-critical Digital Control Systems , 2008 .

[21]  Eldred Nelson,et al.  Estimating software reliability from test data , 1978 .

[22]  Roger C. Cheung,et al.  A User-Oriented Software Reliability Model , 1978, IEEE Transactions on Software Engineering.

[23]  Bojan Cukic,et al.  Early reliability assessment of UML based software models , 2002, WOSP '02.

[24]  John D. Musa,et al.  Software reliability - measurement, prediction, application , 1987, McGraw-Hill series in software engineering and technology.

[25]  Roshanak Roshandel Toward architecture-based reliability estimation , 2004, ICSE 2004.

[26]  Facultad de Ingenier,et al.  Compound and Non Homogeneous Poisson Software Reliability Models , 2010 .

[27]  Kishor S. Trivedi,et al.  Markov reward approach to performability and reliability analysis , 1994, Proceedings of International Workshop on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[28]  Sasikumar Punnekkat,et al.  An industrial survey on the influence of process and product attributes on software product reliability , 2017, 2017 International Conference on Networks & Advances in Computational Technologies (NetACT).

[29]  James A McCall,et al.  Software Reliability, Measurement, and Testing Software Reliability and Test Integration , 1992 .

[30]  Martin L. Shooman,et al.  A many-state Markov model for the estimation and prediction of computer software performance parameters , 1975, Reliable Software.

[31]  Hany H. Ammar,et al.  A scenario-based reliability analysis approach for component-based software , 2004, IEEE Transactions on Reliability.

[32]  Robyn R. Lutz,et al.  Empirical evaluation of reliability improvement in an evolving software product line , 2011, MSR '11.

[33]  Kishor S. Trivedi,et al.  Reliability modeling of life-critical, real-time systems , 1994, Proc. IEEE.

[34]  Mehmet Sahinoglu,et al.  Compound-Poisson Software Reliability Model , 1992, IEEE Trans. Software Eng..

[35]  Swapna S. Gokhale,et al.  Log-logistic software reliability growth model , 1998, Proceedings Third IEEE International High-Assurance Systems Engineering Symposium (Cat. No.98EX231).

[36]  Kai-Yuan Cai,et al.  On the Asymptotic Behavior of Adaptive Testing Strategy for Software Reliability Assessment , 2014, IEEE Transactions on Software Engineering.

[37]  Martin L. Shooman,et al.  Probabilistic Models for Software reliability Prediction , 1972, Statistical Computer Performance Evaluation.

[38]  Domenico Cotroneo,et al.  Combining Operational and Debug Testing for Improving Reliability , 2013, IEEE Transactions on Reliability.

[39]  Shigeru Yamada,et al.  s-Shaped Software Reliability Growth Models and Their Applications , 1984, IEEE Transactions on Reliability.

[40]  Ye Yang,et al.  Incorporating qualitative and quantitative factors for software defect prediction , 2012, EAST '12.

[41]  Arun Sharma,et al.  Reliability of component based systems: a critical survey , 2011, SOEN.

[42]  W. W. Everett,et al.  Software component reliability analysis , 1999, Proceedings 1999 IEEE Symposium on Application-Specific Systems and Software Engineering and Technology. ASSET'99 (Cat. No.PR00122).

[43]  Sasikumar Punnekkat,et al.  How do we collect data for software reliability estimation? , 2010, CompSysTech '10.

[44]  Amrit L. Goel,et al.  Time-Dependent Error-Detection Rate Model for Software Reliability and Other Performance Measures , 1979, IEEE Transactions on Reliability.