An Adaptive Reliability Analysis Using Path Testing for Complex Component-Based Software Systems

With the growing size and complexity of software applications, traditional software reliability methods are insufficient to analyze inter-component interactions of modular software systems. The number of test cases may be extremely large for this application; therefore, it is hard for us to extensively test each software component given resource limitations. In this paper, we propose an adaptive framework of incorporating path testing into reliability estimation for modular software systems. Three estimated methods based on common program structures, namely, sequence, branch, and loop structures, are proposed to calculate the path reliability. Consequently, the derived path reliabilities can be applied to the estimates of software reliability. Some experiments are performed based on two real systems. In addition, the accuracy and correlation with respect to the experiments are investigated by simulation and sensitivity analysis. Experimental results show that the path reliability has a high correlation to the actual software reliability. For software with loop structures, a smaller loop number can be assigned to derive an acceptable estimation of path reliability. Further, the sensitivity analysis can be used to identify critical modules and paths for resource allocation. It can be concluded that the proposed methods are useful and helpful for estimating software reliability and can be adaptively used in the early stages of software development.

[1]  Bev Littlewood,et al.  A Bayesian Reliability Growth Model for Computer Software , 1973 .

[2]  Martin L. Shooman,et al.  Structural models for software reliability prediction , 1976, ICSE '76.

[3]  Eldred Nelson,et al.  Estimating software reliability from test data , 1978 .

[4]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[5]  B. Littlewood Software Reliability Model for Modular Program Structure , 1979, IEEE Transactions on Reliability.

[6]  John D. Musa,et al.  Software reliability measurement , 1984, J. Syst. Softw..

[7]  Roger C. Cheung,et al.  A User-Oriented Software Reliability Model , 1978, IEEE Transactions on Software Engineering.

[8]  Roger S. Pressman,et al.  Software Engineering: A Practitioner's Approach , 1982 .

[9]  Amrit L. Goel,et al.  Software Reliability Models: Assumptions, Limitations, and Applicability , 1985, IEEE Transactions on Software Engineering.

[10]  John D. Musa,et al.  Software reliability - measurement, prediction, application , 1987, McGraw-Hill series in software engineering and technology.

[11]  Thomas A. Mazzuchi,et al.  A Bayes empirical-Bayes model for software reliability , 1988 .

[12]  Peter Kubat,et al.  Assessing reliability of modular software , 1989 .

[13]  K. Misra,et al.  An efficient algorithm to solve integer-programming problems arising in system-reliability design , 1991 .

[14]  Min Xie,et al.  Software Reliability Modelling , 1991, Series on Quality, Reliability and Engineering Statistics.

[15]  Norman F. Schneidewind,et al.  Applying reliability models to the space shuttle , 1992, IEEE Software.

[16]  John D. Musa,et al.  Operational profiles in software-reliability engineering , 1993, IEEE Software.

[17]  Simon P. Wilson,et al.  Software Reliability Modeling , 1994 .

[18]  Michael R. Lyu,et al.  Handbook of software reliability engineering , 1996 .

[19]  Alberto Pasquini,et al.  Sensitivity of reliability-growth models to operational profile errors vs. testing accuracy [software testing] , 1996, IEEE Trans. Reliab..

[20]  Aditya P. Mathur,et al.  On the estimation of reliability of a software system using reliabilities of its components , 1997, Proceedings The Eighth International Symposium on Software Reliability Engineering.

[21]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[22]  Min Xie,et al.  A study of the sensitivity of software release time , 1998, J. Syst. Softw..

[23]  Hoang Pham Software Reliability , 1999 .

[24]  Ye Wu,et al.  An architecture-based software reliability model , 1999, Proceedings 1999 Pacific Rim International Symposium on Dependable Computing.

[25]  Katerina Goseva-Popstojanova,et al.  Architecture-based approach to reliability assessment of software systems , 2001, Perform. Evaluation.

[26]  Paul Jorgensen,et al.  Software Testing: A Craftman's Approach , 2001 .

[27]  Michael R. Lyu,et al.  Effect of code coverage on software reliability measurement , 2001, IEEE Trans. Reliab..

[28]  Michael R. Lyu,et al.  Optimal resource allocation and reliability analysis for component-based software applications , 2002, Proceedings 26th Annual International Computer Software and Applications.

[29]  James M. Bieman,et al.  Software reliability growth with test coverage , 2002, IEEE Trans. Reliab..

[30]  Swapna S. Gokhale,et al.  Reliability prediction and sensitivity analysis based on software architecture , 2002, 13th International Symposium on Software Reliability Engineering, 2002. Proceedings..

[31]  Michael R. Lyu,et al.  A Unified Scheme of Some Nonhomogenous Poisson Process Models for Software Reliability Estimation , 2003, IEEE Trans. Software Eng..

[32]  Michael R. Lyu,et al.  Sensitivity analysis of software reliability for component-based software applications , 2003, Proceedings 27th Annual International Computer Software and Applications Conference. COMPAC 2003.

[33]  Katerina Goseva-Popstojanova,et al.  Assessing uncertainty in reliability of component-based software systems , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[34]  Hany H. Ammar,et al.  A scenario-based reliability analysis approach for component-based software , 2004, IEEE Transactions on Reliability.

[35]  Swapna S. Gokhale,et al.  An analytical approach to architecture-based software performance and reliability prediction , 2004, Perform. Evaluation.

[36]  Glenford J. Myers,et al.  The art of software testing (2. ed.) , 2004 .

[37]  Zhao Li,et al.  Evaluating Web software reliability based on workload and failure data extracted from server logs , 2004, IEEE Transactions on Software Engineering.

[38]  Michael R. Lyu,et al.  Reliability assessment and sensitivity analysis of software reliability growth modeling based on software module structure , 2005, J. Syst. Softw..

[39]  Bojan Cukic,et al.  Error propagation in the reliability analysis of component based systems , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[40]  Michael R. Lyu,et al.  Optimal testing resource allocation, and sensitivity analysis in software development , 2005, IEEE Transactions on Reliability.

[41]  Swapna S. Gokhale,et al.  A simulation approach to structure-based software reliability analysis , 2005, IEEE Transactions on Software Engineering.

[42]  V. E. Johnson,et al.  A hierarchical model for estimating the early reliability of complex systems , 2005, IEEE Transactions on Reliability.

[43]  C.-T. Lin,et al.  Software Reliability Analysis by Considering Fault Dependency and Debugging Time Lag , 2006, IEEE Transactions on Reliability.

[44]  Dai Pan,et al.  Architecture-based software reliability modeling , 2006, J. Syst. Softw..

[45]  Katerina Goseva-Popstojanova,et al.  Architecture-Based Software Reliability: Why Only a Few Parameters Matter? , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[46]  Swapna S. Gokhale,et al.  Architecture-Based Software Reliability Analysis: Overview and Limitations , 2007, IEEE Transactions on Dependable and Secure Computing.

[47]  Michiel van Genuchten,et al.  Using Software Reliability Growth Models in Practice , 2007, IEEE Software.

[48]  Norman F. Schneidewind,et al.  Integrating testing with reliability , 2009, Softw. Test. Verification Reliab..

[49]  Francesca Saglietti,et al.  Integration and Reliability Testing for Component-Based Software Systems , 2009, 2009 35th Euromicro Conference on Software Engineering and Advanced Applications.

[50]  Zibin Zheng,et al.  An adaptive QoS-aware fault tolerance strategy for web services , 2010, Empirical Software Engineering.

[51]  Roger Pressman,et al.  Software Engineering: A Practitioner's Approach, 7Th Edition , 2009 .

[52]  Kishor S. Trivedi,et al.  Software Reliability and Testing Time Allocation: An Architecture-Based Approach , 2010, IEEE Transactions on Software Engineering.

[53]  Chin-Yu Huang,et al.  Analysis of Software Reliability Modeling Considering Testing Compression Factor and Failure-to-Fault Relationship , 2010, IEEE Transactions on Computers.

[54]  Chin-Yu Huang,et al.  Enhancing software reliability modeling and prediction through the introduction of time-variable fault reduction factor , 2011 .