The reliability estimation, prediction and measuring of component-based software

Reliability is a key driver of safety-critical systems such as health-care systems and traffic controllers. It is also one of the most important quality attributes of the systems embedded into our surroundings, e.g. sensor networks that produce information for business processes. Therefore, the design decisions that have a great impact on the reliability of a software system, i.e. architecture and components, need to be thoroughly evaluated. This paper addresses software reliability evaluation during the design and implementation phases; it provides a coherent approach by combining both predicted and measured reliability values with heuristic estimates in order to facilitate a smooth reliability evaluation process. The approach contributes by integrating the component-level reliability evaluation activities (i.e. the heuristic reliability estimation, model-based reliability prediction and model-based reliability measuring of components) and the system-level reliability prediction activity to support the incremental and iterative development of reliable component-based software systems. The use of the developed reliability evaluation approach with the supporting tool chain is illustrated by a case study. The paper concludes with a summary of lessons learnt from the case studies.

[1]  Scott D. Stoller,et al.  Model-checking multi-threaded distributed Java programs , 2000, International Journal on Software Tools for Technology Transfer.

[2]  Vincenzo Grassi,et al.  A Modeling Approach to Analyze the Impact of Error Propagation on Reliability of Component-Based Systems , 2007, CBSE.

[3]  Eila Niemelä,et al.  Modeling Quality Attribute Variability , 2008, ENASE.

[4]  Marko Palviainen,et al.  Knowledge based quality-driven architecture design and evaluation , 2010, Inf. Softw. Technol..

[5]  Nick Mitchell,et al.  Visualizing the Execution of Java Programs , 2001, Software Visualization.

[6]  Dan Rubel The Heart of Eclipse , 2006, ACM Queue.

[7]  W. W. Everett,et al.  Software component reliability analysis , 1999, Proceedings 1999 IEEE Symposium on Application-Specific Systems and Software Engineering and Technology. ASSET'99 (Cat. No.PR00122).

[8]  Katerina Goseva-Popstojanova,et al.  Architecture-based approach to reliability assessment of software systems , 2001, Perform. Evaluation.

[9]  Klaus Havelund,et al.  Model Checking Programs , 2004, Automated Software Engineering.

[10]  Hausi A. Müller,et al.  Shimba—an environment for reverse engineering Java software systems , 2001, Softw. Pract. Exp..

[11]  Jan Bosch,et al.  Architecture-level modifiability analysis (ALMA) , 2004, J. Syst. Softw..

[12]  Viktor Schuppan,et al.  JNuke: Efficient Dynamic Analysis for Java , 2004, CAV.

[13]  Shunji Osaki,et al.  Software Reliability Growth Modeling: Models and Applications , 1985, IEEE Transactions on Software Engineering.

[14]  Jerry J. Harrow Runtime Checking of Multithreaded Applications with Visual Threads , 2000, SPIN.

[15]  Harry M. Sneed,et al.  Program comprehension for the purpose of testing , 2004, Proceedings. 12th IEEE International Workshop on Program Comprehension, 2004..

[16]  Martin Burtscher,et al.  The VPC trace-compression algorithms , 2005, IEEE Transactions on Computers.

[17]  Philippe Kruchten,et al.  The 4+1 View Model of Architecture , 1995, IEEE Softw..

[18]  Marko Palviainen,et al.  A Dynamic Behaviour and Reliability Evaluation Method for Applications That Are Based on Asynchronous Processing Nodes , 2010, 2010 Third International Conference on Software Testing, Verification, and Validation Workshops.

[19]  Katerina Goseva-Popstojanova,et al.  Software reliability estimation under certainty: generalization of the method of moments , 2004, Eighth IEEE International Symposium on High Assurance Systems Engineering, 2004. Proceedings..

[20]  Janice Singer,et al.  Proceedings of the 2004 conference of the Centre for Advanced Studies on Collaborative research, October 5-7, 2004, Markham, Ontario, Canada , 2004, CASCON.

[21]  Eila Niemelä,et al.  Survey of reliability and availability prediction methods from the viewpoint of software architecture , 2007, Software & Systems Modeling.

[22]  Paul C. Jorgensen,et al.  Object-oriented integration testing , 1994, CACM.

[23]  Daniel Hoffman,et al.  Tool Support for Testing Concurrent Java Components , 2003, IEEE Trans. Software Eng..

[24]  Paul Clements,et al.  Software architecture in practice , 1999, SEI series in software engineering.

[25]  Eila Niemelä,et al.  OntoArch Approach for Reliability-Aware Software Architecture Development , 2008, 2008 32nd Annual IEEE International Computer Software and Applications Conference.

[26]  David S. Rosenblum,et al.  Reliability prediction in model-driven development , 2005, MoDELS'05.

[27]  Joost-Pieter Katoen Quantitative Evaluation in Embedded System Design: Trends in Modeling and Analysis Techniques , 2008, 2008 Design, Automation and Test in Europe.

[28]  Claes Wohlin,et al.  An additive reliability model for the analysis of modular software failure data , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[29]  Brendan Murphy,et al.  Measuring Reliability of Software Products , 2004 .

[30]  Siddhartha R. Dalal,et al.  Model-based testing in practice , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[31]  Ralf H. Reussner,et al.  Reliability prediction for component-based software architectures , 2003, J. Syst. Softw..

[32]  Kai Koskimies,et al.  Automated Support for Modeling OO Software , 1998, IEEE Softw..

[33]  Ipek Ozkaya,et al.  Analysis of architecture evaluation data , 2008, J. Syst. Softw..

[34]  Abdelwahab Hamou-Lhadj,et al.  A survey of trace exploration tools and techniques , 2004, CASCON.

[35]  Jan Bosch Architecture in the Age of Compositionality , 2010, ECSA.

[36]  Leonard J. Bass,et al.  A Basis for Analyzing Software Architecture Analysis Methods , 2005, Software Quality Journal.

[37]  Bojan Cukic,et al.  Early reliability assessment of UML based software models , 2002, WOSP '02.

[38]  Eila Niemelä,et al.  A Survey on Software Architecture Analysis Methods , 2002, IEEE Trans. Software Eng..

[39]  Anne Immonen,et al.  A tool for reliability and availability prediction , 2005 .

[40]  Edwin V. Bonilla,et al.  Predicting best design trade-offs: A case study in processor customization , 2012, 2012 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[41]  Grigore Rosu,et al.  An Overview of the Runtime Verification Tool Java PathExplorer , 2004, Formal Methods Syst. Des..

[42]  A. Immonen,et al.  Trustworthiness Evaluation and Testing of Open Source Components , 2007 .

[43]  Mari Matinlassi,et al.  Quality-Driven Software Architecture Model Transformation , 2005, 5th Working IEEE/IFIP Conference on Software Architecture (WICSA'05).