Reproducibility of Environment-Dependent Software Failures: An Experience Report

We investigate the dependence of software failure reproducibility on the environment in which the software is executed. The existence of such dependence is ascertained in literature, but so far it is not fully characterized. In this paper we pinpoint some of the environmental components that can affect the reproducibility of a failure and show this influence through an experimental campaign conducted on the My SQL Server software system. The set of failures of interest is drawn from My SQL's failure reports database and an experiment is designed for each of these failures. The experiments expose the influence of disk usage and level of concurrency on My SQL failure reproducibility. Furthermore, the results show that high levels of usage of these factors increase the probabilities of failure reproducibility.

[1]  Jim Gray,et al.  A census of Tandem system availability between 1985 and 1990 , 1990 .

[2]  Kishor S. Trivedi,et al.  Fighting bugs: remove, retry, replicate, and rejuvenate , 2007, Computer.

[3]  Ram Chillarege,et al.  Test and development process retrospective - a case study using ODC triggers , 2002, Proceedings International Conference on Dependable Systems and Networks.

[4]  Kishor S. Trivedi,et al.  Software Faults, Software Aging and Software Rejuvenation( New Development of Software Reliability Engineering) , 2005 .

[5]  Kishor S. Trivedi,et al.  Fault triggers in open-source software: An experience report , 2013, 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE).

[6]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[7]  Laurie A. Williams,et al.  Does Hardware Configuration and Processor Load Impact Software Fault Observability? , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[8]  Dong Seong Kim,et al.  Recovery from Failures Due to Mandelbugs in IT Systems , 2011, 2011 IEEE 17th Pacific Rim International Symposium on Dependable Computing.

[9]  Alpana Dubey.  Towards adopting ODC in automation application development projects , 2012, ISEC.

[10]  Stefan Wagner,et al.  Defect classification and defect types revisited , 2008, DEFECTS '08.

[11]  Inderpal S. Bhandari,et al.  Orthogonal Defect Classification - A Concept for In-Process Measurements , 1992, IEEE Trans. Software Eng..

[12]  Robert B. Grady,et al.  Software Failure Analysis for High-Return Process Improvement Decisions , 1996 .

[13]  Peter M. Chen,et al.  Whither generic recovery from application faults? A fault study using open-source software , 2000, Proceeding International Conference on Dependable Systems and Networks. DSN 2000.

[14]  Jim Gray,et al.  Why Do Computers Stop and What Can Be Done About It? , 1986, Symposium on Reliability in Distributed Software and Database Systems.

[15]  Cheng Li,et al.  A study of the internal and external effects of concurrency bugs , 2010, 2010 IEEE/IFIP International Conference on Dependable Systems & Networks (DSN).

[16]  Kishor S. Trivedi,et al.  An empirical investigation of fault repairs and mitigations in space mission system software , 2013, 2013 43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[17]  Kishor S. Trivedi,et al.  A Classification of Software Faults , 2011 .

[18]  Domenico Cotroneo,et al.  On the Aging Effects Due to Concurrency Bugs: A Case Study on MySQL , 2012, 2012 IEEE 23rd International Symposium on Software Reliability Engineering.

[19]  Mark Butcher,et al.  Improving software testing via ODC: Three case studies , 2002, IBM Syst. J..

[20]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[21]  Kishor S. Trivedi,et al.  An empirical investigation of fault types in space mission system software , 2010, 2010 IEEE/IFIP International Conference on Dependable Systems & Networks (DSN).

[22]  Karen A. F. Copeland.  Design and Analysis of Experiments, 5th Ed. , 2001 .

[23]  Ravishankar K. Iyer,et al.  Software Dependability in the Tandem GUARDIAN System , 1995, IEEE Trans. Software Eng..