Evaluating the Use of Reference Run Models in Fault Injection Analysis

Fault injection (FI) has been shown to be an effective approach to assessing the dependability of software systems. To determine the impact of faults injected during FI, a given oracle is needed. Oracles can take a variety of forms, including (i) specifications, (ii) error detection mechanisms and (iii) golden runs. Focusing on golden runs, in this paper we show that there are classes of software which a golden run based approach can not be used to analyse. Specifically, we demonstrate that a golden run based approach can not be used in the analysis of systems which employ a main control loop with an irregular period. Further, we show how a simple model, which has been refined using FI experiments, can be employed as an oracle in the analysis of such a system.

[1]  Barry W. Johnson.  Design & analysis of fault tolerant digital systems , 1988 .

[2]  William H. Sanders,et al.  Loki: a state-driven fault injector for distributed systems , 2000, Proceeding International Conference on Dependable Systems and Networks. DSN 2000.

[3]  Johan Karlsson,et al.  Experimental dependability evaluation of a fail-bounded jet engine control system for unmanned aerial vehicles , 2005, 2005 International Conference on Dependable Systems and Networks (DSN'05).

[4]  Ravishankar K. Iyer,et al.  DEPEND: A Simulation-Based Environment for System Level Dependability Analysis , 1997, IEEE Trans. Computers.

[5]  Ravishankar K. Iyer,et al.  NFTAPE: a framework for assessing dependability in distributed systems with lightweight fault injectors , 2000, Proceedings IEEE International Computer Performance and Dependability Symposium. IPDS 2000.

[6]  Neeraj Suri,et al.  On the placement of software mechanisms for detection of data errors , 2002, Proceedings International Conference on Dependable Systems and Networks.

[7]  Alfredo Benso,et al.  A C/C++ source-to-source compiler for dependable applications , 2000, Proceeding International Conference on Dependable Systems and Networks. DSN 2000.

[8]  Jacob A. Abraham,et al.  FERRARI: A Flexible Software-Based Fault and Error Injection System , 1995, IEEE Trans. Computers.

[9]  Volkmar Sieh,et al.  VERIFY: evaluation of reliability using VHDL-models with embedded fault descriptions , 1997, Proceedings of IEEE 27th International Symposium on Fault Tolerant Computing.

[10]  Jeffrey Voas,et al.  Software Fault Injection , 1998 .

[11]  Neeraj Suri,et al.  PROPANE: an environment for examining the propagation of errors in software , 2002, ISSTA '02.

[12]  Johan Karlsson,et al.  Reducing critical failures for control algorithms using executable assertions and best effort recovery , 2001, 2001 International Conference on Dependable Systems and Networks.

[13]  Johan Karlsson,et al.  On the design of robust integrators for fail-bounded control systems , 2003, 2003 International Conference on Dependable Systems and Networks, 2003. Proceedings..

[14]  Ravishankar K. Iyer,et al.  Measuring Fault Tolerance with the FTAPE Fault Injection Tool , 1995, MMB.

[15]  Adam A. Porter,et al.  Specification-based Testing of Reactive Software: Tools and Experiments Experience Report , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[16]  Martin Hiller,et al.  A Software Profiling Methodology for Design and Assessment of Dependable Software , 2002 .

[17]  Daniel P. Siewiorek,et al.  Fault Injection Experiments Using FIAT , 1990, IEEE Trans. Computers.

[18]  David A. Carrington,et al.  A Framework for Specification-Based Testing , 1996, IEEE Trans. Software Eng..

[19]  Neeraj Suri,et al.  An approach for analysing the propagation of data errors in software , 2001, 2001 International Conference on Dependable Systems and Networks.

[20]  Nancy G. Leveson,et al.  The Use of Self Checks and Voting in Software Error Detection: An Empirical Study , 1990, IEEE Trans. Software Eng..

[21]  Jean Arlat,et al.  Building dependable COTS microkernel-based systems using MAFALDA , 2000, Proceedings. 2000 Pacific Rim International Symposium on Dependable Computing.

[22]  Debra J. Richardson,et al.  Specification-based test oracles for reactive systems , 1992, International Conference on Software Engineering.

[23]  John A. Clark,et al.  Software fault injection , 1998 .