Emulation of software faults by educated mutations at machine-code level

This paper proposes a new technique to emulate software faults by educated mutations introduced at the machine-code level and presents an experimental study on the accuracy of the injected faults. The proposed method consists of finding key programming structures at the machine code-level where high-level software faults can be emulated. The main advantage of emulating software faults at the machine-code level is that software faults can be injected even when the source code of the target application is not available, which is very important for the evaluation of COTS components or for the validation of software fault tolerance techniques in COTS based systems. The technique was evaluated using several real programs and different types of faults and, additionally, it includes our study on the key aspects that may impact on the technique accuracy. The portability of the technique is also addressed. The results show that classes of faults such as assignment, checking, interface, and simple algorithm faults can be directly emulated using this technique.

[1]  Ravishankar K. Iyer,et al.  Software Dependability in the Tandem GUARDIAN System , 1995, IEEE Trans. Software Eng..

[2]  Jeffrey M. Voas,et al.  Predicting How Badly "Good" Software Can Behave , 1997, IEEE Softw..

[3]  Henrique Madeira,et al.  Xception: Software Fault Injection and Monitoring in Processor Functional Units1 , 1995 .

[4]  Johan Karlsson,et al.  GOOFI: generic object-oriented fault injection tool , 2001, 2001 International Conference on Dependable Systems and Networks.

[5]  R.A. DeMillo,et al.  An extended overview of the Mothra software testing environment , 1988, [1988] Proceedings. Second Workshop on Software Testing, Verification, and Analysis.

[6]  Peter M. Chen,et al.  The systematic improvement of fault tolerance in the Rio file cache , 1999, Digest of Papers. Twenty-Ninth Annual International Symposium on Fault-Tolerant Computing (Cat. No.99CB36352).

[7]  Douglas M. Blough,et al.  Fault-injection-based testing of fault-tolerant algorithms in message-passing parallel computers , 1997, Proceedings of IEEE 27th International Symposium on Fault Tolerant Computing.

[8]  Philip Koopman,et al.  Dependability Benchmarking: making choices in an n-dimensional problem space , 2001 .

[9]  Taghi M. Khoshgoftaar,et al.  Process measures for predicting software quality , 1997, Proceedings 1997 High-Assurance Engineering Workshop.

[10]  David A. Patterson,et al.  Towards Availability Benchmarks: A Case Study of Software RAID Systems , 2000, USENIX Annual Technical Conference, General Track.

[11]  Jeffrey M. Voas,et al.  A 'Crystal Ball' for Software Liability , 1997, Computer.

[12]  Peter M. Chen,et al.  Comparing disk and memory's resistance to operating system crashes , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[13]  John D. Musa,et al.  Software Reliability Engineering , 1998 .

[14]  Michael R. Lyu,et al.  Handbook of software reliability engineering , 1996 .

[15]  Ravishankar K. Iyer,et al.  Experimental evaluation , 1995 .

[16]  Marco Vieira,et al.  On the emulation of software faults by software fault injection , 2000, Proceeding International Conference on Dependable Systems and Networks. DSN 2000.

[17]  Jim Gray,et al.  A census of Tandem system availability between 1985 and 1990 , 1990 .

[18]  Pascale Thévenod-Fosse,et al.  Software error analysis: a real case study involving real faults and mutations , 1996, ISSTA '96.

[19]  Ravishankar K. Iyer,et al.  An approach towards benchmarking of fault-tolerant commercial systems , 1996, Proceedings of Annual Symposium on Fault Tolerant Computing.

[20]  Mark Sullivan,et al.  Software defects and their impact on system availability-a study of field failures in operating systems , 1991, [1991] Digest of Papers. Fault-Tolerant Computing: The Twenty-First International Symposium.

[21]  Wei-Lun Kao,et al.  Experimental study of software dependability , 1995 .

[22]  Marco Vieira,et al.  ESFFI-a novel technique for the emulation of software faults in COTS components , 2001, Proceedings. Eighth Annual IEEE International Conference and Workshop On the Engineering of Computer-Based Systems-ECBS 2001.

[23]  Ram Chillarege,et al.  Generation of an error set that emulates software faults based on field data , 1996, Proceedings of Annual Symposium on Fault Tolerant Computing.

[24]  Jacob A. Abraham,et al.  FERRARI: a tool for the validation of system dependability properties , 1992, [1992] Digest of Papers. FTCS-22: The Twenty-Second International Symposium on Fault-Tolerant Computing.

[25]  James M. Bieman,et al.  Using fault injection to increase software test coverage , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[26]  Ram Chillarege,et al.  Orthogonal defect classification , 1996 .

[27]  Andrew Koenig,et al.  C Traps and Pitfalls , 1989 .

[28]  Jr. Edward William Krauser.  Compiler-integrated software testing , 1992 .

[29]  Jean-Claude Laprie,et al.  Qualitative and Quantitative Reliability Assessment , 1997, IEEE Softw..

[30]  Jean Arlat,et al.  Fault Injection and Dependability Evaluation of Fault-Tolerant Systems , 1993, IEEE Trans. Computers.

[31]  Wendell D. Jones,et al.  EMERALD: a case study in enhancing software reliability , 1997, Proceedings The Eighth International Symposium on Software Reliability Engineering - Case Studies -.

[32]  D. P. Siewiorek,et al.  Evaluation and comparison of fault-tolerant software techniques , 1993 .

[33]  Neeraj Suri,et al.  DBench (Dependability Benchmarking) , 2001 .