Using fault injection to increase software test coverage

During testing, it is nearly impossible to run all statements or branches of a program. It is especially difficult to test the code used to respond to exceptional conditions. This untested code, often the error recovery code, will tend to be an error prone part of a system. We show that test coverage can be increased through an "assertion violation" technique for injecting software faults during execution. Using our prototype tool, Visual C-Patrol (VCP), we were able to substantially increase test branch coverage in four software systems studied.

[1]  A. Jefferson Offutt,et al.  An Empirical Evaluation of Weak Mutation , 1994, IEEE Trans. Software Eng..

[2]  Ram Chillarege,et al.  Understanding large system failures-a fault injection experiment , 1989, [1989] The Nineteenth International Symposium on Fault-Tolerant Computing. Digest of Papers.

[3]  William E. Howden,et al.  A functional approach to program testing and analysis , 1986, IEEE Transactions on Software Engineering.

[4]  William E. Howden,et al.  Weak Mutation Testing and Completeness of Test Sets , 1982, IEEE Transactions on Software Engineering.

[5]  John F. Passafiume,et al.  Software testing and evaluation , 1987 .

[6]  Cliff B. Jones,et al.  Systematic software development using VDM , 1986, Prentice Hall International Series in Computer Science.

[7]  Daniel P. Siewiorek,et al.  Fault Injection Experiments Using FIAT , 1990, IEEE Trans. Computers.

[8]  James M. Bieman,et al.  Improving software testability with assertion insertion , 1994, Proceedings., International Test Conference.

[9]  R.A. DeMillo,et al.  An extended overview of the Mothra software testing environment , 1988, [1988] Proceedings. Second Workshop on Software Testing, Verification, and Analysis.

[10]  Richard J. Lipton,et al.  Social processes and proofs of theorems and programs , 1979, CACM.

[11]  J. Gerardin.  The :20DEF.injector” test instrument, assistance in the design of reliable and safe systems , 1989 .

[12]  Stephen J. Garland,et al.  Debugging Larch Shared Language Specifications , 1990, IEEE Trans. Software Eng..

[13]  Ravishankar K. Iyer,et al.  FINE: A Fault Injection and Monitoring Environment for Tracing the UNIX System Behavior under Faults , 1993, IEEE Trans. Software Eng..

[14]  Anthony Hall,et al.  Seven myths of formal methods , 1990, IEEE Software.