Research on Software Fault Injection Testing

The software fault injection testing (SFIT) technique has been developed for thirty years. It is one of the most active parts in software testing research. As a non-traditional testing technique, it plays a very important role in enhancing software quality, eliminating software failures and improving the process of software development. A detailed review of the research on SFIT is presented based on the survey and classification of the current SFIT techniques. Then, some important testing frameworks and tools that are effective at present are also discussed. Meanwhile, a brief description of the testing system CSTS (Component Security Testing System) is provided as well. Based on the precise investigation on SFIT, the issues and challenges of SFIT are pointed out and

[1]  Wenliang Du,et al.  Testing for software vulnerability using environment perturbation , 2002 .

[2]  Keith W. Miller,et al.  PISCES: a tool for predicting software testability , 1992, [1992] Proceedings of the Second Symposium on Assessment of Quality Software Development Tools.

[3]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[4]  Henrique Madeira,et al.  Xception: A Technique for the Experimental Evaluation of Dependability in Modern Computers , 1998, IEEE Trans. Software Eng..

[5]  Jeffrey M. Voas.  Fault Injection for the Masses , 1997, Computer.

[6]  Ravishankar K. Iyer,et al.  DEPEND: A Simulation-Based Environment for System Level Dependability Analysis , 1997, IEEE Trans. Computers.

[7]  Jean Arlat,et al.  Fault Injection for Dependability Validation: A Methodology and Some Applications , 1990, IEEE Trans. Software Eng..

[8]  Jie Xu,et al.  A comparison of network level fault injection with code insertion , 2005, 29th Annual International Computer Software and Applications Conference (COMPSAC'05).

[9]  Anup K. Ghosh,et al.  Techniques for evaluating the robustness of Windows NT software , 2000, Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00.

[10]  J.H. Andrews,et al.  Is mutation an appropriate tool for testing experiments? [software testing] , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[11]  Jinhui Shan,et al.  An Approach to Test Data Generation for Killing Multiple Mutants , 2006, 2006 22nd IEEE International Conference on Software Maintenance.

[12]  Sudipto Ghosh,et al.  Bytecode fault injection for Java software , 2008, J. Syst. Softw..

[13]  Ravishankar K. Iyer,et al.  An approach towards benchmarking of fault-tolerant commercial systems , 1996, Proceedings of Annual Symposium on Fault Tolerant Computing.

[14]  James M. Bieman,et al.  Using fault injection to increase software test coverage , 1996, Proceedings of ISSRE '96: 7th International Symposium on Software Reliability Engineering.

[15]  Li Ai.  An Approach for Identifying Software Vulnerabilities Based on Error Propagation Analysis , 2007 .

[16]  A. Jefferson Offutt,et al.  An Empirical Evaluation of Weak Mutation , 1994, IEEE Trans. Software Eng..

[17]  Henrique Madeira,et al.  Emulation of Software Faults: A Field Data Study and a Practical Approach , 2006, IEEE Transactions on Software Engineering.

[18]  Anup K. Ghosh,et al.  An approach to testing COTS software for robustness to operating system exceptions and errors , 1999, Proceedings 10th International Symposium on Software Reliability Engineering (Cat. No.PR00443).

[19]  Jeffrey M. Voas,et al.  Predicting How Badly "Good" Software Can Behave , 1997, IEEE Softw..

[20]  Johan Karlsson,et al.  GOOFI: generic object-oriented fault injection tool , 2001, 2001 International Conference on Dependable Systems and Networks.

[21]  Nuno Silva,et al.  Verification and Validation of (Real Time) COTS Products using Fault Injection Techniques , 2007, 2007 Sixth International IEEE Conference on Commercial-off-the-Shelf (COTS)-Based Software Systems (ICCBSS'07).

[22]  Jinfu Chen,et al.  Testing approach of component security based on dynamic monitoring , 2007 .

[23]  Shih-Kun Huang,et al.  Web application security assessment by fault injection and behavior monitoring , 2003, WWW '03.

[24]  Barton P. Miller,et al.  An empirical study of the reliability of UNIX utilities , 1990, Commun. ACM.

[25]  Frank Hill,et al.  Wrapping windows NT software for robustness , 1999, Digest of Papers. Twenty-Ninth Annual International Symposium on Fault-Tolerant Computing (Cat. No.99CB36352).

[26]  Keith W. Miller,et al.  Defining an adaptive software security metric from a dynamic software failure tolerance measure , 1996, Proceedings of 11th Annual Conference on Computer Assurance. COMPASS '96.

[27]  Jean Arlat,et al.  Fault injection for dependability validation of fault-tolerant computing systems , 1989, [1989] The Nineteenth International Symposium on Fault-Tolerant Computing. Digest of Papers.

[28]  Gregg Rothermel,et al.  An experimental determination of sufficient mutant operators , 1996, TSEM.

[29]  Aditya P. Mathur,et al.  Interface Mutation: An Approach for Integration Testing , 2001, IEEE Trans. Software Eng..

[30]  R.A. DeMillo,et al.  An extended overview of the Mothra software testing environment , 1988, [1988] Proceedings. Second Workshop on Software Testing, Verification, and Analysis.

[31]  Daniel P. Siewiorek,et al.  Comparing operating systems using robustness benchmarks , 1997, Proceedings of SRDS'97: 16th IEEE Symposium on Reliable Distributed Systems.

[32]  Johan Karlsson,et al.  Evaluation of error detection schemes using fault injection by heavy-ion radiation , 1989, [1989] The Nineteenth International Symposium on Fault-Tolerant Computing. Digest of Papers.

[33]  Jean Arlat,et al.  Fault injection for formal testing of fault tolerance , 1996, IEEE Trans. Reliab..

[34]  Ying Jiang,et al.  Contract-based mutation for testing components , 2005, 21st IEEE International Conference on Software Maintenance (ICSM'05).

[35]  A. Jefferson Offutt,et al.  The dynamic domain reduction procedure for test data generation , 1999, Softw. Pract. Exp..

[36]  William E. Howden,et al.  Weak Mutation Testing and Completeness of Test Sets , 1982, IEEE Transactions on Software Engineering.

[37]  Anup K. Ghosh,et al.  Testing the robustness of Windows NT software , 1998, Proceedings Ninth International Symposium on Software Reliability Engineering (Cat. No.98TB100257).

[38]  Jinfu Chen,et al.  Testing Approach of Component Security Based on Fault Injection , 2007 .

[39]  Gary McGraw,et al.  Software Penetration Testing , 2005, IEEE Secur. Priv..

[40]  Jeffrey M. Voas,et al.  PIE: A Dynamic Failure-Based Technique , 1992, IEEE Trans. Software Eng..

[41]  Gary McGraw,et al.  Software fault injection: inoculating programs against errors , 1997 .

[42]  A. Jefferson Offutt,et al.  Constraint-Based Automatic Test Data Generation , 1991, IEEE Trans. Software Eng..

[43]  A. Jefferson Offutt,et al.  Detecting equivalent mutants and the feasible path problem , 1996, Proceedings of 11th Annual Conference on Computer Assurance. COMPASS '96.