Mutation at System and Functional Levels

Mutation analysis has been applied to many testing problems, including functional programs in numerous languages, specifications, network protocols, web services, and security policies. Program mutation, where mutation analysis is applied to programs, has been applied to the unit level (functions and methods), integration of pairs of functions, and individual classes. However, program mutation has not been applied to the problem of testing multiple classes or entire software programs, that is, there is no system level mutation. This paper introduces a project on the problem of multi-class and system level mutation testing. The technical differences between using mutation to test single classes and multiple classes are explored, and new system level mutation operators are defined. A new execution style for detecting killed mutants, Flexible Weak Mutation, is introduced. A support tool, Bacterio, is currently under construction.

[1]  Aditya P. Mathur,et al.  Interface Mutation: An Approach for Integration Testing , 2001, IEEE Trans. Software Eng..

[2]  Brian Marick,et al.  The weak mutation hypothesis , 1991, TAV4.

[3]  A. Jefferson Offutt,et al.  Generating test cases for web services using data perturbation , 2004, SOEN.

[4]  A. Jefferson Offutt,et al.  Constraint-Based Automatic Test Data Generation , 1991, IEEE Trans. Software Eng..

[5]  Andreas Zeller,et al.  Javalanche: efficient mutation testing for Java , 2009, ESEC/SIGSOFT FSE.

[6]  A. Jefferson Offutt,et al.  MuJava: an automated class mutation system , 2005, Softw. Test. Verification Reliab..

[7]  OffuttJeff,et al.  Generating test cases for web services using data perturbation , 2004 .

[8]  A. Jefferson Offutt,et al.  Introduction to Software Testing , 2008 .

[9]  Mark Harman,et al.  Using program slicing to assist in the detection of equivalent mutants , 1999, Softw. Test. Verification Reliab..

[10]  A. Jefferson Offutt,et al.  An Empirical Evaluation of Weak Mutation , 1994, IEEE Trans. Software Eng..

[11]  Yu-Seung Ma.  Description of Method-level Mutation Operators for Java , 2011 .

[12]  A. J. Offutt.  A practical system for mutation testing: help for the common programmer , 1994, Proceedings., International Test Conference.

[13]  A. Jefferson Offutt,et al.  Detecting equivalent mutants and the feasible path problem , 1996, Proceedings of 11th Annual Conference on Computer Assurance. COMPASS '96.

[14]  Mario Piattini,et al.  Decreasing the cost of mutation testing with second-order mutants , 2009 .

[15]  Wynne Hsu,et al.  DESIGN OF MUTANT OPERATORS FOR THE C PROGRAMMING LANGUAGE , 2006 .

[16]  William E. Howden,et al.  Weak Mutation Testing and Completeness of Test Sets , 1982, IEEE Transactions on Software Engineering.

[17]  A. Jefferson Offutt,et al.  Mutation analysis using mutant schemata , 1993, ISSTA '93.

[18]  Richard A. DeMillo,et al.  Compiler-integrated program mutation , 1991, [1991] Proceedings The Fifteenth Annual International Computer Software & Applications Conference.

[19]  S. Inglis,et al.  Jumble Java Byte Code to Measure the Effectiveness of Unit Tests , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[20]  M. R. Woodward,et al.  From weak to strong, dead or alive? an analysis of some mutation testing issues , 1988, [1988] Proceedings. Second Workshop on Software Testing, Verification, and Analysis.

[21]  A. Jefferson Offutt,et al.  Automatically detecting equivalent mutants and infeasible paths , 1997 .

[22]  Leonardo Bottaci,et al.  Efficiency of mutation operators and selective mutation strategies: an empirical study , 1999, Softw. Test. Verification Reliab..

[23]  Mario Piattini,et al.  Integrating techniques and tools for testing automation , 2007, Softw. Test. Verification Reliab..

[24]  Javier Tuya,et al.  Mutating database queries , 2007, Inf. Softw. Technol..

[25]  Auri Marcelo Rizzo Vincenzi,et al.  Toward the determination of sufficient mutant operators for C † , 2001, Softw. Test. Verification Reliab..

[26]  José Carlos Maldonado,et al.  Proteum/IM 2.0: An Integrated Mutation Testing Environment , 2001 .

[27]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[28]  K. N. King,et al.  A fortran language system for mutation‐based software testing , 1991, Softw. Pract. Exp..

[29]  Martin R. Woodward,et al.  An integrated system for program testing using weak mutation and data flow analysis , 1985, ICSE '85.

[30]  Andreas Zeller,et al.  The Impact of Equivalent Mutants , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[31]  Anna Derezinska.  Advanced mutation operators applicable in C# programs , 2006, SET.

[32]  J. Offutt,et al.  Mutation testing implements grammar-based testing , 2006, Second Workshop on Mutation Analysis (Mutation 2006 - ISSRE Workshops 2006).

[33]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.