A Lightweight Program Dependence Based Approach to Concurrent Mutation Analysis

Mutation analysis is a classical software testing approach which attempts to imitate faults using a set of mutants. It has been advocated to be an appropriate technique for evaluating the quality of test suites as well as the effectiveness of a testing method. However, the applicability of mutation analysis, especially in many practical situations, has been hindered due to the high computation cost and the long execution time, which are mainly caused by the large number of mutants. Numerous studies, particularly those based on parallel computing, have been conducted to reduce the overhead of mutation analysis. In this paper, we aim to improve the efficiency of mutation analysis from a different perspective. We make use of lightweight program analysis techniques to identify a group of mutants that share the common execution traces before the mutation location, and then merge them into a synthesized program with the concurrent mechanism, on which mutation analysis can be efficiently executed without the duplicate execution of common traces. Our empirical study demonstrates that our approach can significantly decrease the computation overhead as well as shorten the execution time of mutation analysis, without jeopardizingitseffectiveness. Thein-depthanalysisfurthershows thattheeffectivenessofourapproachispositivelycorrelatedwith thenumberofbranchesintheprogramundertest. Ourapproach makes it possible to efficiently execute mutation analysis even without the need of advanced computer architectures.

[1]  M. R. Woodward,et al.  From weak to strong, dead or alive? an analysis of some mutation testing issues , 1988, [1988] Proceedings. Second Workshop on Software Testing, Verification, and Analysis.

[2]  David A. Freedman,et al.  Statistical Models: Theory and Practice: References , 2005 .

[3]  Aditya P. Mathur,et al.  Modeling mutation and a vector processor , 1988, Proceedings. [1989] 11th International Conference on Software Engineering.

[4]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[5]  Vernon Rego,et al.  Concurrency Enhancement through Program Unification: A Performance Analysis , 1990, J. Parallel Distributed Comput..

[6]  K. N. King,et al.  A fortran language system for mutation‐based software testing , 1991, Softw. Pract. Exp..

[7]  Lionel C. Briand,et al.  Is mutation an appropriate tool for testing experiments? , 2005, ICSE.

[8]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[9]  Alex Groce,et al.  Topsy-Turvy: A Smarter and Faster Parallelization of Mutation Analysis , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[10]  Roy P. Pargas,et al.  Mutation Testing of Software Using MIMD Computer , 1992, ICPP.

[11]  Khaled Nagi,et al.  HadoopMutator: A Cloud-Based Mutation Testing Framework , 2015, ICSR.

[12]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[13]  A. Jefferson Offutt,et al.  An Empirical Evaluation of Weak Mutation , 1994, IEEE Trans. Software Eng..

[14]  René Just,et al.  Higher accuracy and lower run time: efficient mutation analysis using non‐redundant mutation operators , 2015, Softw. Test. Verification Reliab..

[15]  Bo Wang,et al.  Faster mutation analysis via equivalence modulo states , 2017, ISSTA.

[16]  Huai Liu,et al.  A path-aware approach to mutant reduction in mutation testing , 2017, Inf. Softw. Technol..

[17]  Andreas Zeller,et al.  Mutation-Driven Generation of Unit Tests and Oracles , 2012, IEEE Trans. Software Eng..

[18]  Joe D. Warren,et al.  The program dependence graph and its use in optimization , 1984, TOPL.

[19]  Sarfraz Khurshid,et al.  Regression mutation testing , 2012, ISSTA 2012.

[20]  Macario Polo,et al.  Parallel mutation testing , 2013, Softw. Test. Verification Reliab..

[21]  Martin R. Woodward,et al.  An integrated system for program testing using weak mutation and data flow analysis , 1985, ICSE '85.

[22]  Vernon Rego,et al.  High Performance Software Testing on SIMD Machines , 1991, IEEE Trans. Software Eng..

[23]  Michael D. Ernst,et al.  Are mutants a valid substitute for real faults in software testing? , 2014, SIGSOFT FSE.

[24]  Macario Polo,et al.  Mutation at System and Functional Levels , 2010, 2010 Third International Conference on Software Testing, Verification, and Validation Workshops.

[25]  Hiroaki Yoshida,et al.  MuVM: Higher Order Mutation Analysis Virtual Machine for C , 2016, 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST).

[26]  Gregg Rothermel,et al.  An experimental determination of sufficient mutant operators , 1996, TSEM.

[27]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[28]  Márcio Eduardo Delamaro,et al.  Proteum - A Tool for the Assessment of Test Adequacy for C Programs User's guide , 1996 .

[29]  William E. Howden,et al.  Weak Mutation Testing and Completeness of Test Sets , 1982, IEEE Transactions on Software Engineering.