Faster mutation-based fault localization with a novel mutation execution strategy

Checking program entities for finding faults is extremely tedious for developers. Fault localization techniques are designed to give a rank list of the probability that program entities incur faults to assist developers to locate faults. Mutation-based fault localization is a recently proposed fault localization approach via mutation analysis. With improved effectiveness, the mutation-based fault localization also brings huge execution cost. To reduce the execution cost of mutation-based fault localization technique, this paper proposes a dynamic mutation execution strategy which includes execution optimizations on both mutants and test cases. As fewer mutants and test cases are executed with the presented strategy, the fault localization process will be faster. The empirical studies show that mutation-based fault localization with the dynamic strategy we proposed, called Faster-MBFL, can reduce mutant-test execution times by 32.4% to 87% with keeping the fault localization accuracy unchanged; further, the additional run time required by utilizing our strategy can be ignored.

[1]  Xiaohong Su,et al.  A test-suite reduction approach to improving fault-localization effectiveness , 2013, Comput. Lang. Syst. Struct..

[2]  Wes Masri,et al.  Fault localization based on information flow coverage , 2010, Softw. Test. Verification Reliab..

[3]  Michael D. Ernst,et al.  Are mutants a valid substitute for real faults in software testing? , 2014, SIGSOFT FSE.

[4]  Lee Naish,et al.  A model for spectra-based software diagnosis , 2011, TSEM.

[5]  Mary Jean Harrold,et al.  Empirical evaluation of the tarantula automatic fault-localization technique , 2005, ASE.

[6]  Yves Le Traon,et al.  Proteum/FL: A tool for localizing faults using mutation analysis , 2013, 2013 IEEE 13th International Working Conference on Source Code Analysis and Manipulation (SCAM).

[7]  Shin Yoo,et al.  Ask the Mutants: Mutating Faulty Programs for Fault Localization , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[8]  Akbar Siami Namin,et al.  Sufficient mutation operators for measuring test effectiveness , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[9]  Mark Harman,et al.  Constructing Subtle Faults Using Higher Order Mutation Testing , 2008, 2008 Eighth IEEE International Working Conference on Source Code Analysis and Manipulation.

[10]  Yves Le Traon,et al.  Metallaxis‐FL: mutation‐based fault localization , 2015, Softw. Test. Verification Reliab..

[11]  Peter Zoeteweij,et al.  An Evaluation of Similarity Coefficients for Software Fault Localization , 2006, 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC'06).

[12]  Michael D. Ernst,et al.  Efficient mutation analysis by propagating and partitioning infected execution states , 2014, ISSTA 2014.

[13]  Andreas Zeller,et al.  Javalanche: efficient mutation testing for Java , 2009, ESEC/SIGSOFT FSE.

[14]  René Just,et al.  Using Non-redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and Scalable Mutation Analysis , 2012, 2012 IEEE 23rd International Symposium on Software Reliability Engineering.

[15]  René Just,et al.  MAJOR: An efficient and extensible tool for mutation analysis in a Java compiler , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[16]  I. Comparison.  Faster Mutation Testing Inspired by Test Prioritization and Reduction , 2013 .

[17]  Qian Yang,et al.  A Survey of Coverage-Based Testing Tools , 2009, Comput. J..

[18]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[19]  Wang Tiantian,et al.  A test-suite reduction approach to improving fault-localization effectiveness , 2013 .

[20]  Mark Harman,et al.  Higher Order Mutation Testing , 2009, Inf. Softw. Technol..

[21]  Yves Le Traon,et al.  Using Mutants to Locate "Unknown" Faults , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[22]  Lionel C. Briand,et al.  Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria , 2006, IEEE Transactions on Software Engineering.

[23]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[24]  Mary Jean Harrold,et al.  An empirical study of the effects of test-suite reduction on fault localization , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[25]  A. Jefferson Offutt,et al.  Mutation analysis using mutant schemata , 1993, ISSTA '93.

[26]  Yves Le Traon,et al.  Effective fault localization via mutation analysis: a selective mutation approach , 2014, SAC.

[27]  T. Cucinotta,et al.  2012 IEEE Fifth International Conference on Cloud Computing, Honolulu, HI, USA, June 24-29, 2012 , 2012, IEEE CLOUD.

[28]  A. Jefferson Offutt,et al.  Improving logic-based testing , 2013, J. Syst. Softw..

[29]  Rui Abreu,et al.  A Survey on Software Fault Localization , 2016, IEEE Transactions on Software Engineering.

[30]  José Carlos Maldonado,et al.  Proteum/IM 2.0: An Integrated Mutation Testing Environment , 2001 .

[31]  Andreas Zeller,et al.  Lightweight bug localization with AMPLE , 2005, AADEBUG'05.

[32]  Xiangyu Zhang,et al.  Locating faults using multiple spectra-specific models , 2011, SAC.

[33]  René Just,et al.  Higher accuracy and lower run time: efficient mutation analysis using non‐redundant mutation operators , 2015, Softw. Test. Verification Reliab..