MURE: Making Use of MUtations to REfine Spectrum-Based Fault Localization

Locating faults in programs is never an easy task. Spectrum-based fault localization (SBFL) techniques estimate suspicious statements by contrasting the coverage spectra collected from passed and failed program runs. Mutation-based such techniques locate faults by trying different mutates with the aim of finding one that involves less turbulence to program behavior. The latter is empirically known more accurate, but with massive increases in time complexity. In this paper, we propose a new approach, MURE, which uses methodology of the latter to refine results of the former. MURE first drives a stateor-the-art SBFL technique Naish2 to output a list of suspicious statements. It then picks out suspicious statement as candidates, generates mutates for them, and estimates their likelihood of relating to faults. Finally, it refines the resultant list by adjusting part of its ordering. An experiment validates its effectiveness by showing a 30% accuracy improvement over Naish2.

[1]  James A. Jones Fault localization using visualization of test information , 2004, Proceedings. 26th International Conference on Software Engineering.

[2]  Peter Zoeteweij,et al.  An Evaluation of Similarity Coefficients for Software Fault Localization , 2006, 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC'06).

[3]  Xiaoguang Mao,et al.  Alleviating the Impact of Coincidental Correctness on the Effectiveness of SFL by Clustering Test Cases , 2014, 2014 Theoretical Aspects of Software Engineering Conference.

[4]  Yves Le Traon,et al.  Using Mutants to Locate "Unknown" Faults , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[5]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[6]  Shing-Chi Cheung,et al.  Taming coincidental correctness: Coverage refinement with context patterns to improve fault localization , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[7]  Baowen Xu,et al.  A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization , 2013, TSEM.

[8]  Wynne Hsu,et al.  DESIGN OF MUTANT OPERATORS FOR THE C PROGRAMMING LANGUAGE , 2006 .

[9]  Richard G. Hamlet,et al.  Testing Programs with the Aid of a Compiler , 1977, IEEE Transactions on Software Engineering.

[10]  A.J.C. van Gemund,et al.  On the Accuracy of Spectrum-based Fault Localization , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[11]  Lee Naish,et al.  A model for spectra-based software diagnosis , 2011, TSEM.

[12]  Raúl A. Santelices,et al.  Lightweight fault-localization using multiple coverage types , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[13]  Yves Le Traon,et al.  Improving test suites for efficient fault localization , 2006, ICSE.

[14]  Kai-Yuan Cai,et al.  Effective Fault Localization using Code Coverage , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[15]  Debra J. Richardson,et al.  An Analysis of Test Data Selection Criteria Using the RELAY Model of Fault Detection , 1993, IEEE Trans. Software Eng..

[16]  Timothy Alan Budd,et al.  Mutation analysis of program test data , 1980 .

[17]  Jian Zhang,et al.  A theoretical analysis on cloning the failed test cases to improve spectrum-based fault localization , 2017, J. Syst. Softw..

[18]  Peter Zoeteweij,et al.  A practical evaluation of spectrum-based fault localization , 2009, J. Syst. Softw..

[19]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[20]  Byoungju Choi,et al.  A family of code coverage-based heuristics for effective fault localization , 2010, J. Syst. Softw..

[21]  John T. Stasko,et al.  Visualization of test information to assist fault localization , 2002, ICSE '02.

[22]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[23]  Mary Jean Harrold,et al.  Empirical evaluation of the tarantula automatic fault-localization technique , 2005, ASE.

[24]  Brian Marick,et al.  The weak mutation hypothesis , 1991, TAV4.

[25]  A. Jefferson Offutt,et al.  Mutation 2000: uniting the orthogonal , 2001 .

[26]  W. Eric Wong,et al.  An empirical comparison of data flow and mutation‐based test adequacy criteria , 1994, Softw. Test. Verification Reliab..

[27]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[28]  Kai-Yuan Cai,et al.  Exploring the usefulness of unlabelled test cases in software fault localization , 2018, J. Syst. Softw..

[29]  Zheng Li,et al.  Faster mutation-based fault localization with a novel mutation execution strategy , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[30]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[31]  Shin Yoo,et al.  Ask the Mutants: Mutating Faulty Programs for Fault Localization , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[32]  Yu Qi,et al.  Effective program debugging based on execution slices and inter-block data dependency , 2006, J. Syst. Softw..