Interactive fault localization leveraging simple user feedback

Many fault localization methods have been proposed in the literature. These methods take in a set of program execution profiles and output a list of suspicious program elements. The list of program elements ranked by their suspiciousness is then presented to developers for manual inspection. Currently, the suspicious elements are ranked in a batch process where developers' inspection efforts are rarely utilized for ranking. The inaccuracy and static nature of existing fault localization methods prompt us to incorporate user feedback to improve the accuracy of the existing methods. In this paper, we propose an interactive fault localization framework that leverages simple user feedback. Our framework only needs users to label the statements examined as faulty or clean, which does not require additional effort than conventional non-interactive methods. After users label suspicious program elements as faulty or clean, our framework incorporates such information and re-orders the rest of the suspicious program elements, aiming to expose truly faulty elements earlier. We have integrated our solution with three well-known fault localization methods: Ochiai, Tarantula, and Jaccard. The evaluation on five Unix programs and the Siemens test suite shows that our solution achieves significant improvements on fault localization accuracy.

[1]  Alessandro Orso,et al.  Are automated debugging techniques actually helping programmers? , 2011, ISSTA '11.

[2]  David Lo,et al.  Active refinement of clone anomaly reports , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[3]  Tao Xie,et al.  Interactive Fault Localization Using Test Information , 2009, Journal of Computer Science and Technology.

[4]  Andreas Zeller,et al.  Mining metrics to predict component failures , 2006, ICSE.

[5]  John T. Stasko,et al.  Visualization of test information to assist fault localization , 2002, ICSE '02.

[6]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[7]  Chao Liu,et al.  SOBER: statistical model-based bug localization , 2005, ESEC/FSE-13.

[8]  Elaine J. Weyuker,et al.  Programmer-based fault prediction , 2010, PROMISE '10.

[9]  Peter Zoeteweij,et al.  Spectrum-Based Multiple Fault Localization , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[10]  A. Zeller Isolating cause-effect chains from computer programs , 2002, SIGSOFT '02/FSE-10.

[11]  Brent Hailpern,et al.  Software debugging, testing, and verification , 2002, IBM Syst. J..

[12]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[13]  David Lo,et al.  Comprehensive evaluation of association measures for fault localization , 2010, 2010 IEEE International Conference on Software Maintenance.

[14]  Josep Silva,et al.  An optimal strategy for algorithmic debugging , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[15]  Peter Zoeteweij,et al.  A practical evaluation of spectrum-based fault localization , 2009, J. Syst. Softw..

[16]  Zhi-Hua Zhou,et al.  Sample-based software defect prediction with active and semi-supervised learning , 2012, Automated Software Engineering.

[17]  Sigrid Eldh Software Testing Techniques , 2007 .

[18]  Rui Abreu,et al.  Spectrum-based Fault Localization in Embedded Software , 2009 .

[19]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[20]  Andy Podgurski,et al.  Mitigating the confounding effects of program dependences for effective fault localization , 2011, ESEC/FSE '11.

[21]  Xiangyu Zhang,et al.  Towards locating execution omission errors , 2007, PLDI '07.

[22]  Andy Podgurski,et al.  Causal inference for statistical fault localization , 2010, ISSTA '10.

[23]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[24]  T. H. Tse,et al.  On Practical Adequate Test Suites for Integrated Test Case Prioritization and Fault Localization , 2011, 2011 11th International Conference on Quality Software.

[25]  Scott N. Woodfield,et al.  Evaluating the effectiveness of reliability-assurance techniques , 1989, J. Syst. Softw..

[26]  Rongxin Wu,et al.  Sampling program quality , 2010, 2010 IEEE International Conference on Software Maintenance.

[27]  Rajiv Gupta,et al.  Fault localization using value replacement , 2008, ISSTA '08.

[28]  Mary Jean Harrold,et al.  Empirical evaluation of the tarantula automatic fault-localization technique , 2005, ASE.

[29]  Rui Abreu,et al.  Prioritizing tests for software fault diagnosis , 2011, Softw. Pract. Exp..

[30]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[31]  Michael I. Jordan,et al.  Bug isolation via remote program sampling , 2003, PLDI.

[32]  Norman Matloff,et al.  The Art of Debugging with GDB, DDD, and Eclipse , 2008 .