A general noise-reduction framework for fault localization of Java programs

Context: Existing fault-localization techniques combine various program features and similarity coefficients with the aim of precisely assessing the similarities among the dynamic spectra of these program features to predict the locations of faults. Many such techniques estimate the probability of a particular program feature causing the observed failures. They often ignore the noise introduced by other features on the same set of executions that may lead to the observed failures. It is unclear to what extent such noise can be alleviated. Objective: This paper aims to develop a framework that reduces the noise in fault-failure correlation measurements. Method: We develop a fault-localization framework that uses chains of key basic blocks as program features and a noise-reduction methodology to improve on the similarity coefficients of fault-localization techniques. We evaluate our framework on five base techniques using five real-life median-scaled programs in different application domains. We also conduct a case study on subjects with multiple faults. Results: The experimental result shows that the synthesized techniques are more effective than their base techniques by almost 10%. Moreover, their runtime overhead factors to collect the required feature values are practical. The case study also shows that the synthesized techniques work well on subjects with multiple faults. Conclusion: We conclude that the proposed framework has a significant and positive effect on improving the effectiveness of the corresponding base techniques.

[1]  Trishul M. Chilimbi,et al.  HOLMES: Effective statistical debugging via efficient path profiling , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[2]  Fadi A. Zaraket,et al.  Enhancing Fault Localization via Multivariate Visualization , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[3]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[4]  Tsong Yueh Chen,et al.  How well does test case prioritization integrate with statistical fault localization? , 2012, Inf. Softw. Technol..

[5]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[6]  T. H. Tse,et al.  Non-parametric statistical fault localization , 2011, J. Syst. Softw..

[7]  T. H. Tse,et al.  Fault localization through evaluation sequences , 2010, J. Syst. Softw..

[8]  Xiangyu Zhang,et al.  Pruning dynamic slices with confidence , 2006, PLDI '06.

[9]  Laurie Hendren,et al.  Soot: a Java bytecode optimization framework , 2010, CASCON.

[10]  Raúl A. Santelices,et al.  Lightweight fault-localization using multiple coverage types , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[11]  Aritra Bandyopadhyay Mitigating the Effect of Coincidental Correctness in Spectrum Based Fault Localization , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[12]  Frank Tip,et al.  Practical fault localization for dynamic web applications , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[13]  A.J.C. van Gemund,et al.  On the Accuracy of Spectrum-based Fault Localization , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[14]  Zhendong Su,et al.  Context-aware statistical debugging: from bug predictors to faulty control flow paths , 2007, ASE.

[15]  Yves Le Traon,et al.  Improving test suites for efficient fault localization , 2006, ICSE.

[16]  T. H. Tse,et al.  Capturing propagation of infected program states , 2009, ESEC/FSE '09.

[17]  Richard W. Vuduc,et al.  Falcon: fault localization in concurrent programs , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[18]  Peggy Cellier Formal concept analysis applied to fault localization , 2008, ICSE Companion '08.

[19]  Peter Zoeteweij,et al.  A practical evaluation of spectrum-based fault localization , 2009, J. Syst. Softw..

[20]  T. H. Tse,et al.  Adaptive Random Test Case Prioritization , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[21]  Laurie Hendren,et al.  Jimple: Simplifying Java Bytecode for Analyses and Transformations , 1998 .

[22]  Peter Zoeteweij,et al.  Spectrum-Based Multiple Fault Localization , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[23]  Mary Jean Harrold,et al.  Debugging in Parallel , 2007, ISSTA '07.

[24]  Andy Podgurski,et al.  The Probabilistic Program Dependence Graph and Its Application to Fault Diagnosis , 2008, IEEE Transactions on Software Engineering.

[25]  Rajiv Gupta,et al.  Learning universal probabilistic models for fault localization , 2010, PASTE '10.

[26]  Jian Xu,et al.  A Dynamic Fault Localization Technique with Noise Reduction for Java Programs , 2011, 2011 11th International Conference on Quality Software.

[27]  Sudipto Ghosh,et al.  Tester Feedback Driven Fault Localization , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[28]  Gregg Rothermel,et al.  Selecting a Cost-Effective Test Case Prioritization Technique , 2004, Software Quality Journal.

[29]  Mary Jean Harrold,et al.  An empirical study of the effects of test-suite reduction on fault localization , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[30]  James A. Jones,et al.  On the influence of multiple faults on coverage-based fault localization , 2011, ISSTA '11.

[31]  Byoungju Choi,et al.  A family of code coverage-based heuristics for effective fault localization , 2010, J. Syst. Softw..

[32]  Chen Fu,et al.  Exception-Chain Analysis: Revealing Exception Handling Architecture in Java Server Applications , 2007, 29th International Conference on Software Engineering (ICSE'07).

[33]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[34]  T. H. Tse,et al.  Fault Localization Based Only on Failed Runs , 2012, Computer.

[35]  Rajiv Gupta,et al.  Fault localization using value replacement , 2008, ISSTA '08.

[36]  Mary Jean Harrold,et al.  Empirical evaluation of the tarantula automatic fault-localization technique , 2005, ASE.

[37]  Ding Yuan,et al.  SherLog: error diagnosis by connecting clues from run-time logs , 2010, ASPLOS XV.

[38]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[39]  Chao Liu,et al.  Statistical Debugging: A Hypothesis Testing-Based Approach , 2006, IEEE Transactions on Software Engineering.

[40]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[41]  Kai-Yuan Cai,et al.  Effective Fault Localization using Code Coverage , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[42]  Ting Chen,et al.  Statistical debugging using compound boolean predicates , 2007, ISSTA '07.

[43]  Lee Naish,et al.  A model for spectra-based software diagnosis , 2011, TSEM.

[44]  Jeffrey M. Voas,et al.  PIE: A Dynamic Failure-Based Technique , 1992, IEEE Trans. Software Eng..