Boosting spectrum-based fault localization using PageRank

Manual debugging is notoriously tedious and time consuming. Therefore, various automated fault localization techniques have been proposed to help with manual debugging. Among the existing fault localization techniques, spectrum-based fault localization (SBFL) is one of the most widely studied techniques due to being lightweight. A focus of existing SBFL techniques is to consider how to differentiate program source code entities (i.e., one dimension in program spectra); indeed, this focus is aligned with the ultimate goal of finding the faulty lines of code. Our key insight is to enhance existing SBFL techniques by additionally considering how to differentiate tests (i.e., the other dimension in program spectra), which, to the best of our knowledge, has not been studied in prior work. We present PRFL, a lightweight technique that boosts spectrum-based fault localization by differentiating tests using PageRank algorithm. Given the original program spectrum information, PRFL uses PageRank to recompute the spectrum information by considering the contributions of different tests. Then, traditional SBFL techniques can be applied on the recomputed spectrum information to achieve more effective fault localization. Although simple and lightweight, PRFL has been demonstrated to outperform state-of-the-art SBFL techniques significantly (e.g., ranking 42% more real faults within Top-1 compared with the most effective traditional SBFL technique) with low overhead (e.g., around 2 minute average extra overhead on real faults) on 357 real faults from 5 Defects4J projects and 30692 artificial (i.e., mutation) faults from 87 GitHub projects, demonstrating a promising future for considering the contributions of different tests during fault localization.

[1]  Xiangyu Zhang,et al.  Locating faulty code using failure-inducing chops , 2005, ASE.

[2]  Marcelo d'Amorim,et al.  Entropy-based test generation for improved fault localization , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[3]  Martin Monperrus,et al.  Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs , 2018, IEEE Transactions on Software Engineering.

[4]  Andreas Zeller,et al.  Lightweight bug localization with AMPLE , 2005, AADEBUG'05.

[5]  Pedro M. Domingos,et al.  Learning Tractable Probabilistic Models for Fault Localization , 2015, AAAI.

[6]  Lionel C. Briand,et al.  Is mutation an appropriate tool for testing experiments? , 2005, ICSE.

[7]  Martin Monperrus,et al.  Learning to Combine Multiple Ranking Metrics for Fault Localization , 2014, 2014 IEEE International Conference on Software Maintenance and Evolution.

[8]  Alexei Chepelianskii,et al.  Towards physical laws for software architecture , 2010, ArXiv.

[9]  Sarfraz Khurshid,et al.  Injecting mechanical faults to localize developer faults for evolving software , 2013, OOPSLA.

[10]  Xiangyu Zhang,et al.  A study of effectiveness of dynamic slicing in locating real faults , 2006, Empirical Software Engineering.

[11]  Marcelo d'Amorim,et al.  Fault-localization using dynamic slicing and change impact analysis , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[12]  Michael D. Ernst,et al.  Evaluating & improving fault localization techniques , 2016 .

[13]  Stephen McCamant,et al.  The Daikon system for dynamic detection of likely invariants , 2007, Sci. Comput. Program..

[14]  Yves Le Traon,et al.  Assessing and Comparing Mutation-based Fault Localization Techniques , 2016, ArXiv.

[15]  Shin Hong,et al.  Mutation-Based Fault Localization for Real-World Multilingual Programs (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[16]  Xiangyu Zhang,et al.  Locating faulty code by multiple points slicing , 2007, Softw. Pract. Exp..

[17]  Michael D. Ernst,et al.  Defects4J: a database of existing faults to enable controlled testing studies for Java programs , 2014, ISSTA 2014.

[18]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[19]  Sarfraz Khurshid,et al.  Software fault localization using feature selection , 2011, MALETS '11.

[20]  Ali Mesbah,et al.  AutoFLox: An Automatic Fault Localizer for Client-Side JavaScript , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[21]  Yves Le Traon,et al.  Proteum/FL: A tool for localizing faults using mutation analysis , 2013, 2013 IEEE 13th International Working Conference on Source Code Analysis and Manipulation (SCAM).

[22]  W. Eric Wong,et al.  The DStar Method for Effective Software Fault Localization , 2014, IEEE Transactions on Reliability.

[23]  Lee Naish,et al.  A model for spectra-based software diagnosis , 2011, TSEM.

[24]  Yuhua Qi,et al.  The strength of random search on automated program repair , 2014, ICSE.

[25]  Claire Le Goues,et al.  GenProg: A Generic Method for Automatic Software Repair , 2012, IEEE Transactions on Software Engineering.

[26]  Shin Yoo,et al.  Ask the Mutants: Mutating Faulty Programs for Fault Localization , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[27]  Rui Abreu,et al.  Threats to the validity and value of empirical assessments of the accuracy of coverage-based fault locators , 2013, ISSTA.

[28]  Na Meng,et al.  How Does Execution Information Help with Information-Retrieval Based Bug Localization? , 2017, 2017 IEEE/ACM 25th International Conference on Program Comprehension (ICPC).

[29]  Serge Demeyer,et al.  Fine-tuning spectrum based fault localisation with frequent method item sets , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[30]  David Grove,et al.  Optimization of Object-Oriented Programs Using Static Class Hierarchy Analysis , 1995, ECOOP.

[31]  John T. Stasko,et al.  Visualization of test information to assist fault localization , 2002, ICSE '02.

[32]  Shujuan Jiang,et al.  Fault localization and repair for Java runtime exceptions , 2009, ISSTA.

[33]  Yves Le Traon,et al.  Effective fault localization via mutation analysis: a selective mutation approach , 2014, SAC.

[34]  Matias Martinez,et al.  Automatic repair of real bugs in java: a large-scale experiment on the defects4j dataset , 2016, Empirical Software Engineering.

[35]  Martin Monperrus,et al.  Automatic repair of buggy if conditions and missing preconditions with SMT , 2014, CSTVA 2014.

[36]  Ming Li,et al.  Constrained feature selection for localizing faults , 2015, 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[37]  Xiangyu Zhang,et al.  Locating faults through automated predicate switching , 2006, ICSE.

[38]  David Lo,et al.  Extended comprehensive study of association measures for fault localization , 2014, J. Softw. Evol. Process..

[39]  Yves Le Traon,et al.  Metallaxis‐FL: mutation‐based fault localization , 2015, Softw. Test. Verification Reliab..

[40]  Alessandro Orso,et al.  Are automated debugging techniques actually helping programmers? , 2011, ISSTA '11.

[41]  Michalis Faloutsos,et al.  Graph-based analysis and prediction for software evolution , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[42]  Michael D. Ernst,et al.  Are mutants a valid substitute for real faults in software testing? , 2014, SIGSOFT FSE.

[43]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[44]  Gordon Fraser,et al.  Do Automatically Generated Unit Tests Find Real Faults? An Empirical Study of Effectiveness and Challenges (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[45]  Ali Mesbah,et al.  Guided Mutation Testing for JavaScript Web Applications , 2015, IEEE Transactions on Software Engineering.

[46]  Alex Groce,et al.  Extended program invariants: applications in testing and fault localization , 2012, WODA 2012.

[47]  Shin Yoo,et al.  Evolving Human Competitive Spectra-Based Fault Localisation Techniques , 2012, SSBSE.

[48]  Xiangyu Zhang,et al.  Experimental evaluation of using dynamic slices for fault location , 2005, AADEBUG'05.

[49]  Rajiv Gupta,et al.  Learning universal probabilistic models for fault localization , 2010, PASTE '10.

[50]  Rui Abreu,et al.  A Survey on Software Fault Localization , 2016, IEEE Transactions on Software Engineering.

[51]  Michael D. Ernst,et al.  Evaluating and Improving Fault Localization , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[52]  Martin Monperrus,et al.  Test case purification for improving fault localization , 2014, SIGSOFT FSE.

[53]  Mark Harman,et al.  Provably Optimal and Human-Competitive Results in SBSE for Spectrum Based Fault Localisation , 2013, SSBSE.

[54]  David F. Gleich,et al.  PageRank beyond the Web , 2014, SIAM Rev..

[55]  Yuhua Qi,et al.  Slice-based statistical fault localization , 2014, J. Syst. Softw..

[56]  Sam Shah,et al.  Root cause detection in a service-oriented architecture , 2013, SIGMETRICS '13.

[57]  Xiangyu Zhang,et al.  Locating faults using multiple spectra-specific models , 2011, SAC.

[58]  A.J.C. van Gemund,et al.  On the Accuracy of Spectrum-based Fault Localization , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[59]  Yves Le Traon,et al.  Using Mutants to Locate "Unknown" Faults , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[60]  David Lo,et al.  Information retrieval and spectrum based bug localization: better together , 2015, ESEC/SIGSOFT FSE.

[61]  David Lo,et al.  Practitioners' expectations on automated fault localization , 2016, ISSTA.

[62]  Alessandro Orso,et al.  Evaluating the usefulness of IR-based fault localization techniques , 2015, ISSTA.

[63]  Lars Grunske,et al.  A learning-to-rank based fault localization approach using likely invariants , 2016, ISSTA.

[64]  Gregory Tassey,et al.  Prepared for what , 2007 .

[65]  Fan Long,et al.  Staged program repair with condition synthesis , 2015, ESEC/SIGSOFT FSE.