Explaining Software Failures by Cascade Fault Localization

During software debugging, a significant amount of effort is required for programmers to identify the root cause of a manifested failure. In this article, we propose a cascade fault localization method to help speed up this labor-intensive process via a combination of weakest precondition computation and constraint solving. Our approach produces a cause tree, where each node is a potential cause of the failure and each edge represents a casual relationship between two causes. There are two main contributions of this article that differentiate our approach from existing methods. First, our method systematically computes all potential causes of a failure and augments each cause with a proper context for ease of comprehension by the user. Second, our method organizes the potential causes in a tree structure to enable on-the-fly pruning based on domain knowledge and feedback from the user. We have implemented our new method in a software tool called CaFL, which builds upon the LLVM compiler and KLEE symbolic virtual machine. We have conducted experiments on a large set of public benchmarks, including real applications from GNU Coreutils and Busybox. Our results show that in most cases the user has to examine only a small fraction of the execution trace before identifying the root cause of the failure.

[1]  Alex Groce,et al.  What Went Wrong: Explaining Counterexamples , 2003, SPIN.

[2]  ZellerAndreas Isolating cause-effect chains from computer programs , 2002 .

[3]  Tibor Gyimóthy,et al.  An efficient relevant slicing method for debugging , 1999, ESEC/FSE-7.

[4]  Chris Lattner,et al.  LLVM: AN INFRASTRUCTURE FOR MULTI-STAGE OPTIMIZATION , 2000 .

[5]  Emina Torlak,et al.  What Gives? A Hybrid Algorithm for Error Trace Explanation , 2014, VSTTE.

[6]  Vikram S. Adve,et al.  Using likely invariants for automated software fault localization , 2013, ASPLOS '13.

[7]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[8]  Thomas Wies,et al.  Flow-Sensitive Fault Localization , 2013, VMCAI.

[9]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[10]  Shriram Krishnamurthi,et al.  Automated Fault Localization Using Potential Invariants , 2003, ArXiv.

[11]  Boris Beizer,et al.  Software Testing Techniques , 1983 .

[12]  Satish Narayanasamy,et al.  Offline symbolic analysis for multi-processor execution replay , 2009, 2009 42nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[13]  David W. Binkley,et al.  Program slicing , 2008, 2008 Frontiers of Software Maintenance.

[14]  Chao Wang,et al.  Abstraction and mining of traces to explain concurrency bugs , 2016, FM 2016.

[15]  Mayur Naik,et al.  From symptom to cause: localizing errors in counterexample traces , 2003, POPL '03.

[16]  Inês Lynce,et al.  On Computing Minimum Unsatisfiable Cores , 2004, SAT.

[17]  Zhenkai Liang,et al.  DARWIN: An approach to debugging evolving programs , 2012, TSEM.

[18]  Alex Groce,et al.  SPECIAL S ECTION O N T OOLS A ND A LGORITHMS F OR THE C ONSTRUCTION A ND A NALYSIS O F S YSTEMS , 2005 .

[19]  Thomas Wies,et al.  Error Invariants , 2012, FM.

[20]  Joseph Robert Horgan,et al.  Dynamic program slicing , 1990, PLDI '90.

[21]  Zhenkai Liang,et al.  Golden implementation driven software debugging , 2010, FSE '10.

[22]  Andreas Zeller,et al.  Locating causes of program failures , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[23]  Roderick Bloem,et al.  Repair of Boolean Programs with an Application to C , 2006, CAV.

[24]  Rupak Majumdar,et al.  Cause clue clauses: error localization using maximum satisfiability , 2010, PLDI '11.

[25]  A. Zeller Isolating cause-effect chains from computer programs , 2002, SIGSOFT '02/FSE-10.

[26]  Rupak Majumdar,et al.  Bug-Assist: Assisting Fault Localization in ANSI-C Programs , 2011, CAV.

[27]  Xiangyu Zhang,et al.  Locating faults through automated predicate switching , 2006, ICSE.

[28]  Sharad Malik,et al.  Validating SAT solvers using an independent resolution-based checker: practical implementations and other applications , 2003, 2003 Design, Automation and Test in Europe Conference and Exhibition.

[29]  Shriram Krishnamurthi,et al.  Automated Fault Localization Using Potential Invariants , 2003, ArXiv.

[30]  Hai Zhou,et al.  Parallel CAD: Algorithm Design and Programming Special Section Call for Papers TODAES: ACM Transactions on Design Automation of Electronic Systems , 2010 .

[31]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[32]  Alessandro Orso,et al.  Isolating failure causes through test case generation , 2012, ISSTA 2012.

[33]  Robert Sedgewick,et al.  Algorithms in C , 1990 .

[34]  Alex Groce,et al.  Understanding Counterexamples with explain , 2004, CAV.

[35]  Roderick Bloem,et al.  Automated Fault Localization for C Programs , 2007, V&D@FLoC.

[36]  Gogul Balakrishnan,et al.  PED: Proof-Guided Error Diagnosis by Triangulation of Program Error Causes , 2008, 2008 Sixth IEEE International Conference on Software Engineering and Formal Methods.

[37]  Xiangyu Zhang,et al.  Towards locating execution omission errors , 2007, PLDI '07.

[38]  David L. Dill,et al.  A Decision Procedure for Bit-Vectors and Arrays , 2007, CAV.

[39]  Karem A. Sakallah,et al.  Algorithms for Computing Minimal Unsatisfiable Subsets of Constraints , 2007, Journal of Automated Reasoning.

[40]  Edsger W. Dijkstra,et al.  A Discipline of Programming , 1976 .

[41]  Bing Li,et al.  Automated Program Debugging Via Multiple Predicate Switching , 2010, AAAI.

[42]  Jian Liu,et al.  A Synergistic Analysis Method for Explaining Failed Regression Tests , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[43]  Chao Wang,et al.  Whodunit? Causal Analysis for Counterexamples , 2006, ATVA.

[44]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[45]  William Craig,et al.  Three uses of the Herbrand-Gentzen theorem in relating model theory and proof theory , 1957, Journal of Symbolic Logic.

[46]  Satish Narayanasamy,et al.  Offline symbolic analysis to infer Total Store Order , 2011, 2011 IEEE 17th International Symposium on High Performance Computer Architecture.