Dynamic Analysis for Integration Faults Localization

Software developers usually integrate third party components to build systems providing distinct functionalities like graph drawing, or persistence capabilities. Developers often use grey-box components: software modules they do not know in details because provided without source code or incomplete specifications or both. Lack of source code and specifications makes the integration of such modules difficult and often causes faults that lead to critical failures if not detected at testing time. Lack of such informations complicates faults localization too. Existing static analysis and debugging techniques rely on source code or specifications, for this reason their applicability is often limited when grey-box components are used. Dynamic analysis techniques do not need such information: they monitor components interfaces, thus being applicable even when this information is missing. Dynamic analysis approaches identify violations of models inferred from data recorded during monitored executions. Unfortunately these techniques suffers from limitations too: they are capable of identifying only specific kinds of faults, their results are often affected by false positives, and they present scalability issues depending on the huge amount of data collected during training or on the identification of many violations during debugging. This paper presents Behaviour Capture and Test (BCT), a dynamic analysis technique that overcomes the limitations of the existing dynamic analysis techniques. BCT uses different kinds of models to localize different types of faults, it prunes false positives, it incrementally builds models to save disk space and guides developers when many models violations are identified. Successful results obtained when applying the technique on injected and real faults are reported in the paper: the considered case studies regard both regression faults and faults depending on rare events sequences not stressed at testing time. ∗Mauro Pezzè is also professor at the University of Lugano, Faculty of Informatics, via Buffi, 13 6900 Lugano (Switzerland).

[1]  Jeffrey S. Foster,et al.  A comparison of bug finding tools for Java , 2004, 15th International Symposium on Software Reliability Engineering.

[2]  Peter G. Bishop,et al.  An exploration of software faults and failure behaviour in a large population of programs , 2004, 15th International Symposium on Software Reliability Engineering.

[3]  David J. Carney,et al.  Isolating faults in complex COTS-based systems , 1999, J. Softw. Maintenance Res. Pract..

[4]  Nancy G. Leveson,et al.  Role of Software in Spacecraft Accidents , 2004 .

[5]  James R. Larus,et al.  Exploiting hardware performance counters with flow and context sensitive profiling , 1997, PLDI '97.

[6]  Marc J. Balcer,et al.  The category-partition method for specifying and generating fuctional tests , 1988, CACM.

[7]  U. Kroszynski The CAD*I software tools , 1989 .

[8]  David Hovemeyer,et al.  Finding bugs is easy , 2004, SIGP.

[9]  Alexander L. Wolf,et al.  Discovering models of software processes from event-based data , 1998, TSEM.

[10]  Maurizio Morisio,et al.  Investigating and improving a COTS-based software development process , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[11]  David Leon,et al.  An Empirical Study of Test Case Filtering Techniques Based on Exercising Information Flows , 2007, IEEE Transactions on Software Engineering.

[12]  Alessandro Orso,et al.  A Technique for Enabling and Supporting Debugging of Field Failures , 2007, 29th International Conference on Software Engineering (ICSE'07).

[13]  Elaine J. Weyuker,et al.  Testing Component-Based Software: A Cautionary Tale , 1998, IEEE Softw..

[14]  Steven P. Reiss,et al.  Encoding program executions , 2001, Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001.

[15]  Rajesh Parekh,et al.  A Polynominal Time Incremental Algorithm for Learning DFA , 1998, ICGI.

[16]  Jerome A. Feldman,et al.  Learning automata from ordered examples , 1991, COLT '88.

[17]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[18]  J. William Ahwood,et al.  CLASSIFICATION , 1931, Foundations of Familiar Language.

[19]  Andreas Zeller,et al.  Why Programs Fail: A Guide to Systematic Debugging , 2005 .

[20]  Gregor Kiczales,et al.  Aspect-oriented programming , 2001, ESEC/FSE-9.

[21]  Andreas Zeller,et al.  Detecting object usage anomalies , 2007, ESEC-FSE '07.

[22]  Chao Liu,et al.  Failure proximity: a fault localization-based approach , 2006, SIGSOFT '06/FSE-14.

[23]  Richard Lippmann,et al.  Testing static analysis tools using exploitable buffer overflows from open source code , 2004, SIGSOFT '04/FSE-12.

[24]  Sudheendra Hangal,et al.  Tracking down software bugs using automatic anomaly detection , 2002, ICSE '02.

[25]  Peter M. Chen,et al.  Whither generic recovery from application faults? A fault study using open-source software , 2000, Proceeding International Conference on Dependable Systems and Networks. DSN 2000.

[26]  Mary Shaw,et al.  Semantic anomaly detection in online data sources , 2002, ICSE '02.

[27]  Jerome A. Feldman,et al.  On the Synthesis of Finite-State Machines from Samples of Their Behavior , 1972, IEEE Transactions on Computers.

[28]  John T. Stasko,et al.  Visualization of test information to assist fault localization , 2002, ICSE '02.

[29]  Lionel C. Briand,et al.  Using Machine Learning to Support Debugging with Tarantula , 2007, The 18th IEEE International Symposium on Software Reliability (ISSRE '07).

[30]  Michael D. Ernst,et al.  Efficient incremental algorithms for dynamic detection of likely invariants , 2004, SIGSOFT '04/FSE-12.

[31]  A. Zeller Isolating cause-effect chains from computer programs , 2002, SIGSOFT '02/FSE-10.

[32]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[33]  William G. Griswold,et al.  Dynamically discovering likely program invariants to support program evolution , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[34]  Pierre Dupont,et al.  Incremental regular inference , 1996, ICGI.

[35]  Sigrid Eldh,et al.  Component Testing Is Not Enough - A Study of Software Faults in Telecom Middleware , 2007, TestCom/FATES.