Extended program invariants: applications in testing and fault localization

Invariants are powerful tools for program analysis and reasoning.Several tools and techniques have been developed to infer invariants of a program. Given a test suite for a program, an invariant detection tool (IDT) extracts (potential) invariants from the program execution on test cases of the test suite. The resultant invariants contain relations only over variables and constants that are visible to the IDT. IDTs are usually unable to extract invariants about execution features like taken branches, since programs usually do not have state variables for such features. Thus, the IDT has no information about such features in order to infer relations between them. We speculate that invariants about execution features are useful for understanding test suites; we call these invariants, extended invariants. In this paper, we discuss potential applications of extended invariants in understanding of test suites, and fault localization. We illustrate the usefulness of extended invariants with some small examples that use basic block count as the execution feature in extended invariants. We believe extended invariants provide useful information about execution of programs that can be utilized in program analysis and testing.

[1]  William G. Griswold,et al.  Dynamically discovering likely program invariants to support program evolution , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[2]  Bojan Cukic,et al.  Comparing Partition and Random Testing via Majorization and Schur Functions , 2003, IEEE Trans. Software Eng..

[3]  Michal Young,et al.  Residual test coverage monitoring , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[4]  Yannis Smaragdakis,et al.  Dynamically discovering likely interface invariants , 2006, ICSE '06.

[5]  Xiangyu Zhang,et al.  Locating faulty code using failure-inducing chops , 2005, ASE.

[6]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[7]  Nicolas Halbwachs,et al.  Automatic discovery of linear restraints among variables of a program , 1978, POPL.

[8]  Andrei Voronkov,et al.  Finding Loop Invariants for Programs over Arrays Using a Theorem Prover , 2009, FASE.

[9]  Monica S. Lam,et al.  Automatic extraction of object-oriented component interfaces , 2002, ISSTA '02.

[10]  Shriram Krishnamurthi,et al.  Automated Fault Localization Using Potential Invariants , 2003, ArXiv.

[11]  Michael D. Ernst,et al.  Improving test suites via operational abstraction , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[12]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[13]  James R. Larus,et al.  Mining specifications , 2002, POPL '02.

[14]  John T. Stasko,et al.  Visualization of test information to assist fault localization , 2002, ICSE '02.

[15]  Alex Groce,et al.  Swarm testing , 2012, ISSTA 2012.

[16]  Barton P. Miller,et al.  An empirical study of the robustness of MacOS applications using random testing , 2007, OPSR.

[17]  Simeon C. Ntafos,et al.  An Evaluation of Random Testing , 1984, IEEE Transactions on Software Engineering.

[18]  Andreas Zeller,et al.  Mining object behavior with ADABU , 2006, WODA '06.

[19]  Eugene H. Spafford,et al.  An execution-backtracking approach to debugging , 1991, IEEE Software.

[20]  Shriram Krishnamurthi,et al.  Automated Fault Localization Using Potential Invariants , 2003, ArXiv.

[21]  Michael I. Jordan,et al.  Bug isolation via remote program sampling , 2003, PLDI.

[22]  Andreas Zeller,et al.  Locating causes of program failures , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..