Generating test cases for specification mining

Dynamic specification mining observes program executions to infer models of normal program behavior. What makes us believe that we have seen sufficiently many executions? The typestate miner generates test cases that cover previously unobserved behavior, systematically extending the execution space and enriching the specification. To our knowledge, this is the first combination of systematic test case generation and typestate mining--a combination with clear benefits: On a sample of 800 defects seeded into six Java subjects, a static typestate verifier fed with enriched models would report significantly more true positives, and significantly fewer false positives than the initial models.

[1]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[2]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[3]  Robert E. Strom,et al.  Typestate: A programming language concept for enhancing software reliability , 1986, IEEE Transactions on Software Engineering.

[4]  Alexander L. Wolf,et al.  Discovering models of software processes from event-based data , 1998, TSEM.

[5]  William G. Griswold,et al.  Dynamically discovering likely program invariants to support program evolution , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[6]  James R. Larus,et al.  Mining specifications , 2002, POPL '02.

[7]  Monica S. Lam,et al.  Automatic extraction of object-oriented component interfaces , 2002, ISSTA '02.

[8]  David Notkin,et al.  Mutually Enhancing Test Generation and Specification Inference , 2003, FATES.

[9]  Neelam Gupta,et al.  A new structural coverage criterion for dynamic detection of program invariants , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[10]  Robert DeLine,et al.  Typestates for Objects , 2004, ECOOP.

[11]  TonellaPaolo.  Evolutionary testing of classes , 2004 .

[12]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[13]  Paolo Tonella,et al.  Evolutionary testing of classes , 2004, ISSTA '04.

[14]  Margus Veanes,et al.  Online testing with model programs , 2005, ESEC/FSE-13.

[15]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[16]  K. Rustan M. Leino,et al.  The Spec# Programming System: Challenges and Directions , 2005, VSTTE.

[17]  Manu Sridharan,et al.  Refinement-based context-sensitive points-to analysis for Java , 2006, PLDI '06.

[18]  Eran Yahav,et al.  Continuous code-quality assurance with SAFE , 2006, PEPM '06.

[19]  Andreas Zeller,et al.  Mining object behavior with ADABU , 2006, WODA '06.

[20]  Eran Yahav,et al.  Effective typestate verification in the presence of aliasing , 2006, TSEM.

[21]  Andreas Zeller,et al.  Detecting object usage anomalies , 2007, ESEC-FSE '07.

[22]  Jonathan Aldrich,et al.  Modular typestate checking of aliased objects , 2007, OOPSLA.

[23]  Sarfraz Khurshid,et al.  Korat: A Tool for Generating Structurally Complex Test Inputs , 2007, 29th International Conference on Software Engineering (ICSE'07).

[24]  Suresh Jagannathan,et al.  Static specification inference using predicate mining , 2007, PLDI '07.

[25]  Rupak Majumdar,et al.  Hybrid Concolic Testing , 2007, 29th International Conference on Software Engineering (ICSE'07).

[26]  Bertrand Meyer,et al.  Experimental assessment of random testing for object-oriented software , 2007, ISSTA '07.

[27]  Leonardo Mariani,et al.  Automatic generation of software behavioral models , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[28]  Eran Yahav,et al.  Static Specification Mining Using Automata-Based Abstractions , 2007, IEEE Transactions on Software Engineering.

[29]  Bertrand Meyer,et al.  Programs That Test Themselves , 2009, Computer.

[30]  Arie van Deursen,et al.  Invariant-based automatic testing of AJAX user interfaces , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[31]  Paola Inverardi,et al.  Automatic synthesis of behavior protocols for composable web-services , 2009, ESEC/FSE '09.

[32]  Andreas Zeller,et al.  Generating Fixes from Object Behavior Anomalies , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[33]  Rance Cleaveland,et al.  Using formal specifications to support testing , 2009, CSUR.