Generation of Test Data Structures Using Constraint Logic Programming

The goal of Bounded-Exhaustive Testing (BET) is the automatic generation of all the test cases satisfying a given invariant, within a given bound. When the input has a complex structure, the development of correct and efficient generators becomes a very challenging task. In this paper we use Constraint Logic Programming (CLP) to systematically develop generators of structurally complex test data. Similarly to filtering-based test generation, we follow a declarative approach which allows us to separate the issue of (i) defining the test structure and invariant, from that of (ii) generating admissible test input instances. This separation helps improve the correctness of the developed test case generators. However, in contrast with filtering approaches, we rely on a symbolic representation and we take advantage of efficient search strategies provided by CLP systems for generating test instances. Through some experiments on examples taken from the literature on BET, we show that CLP, by combining the use of constraints and recursion, allows one to write intuitive and easily understandable test generators. We also show that these generators can be much more efficient than those built using ad-hoc filtering-based test generation tools like Korat.

[1]  V. S. Costa,et al.  Theory and Practice of Logic Programming , 2010 .

[2]  Alberto Pettorossi,et al.  Unfolding - Definition - Folding, in this Order, for Avaoiding Unnecessary Variables in Logic Programs , 1991, PLILP.

[3]  Wim Vanhoof,et al.  Binding-time analysis for Mercury , 2004 .

[4]  J. W. Lloyd,et al.  Foundations of logic programming; (2nd extended ed.) , 1987 .

[5]  Qing Li,et al.  Automatic Test Generation for Dynamic Data Structures , 2007, 5th ACIS International Conference on Software Engineering Research, Management & Applications (SERA 2007).

[6]  R. W. Robinson Counting unlabeled acyclic digraphs , 1977 .

[7]  Sarfraz Khurshid,et al.  TestEra: A tool for testing Java programs using alloy specifications , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[8]  Darko Marinov,et al.  Automatic testing of software with structurally complex inputs , 2005 .

[9]  Ronald L. Rivest,et al.  Introduction to Algorithms, third edition , 2009 .

[10]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[11]  Sarfraz Khurshid,et al.  Korat: A Tool for Generating Structurally Complex Test Inputs , 2007, 29th International Conference on Software Engineering (ICSE'07).

[12]  Peter Sestoft,et al.  Partial evaluation and automatic program generation , 1993, Prentice Hall international series in computer science.

[13]  Jacob M. Howe,et al.  Efficient Groundness Analysis in Prolog , 2003, Theory Pract. Log. Program..

[14]  Alberto Pettorossi,et al.  Program transformation for development, verification, and synthesis of programs , 2011, Intelligenza Artificiale.

[15]  A. Jefferson Offutt,et al.  Constraint-Based Automatic Test Data Generation , 1991, IEEE Trans. Software Eng..

[16]  Andy King,et al.  RedAlert: Determinacy inference for Prolog , 2011, Theory Pract. Log. Program..

[17]  Hong-Yi Wang,et al.  Grammar Based Testing of HTML Injection Vulnerabilities in RSS Feeds , 2009, 2009 Testing: Academic and Industrial Conference - Practice and Research Techniques.

[18]  Sarfraz Khurshid,et al.  Test generation through programming in UDITA , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[19]  Wim Vanhoof,et al.  Towards a Framework for Constraint-Based Test Case Generation , 2009, LOPSTR.

[20]  Michael J. Maher,et al.  Constraint Logic Programming: A Survey , 1994, J. Log. Program..

[21]  John W. Lloyd,et al.  Foundations of Logic Programming, 1st Edition , 1984 .

[22]  Christophe Meudec ATGen: automatic test data generation using constraint logic programming and symbolic execution † , 2001, Softw. Test. Verification Reliab..

[23]  Sarfraz Khurshid,et al.  An Evaluation of Exhaustive Testing for Data Structures , 2003 .

[24]  Peter J. Stuckey,et al.  Programming with Constraints: An Introduction , 1998 .

[25]  Sarfraz Khurshid,et al.  Software assurance by bounded exhaustive testing , 2004, IEEE Transactions on Software Engineering.

[26]  Elvira Albert,et al.  Test case generation for object-oriented imperative languages in CLP , 2010, Theory Pract. Log. Program..

[27]  J. Lloyd Foundations of Logic Programming , 1984, Symbolic Computation.

[28]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[29]  Nazareno Aguirre,et al.  Incorporating Coverage Criteria in Bounded Exhaustive Black Box Test Generation of Structural Inputs , 2011, TAP@TOOLS.

[30]  Charles H. C. Little,et al.  Combinatorial Mathematics V , 1977 .

[31]  Sarfraz Khurshid,et al.  Test input generation with java PathFinder , 2004, ISSTA '04.

[32]  Sarfraz Khurshid,et al.  Optimizing Generation of Object Graphs in Java PathFinder , 2009, 2009 International Conference on Software Testing Verification and Validation.

[33]  Agostino Dovier,et al.  An empirical study of constraint logic programming and answer set programming solutions of combinatorial problems , 2009, J. Exp. Theor. Artif. Intell..

[34]  Neelam Gupta,et al.  Generating test data for functions with pointer inputs , 2002, Proceedings 17th IEEE International Conference on Automated Software Engineering,.

[35]  Manuel V. Hermenegildo,et al.  Energy Consumption Analysis of Programs Based on XMOS ISA-Level Models , 2013, LOPSTR.

[36]  Eda Marchetti,et al.  WS-TAXI: A WSDL-based Testing Tool for Web Services , 2009, 2009 International Conference on Software Testing Verification and Validation.

[37]  Arnaud Gotlieb,et al.  Modelling Dynamic Memory Management in Constraint-Based Testing , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[38]  Alberto Pettorossi,et al.  Transformation Rules for Locally Stratified Constraint Logic Programs , 2004, Program Development in Computational Logic.

[39]  Luís Moniz Pereira,et al.  Computational Logic — CL 2000 , 2000, Lecture Notes in Computer Science.

[40]  Bernard Botella,et al.  A CLP Framework for Computing Structural Test Data , 2000, Computational Logic.