Efficient generation of test data structures using constraint logic programming and program transformation

The goal of Bounded-Exhaustive Testing (BET) is the automatic generation of all test cases satisfying a given invariant, within a given size bound. When the test cases have a complex structure, the development of correct and efficient generators becomes a very challenging task. In this paper we use Constraint Logic Programming (CLP) to systematically develop generators of structurally complex test data structures. We follow a declarative approach which allows us to separate the issue of (i) defining the test data structure in terms of its properties, from that of (ii) efficiently generating data structure instances. This separation helps establish the correctness of the developed test case generators. We rely on a symbolic representation and we take advantage of efficient search strategies provided by CLP systems for generating test instances. Through a running example taken from the literature on BET, we illustrate our test generation framework and we show that CLP allows us to develop easily understandable and efficient test generators. Additionally, we propose a program transformation technique whose goal is to make the evaluation of these CLP-based generators much more efficient and we demonstrate its effectiveness on a number of complex test data structures.

[1]  Hareton K. N. Leung,et al.  A survey of combinatorial testing , 2011, CSUR.

[2]  J. W. Lloyd,et al.  Foundations of logic programming; (2nd extended ed.) , 1987 .

[3]  Sarfraz Khurshid,et al.  Korat: A Tool for Generating Structurally Complex Test Inputs , 2007, 29th International Conference on Software Engineering (ICSE'07).

[4]  A. Jefferson Offutt,et al.  Constraint-Based Automatic Test Data Generation , 1991, IEEE Trans. Software Eng..

[5]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[6]  Arnaud Gotlieb,et al.  Modelling Dynamic Memory Management in Constraint-Based Testing , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[7]  Elena Marchiori,et al.  UvA-DARE ( Digital Academic Repository ) Reasoning about Prolog Programs : from Modes through Types to Assertions , 2005 .

[8]  Peter J. Stuckey,et al.  Programming with Constraints: An Introduction , 1998 .

[9]  Alberto Pettorossi,et al.  Verification of Imperative Programs by Constraint Logic Program Transformation , 2013, Festschrift for Dave Schmidt.

[10]  Alberto Pettorossi,et al.  Using Real Relaxations during Program Specialization , 2011, LOPSTR.

[11]  Koichi Furukawa,et al.  Notes on Transformation Techniques for Generate and Test Logic Programs , 1987, SLP.

[12]  Rance Cleaveland,et al.  Using formal specifications to support testing , 2009, CSUR.

[13]  Lee Naish,et al.  Negation and Control in Prolog , 1986, Lecture Notes in Computer Science.

[14]  John Darlington,et al.  A Transformation System for Developing Recursive Programs , 1977, J. ACM.

[15]  Alberto Pettorossi,et al.  Unfolding - Definition - Folding, in this Order, for Avaoiding Unnecessary Variables in Logic Programs , 1991, PLILP.

[16]  Wim Vanhoof,et al.  Towards a Framework for Constraint-Based Test Case Generation , 2009, LOPSTR.

[17]  John W. Lloyd,et al.  Foundations of Logic Programming, 1st Edition , 1984 .

[18]  Richard S. Bird The promotion and accumulation strategies in transformational programming , 1984, TOPL.

[19]  Elvira Albert,et al.  Test case generation for object-oriented imperative languages in CLP , 2010, Theory Pract. Log. Program..

[20]  Hisao Tamaki,et al.  Unfold/Fold Transformation of Logic Programs , 1984, ICLP.

[21]  Sarfraz Khurshid,et al.  TestEra: A tool for testing Java programs using alloy specifications , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[22]  Sarfraz Khurshid,et al.  Test generation through programming in UDITA , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[23]  Sandro Etalle,et al.  Transformations of CLP Modules , 1996, Theor. Comput. Sci..

[24]  Alberto Pettorossi,et al.  Verifying programs via iterated specialization , 2013, PEPM '13.

[25]  Sarfraz Khurshid,et al.  Software assurance by bounded exhaustive testing , 2004, IEEE Transactions on Software Engineering.

[26]  Alberto Pettorossi,et al.  Generalization strategies for the verification of infinite state systems , 2011, Theory and Practice of Logic Programming.

[27]  Bernard Botella,et al.  A CLP Framework for Computing Structural Test Data , 2000, Computational Logic.

[28]  Alberto Pettorossi,et al.  Improving Reachability Analysis of Infinite State Systems by Specialization , 2011, Fundam. Informaticae.

[29]  Christophe Meudec ATGen: automatic test data generation using constraint logic programming and symbolic execution † , 2001, Softw. Test. Verification Reliab..

[30]  Maurice Bruynooghe,et al.  Under Consideration for Publication in Theory and Practice of Logic Programming Logic Program Specialisation through Partial Deduction: Control Issues , 2022 .

[31]  Mario Cannataro,et al.  Protein-to-protein interactions: Technologies, databases, and algorithms , 2010, CSUR.

[32]  Alberto Pettorossi,et al.  Transformation Rules for Locally Stratified Constraint Logic Programs , 2004, Program Development in Computational Logic.

[33]  J. Lloyd Foundations of Logic Programming , 1984, Symbolic Computation.

[34]  Fabio Fioravanti,et al.  Generation of Test Data Structures Using Constraint Logic Programming , 2012, TAP@TOOLS.

[35]  Bruno Legeard,et al.  A taxonomy of model‐based testing approaches , 2012, Softw. Test. Verification Reliab..

[36]  Maurice Bruynooghe,et al.  Compiling Control , 1989, J. Log. Program..

[37]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[38]  Darko Marinov,et al.  Automatic testing of software with structurally complex inputs , 2005 .