Answer-Set Programming as a new Approach to Event-Sequence Testing

In many applications, faults are triggered by events that occur in a particular order. Based on the assumption that most bugs are caused by the interaction of a low number of events, Kuhn et al. recently introduced sequence covering arrays (SCAs) as suitable designs for event sequence testing. In practice, directly applying SCAs for testing is often impaired by additional constraints, and SCAs have to be adapted to fit application-specific needs. Modifying precomputed SCAs to account for problem variations can be problematic, if not impossible, and developing dedicated algorithms is costly. In this paper, we propose answer-set programming (ASP), a well-known knowledge-representation formalism from the area of artificial intelligence based on logic programming, as a declarative paradigm for computing SCAs. Our approach allows to concisely state complex coverage criteria in an elaboration tolerant way, i.e., small variations of a problem specification require only small modifications of the ASP representation.

[1]  Martin Gebser,et al.  GrinGo : A New Grounder for Answer Set Programming , 2007, LPNMR.

[2]  Yuanyuan Zhou,et al.  Learning from mistakes: a comprehensive study on real world concurrency bug characteristics , 2008, ASPLOS.

[3]  Miroslaw Truszczynski,et al.  The Second Answer Set Programming Competition , 2009, LPNMR.

[4]  Timo Soininen,et al.  Extending and implementing the stable model semantics , 2000, Artif. Intell..

[5]  Martin Gebser,et al.  Conflict-Driven Answer Set Solving , 2007, IJCAI.

[6]  Brian A. Malloy,et al.  Data flow testing of parallelized code , 1992, Proceedings Conference on Software Maintenance 1992.

[7]  Kari J. Nurmela,et al.  Upper bounds for covering arrays by tabu search , 2004, Discret. Appl. Math..

[8]  Charles J. Colbourn,et al.  Prioritized interaction testing for pair-wise coverage with seeding and constraints , 2006, Inf. Softw. Technol..

[9]  Steven David Prestwich,et al.  Constraint Models for the Covering Test Problem , 2006, Constraints.

[10]  Myra B. Cohen,et al.  Interaction testing of highly-configurable systems in the presence of constraints , 2007, ISSTA '07.

[11]  Angelo Gargantini,et al.  A Formal Logic Approach to Constrained Combinatorial Testing , 2010, Journal of Automated Reasoning.

[12]  Agostino Dovier,et al.  An empirical study of constraint logic programming and answer set programming solutions of combinatorial problems , 2009, J. Exp. Theor. Artif. Intell..

[13]  Jeff Yu Lei,et al.  Combinatorial Methods for Event Sequence Testing , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[14]  A. Jefferson Offutt,et al.  Combination testing strategies: a survey , 2005, Softw. Test. Verification Reliab..

[15]  Marius Thomas Lindauer,et al.  Potassco: The Potsdam Answer Set Solving Collection , 2011, AI Commun..

[16]  Yu Lei,et al.  Practical Combinatorial Testing , 2010 .

[17]  Chitta Baral,et al.  Knowledge Representation, Reasoning and Declarative Problem Solving , 2003 .

[18]  Myra B. Cohen,et al.  Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach , 2008, IEEE Transactions on Software Engineering.

[19]  Hareton K. N. Leung,et al.  A survey of combinatorial testing , 2011, CSUR.

[20]  Katsumi Inoue,et al.  Generating Combinatorial Test Cases by Efficient SAT Encodings Suitable for CDCL SAT Solvers , 2010, LPAR.

[21]  Shan Lu,et al.  A study of interleaving coverage criteria , 2007, ESEC-FSE companion '07.

[22]  Michael L. Fredman,et al.  The AETG System: An Approach to Testing Based on Combinatiorial Design , 1997, IEEE Trans. Software Eng..

[23]  Myra B. Cohen,et al.  Constructing test suites for interaction testing , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..