On the Small-Scope Hypothesis for Testing Answer-Set Programs

In software testing, the small-scope hypothesis states that a high proportion of errors can be found by testing a program for all test inputs within some small scope. In this paper, we evaluate the small-scope hypothesis for answer-set programming (ASP). To this end, we follow work in traditional testing and base our evaluation on mutation analysis. In fact, we show that a rather limited scope is sufficient for testing ASP encodings from a representative set of benchmark problems. Our experimental evaluation facilitates effective methods for testing in ASP. Also, it gives some justification to analyse programs at the propositional level after grounding them over a small domain.

[1]  A Jeeerson Ooutt,et al.  Subsumption of Condition Coverage Techniques by Mutation Testing , 1996 .

[2]  Tomi Janhunen,et al.  A Translation-based Approach to the Verification of Modular Equivalence , 2009, J. Log. Comput..

[3]  Hans Tompits,et al.  Random vs. Structure-Based Testing of Answer-Set Programs: An Experimental Comparison , 2011, LPNMR.

[4]  Martin Gebser,et al.  On the Input Language of ASP Grounder Gringo , 2009, LPNMR.

[5]  Bart Selman,et al.  Model Counting: A New Strategy for Obtaining Good Bounds , 2006, AAAI.

[6]  Gregg Rothermel,et al.  An experimental evaluation of selective mutation , 1993, Proceedings of 1993 15th International Conference on Software Engineering.

[7]  Alexander A. Razborov,et al.  Why are there so many loop formulas? , 2006, TOCL.

[8]  Stefan Woltran,et al.  Testing Relativised Uniform Equivalence under Answer-Set Projection in the System ccT , 2007, INAP/WLP.

[9]  Wolfgang Faber,et al.  The Diagnosis Frontend of the dlv System , 1999, AI Commun..

[10]  Nicola Leone,et al.  Unit Testing in ASPIDE , 2011, INAP/WLP.

[11]  Vladimir Lifschitz,et al.  Weight constraints as nested expressions , 2003, Theory and Practice of Logic Programming.

[12]  Martin Gebser,et al.  GrinGo : A New Grounder for Answer Set Programming , 2007, LPNMR.

[13]  Esra Erdem,et al.  Inferring Phylogenetic Trees Using Answer Set Programming , 2007, Journal of Automated Reasoning.

[14]  Daniel Jackson,et al.  Elements of style: analyzing a software design feature with a counterexample detector , 1996, ISSTA '96.

[15]  Marina De Vos,et al.  TOAST: Applying Answer Set Programming to Superoptimisation , 2006, ICLP.

[16]  K. N. King,et al.  A fortran language system for mutation‐based software testing , 1991, Softw. Pract. Exp..

[17]  Chitta Baral,et al.  Reasoning agents in dynamic domains , 2000 .

[18]  Oded Shmueli.  Decidability and Expressiveness of Logic Queries. , 1987, PODS 1987.

[19]  Timo Soininen,et al.  Extending and implementing the stable model semantics , 2000, Artif. Intell..

[20]  Torsten Schaub,et al.  Modelling Biological Networks by Action Languages Via Answer Set Programming , 2006, ICLP.

[21]  Esra Erdem,et al.  Temporal phylogenetic networks and logic programming , 2005, Theory and Practice of Logic Programming.

[22]  Hans Tompits,et al.  On Testing Answer-Set Programs , 2010, ECAI.

[23]  Ilkka Niemelä,et al.  Developing a Declarative Rule Language for Applications in Product Configuration , 1999, PADL.

[24]  A. Jefferson Offutt,et al.  Mutation Operators for Ada , 1996 .

[25]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[26]  John A. Clark,et al.  The Rigorous Generation of Java Mutation Operators Using HAZOP , 1999 .

[27]  Wynne Hsu,et al.  DESIGN OF MUTANT OPERATORS FOR THE C PROGRAMMING LANGUAGE , 2006 .

[28]  Alexandr Andoni,et al.  Evaluating the “ Small Scope Hypothesis ” , 2002 .

[29]  Fangzhen Lin,et al.  ASSAT: computing answer sets of a logic program by SAT solvers , 2002, Artif. Intell..

[30]  Stefan Woltran,et al.  Facts Do Not Cease to Exist Because They Are Ignored: Relativised Uniform Equivalence with Answer-Set Projection , 2007, AAAI.

[31]  Michael Gelfond,et al.  An A Prolog decision support system for the Space Shuttle , 2001, Answer Set Programming.

[32]  Mario Alviano,et al.  The Third Answer Set Programming Competition: Preliminary Report of the System Competition Track , 2011, LPNMR.

[33]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[34]  Oded Shmueli,et al.  Decidability and expressiveness aspects of logic queries , 1987, XP7.52 Workshop on Database Theory.

[35]  Wolfgang Faber,et al.  Planning under Incomplete Knowledge , 2000, Computational Logic.

[36]  Stefan Woltran,et al.  ccT on Stage: Generalised Uniform Equivalence Testing for Verifying Student Assignment Solutions , 2009, LPNMR.

[37]  Tomi Janhunen,et al.  LPEQ and DLPEQ - Translators for Automated Equivalence Testing of Logic Programs , 2004, LPNMR.

[38]  Sarfraz Khurshid,et al.  An Evaluation of Exhaustive Testing for Data Structures , 2003 .

[39]  Axel Polleres.  Semantic Web Languages and Semantic Web Services as Application Areas for Answer Set Programming , 2005, Nonmonotonic Reasoning, Answer Set Programming and Constraints.