A unified framework for declarative debugging and testing

Abstract Context: Debugging is the most challenging and time consuming task in software development. However, it is not properly integrated in the software development cycle, because the result of so much effort is not available in further iterations of the cycle, and the debugging process itself does not benefit from the outcome of other phases such as testing. Objective: We propose to integrate debugging and testing within a single unified framework where each phase generates useful information for the other and the outcomes of each phase are reused. Method: We consider a declarative debugging setting that employs tests to automatically entail the validity of some subcomputations, thus decreasing the time and effort needed to find a bug. Additionally, the debugger stores as new tests the information collected from the user during the debugging phase. This information becomes part of the program test suite, and can be used in future debugging sessions, and also as regression tests. Results: We define a general framework where declarative debugging establishes a bidirectional collaboration with testing. The new setting preserves the properties of the underlying declarative debugging framework (weak completeness and soundness) while generating test cases that can be used later in other debugging sessions or even in other cycles of the software development. The proposed framework is general enough to be instantiated to very different programming languages: Erlang (functional), Java (imperative, object-oriented), and SQL (data query); and the experimental results obtained for Erlang programs validate the effectiveness of the framework. Conclusion: We propose a general unified framework for debugging and testing that simplifies each phase and maximizes the reusability of the outcomes in the different phases of the software development cycle, therefore reducing the overall effort.

[1]  Wlodzimierz Drabent,et al.  Declarative Diagnosis of Contraint Programs: An Assertion-Based Approach , 1997, AADEBUG.

[2]  C. J. Date.  A Guide to the SQL Standard , 1987 .

[3]  Peter Fritzson,et al.  Generalized algorithmic debugging and testing , 1991, LOPL.

[4]  Joe Armstrong,et al.  Concurrent programming in ERLANG , 1993 .

[5]  Adrián Riesco,et al.  A zoom-declarative debugger for sequential Erlang programs , 2015, Sci. Comput. Program..

[6]  K. Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2000, ICFP '00.

[7]  Neil Maiden,et al.  Scenarios, Stories, Use Cases: Through the Systems Development Life-Cycle , 2004 .

[8]  Adrián Riesco,et al.  A core Erlang semantics for declarative debugging , 2019, J. Log. Algebraic Methods Program..

[9]  Rafael Caballero,et al.  Declarative Debugging of Wrong and Missing Answers for SQL Views , 2012, FLOPS.

[10]  Konstantinos Sagonas,et al.  A PropEr integration of types and function specifications with property-based testing , 2011, Erlang Workshop.

[11]  Henrik Nilsson,et al.  How to look busy while being as lazy as ever: the Implementation of a lazy functional debugger , 2001, Journal of Functional Programming.

[12]  Chao Liu,et al.  Statistical Debugging: A Hypothesis Testing-Based Approach , 2006, IEEE Transactions on Software Engineering.

[13]  Adrián Riesco,et al.  Debugging Meets Testing in Erlang , 2016, TAP@STAF.

[14]  Matt Bishop,et al.  Property-based testing: a new approach to testing for assurance , 1997, SOEN.

[15]  Adrián Riesco,et al.  EDD: A Declarative Debugger for Sequential Erlang Programs , 2014, TACAS.

[16]  John Hughes,et al.  Software Testing with QuickCheck , 2009, CEFP.

[17]  Paul W. P. J. Grefen,et al.  A multi-set extended relational algebra: a formal approach to a practical issue , 1994, Proceedings of 1994 IEEE 10th International Conference on Data Engineering.

[18]  Herbert Kuchen,et al.  Algorithmic Debugging of Java Programs , 2007, WFLP.

[19]  Francesco Russo,et al.  A Declarative Debugging Environment for DATALOG , 1991, RCLP.

[20]  James Cheney,et al.  Provenance in Databases: Why, How, and Where , 2009, Found. Trends Databases.

[21]  Richard Carlsson,et al.  EUnit: a lightweight unit testing framework for Erlang , 2006, ERLANG '06.

[22]  Adrián Riesco,et al.  Declarative debugging of concurrent Erlang programs , 2018, J. Log. Algebraic Methods Program..

[23]  Marc J. Balcer,et al.  The category-partition method for specifying and generating fuctional tests , 1988, CACM.

[24]  Josep Silva,et al.  Algorithmic debugging generalized , 2018, J. Log. Algebraic Methods Program..

[25]  Norman Matloff,et al.  The Art of Debugging with GDB, DDD, and Eclipse , 2008 .

[26]  Lee Naish.  A Declarative Debugging Scheme , 1997, J. Funct. Log. Program..

[27]  Gabriella Kókai,et al.  Algorithmic Debugging and Testing of Prolog Programs , 1997, LPE.

[28]  Adrián Riesco,et al.  A Survey of Algorithmic Debugging , 2017, ACM Comput. Surv..

[29]  Per Runeson,et al.  A survey of unit testing practices , 2006, IEEE Software.

[30]  Rafael Caballero,et al.  Debugging of wrong and missing answers for datalog programs with constraint handling rules , 2015, PPDP.

[31]  Thom W. Frühwirth,et al.  Theory and Practice of Constraint Handling Rules , 1998, J. Log. Program..

[32]  Josep Silva,et al.  An algorithmic debugger for Java , 2010, 2010 IEEE International Conference on Software Maintenance.

[33]  Josep Silva,et al.  Implementation of an Optimal Strategy for Algorithmic Debugging , 2012, Electron. Notes Theor. Comput. Sci..

[34]  Josep Silva.  A survey on algorithmic debugging strategies , 2011, Adv. Eng. Softw..

[35]  John Hughes,et al.  Testing telecoms software with quviq QuickCheck , 2006, ERLANG '06.

[36]  Mark Harman,et al.  The Oracle Problem in Software Testing: A Survey , 2015, IEEE Transactions on Software Engineering.