The ContractLog Approach Towards Test-driven Verification and Validation of Rule Bases-A Homogeneous Integration of Test Cases and Integrity Constraints into Evolving Logic Programs and Rule Markup Languages (RuleML)

Rules are often being used as a (declarative) programming language to describe real-world decision logic such as business rules or contract/policy logic and create production systems upon (see e.g. RBSLA/ContractLog project hosted at Sourceforge). For this reason, it is important to support testing mechanisms, that can help rule programmers to determine the reliability of the results produced by their rule systems. Different approaches and methodologies to verification and validation (V&V) of rule-based systems have been proposed in the literature. Simple operational debugging approaches which instrument the rule system and explore its execution trace using e.g., spy and trace commands, place a huge cognitive load on the user, who needs to analyze each step of the inference process. On the other hand, typical heavy-weight validation methodologies such as waterfall-based approaches are often not suitable for rule-based systems, because they induce high costs of change and do not facilitate evolutionary modelling of rules with collaborations of different roles such as domain experts, system developers and knowledge engineers. In this paper, we adapt a recent trend in software engineering (SE), namely test driven development, to define self validating rule bases in the domain of logic programming, one of the most prominent and successful approaches in declarative rule based programming. We argue that test cases, an agile development methodology of Extreme Programming (XP), have a huge potential to be a successful tool for declarative verification and validation of LP-based rule bases. Test cases in combination with other XP related techniques such as test coverage measurement or evolving rule base refinements (by applying refactorings which improve the existing rule code, e.g., via removing inconsistencies, redundancy or missing knowledge without breaking its functionality) are suitable for typically frequently changing requirements and models of rule-based projects. Due to their inherent simplicity test cases better facilitate rule interchange in distributed applications and support different roles which are involved during the rule engineering process. The presence of test cases safeguards the life cycle of rules, e.g. enabling V&V at design time 2 Verification and Validation of Rule Bases but also dynamic testing of (transactional) self-updates of the extensional facts and intensional rules at runtime. Here the test cases are used as highly expressive integrity constraints, e.g. as post conditional tests in ECA(P) rules. If the post condition test in an ECAP rule fails the update action which adds or removes rules or facts from the KB is completely ”rolled back”. Another very interesting application domain of test cases for logic programs (LPs) is rule interchange. Test cases can be used to ensure correct execution of an interchanged LP in a target execution environment by validating the LP in the target inference engine with the attached and interchanged test cases. Meta annotations of the inference engine and the interchanged LPs disclosing the supported/intended semantics (e.g. well-founded semantics, stable model semantics etc.) of both the engine and the program together with appropriate test case variants for different semantics can be used to decide wether the interchanged rule base can be executed correctly in the target rule system. In case such annotations are not given, test cases with meta test programs (meta test rules) as assertions allow analyzing typical properties of LP based semantics. An adequate combination of meta tests for semantic properties uniquely characterizes a particular semantics and hence allows automatic determination of the semantics supported by the target inference engine. We show that test cases can be represented in a homogeneous LP representation language. The major advantage of this homogenous integration is, that the test cases and the test suites can be managed, maintained and executed within the same rule based environment, based on a welldefined declarative LP-related semantical basis. We further investigate, how the concept of test coverage from XP can be adapted to logic programming, in order to quantify the quality of test cases for V&V of a particular LP and use the novel notion of test coverage to give feedback and hints on how to optimize and refine the test cases and the rule code in an iterative process. We discuss the implementation of these ideas in the ContracLog KR and present concrete syntax and semantics of test cases for general LPs. We further extend this approach to support welladopted SE test frameworks like JUnit [23] and to facilitate a very tight integration of tests into existing IDEs such as Eclipse and tools such as Ant. In addition to the representation of test cases in a Prolog-like scripting syntax, we introduce a markup serialization as an extension to the emerging Semantic Web Rule Markup Language RuleML.