Testing Methods for Decision Support Systems

Decision support systems (DSS) have proved to be efficient for helping humans to make a decision in various domains such as health (Dorr et al., 2007). However, before being used in practice, these systems need to be extensively evaluated to ensure their validity and their efficiency. DSS evaluation usually includes two steps: first, testing the DSS under controlled conditions, and second, evaluating the DSS in real use, during a randomised trial. In this chapter, we will focus on the first step. The test of decision support systems uses various methods aimed at detecting errors in a DSS without having to use the DSS under real use conditions; several of these methods were initially developed in the field of expert systems, or software testing (Meyer, 2008). DSS testing methods are usually classified in two categories (Preece, 1994): •

[1]  Johanna D. Moore,et al.  Detecting Knowledge Base Inconsistencies Using Automated Generation of Text and Examples , 1996, AAAI/IAAI, Vol. 1.

[2]  Moonis Ali,et al.  Multiple Approaches to Intelligent Systems , 1999, Lecture Notes in Computer Science.

[3]  A. Preece.  Building the Right System Right Evaluating V & V Methods in Knowledge Engineering , 1998 .

[4]  Alun D. Preece,et al.  Validation of Knowledge-Based Systems: The State-of-the-Art in North America , 1994 .

[5]  Bertrand Meyer,et al.  Seven Principles of Software Testing , 2008, Computer.

[6]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[7]  Amy N. Cohen,et al.  Review paper: Informatics Systems to Promote Improved Care for Chronic Illness: A Literature Review , 2007, J. Am. Medical Informatics Assoc..

[8]  Silvia Miksch,et al.  Knowledge-based verification of clinical guidelines by detection of anomalies , 2001, Artif. Intell. Medicine.

[9]  Alun D. Preece,et al.  Foundation and application of knowledge base verification , 1994, Int. J. Intell. Syst..

[10]  Silvia Miksch,et al.  Verification of temporal scheduling constraints in clinical practice guidelines , 2002, Artif. Intell. Medicine.

[11]  Amparo Alonso-Betanzos,et al.  Empirical evaluation of a hybrid intelligent monitoring system using different measures of effectiveness , 2002, Artif. Intell. Medicine.

[12]  Rodger Knaus,et al.  VERIFICATION, VALIDATION, AND EVALUATION OF EXPERT SYSTEMS. VOLUME 1 : AN FHWA HANDBOOK , 1997 .

[13]  Ruddy Lelouche,et al.  Test Case Generation using KBS Strategy , 1993, IJCAI.

[14]  Alicia Perez,et al.  Evaluation of Taxonomic Knowledge in Ontologies and Knowledge Bases , 1999 .

[15]  Jean-Daniel Zucker,et al.  Use of the C4.5 machine learning algorithm to test a clinical guideline-based decision support system , 2008, MIE.

[16]  A. McCray,et al.  Yearbook of Medical Informatics , 2013, Yearbook of Medical Informatics.

[17]  E Coiera,et al.  Section 1: Health and Clinical Mangement: The Safety and Quality of Decision Support Systems , 2006, Yearbook of Medical Informatics.

[18]  Richard S. Sojda,et al.  Empirical evaluation of decision support systems: Needs, definitions, potential methods, and an example pertaining to waterfowl management , 2007, Environ. Model. Softw..