How the testing techniques for a decision support system changed over eight years

Since 1985 we have been developing a decision support system to help speech clinicians with the diagnosis and clinical management of small children who may have begun to stutter. This paper describes how our testing of the system has evolved over these eight years. The test measures are described and the conclusion shows that human experts are less consistent than the decision support system and that humans are often biased.<<ETX>>

[1]  R D Yee,et al.  On-line analysis of eye movements using a digital computer. , 1980, Aviation, space, and environmental medicine.

[2]  Reinhold Kliegl,et al.  Automated and interactive analysis of eye fixation data in reading , 1981 .

[3]  Margaret C. Thompson,et al.  Monitoring animals’ movements using digitized video images , 1988 .

[4]  Ritu Agarwal,et al.  Formal validation of a knowledge-based system using a variation of the Turing test , 1993 .

[5]  K. Bharathan,et al.  Knowledge extraction changes the way an expert thinks , 1992, [Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics.

[6]  J Merchant,et al.  Remote measurement of eye direction allowing subject motion over one cubic foot of space. , 1974, IEEE transactions on bio-medical engineering.

[7]  A. Terry Bahill,et al.  Interactive verification of knowledge-based systems , 1993, IEEE Expert.

[8]  A. Terry Bahill,et al.  Verifying and Validating Personal Computer-Based Expert Systems , 1991 .

[9]  Leonard Adelman,et al.  Evaluating decision support and expert systems , 1991 .

[10]  G. Barnes.  A procedure for the analysis of nystagmus and other eye movements. , 1981, Aviation, space, and environmental medicine.

[11]  Lawrence W. Stark,et al.  Eye monitor: microcomputer-based instrument uses an internal mode to track the eye , 1991, Computer.

[12]  Hanqi Zhuang,et al.  Corner detection by a cost minimization approach , 1993, Pattern Recognit..

[13]  Marc Ayel,et al.  Validation, verification and test of knowledge-based systems , 1991 .

[14]  L. Young,et al.  Survey of eye movement recording methods , 1975 .

[15]  Mark Chignell,et al.  Intelligent database tools and applications - hyperinformation access, data quality, visualization, automatic discovery , 1993, Wiley professional computing.

[16]  Mark S. Nixon,et al.  Eye Spacing Measurement for Facial Recognition , 1985, Optics & Photonics.

[17]  Kamran Parsaye.  Acquiring & verifying knowledge , 1988 .

[18]  Hanqi Zhuang,et al.  On improving eye feature extraction using deformable templates , 1994, Pattern Recognit..

[19]  Andrew P. Sage.  Decision support systems engineering , 1991 .

[20]  Uma G. Gupta.  Validating and Verifying Knowledge-Based Systems , 1991 .

[21]  Hiroyuki Nakamura,et al.  Design of eye movement monitoring system for practical environment , 1991, Medical Imaging.

[22]  Suzanne Smith,et al.  Verification and validation of rule-based expert systems , 1993 .

[23]  Yue Kang,et al.  A tool for detecting expert-system errors , 1989 .

[24]  Stephen J. Andriole,et al.  Software validation, verification, testing, and documentation , 1986 .