Cognitive Task Analysis: An Approach to Knowledge Acquisition for Intelligent System Design

Publisher Summary This chapter discusses some of the common pitfalls that arise in building intelligent support systems and describe a pragmatic knowledge acquisition approach for defining and building effective intelligent support systems. The cognitive task analysis provides an umbrella structure of domain semantics that organizes and makes explicit what particular pieces of knowledge mean about problem-solving in the domain. Acquiring and using such a domain semantics is essential (l) to specify what kinds of cognitive support functions are needed, (2) to specify what kinds of computational mechanisms are capable of providing such functions, (3) to clearly delineate machine performance boundaries, and (4) to build less brittle machine problem-solvers, for example, through features that enable the human problem-solver to extend and adapt the capability of the system to handle unanticipated situations. This is in contrast to technology-driven approaches where knowledge acquisition focuses on describing domain knowledge in terms of the syntax of particular computational mechanisms. In other words, the language of implementation is used as a substitute for a cognitive language of description. The cognitive task analysis approach redefines the knowledge acquisition problem: knowledge acquisition, first, is about deciding what kinds of intelligent systems would make a difference and, second, about what domain specific knowledge is needed to fuel those systems.

[1]  Benjamin Kuipers,et al.  How to Discover a Knowledge Representation for Causal Reasoning by Studying an Expert Physician , 1983, IJCAI.

[2]  Jacques Leplat,et al.  Evaluation of Different Modalities of Verbalization in a Sorting Task , 1983, Int. J. Man Mach. Stud..

[3]  J. C. Flanagan Psychological Bulletin THE CRITICAL INCIDENT TECHNIQUE , 2022 .

[4]  James L. Alty,et al.  Face-to-face guidance of university computer users—II. Characterizing advisory interactions , 1980 .

[5]  John M. Carroll,et al.  The answer is in the question: a protocol study of intelligent help , 1987 .

[6]  James B. Moen,et al.  Garden Path Errors in Diagnostic Reasoning , 1988 .

[7]  William J. Clancey Methodology for building an intelligent tutoring system , 1987 .

[8]  J. R. Miller,et al.  Justified advice: a semi-naturalistic study of advisory strategies , 1988, CHI '88.

[9]  Larry J. Eshelman,et al.  MOLE: A Tenacious Knowledge-Acquisition Tool , 1987, Int. J. Man Mach. Stud..

[10]  William J. Clancey,et al.  Heuristic Classification , 1986, Artif. Intell..

[11]  John M. Carroll,et al.  Interface design issues for advice-giving expert systems , 1987, CACM.

[12]  Emilie M. Roth,et al.  The effect of context on the structure of categories , 1983, Cognitive Psychology.

[13]  Erik Hollnagel,et al.  Mapping Cognitive Demands in Complex Problem-Solving Worlds , 1987, Int. J. Man Mach. Stud..

[14]  Christine M. Mitchell,et al.  A Discrete Control Model of Operator Function: A Methodology for Information Display Design , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[15]  James L. Alty,et al.  Expert Systems: An Alternative Paradigm , 1984, Int. J. Man Mach. Stud..

[16]  Daniel Kahneman,et al.  Availability: A heuristic for judging frequency and probability , 1973 .

[17]  H. H. Clark,et al.  References in Conversation Between Experts and Novices , 1987 .

[18]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[19]  J. Alty,et al.  Face-to-face guidance of university computer users—I. A study of advisory services , 1980 .

[20]  Jeffrey M. Bradshaw,et al.  Expertise Transfer and Complex Problems: Using AQUINAS as a Knowledge-Acquisition Workbench for Knowledge-Based Systems , 1987, Int. J. Man Mach. Stud..

[21]  D. Broadbent,et al.  Explanation and Verbalization in a Computer-Assisted Search Task , 1987 .

[22]  Allen Newell,et al.  The Knowledge Level , 1989, Artif. Intell..

[23]  R. Glaser,et al.  Expertise in a complex skill: Diagnosing x-ray pictures. , 1988 .

[24]  David Perkins,et al.  Fragile knowledge and neglected strategies in novice programmers , 1985 .

[25]  E. M. Roth,et al.  Toward Joint Person-Machine Cognitive Systems: A Prototype Expert System for Electronics Troubleshooting , 1985 .

[26]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[27]  Neville Moray Intelligent Aids, Mental Models, and the Theory of Machines , 1987, Int. J. Man Mach. Stud..

[28]  L. Quinn,et al.  Intelligent interfaces: user models and planners , 1986, CHI '86.

[29]  Gary S. Kahn,et al.  A Mixed-Initiative Workbench for Knowledge Acquisition , 1987, Int. J. Man Mach. Stud..

[30]  Kevin B. Bennett,et al.  Human Interaction with an "Intelligent" Machine , 1987, Int. J. Man Mach. Stud..

[31]  Perry L. Miller,et al.  ATTENDING: Critiquing a Physician's Management Plan , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  John D. Bransford,et al.  Teaching thinking and problem solving: Research foundations. , 1986 .

[33]  Edward H. Shortliffe,et al.  Adapting a Consultation System to Critique User Plans , 1983, Int. J. Man Mach. Stud..

[34]  Robert L. Osborne Online, Artificial Intelligence-Based Turbine Generator Diagnostics , 1986, AI Mag..

[35]  J. B. Brooke,et al.  Effects of system display format on performance in a fault location task , 1981 .

[36]  J Leplat,et al.  The elicitation of expert knowledge , 1986 .

[37]  D D Woods,et al.  Paradigms for intelligent decision support , 1986 .

[38]  J. McKendree,et al.  Advising roles of a computer consultant , 1986, CHI '86.

[39]  David S. Prerau,et al.  Knowledge acquisition in expert system development , 1987 .

[40]  Robert R. Hoffman,et al.  The Problem of Extracting the Knowledge of Experts from the Perspective of Experimental Psychology , 1987, AI Mag..

[41]  B. Kuipers,et al.  Critical Decisions under Uncertainty: Representation and Structure , 1990, Cogn. Sci..

[42]  John P. McDermott,et al.  R1 Revisited: Four Years in the Trenches , 1984, AI Mag..

[43]  D. Woods Coping with complexity: the psychology of human behaviour in complex systems , 1988 .

[44]  Nicholas J. Belkin,et al.  Knowledge Elicitation Using Discourse Analysis , 1987, Int. J. Man Mach. Stud..

[45]  John H. Boose,et al.  Personal Construct Theory and the Transfer of Human Expertise , 1984, AAAI.

[46]  David Hawkins,et al.  An Analysis of Expert Thinking , 1983, Int. J. Man Mach. Stud..

[47]  Paul R. Cohen,et al.  Design for Acquisition: Principles of Knowledge-System Design to Facilitate Knowledge Acquisition , 1987, Int. J. Man Mach. Stud..

[48]  Dorothea P. Simon,et al.  Expert and Novice Performance in Solving Physics Problems , 1980, Science.

[49]  E. Rosch,et al.  Family resemblances: Studies in the internal structure of categories , 1975, Cognitive Psychology.

[50]  Dianne C. Berry,et al.  Expert systems and the man–machine interface , 1986 .

[51]  Baruch Fischhoff,et al.  Judgment under uncertainty: For those condemned to study the past: Heuristics and biases in hindsight , 1982 .

[52]  Julia Hirschberg,et al.  User Participation in the Reasoning Processes of Expert Systems , 1982, AAAI.

[53]  E. Loftus,et al.  Reconstruction of automobile destruction: An example of the interaction between language and memory , 1974 .

[54]  M. J. Loftus,et al.  The Problem of Explanation: Placing Computer Generated Answers in Context , 1985 .