Learning to Troubleshoot: A New Theory-Based Design Architecture

Troubleshooting is a common form of problem solving. Technicians (e.g., automotive mechanics, electricians) and professionals (physician, therapists, ombudspersons) diagnose faulty systems and take direct, corrective action to eliminate any faults in order to return the systems to their normal states. Traditional approaches to troubleshooting instruction have emphasized either theoretical or domain knowledge about the system or specific troubleshooting procedures. These methods have failed to develop transferable troubleshooting skills in learners. In this article, we propose an architecture for designing learning environments for troubleshooting. The architecture integrates experiential, domain, and device knowledge in a learning system that enables learners to generate and test hypotheses for every action they take, relate every action to a conceptual model of the system, and query experienced troubleshooters about what they would do. The architecture includes three essential components: A multi-layered conceptual model of the system that includes topographic, function, strategic, and procedural representations; a simulator that requires the learner to generate hypotheses, reconcile the hypotheses to the system mode, test the hypotheses, and interpret the results from the test; and a case library that uses a case-based reasoning engine to access relevant stories of troubleshooting experiences as advice for the learner. This novel architecture can be used to develop learning environments for different kinds of troubleshooting.

[1]  John A. Allen,et al.  The effects of network size and fault intermittency on troubleshooting performance , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[2]  A. Collins,et al.  Situated Cognition and the Culture of Learning , 1989 .

[3]  Janet L. Kolodner,et al.  Case-Based Reasoning , 1989, IJCAI 1989.

[4]  P. Thagard,et al.  Coherence in Thought and Action , 2000 .

[5]  Denis Besnard,et al.  Expert error in trouble-shooting: an exploratory study in electronics , 1999, Int. J. Hum. Comput. Stud..

[6]  D. Bobrow.  Qualitative Reasoning about Physical Systems , 1985 .

[7]  John Seely Brown,et al.  SOPHIE: A Step Toward Creating a Reactive Learning Environment , 1975, Int. J. Man Mach. Stud..

[8]  F. Paas,et al.  Cognitive Load Theory and Instructional Design: Recent Developments , 2003 .

[9]  David H. Jonassen,et al.  Case-based reasoning and instructional design: Using stories to support problem solving , 2002 .

[10]  Timothy E Hewett,et al.  Taking the load off. , 2004, Rehab management.

[11]  Philip Henning,et al.  Mental Models: Knowledge in the Head and Knowledge in the World , 1996, ICLS.

[12]  Nancy J. Cooke,et al.  Measuring mental models: Choosing the right tools for the job , 1995 .

[13]  R. Sternberg,et al.  Complex Problem Solving : Principles and Mechanisms , 1992 .

[14]  L. Shulman,et al.  Medical Problem Solving: An Analysis of Clinical Reasoning , 1978 .

[15]  R. Mayer,et al.  Nine Ways to Reduce Cognitive Load in Multimedia Learning , 2003 .

[16]  David J. Bryant,et al.  Spatial mental models from descriptions , 1994 .

[17]  J. Patrick.  COGNITIVE ASPECTS OF FAULT-FINDING TRAINING AND TRANSFER* , 1993 .

[18]  Robert W. Swezey,et al.  Readings in training and simulation : a 30-year perspective , 2001 .

[19]  Nancy J. Cooke,et al.  Toward an on-line knowledge assessment methodology: Building on the relationship between knowing and doing , 1996 .

[20]  Henk G. Schmidt,et al.  On the Role of Biomedical Knowledge in Clinical Reasoning by Experts, Intermediates and Novices , 1992, Cogn. Sci..

[21]  Ok-choon Park,et al.  Selective use of animation and feedback in computer-based instruction , 1992 .

[22]  Joseph Psotka,et al.  Intelligent tutoring systems : lessons learned , 1988 .

[23]  Susanne P. Lajoie,et al.  Dynamic Assessment of Proficiency for Solving Procedural Knowledge Tasks , 1992 .

[24]  J. Kent Davis,et al.  Field Independence and Hypothesis Testing , 1976 .

[25]  K. A. Ericsson,et al.  Toward a general theory of expertise : prospects and limits , 1991 .

[26]  Udo Konradt,et al.  Strategies of failure diagnosis in computer-controlled manufacturing systems: empirical analysis and implications for the design of adaptive decision support system , 1995, Int. J. Hum. Comput. Stud..

[27]  J. Sweller,et al.  Structuring Effective Worked Examples , 1990 .

[28]  Jan Maarten Schraagen,et al.  Cognitive Task Analysis and Innovation of Training: The Case of Structured Troubleshooting , 2000, Hum. Factors.

[29]  Dorothea P. Simon,et al.  Expert and Novice Performance in Solving Physics Problems , 1980, Science.

[30]  K. J. J. Gilhooly,et al.  Human and Machine Problem Solving , 1989 .

[31]  David H. Jonassen,et al.  Causal Modelling for Structuring Case-Based Learning Environments , 1996, Intell. Tutoring Media.

[32]  Randall Davis,et al.  Reasoning from First Principles in Electronic Troubleshooting , 1983, Int. J. Man Mach. Stud..

[33]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[34]  B. Tversky.  Spatial mental models , 1991 .

[35]  Donald A. Schön.  The reflective practitioner : how professionals think in action , 1986 .

[36]  Jeffrey W. Flesher,et al.  An Exploration of Technical Troubleshooting Expertise in Design, Manufacturing, and Repair Contexts. , 1993 .

[37]  J. Sweller,et al.  Learning to Solve Compare Word Problems: The Effect of Example Format and Generating Self-Explanations , 1998 .

[38]  David H. Jonassen,et al.  Toward a design theory of problem solving , 2000 .

[39]  Randall T. MacPherson.  Factors Affecting Technological Trouble Shooting Skills. , 1998 .

[40]  Robert Glaser,et al.  A naturalistic study of transfer: adaptive expertise in technical domains , 1996 .

[41]  Sherrie P. Gott,et al.  A Procedural Guide to Cognitive Task Analysis: The PARI Methodology. , 1995 .

[42]  Scott D. Johnson.  Understanding Troubleshooting Styles To Improve Training Methods. , 1995 .

[43]  Fred Paas,et al.  High versus low contextual interference in simulation-based training of troubleshooting skills: effects on transfer performance and invested mental effort , 1998 .

[44]  C McGuinness,et al.  Problem representation: The effects of spatial arrays , 1986, Memory & cognition.

[45]  P. Kirschner,et al.  Taking the Load Off a Learner's Mind: Instructional Design for Complex Learning , 2003 .

[46]  L. Buffardi,et al.  Maintenance Training Simulator Fidelity and Individual Differences in Transfer of Training , 1986 .

[47]  David H. Jonassen,et al.  The effects of case libraries on problem solving , 2003, J. Comput. Assist. Learn..

[48]  R. Pokorny,et al.  Analyzing components of work samples to evaluate performance. , 1996, Military psychology : the official journal of the Division of Military Psychology, American Psychological Association.

[49]  Susan R. Bereiter,et al.  A field-based study of troubleshooting in computer-controlled manufacturing systems , 1989, IEEE Trans. Syst. Man Cybern..

[50]  Joseph Psotka,et al.  Intelligent Instruction by Computer: Theory and Practice. , 1992 .

[51]  Sandra H. Rouse,et al.  A Rule-Based Model of Human Problem Solving Performance in Fault Diagnosis Tasks , 1980, IEEE Transactions on Systems, Man, and Cybernetics.

[52]  Valerie L. Shalin,et al.  Cognitive task analysis , 2000 .

[53]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[54]  M. Chi,et al.  The Content of Physics Self-Explanations , 1991 .

[55]  Norman Frederiksen,et al.  Implications of Cognitive Theory for Instruction in Problem Solving , 1984 .

[56]  Scott D. Johnson,et al.  The Effect of Functional Flow Diagrams on Apprentice Aircraft Mechanics' Technical System Understanding , 2008 .

[57]  William B. Rouse,et al.  Review and Evaluation of Empirical Research in Troubleshooting , 1985 .

[58]  Scott D. Johnson.  Cognitive Analysis of Expert and Novice Troubleshooting Performance , 2008 .

[59]  D. Schoen,et al.  The Reflective Practitioner: How Professionals Think in Action , 1985 .

[60]  Barbara Means,et al.  Cognitive task analysis as a basis for tutor development: Articulating abstract knowledge representations. , 1988 .

[61]  R. A. Tarmizi,et al.  Guidance during Mathematical Problem Solving. , 1988 .

[62]  J. Sweller,et al.  Effects of schema acquisition and rule automation on mathematical problem-solving transfer. , 1987 .

[63]  J. Sweller,et al.  The Use of Worked Examples as a Substitute for Problem Solving in Learning Algebra , 1985 .

[64]  Matthew W. Lewis,et al.  Self-Explonations: How Students Study and Use Examples in Learning to Solve Problems , 1989, Cogn. Sci..

[65]  D. Gaba.  Dynamic Decision-Making in Anesthesiology: Cognitive Models and Training Approaches , 1992 .

[66]  William B. Rouse,et al.  Training Maintenance Technicians for Troubleshooting: Two Experiments with Computer Simulations , 1982 .

[67]  Jens Rasmussen,et al.  Information Processing and Human-Machine Interaction: An Approach to Cognitive Engineering , 1986 .

[68]  R. Evans,et al.  Reliability of the "Draw-a-Man" test. , 1975 .

[69]  Jean-Michel Hoc.  A Method to Describe Human Diagnostic Strategies in Relation to the Design of Human-Machine Cooperation , 2000 .

[70]  R. Atkinson,et al.  Structuring the Transition From Example Study to Problem Solving in Cognitive Skill Acquisition: A Cognitive Load Perspective , 2003 .

[71]  William B. Johnson,et al.  Modeling student performance in diagnostic tasks: A decade of evolution , 1992 .

[72]  S. Chipman,et al.  Cognitively diagnostic assessment , 1995 .

[73]  John Sweller,et al.  Cognitive load theory and instructional design , 1997 .

[74]  Christopher K. Riesbeck,et al.  Experience, Memory and Reasoning , 1986 .

[75]  Scott D. Johnson,et al.  Enhancing Electrical Troubleshooting Skills in a Computer-Coached Practice Environment , 1993, Interact. Learn. Environ..

[76]  David E. Kieras,et al.  The Role of a Mental Model in Learning to Operate a Device. , 1984 .

[77]  Aidan P. Moran,et al.  Field Independence and Proficiency in Electrical Fault Diagnosis , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[78]  Maddy D. Brouwer-Janse,et al.  Human-Machine Communication for Educational Systems Design , 1994 .

[79]  Derek Hodson,et al.  Setting the Stage for Developing Pre‐service Teachers’ Conceptions of Good Science Teaching: The role of classroom videos , 2006 .

[80]  Susanne P. Lajoie,et al.  Apprenticeship training in the workplace: computer-coached practice environment as a new form of apprenticeship , 1989 .

[81]  K. Holyoak,et al.  Learning subgoals and methods for solving probability problems , 1990, Memory & cognition.

[82]  Royce R. Ronning,et al.  Individual differences: A third component in problem‐solving instruction , 1984 .

[83]  V. Shute,et al.  Cognitive Approaches To Automated Instruction , 1992 .

[84]  David N. Perkins,et al.  Models and Moves: Focusing on Dimensions of Causal Complexity To Achieve Deeper Scientific Understanding. , 2000 .

[85]  J. Prochaska,et al.  In Search of How People Change: Applications to Addictive Behaviors , 1992, The American psychologist.

[86]  Robert W. Swezey,et al.  Effects of instructional delivery systems and training parameter manipulations on electromechanical maintenance performance , 1988 .

[87]  H. Kelley.  The processes of causal attribution. , 1973 .

[88]  Agnar Aamodt,et al.  Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches , 1994, AI Commun..

[89]  John R. Frederiksen.  Implicit testing within an intelligent tutoring system , 1987 .

[90]  Drew H. Gitomer,et al.  Individual Differences in Technical Troubleshooting , 1988 .

[91]  Valerie J. Shute,et al.  Taxonomy of Learning Skills , 1988 .

[92]  H. Dreyfus.  Mind Over Machine , 1986 .

[93]  Scott D. Johnson.  A Description of Expert and Novice Performance Differences on Technical Troubleshooting Tasks. , 1989 .

[94]  John Patrick,et al.  Training and transfer of fault-finding skill , 1988 .

[95]  H. Schmidt,et al.  On acquiring expertise in medicine , 1993 .

[96]  R. Sternberg,et al.  Transfer on Trial: Intelligence, Cognition and Instruction , 1993 .

[97]  R. Schank,et al.  Tell Me a Story: Narrative and Intelligence , 1991 .

[98]  Johan de Kleer,et al.  How Circuits Work , 1984, Artif. Intell..

[99]  David H. Jonassen,et al.  Conceptual Understanding of Causal Reasoning in Physics , 2006 .

[100]  Kathryn T. Spoehr,et al.  Knowledge Organization and the Acquisition of Procedural Expertise , 1989 .