The Implications of Research on Expertise for Curriculum and Pedagogy

Instruction on problem solving in particular domains typically relies on explanations from experts about their strategies. However, research indicates that such self-reports often are incomplete or inaccurate (e.g., Chao & Salvendy, 1994; Cooke & Breedin, 1994). This article evaluates research on experts’ cognition, the accuracy of experts’ self-reports, and the efficacy of instruction based on experts’ self-reports. Analysis of this evidence indicates that experts’ free recall of strategies introduces errors and omissions into instructional materials that hinder student success. In contrast, when experts engage in structured knowledge elicitation techniques (e.g., cognitive task analysis), the resultant instruction is more effective. Based on these findings, the article provides a theoretical explanation of experts’ self-report errors and discusses implications for the continued improvement of instructional design processes.

[1]  N. Maier Reasoning in humans. II. The solution of a problem and its appearance in consciousness. , 1931 .

[2]  K. Duncker,et al.  On problem-solving , 1945 .

[3]  George Sperling,et al.  The information available in brief visual presentations. , 1960 .

[4]  G. Miller,et al.  Plans and the structure of behavior , 1960 .

[5]  P. Slovic,et al.  An analysis-of-variance model for the assessment of configural cue utilization in clinical judgment. , 1968, Psychological bulletin.

[6]  A. Inkeles,et al.  International Encyclopedia of the Social Sciences. , 1968 .

[7]  G. Miller,et al.  Plans and the structure of behavior , 1960 .

[8]  G. Bateson Steps to an Ecology of Mind , 1972 .

[9]  H. Simon,et al.  The mind's eye in chess. , 1973 .

[10]  W. Chase,et al.  Visual information processing. , 1974 .

[11]  H. J. Einhorn Expert judgment: Some necessary conditions and an example. , 1974 .

[12]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[13]  R. Shiffrin,et al.  Controlled and automatic human information processing: I , 1977 .

[14]  Walter Schneider,et al.  Controlled and Automatic Human Information Processing: 1. Detection, Search, and Attention. , 1977 .

[15]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[16]  D. Broadbent Levels, Hierarchies, and the Locus of Control* , 1977 .

[17]  Walter Schneider,et al.  Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. , 1977 .

[18]  Timothy D. Wilson,et al.  The accuracy of verbal reports about the effects of stimuli on evaluations and behavior , 1978 .

[19]  K. Holyoak,et al.  Analogical problem solving , 1980, Cognitive Psychology.

[20]  Dorothea P. Simon,et al.  Expert and Novice Performance in Solving Physics Problems , 1980, Science.

[21]  Herbert A. Simon,et al.  Models of Competence in Solving Physics Problems , 1980, Cogn. Sci..

[22]  Gary Cantor,et al.  On the Relationship Between Implicit and Explicit Modes in the Learning of a Complex Rule Structure , 1980 .

[23]  K. A. Ericsson,et al.  Verbal reports as data. , 1980 .

[24]  B. Adelson Problem solving and the development of abstract categories in programming languages , 1981, Memory & cognition.

[25]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[26]  William B. Rouse,et al.  Human Detection and Diagnosis of System Failures , 1981 .

[27]  Michelene T. H. Chi,et al.  Expertise in Problem Solving. , 1981 .

[28]  Lisanne Bainbridge,et al.  Mathematical Equations or Processing Routines , 1981 .

[29]  R. Sternberg Advances in the psychology of human intelligence , 1982 .

[30]  John R. Anderson Acquisition of cognitive skill. , 1982 .

[31]  P E Johnson,et al.  What kind of expert should a system be? , 1983, The Journal of medicine and philosophy.

[32]  K. Holyoak,et al.  Schema induction and analogical transfer , 1983, Cognitive Psychology.

[33]  P. Johnson-Laird Mental models , 1989 .

[34]  W. Schneider,et al.  Automatic category search and its transfer. , 1984, Journal of experimental psychology. Learning, memory, and cognition.

[35]  G. Logan On the ability to inhibit thought and action , 1984 .

[36]  G. Hatano,et al.  TWO COURSES OF EXPERTISE , 1984 .

[37]  D. Broadbent,et al.  On the Relationship between Task Performance and Associated Verbalizable Knowledge , 1984 .

[38]  Yuichiro Anzai,et al.  Internal Models in Physics Problem Solving , 1984 .

[39]  Herbert J. Walberg,et al.  Adapting Instruction to Individual Differences , 1985 .

[40]  David L. Lohman Predicting Mathemathanic Effects in the Teaching of Higher-Order Thinking Skills , 1986 .

[41]  D. Broadbent,et al.  Implicit and explicit knowledge in the control of complex systems , 1986 .

[42]  Harold Maurice Collins,et al.  Where's the expertise? Expert systems as a medium of knowledge transfer , 1986 .

[43]  Jeffrey K. Liker,et al.  A day at the races: A study of IQ, expertise, and cognitive complexity. , 1986 .

[44]  K. Hakuta,et al.  Child development and education in Japan. , 1986 .

[45]  John R. Anderson,et al.  Skill Acquisition: Compilation of Weak-Method Problem Solutions. , 1987 .

[46]  W. D. Gray,et al.  Transfer of Cognitive Skills , 1987 .

[47]  Jörg Doll,et al.  Intelligenz und Schachleistung—eine Untersuchung an Schachexperten. , 1987 .

[48]  James E. McDonald,et al.  The Application of Psychological Scaling Techniques to Knowledge Elicitation for Knowledge-Based Systems , 1987, Int. J. Man Mach. Stud..

[49]  J. Sweller,et al.  Effects of schema acquisition and rule automation on mathematical problem-solving transfer. , 1987 .

[50]  G. Logan Toward an instance theory of automatization. , 1988 .

[51]  P. Ackerman Determinants of individual differences during skill acquisition: Cognitive abilities and information processing. , 1988 .

[52]  Barbara Rogoff,et al.  Influence of Adult and Peer Collaborators on Children's Planning Skills. , 1988 .

[53]  John Clement,et al.  Observed Methods for Generating Analogies in Scientific Problem Solving , 1987, Cogn. Sci..

[54]  H. Simon,et al.  Skill in Chess , 1988 .

[55]  Theresa M. Mullin,et al.  Experts' estimation of uncertain quantities and its implications for knowledge acquisition , 1989, IEEE Trans. Syst. Man Cybern..

[56]  Thomas R. Gruber,et al.  The Acquisition of Strategic Knowledge , 1989 .

[57]  Sandra L. Newsome,et al.  Presenting Abstract versus Concrete Information in Expert Systems: What is the Impact on User Performance? , 1989, Int. J. Man Mach. Stud..

[58]  A F Kramer,et al.  Development and transfer of automatic processing. , 1990, Journal of experimental psychology. Human perception and performance.

[59]  Charles L. Hulin,et al.  Adding a Dimension: Time as a Factor in the Generalizability of Predictive Relationships , 1990 .

[60]  P. Chandler,et al.  Cognitive load as a factor in the structuring of technical material. , 1990 .

[61]  John R. Anderson,et al.  Abstract Planning and Perceptual Chunks: Elements of Expertise in Geometry , 1990, Cogn. Sci..

[62]  James L. McClelland,et al.  On the control of automatic processes: a parallel distributed processing account of the Stroop effect. , 1990, Psychological review.

[63]  A. D. Fisk,et al.  Recombination of Automatic Processing components: The Effects of Transfer, Reversal, and Conflict Situations , 1991, Human factors.

[64]  Barbara Rogoff,et al.  Children's guided participation in planning imaginary errands with skilled adult or peer partners. , 1991 .

[65]  P. Chandler,et al.  Cognitive Load Theory and the Format of Instruction , 1991 .

[66]  Nancy J. Cooke,et al.  Modeling human expertise in expert systems , 1992 .

[67]  P. Lewicki,et al.  Nonconscious acquisition of information. , 1992, The American psychologist.

[68]  J. Anderson,et al.  Automaticity and the ACT* theory. , 1992, The American journal of psychology.

[69]  L. Squire,et al.  The structure and organization of memory. , 1993, Annual review of psychology.

[70]  B Crandall,et al.  Critical decision method: A technique for eliciting concrete assessment indicators from the intuition of NICU nurses , 1993, ANS. Advances in nursing science.

[71]  Irving Biederman,et al.  Visual object recognition , 1993 .

[72]  M. Scardamalia,et al.  Surpassing Ourselves: An Inquiry into the Nature and Implications of Expertise , 1993 .

[73]  Nancy J. Cooke,et al.  Role of high-level knowledge in memory for chess positions , 1993 .

[74]  Robert J. Crutcher,et al.  The role of deliberate practice in the acquisition of expert performance. , 1993 .

[75]  John R. Anderson Problem solving and learning. , 1993 .

[76]  William F. Brewer,et al.  The Role of Anomalous Data in Knowledge Acquisition: A Theoretical Framework and Implications for Science Instruction , 1993 .

[77]  N. J. Cook,et al.  Constructing naive theories of motion on the fly , 1994, Memory & cognition.

[78]  Robert J. Sternberg,et al.  Cognitive conceptions of expertise , 1997 .

[79]  S. Casner Understanding the Determinants of Problem-Solving Behavior in a Complex Environment , 1994 .

[80]  Nancy J. Cooke,et al.  Varieties of knowledge elicitation techniques , 1994, Int. J. Hum. Comput. Stud..

[81]  David A. Kareken,et al.  Human judgment and estimation of premorbid intellectual function. , 1994 .

[82]  Marin Marinov,et al.  A Symbolic Model of the Nonconscious Acquisition of Information , 1994, Cogn. Sci..

[83]  Gavriel Salvendy,et al.  Percentage of procedural knowledge acquired as a function of the number of experts from whom knowledge is acquired for diagnosis, debugging, and interpretation tasks , 1994, Int. J. Hum. Comput. Interact..

[84]  N. Charness,et al.  Expert Performance Its Structure and Acquisition , 2002 .

[85]  K. A. Ericsson,et al.  Long-term working memory. , 1995, Psychological review.

[86]  K. A. Ericsson,et al.  The Road To Excellence: The Acquisition of Expert Performance in the Arts and Sciences, Sports, and Games , 1996 .

[87]  K. A. Ericsson,et al.  Expert and exceptional performance: evidence of maximal adaptation to task constraints. , 1996, Annual review of psychology.

[88]  Gordon D. Logan,et al.  ATTENTION IN THE ACQUISITION AND EXPRESSION OF AUTOMATICITY , 1996 .

[89]  L. Reder,et al.  Metacognition does not imply awareness: Strategy choice is governed by implicit learning and memory , 1996 .

[90]  R. Stofflett,et al.  Representation of the conceptual change model in science teacher education , 1996 .

[91]  L. Reder Implicit memory and metacognition , 1996 .

[92]  Neil Charness,et al.  The role of practice and coaching in entrepreneurial skill domains: An international comparison of life-span chess skill acquisition. , 1996 .

[93]  John R. Anderson,et al.  How people learn to skip steps. , 1996 .

[94]  John R. Anderson,et al.  History of Success and Current Context in Problem Solving Combined Influences on Operator Selection , 1996, Cognitive Psychology.

[95]  J. Starkes,et al.  Deliberate practice in sports: What is it anyway? , 1996 .

[96]  Herbert A. Simon,et al.  Templates in Chess Memory: A Mechanism for Recalling Several Boards , 1996, Cognitive Psychology.

[97]  H. A. Mcallister,et al.  Self-serving bias in the classroom: Who shows it? Who knows it? , 1996 .

[98]  Robyn M. Dawes,et al.  House of Cards , 1996 .

[99]  R. Clark,et al.  Cognitive task analysis for training , 1996 .

[100]  A. D. D. Groot,et al.  Perception and memory in chess: Studies in the heuristics of the prodessional eye , 1996 .

[101]  Philip J. Stroffolino,et al.  To calculate or not to calculate: A source activation confusion model of problem familiarity's role in strategy selection , 1997 .

[102]  Hélène Eyrolle,et al.  Expertise in dynamic environments , 1997 .

[103]  Colleen M. Zeitz,et al.  Some concrete advantages of abstraction: how experts' representations facilitate reasoning , 1997 .

[104]  David H. Jonassen,et al.  Task Analysis Methods for Instructional Design , 1998 .

[105]  R. Baumeister,et al.  Ego depletion: is the active self a limited resource? , 1998, Journal of personality and social psychology.

[106]  Sam Wineburg,et al.  Reading Abraham Lincoln: An Expert/Expert Study in the Interpretation of Historical Texts , 1998, Cogn. Sci..

[107]  F. Gobet Expert memory: a comparison of four theories , 1998, Cognition.

[108]  Daniel L. Schwartz,et al.  A time for telling , 1998 .

[109]  A. S. Dhillon Individual differences within problem‐solving strategies used in physics , 1998 .

[110]  A. J. Wells,et al.  Turing's Analysis of Computation and Theories of Cognitive Architecture , 1998, Cogn. Sci..

[111]  K J Vicente,et al.  An ecological theory of expertise effects in memory recall. , 1998, Psychological review.

[112]  D O'Hare,et al.  Cognitive task analyses for decision centred design and training. , 1998, Ergonomics.

[113]  H. Simon,et al.  Expert chess memory: revisiting the chunking hypothesis. , 1998, Memory.

[114]  Joseph A. Horvath,et al.  Cognitive conceptions of expertise and their relations to giftedness. , 1998 .

[115]  K. A. Ericsson,et al.  The Scientific Study of Expert Levels of Performance: general implications for optimal learning and creativity 1 , 1998 .

[116]  Nigel Shadbolt,et al.  Use of the Critical Decision Method to Elicit Expert Knowledge: A Case Study in the Methodology of Cognitive Task Analysis , 1998, Hum. Factors.

[117]  Dean Keith Simonton,et al.  Talent and its development: An emergenic and epigenetic model. , 1999 .

[118]  Lisanne Bainbridge,et al.  Verbal reports as evidence of the process operator's knowledge , 1999, Int. J. Hum. Comput. Stud..

[119]  Tor Nørretranders,et al.  The user illusion , 1999 .

[120]  S. Chaiken,et al.  Dual-process theories in social psychology , 1999 .

[121]  Denis Besnard,et al.  Expert error in trouble-shooting: an exploratory study in electronics , 1999, Int. J. Hum. Comput. Stud..

[122]  T. Chartrand,et al.  THE UNBEARABLE AUTOMATICITY OF BEING , 1999 .

[123]  J. Tuovinen,et al.  A comparison of cognitive load associated with discovery learning and worked examples , 1999 .

[124]  J. Bargh The cognitive monster: The case against the controllability of automatic stereotype effects. , 1999 .

[125]  K. Taylor,et al.  Accessing Problem-Solving Strategy Knowledge: The Complementary Use of Concurrent Verbal Protocols and Retrospective Debriefing. , 2000 .

[126]  Valerie L. Shalin,et al.  Cognitive task analysis , 2000 .

[127]  Thomas L. Seamster,et al.  A Skill-Based Cognitive Task Analysis Framework , 2000 .

[128]  K. A. Ericsson,et al.  Shortcomings of generic retrieval structures with slots of the type that Gobet (1993) proposed and modelled. , 2000, British journal of psychology.

[129]  Vladimir M. Sloutsky,et al.  Problem Representation in Experts and Novices: Part 2. Underlying Processing Mechanisms , 2000 .

[130]  Vladimir M. Sloutsky,et al.  Problem Representation in Experts and Novices: Part 1. Differences in the Content of Representation , 2000 .

[131]  M. Ferguson,et al.  Beyond behaviorism: on the automaticity of higher mental processes. , 2000, Psychological bulletin.

[132]  H. Aarts,et al.  Habits as knowledge structures: Automaticity in goal-directed behavior , 2000 .

[133]  Denis Besnard Expert Error: The Case of Trouble-Shooting in Electronics , 2000, SAFECOMP.

[134]  K J Vicente,et al.  Revisiting the constraint attunement hypothesis: reply to Ericsson, Patel, and Kintsch (2000) and Simon and Gobet (2000). , 2000, Psychological review.

[135]  Valerie J. Shute,et al.  State-of-the-art review of cognitive task analysis techniques , 2000 .

[136]  Derek H. Sleeman,et al.  Accommodating surprise in taxonomic tasks: the role of expertise , 2000, Cogn. Sci..

[137]  John D. Van Horn,et al.  Characterizing mature human intelligence: Expertise development , 2000 .

[138]  Geert Crombez,et al.  Automatic attitude activation and efficiency: The fourth horseman of automaticity. , 2000 .

[139]  Jan Maarten Schraagen,et al.  Cognitive Task Analysis and Innovation of Training: The Case of Structured Troubleshooting , 2000, Hum. Factors.

[140]  D. Wegner,et al.  Automaticity of Action, Psychology of , 2001 .

[141]  Pamela J. Hinds,et al.  Bothered by abstraction: the effect of expertise on knowledge transfer and subsequent novice performance. , 2001, The Journal of applied psychology.

[142]  N. Smelser,et al.  International Encyclopedia of the Social and Behavioral Sciences , 2001 .

[143]  Denis Besnard,et al.  Troubleshooting in Mechanics: A Heuristic Matching Process , 2001, Cognition, Technology & Work.

[144]  J. Bargh,et al.  The automated will: nonconscious activation and pursuit of behavioral goals. , 2001, Journal of personality and social psychology.

[145]  L. Phillips,et al.  Mental planning and the Tower of London task , 2001, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[146]  Frank J. Lee,et al.  Does Learning a Complex Task Have to Be Complex?: A Study in Learning Decomposition , 2001, Cognitive Psychology.

[147]  D. McAdams,et al.  The Psychology of Life Stories , 2001 .

[148]  J. D. Horn,et al.  Expertise and age-related changes in components of intelligence. , 2001, Psychology and aging.

[149]  Sian L. Beilock,et al.  Expertise, attention, and memory in sensorimotor skill execution: Impact of novel task constraints on dual-task performance and episodic memory , 2002, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[150]  Christopher D. Wickens,et al.  Situation Awareness and Workload in Aviation , 2002 .

[151]  M. David Merrill A Pebble‐in‐the‐Pond Model For Instructional Design , 2002 .

[152]  D. Wegner The Illusion of Conscious Will , 2018, The MIT Press.

[153]  R. Parncutt,et al.  The Science and Psychology of Music Performance Creative Strategies for Teaching and Learning , 2002 .

[154]  S W Brown,et al.  The role of practice and automaticity in temporal and nontemporal dual-task performance , 2002, Psychological research.

[155]  Mark J. Lauer,et al.  Tools of the trade: Survey research , 2002 .

[156]  Ap Dijksterhuis,et al.  When stereotypes get in the way: stereotypes obstruct stereotype-inconsistent trait inferences. , 2003, Journal of personality and social psychology.

[157]  Slava Kalyuga,et al.  The Expertise Reversal Effect , 2003 .

[158]  H. Aarts,et al.  The silence of the library: environment, situational norm, and social behavior. , 2003, Journal of personality and social psychology.

[159]  Slava Kalyuga,et al.  Measuring Knowledge to Optimize Cognitive Load Factors During Instruction. , 2004 .

[160]  Stephanie M. Doane,et al.  Memory Processes of Flight Situation Awareness: Interactive Roles of Working Memory Capacity, Long-Term Working Memory, and Expertise , 2004, Hum. Factors.

[161]  K. A. Ericsson,et al.  Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. , 2004, Academic medicine : journal of the Association of American Medical Colleges.

[162]  R. Mayer Should there be a three-strikes rule against pure discovery learning? The case for guided methods of instruction. , 2004, The American psychologist.

[163]  Timothy D. Wilson,et al.  Self-knowledge: its limits, value, and potential for improvement. , 2004, Annual review of psychology.

[164]  George C Velmahos,et al.  Cognitive task analysis for teaching technical skills in an inanimate surgical skills laboratory. , 2004, American journal of surgery.

[165]  Suzanne C. Thompson,et al.  Illusory Control and Motives for Control: The Role of Connection and Intentionality , 2004 .

[166]  Richard E. Clark,et al.  Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching , 2006 .