Characterizing novice-expert differences in macrocognition: an exploratory study of cognitive work in the emergency department.

STUDY OBJECTIVE The objectives of this study are to elicit and document descriptions of emergency physician expertise, to characterize cognitive differences between novice and expert physicians, and to identify areas in which novices' skill and knowledge gaps are most pronounced. The nature of the differences between novices and experts needs to be explored to develop effective instructional modalities that accelerate the learning curve of inexperienced physicians who work in high-complexity environments. METHODS We interviewed novice emergency physicians (first-year residents) and attending physicians with significant expertise, working in an academic Level I trauma center in Southern California. With cognitive task analysis, we used task diagrams to capture nonroutine critical incidents that required the use of complex cognitive skills. Timelines were constructed to develop a detailed understanding of challenging incidents and the decisions involved as the incident unfolded, followed by progressive deepening to tease out situation-specific cues, knowledge, and information that experts and novices used. A thematic analysis of the interview transcripts was conducted to identify key categories. Using classification techniques for data reduction, we identified a smaller set of key themes, which composed the core findings of the study. RESULTS Five interns and 6 attending physicians participated in the interviews. Novice physicians reported having difficulties representing the patient's story to attending physicians and other health care providers. Overrelying on objective data, novice physicians use linear thinking to move to diagnosis quickly and are likely to discount and explain away data that do not "fit" the frame. Experienced physicians draw on expertise to recognize cues and patterns while leaving room for altering or even changing their initial diagnosis. Whereas experts maintain high levels of spatial, temporal, and organizational systems awareness when overseeing treatment modalities of multiple patients, novices have difficulty seeing and maintaining the "big picture." CONCLUSION Novice physicians use sense-making styles that differ from those of experts. Training novices to respond to the high cognitive demand of complex environments early in their careers requires instructional modalities that not only increase their knowledge base but also accelerate the integration of knowledge and practice. Simulation and custom-designed avatar-mediated virtual worlds are a promising new technology that may facilitate such training. Future research should expand on the results of this study through the use of larger sample sizes and interviews conducted at multiple sites to increase generalizability.

[1]  Rosaline S Barbour,et al.  Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? , 2001, BMJ : British Medical Journal.

[2]  P. Croskerry,et al.  Emergency medicine: A practice prone to error? , 2001, CJEM.

[3]  Stephanie Guerlain,et al.  How Residents Think and Make Medical Decisions: Implications for Education and Patient Safety , 2007, The American surgeon.

[4]  Gary Klein,et al.  Naturalistic Decision Making , 2008, Hum. Factors.

[5]  Beth Crandall,et al.  Critical care physician cognitive task analysis: an exploratory study , 2009, Critical care.

[6]  D. R. Johnson,et al.  Medical problem solving and uncertainty in the emergency department. , 1991, Annals of emergency medicine.

[7]  T. Krummel,et al.  Virtual reality and simulation: training the future emergency physician. , 2002, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[8]  Stephen M. Fiore,et al.  Toward an Understanding of Macrocognition in Teams: Predicting Processes in Complex Collaborative Contexts , 2010, Hum. Factors.

[9]  T. Ormerod,et al.  Naturalistic Decision Making and Macrocognition , 2012 .

[10]  W. L. Heinrichs,et al.  Design, Development, and Evaluation of an Online Virtual Emergency Department for Training Trauma Teams , 2008, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[11]  Jenny W. Rudolph,et al.  The Dynamics of Action-Oriented Problem Solving: Linking Interpretation and Choice , 2009 .

[12]  Raanan Lipshitz,et al.  How professionals make decisions. , 2004 .

[13]  Eduardo Salas,et al.  Towards an understanding of macrocognition in teams: developing and defining complex collaborative processes and products , 2010 .

[14]  Kim J. Vicente,et al.  Local applications of the ecological approach to human-machine systems , 1995 .

[15]  Jens Rasmussen,et al.  Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[16]  Ying Liu,et al.  Opportunistic decision making and complexity in emergency care , 2011, J. Biomed. Informatics.

[17]  David M. Irby,et al.  Educating Physicians: A Call for Reform of Medical School and Residency , 2010 .

[18]  Gary Klein,et al.  Developing Expertise in Decision Making , 1997 .

[19]  Joann Keyton,et al.  Perspectives: Examining Communication as Macrocognition in STS , 2010, Hum. Factors.

[20]  Robert R. Hoffman,et al.  Studying Cognitive Systems in Context: Preface to the Special Section , 2000, Hum. Factors.

[21]  Karol G. Ross,et al.  Macrocognition , 2003, IEEE Intell. Syst..

[22]  Jens Rasmussen Ecological Interface Design for Reliable Human-Machine Systems , 1999 .

[23]  I. Newman,et al.  Qualitative-quantitative research methodology , 1998 .

[24]  Gary Klein,et al.  Making Sense of Sensemaking 2: A Macrocognitive Model , 2006, IEEE Intelligent Systems.

[25]  J M Schraagen,et al.  A Method for Cognitive Task Analysis , 1992 .

[26]  D. Woods,et al.  Studying Cognitive Systems in Context: The Cognitive Systems Triad , 2006 .

[27]  D. Silverman Interpreting Qualitative Data , 1993 .

[28]  M. Merialdi,et al.  Learning in a Virtual World: Experience With Using Second Life for Medical Education , 2010, Journal of medical Internet research.

[29]  Pat Croskerry,et al.  Profiles in patient safety: A "perfect storm" in the emergency department. , 2007, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[30]  Pat Croskerry,et al.  A universal model of diagnostic reasoning. , 2009, Academic medicine : journal of the Association of American Medical Colleges.

[31]  N. Britten,et al.  Using Reflexivity to Optimize Teamwork in Qualitative Research , 1999, Qualitative health research.

[32]  Lynda Berends,et al.  Using multiple coders to enhance qualitative analysis: The case of interviews with consumers of drug treatment , 2005 .

[33]  P. Croskerry The cognitive imperative: thinking about how we think. , 2000, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[34]  Sterling Wiggins,et al.  Team sensemaking , 2010 .

[35]  Gary Klein,et al.  Making Sense of Sensemaking 1: Alternative Perspectives , 2006, IEEE Intelligent Systems.

[36]  B. Brehmer Dynamic decision making: human control of complex systems. , 1992, Acta psychologica.

[37]  Joann Keyton,et al.  Perspective: Examining Communication as Macrocognition , 2010 .

[38]  Yasuharu Okuda,et al.  The utility of simulation in medical education: what is the evidence? , 2009, The Mount Sinai journal of medicine, New York.

[39]  G. Klein,et al.  The macrocognition framework of naturalistic decision making , 2008 .

[40]  Anthony Peter Macmillan Coxon,et al.  Sorting Data: Collection and Analysis , 1999 .

[41]  Jean Hartley,et al.  Case study research , 2004 .

[42]  Gary Klein,et al.  Working Minds: A Practitioner's Guide to Cognitive Task Analysis , 2006 .

[43]  Gary Klein Sources of Power , 1998 .

[44]  V L Patel,et al.  Cognition and expertise: acquisition of medical competence. , 2000, Clinical and investigative medicine. Medecine clinique et experimentale.

[45]  H. Simon,et al.  Models Of Man : Social And Rational , 1957 .

[46]  Brendan J Leary Recognition-Primed Decisions, Ethical Intuition and Borrowing Experience , 2012 .

[47]  P Croskerry,et al.  Clinical decision making: an emergency medicine perspective. , 1999, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[48]  D. Norman,et al.  New technology and human error , 1989 .

[49]  K. A. Ericsson,et al.  The making of an expert. , 2007, Harvard business review.

[50]  K. A. Ericsson,et al.  Recent advances in expertise research: a commentary on the contributions to the special issue , 2005 .

[51]  Nigel Shadbolt,et al.  Use of the Critical Decision Method to Elicit Expert Knowledge: A Case Study in the Methodology of Cognitive Task Analysis , 1998, Hum. Factors.

[52]  S. Kvale Interviews : an introduction to qualitative research interviewing , 1996 .