Skilled and Unaware: The Interactive Effects of Team Cognition, Team Metacognition, and Task Confidence on Team Performance

The purpose of this study was to evaluate the independent and joint effects of objective and subjective measures of situation awareness on team performance. We propose that objective and subjective measures are types of metacognitive experiences, with objective measures capturing actual situation knowledge and subjective measures capturing perceived situation knowledge. These metacognitive experiences are theorized to work in tandem with task confidence to predict performance. Data were collected from 72 three-person teams working in a simulated task environment of emergency management teams. Objective situation awareness, subjective situation awareness, and task confidence were each measured through the Situation Awareness Global Awareness Technique, the Mission Awareness Rating Scale, and collective efficacy, respectively. Results showed that all three metacognitive experiences interacted in predicting team performance, such that task confidence strengthened the relationship between actual situation knowledge and performance when the team’s perceived situation awareness was low and weakened the relationship when the team’s perceived situation awareness was high. These results suggest that it is always important to have the necessary actual situation knowledge to complete a task, but there are times in which aspects of awareness in relation to judgments or feelings should be low in order to create the necessary drive to regulate one’s behavior.

[1]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[2]  E. Sundstrom,et al.  Work teams: Applications and effectiveness. , 1990 .

[3]  Anthony J. Nyberg,et al.  Collective efficacy, group potency, and group performance: meta-analyses of their relationships, and test of a mediation model. , 2009, The Journal of applied psychology.

[4]  Michael A. West,et al.  Reflexivity, revolution and innovation in work teams , 2000 .

[5]  Herman Aguinis,et al.  Best-practice recommendations for estimating interaction effects using moderated multiple regression , 2010 .

[6]  R. M. Taylor,et al.  Situational Awareness Rating Technique (Sart): The Development of a Tool for Aircrew Systems Design , 2017 .

[7]  Mica R. Endsley,et al.  Use of Real-Time Probes for Measuring Situation Awareness , 2004 .

[8]  T. O. Nelson,et al.  Why investigate metacognition , 1994 .

[9]  Larry E. Toothaker,et al.  Multiple Regression: Testing and Interpreting Interactions , 1991 .

[10]  Michael D. McNeese,et al.  NeoCITIES: A Simulated Command and Control Task Environment for Experimental Research , 2010 .

[11]  Michael D. McNeese,et al.  Using the Neocities 3.1 Simulation to Study and Measure Team Cognition , 2010 .

[12]  John D. Lee.  Measuring Driver Adaptation to In-Vehicle Information Systems: Disassociation of Subjective and Objective Situation Awareness Measures , 1999 .

[13]  Daniel J. Garland,et al.  Situation Awareness Analysis and Measurement , 2009 .

[14]  Mica R. Endsley,et al.  Measurement of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[15]  Mary J. Waller,et al.  Team Cognition and Adaptability in Dynamic Settings: A review of Pertinent Work , 2010 .

[16]  Thomas D. Hardiman,et al.  A Comparative Analysis of Sagat and Sart for Evaluations of Situation Awareness , 1998 .

[17]  S. Kozlowski,et al.  Multilevel Theory, Research, a n d M e t h o d s i n Organizations Foundations, Extensions, and New Directions , 2022 .

[18]  A. Efklides.  Metacognition and Affect: What Can Metacognitive Experiences Tell Us about the Learning Process?. , 2006 .

[19]  Adel Guitouni,et al.  The role of metacognition in the relationship between objective and subjective measures of situation awareness , 2010 .

[20]  S. Kozlowski,et al.  Active learning: effects of core training design elements on self-regulatory processes, learning, and adaptability. , 2008, The Journal of applied psychology.

[21]  Christopher D. Wickens,et al.  Exploring the Concept of Team Situation Awareness in a Simulated Air Combat Environment , 2009 .

[22]  Susan Mohammed,et al.  An Experimental Evaluation of the Effectiveness of Endogenous and Exogenous Fantasy in Computer-Based Simulation Training , 2013, Int. J. Gaming Comput. Mediat. Simulations.

[23]  M. Endsley.  Situation Awareness Misconceptions and Misunderstandings , 2015 .

[24]  Daan van Knippenberg,et al.  To reflect or not to reflect: Prior team performance as a boundary condition of the effects of reflexivity on learning and final team performance , 2013 .

[25]  Akiko Nakata,et al.  Situation awareness in air traffic control : enhanced displays for advanced operations , 2000 .

[26]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[27]  W. T. Powers.  Behavior, the control of perception , 1973 .

[28]  M. Frese,et al.  Self-regulation in error management training: emotion control and metacognition as mediators of performance effects. , 2005, The Journal of applied psychology.

[29]  A. Bandura.  Self-Efficacy: The Exercise of Control , 1997, Journal of Cognitive Psychotherapy.

[30]  F. Lichacz.  Augmenting understanding of the relationship between situation awareness and confidence using calibration analysis , 2008, Ergonomics.

[31]  D. Chan.  Functional Relations among Constructs in the Same Content Domain at Different Levels of Analysis: A Typology of Composition Models , 1998 .

[32]  Kerri L. Johnson,et al.  Why People Fail to Recognize Their Own Incompetence , 2003 .

[33]  Lelyn D. Saner,et al.  Measuring and Predicting Shared Situation Awareness in Teams , 2009 .

[34]  Anastasia Efklides,et al.  Metacognition: Defining its facets and levels of functioning in relation to self- and co-regulation , 2008 .

[35]  Traci Sitzmann,et al.  A Meta‐Analytic Investigation of the Within‐Person Self‐Efficacy Domain: Is Self‐Efficacy a Product of Past Performance or a Driver of Future Performance? , 2013 .

[36]  P. Bliese.  Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. , 2000 .

[37]  Neville Stanton,et al.  Situation awareness measurement: a review of applicability for C4i environments. , 2006, Applied ergonomics.

[38]  J. Flavell.  Metacognition and Cognitive Monitoring: A New Area of Cognitive-Developmental Inquiry. , 1979 .

[39]  Nancy J. Cooke,et al.  Measuring Team Knowledge During Skill Acquisition of a Complex Task , 2001 .

[40]  Richard P. DeShon,et al.  Prior Performance and Goal Progress as Moderators of the Relationship Between Self-Efficacy and Performance , 2009 .

[41]  L. James,et al.  rwg: An assessment of within-group interrater agreement. , 1993 .

[42]  A. Efklides.  Metacognition Defining : Its Facets and Levels of Functioning in Relation to Self-Regulation and Co-regulation , 2008 .

[43]  J. Kruger,et al.  Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. , 1999, Journal of personality and social psychology.

[44]  J. Diefendorff,et al.  Self-regulation at work. , 2010, Annual review of psychology.

[45]  Michael D. McNeese.  Socio-Cognitive Factors in the Acquisition and Transfer of Knowledge , 2000, Cognition, Technology & Work.

[46]  R. Edwards,et al.  Medical student self-assessment of performance on an obstetrics and gynecology clerkship. , 2003, American journal of obstetrics and gynecology.

[47]  Kerri L. Johnson,et al.  Why the Unskilled are Unaware: Further Explorations of (Absent) Self-Insight Among the Incompetent , 2006, Organizational behavior and human decision processes.

[48]  Rashaad E. T. Jones,et al.  The Neocities Simulation: Understanding the Design and Experimental Methodology Used to Develop a Team Emergency Management Simulation , 2005 .

[49]  Michael D. Matthews,et al.  Assessing Situation Awareness in Field Training Exercises , 2002 .

[50]  L. James.  Aggregation Bias in Estimates of Perceptual Agreement. , 1982 .

[51]  B. McGuinness,et al.  Quantitative Analysis of Situational Awareness (QUASA): Applying Signal Detection Theory to True/False Probes and Self-Ratings , 2004 .

[52]  A. Efklides.  How Does Metacognition Contribute to the Regulation of Learning? An Integrative Approach , 2014 .

[53]  David Benjamin Hellar.  An Investigation of Data Overload in Team-Based Distributed Cognition Systems. , 2009 .

[54]  Mica R. Endsley,et al.  Theoretical Underpinnings of Situation Awareness, A Critical Review , 2000 .