From "Knowing What" to "Knowing When": Exploring a Concept of Situation Awareness Synchrony for Evaluating SA Dynamics in Teams

This concept paper presents an initial exploration of measuring Situation Awareness (SA) dynamics in team settings. SA dynamics refer to the evaluation of SA’s temporal evolution of one or more teammates. We discuss why current methods are inherently limited by the subjective nature of SA and why it is important to identify measures for the temporal evolution of SA. Most current approaches focus on measuring the accuracy of SA (i.e., What? is known)and the similarity in the context of shared SA, often resulting in rather qualitative assessments. However, quantitative assessments are important to address temporal aspects (i.e., When? and ultimately For how long?). Thus, we propose, as a complementary approach to accuracy and similarity of SA, to consider the concept of SA synchrony as a quantitative metric of SA dynamics in teams. Specifically, we highlight the existence of three latencies with high relevance to shared SA dynamics and discuss options for their assessment.

[1]  Simone Sulpizio,et al.  Analyzing spatial data from mouse tracker methodology: An entropic approach , 2017, Behavior research methods.

[2]  Luís Meireles,et al.  Conceptualization and Measurement of Individual Situation Awareness (SA) in Expert Populations across Operational Domains: a systematic review of the literature with a practical purpose on our minds , 2018, Proceedings of the Human Factors and Ergonomics Society Annual Meeting.

[3]  M. R. Houck,et al.  Tools for assessing situational awareness in an operational fighter environment. , 1994, Aviation, space, and environmental medicine.

[4]  Nancy J. Cooke,et al.  On Teams, Teamwork, and Team Performance: Discoveries and Developments , 2008, Hum. Factors.

[5]  David Schuster,et al.  A Quantitative Measure for Shared and Complementary Situation Awareness , 2016 .

[6]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[7]  Albert A. Nofi Defining and Measuring Shared Situational Awareness , 2000 .

[8]  Endsley,et al.  A model of inter- and intrateam situation awareness: implications for design, training and measurement , 2001 .

[9]  Guy H. Walker,et al.  What really is going on? Review of situation awareness models for individuals and teams , 2008 .

[10]  Danko Nikolić,et al.  SITUATION AWARENESS AS A PREDICTOR OF PERFORMANCE IN EN ROUTE AIR TRAFFIC CONTROLLERS , 1998 .

[11]  Mark W. Scerbo,et al.  Theoretical Perspectives on Adaptive Automation , 2019, Human Performance in Automated and Autonomous Systems.

[12]  Eduardo Salas,et al.  A Knowledge Elicitation Approach to the Measurement of Team Situation Awareness , 2017 .

[13]  Michael D. Matthews,et al.  Assessing Situation Awareness in Field Training Exercises , 2002 .

[14]  Jonathan B Freeman,et al.  MouseTracker: Software for studying real-time mental processing using a computer mouse-tracking method , 2010, Behavior research methods.

[15]  Guy H. Walker,et al.  State-of-science: situation awareness in individuals, teams and systems , 2017, Ergonomics.

[16]  Christopher D. Wickens,et al.  Exploring the Concept of Team Situation Awareness in a Simulated Air Combat Environment , 2009 .

[17]  Jonas M. B. Haslbeck,et al.  Mouse-Tracking , 2019, A Handbook of Process Tracing Methods.

[18]  Robert L. Helmreich,et al.  Group interaction and flight crew performance , 1988 .

[19]  Cheryl Z. Qian,et al.  Designing a Situational Awareness Information Display: Adopting an Affordance-Based Framework to Amplify User Experience in Environmental Interaction Design , 2016, Informatics.

[20]  Michael A. Vidulich,et al.  Testing a Subjective Metric of Situation Awareness , 1991 .

[21]  Nancy J. Cooke,et al.  Measuring team situation awareness in decentralized command and control environments , 2006, Ergonomics.

[22]  R. M. Taylor,et al.  Situational Awareness Rating Technique (Sart): The Development of a Tool for Aircrew Systems Design , 2017 .

[23]  Peter Berggren,et al.  Using shared priorities to measure shared situation awareness , 2010, ISCRAM.

[24]  Mica R. Endsley,et al.  Situation awareness global assessment technique (SAGAT) , 1988, Proceedings of the IEEE 1988 National Aerospace and Electronics Conference.

[25]  Kane J. Middleton,et al.  The application of subjective job task analysis techniques in physically demanding occupations: evidence for the presence of self-serving bias , 2017, Ergonomics.

[26]  Eduardo Salas,et al.  Broadening the Measurement of Situation Awareness through Cognitive Engineering Methods , 1997 .

[27]  A. Tomarken A psychometric perspective on psychophysiological measures , 1995 .

[28]  M R Endsley SITUATION AWARENESS IN AVIATION SYSTEMS. IN: HANDBOOK OF AVIATION HUMAN FACTORS , 1999 .

[29]  Björn Johansson,et al.  An exploratory study of a low-level shared awareness measure using mission-critical locations during an emergency exercise , 2015 .

[30]  Guy H. Walker,et al.  Is it really better to share? Distributed situation awareness and its implications for collaborative system design , 2010 .

[31]  A. H. Roscoe,et al.  Heart rate as a psychophysiological measure for in-flight workload assessment. , 1993, Ergonomics.

[32]  Daniel J. Garland,et al.  Situation Awareness Analysis and Measurement , 2009 .

[33]  Rolf Zon,et al.  Eye Movements as an Indicator of Situation Awareness in a Flight Simulator Experiment , 2012 .

[34]  Leo Gugerty,et al.  Development of a Novel Measure of Situation Awareness: The Case for Eye Movement Analysis , 2010 .

[35]  Marilyn Jager Adams,et al.  Situation Awareness and the Cognitive Management of Complex Systems , 1995, Hum. Factors.

[36]  E. Salas,et al.  The Assessment of Team Performance: Observations and Needs , 2017 .

[37]  Neville Stanton,et al.  Situation awareness measurement: a review of applicability for C4i environments. , 2006, Applied ergonomics.

[38]  Debra G. Jones,et al.  Designing for Situation Awareness : An Approach to User-Centered Design , 2003 .

[39]  Mica R. Endsley,et al.  Situation Awareness Oriented Design: From User's Cognitive Requirements to Creating Effective Supporting Technologies , 2003 .

[40]  Brian F. Gore,et al.  Modeling Pilot Situation Awareness , 2011 .

[41]  Neville A. Stanton,et al.  Situation awareness based on eye movements in relation to the task environment , 2018, Cognition, Technology & Work.

[42]  Eduardo Salas,et al.  Situation Awareness in Team Performance: Implications for Measurement and Training , 1995, Hum. Factors.

[43]  Lakhmi C. Jain,et al.  Monitoring pilot "Situation Awareness" , 2013, J. Intell. Fuzzy Syst..

[44]  Tom Ziemke,et al.  Situation awareness in human-machine interactive systems , 2017, Cognitive Systems Research.

[45]  Stefan Scherbaum,et al.  Unraveling the sub-processes of selective attention: insights from dynamic modeling and continuous behavior , 2015, Cognitive Processing.

[46]  Thomas D. Hardiman,et al.  A Comparative Analysis of Sagat and Sart for Evaluations of Situation Awareness , 1998 .

[47]  Sven Fuchs,et al.  Multidimensional Real-Time Assessment of User State and Performance to Trigger Dynamic System Adaptation , 2017, HCI.

[48]  Mohamed Chetouani,et al.  Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines , 2012, IEEE Transactions on Affective Computing.

[49]  Mary C. Whitton,et al.  Designing to support situation awareness across distances: an example from a scientific collaboratory , 2004, Inf. Process. Manag..

[50]  P G Jorna,et al.  Heart rate and workload variations in actual and simulated flight. , 1993, Ergonomics.

[51]  J. Veltman,et al.  Physiological workload reactions to increasing levels of task difficulty. , 1998, Ergonomics.

[52]  Lelyn D. Saner,et al.  Measuring and Predicting Shared Situation Awareness in Teams , 2009 .

[53]  Florian Jentsch,et al.  The importance of shared mental models and shared situation awareness for transforming robots from tools to teammates , 2012, Defense, Security, and Sensing.