Situation awareness measurement: a review of applicability for C4i environments.

The construct of situation awareness (SA) has become a core theme within the human factors (HF) research community. Consequently, there have been numerous attempts to develop reliable and valid measures of SA but there is a lack of techniques developed specifically for the assessment of SA in command, control, communication, computers and intelligence (C4i) environments. During the design, development and evaluation of novel systems, technology and procedures, valid and reliable situation awareness measurement techniques are required for the assessment of individual and team SA, in order to determine the improvements (or in some cases decrements) resulting from proposed design and technological interventions. The paper presents a review of existing situation awareness measurement techniques for their suitability for use in the assessment of SA in C4i environments. Seventeen SA measures were evaluated against a set of HF methods criteria. It was concluded that current SA measurement techniques are inadequate by themselves for use in the assessment of SA in C4i environments, and a multiple-measure approach utilising different approaches is recommended.

[1]  Guy H. Walker,et al.  Human Factors Design and Evaluation Methods Review , 2004 .

[2]  Barry Kirwan,et al.  A Guide To Task Analysis: The Task Analysis Working Group , 1992 .

[3]  Mica R. Endsley,et al.  Theoretical Underpinnings of Situation Awareness, A Critical Review , 2000 .

[4]  Richard L Wampler,et al.  Modeling and Measuring Situation Awareness in the Infantry Operational Environment , 2000 .

[5]  L. Gugerty.  Situation Awareness During Driving: Explicit and Implicit Knowledge in Dynamic Spatial Memory , 1997 .

[6]  Christian Lebiere,et al.  Cockpit System Situational Awareness Modeling Tool , 2004 .

[7]  Kip Smith,et al.  Situation Awareness Is Adaptive, Externally Directed Consciousness , 1995, Hum. Factors.

[8]  M. R. Houck,et al.  Tools for assessing situational awareness in an operational fighter environment. , 1994, Aviation, space, and environmental medicine.

[9]  Mica R. Endsley,et al.  Predictive Utility of an Objective Measure of Situation Awareness , 1990 .

[10]  Eduardo Salas,et al.  Situation Awareness in Team Performance: Implications for Measurement and Training , 1995, Hum. Factors.

[11]  N A Stanton,et al.  Distributed situation awareness in dynamic systems: theoretical development and application of an ergonomics methodology , 2006, Ergonomics.

[12]  S. Stevenage,et al.  Learning to predict human error: issues of acceptability, reliability and validity. , 1998, Ergonomics.

[13]  Neville A. Stanton,et al.  Error by design: methods for predicting device usability , 2002 .

[14]  B Kirwan,et al.  Human error identification in human reliability assessment. Part 2: Detailed comparison of techniques. , 1992, Applied ergonomics.

[15]  Klaus Eyferth,et al.  Securing future ATM-concepts' safety by measuring situation awareness in ATC , 2003 .

[16]  Ernest J. Savoie.  Tapping the Power of Teams , 2002 .

[17]  Michael A. Vidulich,et al.  Testing a Subjective Metric of Situation Awareness , 1991 .

[18]  Janice Redish,et al.  Task analysis , 2002 .

[19]  Guy H. Walker,et al.  Measuring and predicting SA in C4I; development and testing of a refined SA measurement technique, and a new concept for SA prediction , 2004 .

[20]  H. Artman,et al.  Situation Awareness as Distributed Cognition , 2004 .

[21]  G. Bedny,et al.  Theory of Activity and Situation Awareness , 1999 .

[22]  Mica R. Endsley,et al.  Experimental Analysis and Measurement of Situation Awareness , 1995 .

[23]  John Piggott,et al.  Situational awareness and safety , 2001 .

[24]  Mica R. Endsley,et al.  A Survey of Situation Awareness Requirements in Air-to-Air Combat Fighters , 1993 .

[25]  David N. Hogg,et al.  Development of a situation awareness measure to evaluate advanced alarm systems in nuclear power plant control rooms , 1995 .

[26]  C Baber,et al.  Human error identification techniques applied to public technology: predictions compared with observed use. , 1996, Applied ergonomics.

[27]  C. J. Harris,et al.  Advances in Command, Control and Communication Systems , 1987 .

[28]  Mark S. Young,et al.  What price ergonomics? , 1999, Nature.

[29]  K. Dennehy,et al.  Cranfield situation awareness scale , 1997 .

[30]  E. Jeannot,et al.  The Development of Situation Awareness Measures in ATM Systems , 2003 .

[31]  Guy H. Walker,et al.  Verbal Protocol Analysis , 2004, Encyclopedia of Personality and Individual Differences.

[32]  B Kirwan,et al.  Human error identification techniques for risk assessment of high risk systems--part 2: towards a framework approach. , 1998, Applied ergonomics.

[33]  Danko Nikolić,et al.  SITUATION AWARENESS AS A PREDICTOR OF PERFORMANCE IN EN ROUTE AIR TRAFFIC CONTROLLERS , 1998 .

[34]  B. McGuinness,et al.  Quantitative Analysis of Situational Awareness (QUASA): Applying Signal Detection Theory to True/False Probes and Self-Ratings , 2004 .

[35]  Debra Jones,et al.  Situation Awareness Measurement and the Situation Awareness Global Assessment Technique , 2004 .

[36]  G. Klein,et al.  Cognitive Task Analysis of Teams , 2000 .

[37]  Barry McGuinness,et al.  Assessing Human Factors in Command and Control: Workload and Situational Awareness Metrics , 2002 .

[38]  B Kirwan,et al.  Human error identification techniques for risk assessment of high risk systems--Part 1: Review and evaluation of techniques. , 1998, Applied ergonomics.

[39]  Mica R. Endsley,et al.  Situation awareness global assessment technique (SAGAT) , 1988, Proceedings of the IEEE 1988 National Aerospace and Electronics Conference.

[40]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[41]  Mica R Endsley,et al.  SITUATION AWARENESS IN AIR TRAFFIC CONTROL: THE PICTURE. , 1998 .

[42]  Mica R. Endsley,et al.  Measurement of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[43]  I D Brown,et al.  A REVIEW OF THE 'LOOKED BUT FAILED TO SEE' ACCIDENT CAUSATION FACTOR , 2002 .

[44]  Xianjun Sam Zheng,et al.  Dynamic monitoring of traffic flow: The driver's situation awareness , 2004 .

[45]  Nancy J. Cooke,et al.  Measuring Team Knowledge , 2000, Hum. Factors.

[46]  Michael D. Matthews,et al.  Assessing Situation Awareness in Field Training Exercises , 2002 .

[47]  Neville A Stanton,et al.  Giving ergonomics away? The application of ergonomics methods by novices. , 2003, Applied ergonomics.

[48]  Valerie L. Shalin,et al.  Cognitive task analysis , 2000 .

[49]  P. Hancock,et al.  Human Mental Workload , 1988 .

[50]  Earl S. Stein,et al.  Human factors in air traffic control , 1998 .