Prediction in Situation Awareness: Confidence Bias and Underlying Cognitive Abilities

Sixteen military fighter pilots performed and air combat simulation, where their Levels 1, 2, and 3 situational awareness (SA) was assessed using the Situation Awareness Global Assessment Technique (SAGAT). Their confidence levels on SA were also elicited to derive overconfidence scores. The pilots had the lowest accuracy and highest overconfidence in their answers to prediction questions (Level 3 SA). The cognitive abilities associated with prediction were different from those associated with the lower levels of SA, in which a math aptitude (reasoning) test was highly associated with prediction. Through regression analysis, we found that both prediction and overconfidence bias were significant predictors of pilots' performance measured by mission survivability.

[1]  D. Segal,et al.  Comprehensive handbook of psychological assessment, Vol. 2: Personality assessment. , 2004 .

[2]  M. Endsley,et al.  Individual Differences in Pilot Situation Awareness , 1994 .

[3]  Mica R. Endsley,et al.  Situation awareness global assessment technique (SAGAT) , 1988, Proceedings of the IEEE 1988 National Aerospace and Electronics Conference.

[4]  Earl L. Wiener.  Controlled Flight into Terrain Accidents: System-Induced Errors , 1977 .

[5]  David Crundall,et al.  What happens next? Predicting other road users' behaviour as a function of driving experience and processing time , 2009, Ergonomics.

[6]  R. Hogarth,et al.  Confidence in judgment: Persistence of the illusion of validity. , 1978 .

[7]  Danko Nikolić,et al.  Expertise and chess: A pilot study comparing situation awareness methodologies , 1995 .

[8]  Mica R. Endsley,et al.  Direct Measurement of Situation Awareness: Validity and Use of SAGAT , 2000 .

[9]  K. A. Ericsson,et al.  Long-term working memory. , 1995, Psychological review.

[10]  Derek J. Koehler,et al.  Heuristics and Biases: The Calibration of Expert Judgment: Heuristics and Biases Beyond the Laboratory , 2002 .

[11]  Nassim Nicholas Taleb,et al.  The Black Swan: The Impact of the Highly Improbable , 2007 .

[12]  S. Lichtenstein,et al.  Do those who know more also know more about how much they know?*1 , 1977 .

[13]  Mica R. Endsley,et al.  Measurement of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[14]  Stephanie M. Doane,et al.  Memory Processes of Flight Situation Awareness: Interactive Roles of Working Memory Capacity, Long-Term Working Memory, and Expertise , 2004, Hum. Factors.

[15]  D B Kaber,et al.  Situation awareness and driving performance in a simulated navigation task , 2007, Ergonomics.

[16]  H. Artman,et al.  Situation Awareness as Distributed Cognition , 2004 .

[17]  T R Carretta,et al.  Prediction of situational awareness in F-15 pilots. , 1996, The International journal of aviation psychology.

[18]  Andrew R. Dattel,et al.  Does Situation Awareness Add to the Validity of Cognitive Tests? , 2006, Hum. Factors.

[19]  M R Endsley,et al.  Sources of situation awareness errors in aviation. , 1996, Aviation, space, and environmental medicine.

[20]  Stephanie M. Doane,et al.  Pilot Ability to Anticipate the Consequences of Flight Actions as a Function of Expertise , 2004, Hum. Factors.

[21]  D O'Hare,et al.  Situational awareness ability and cognitive skills training in a complex real-world task , 2007, Ergonomics.

[22]  P. Tetlock.  Expert Political Judgment: How Good Is It? How Can We Know? , 2005 .

[23]  David Woods,et al.  Situation Awareness: A Critical But Ill-Defined Phenomenon , 1991 .

[24]  B. Fischhoff,et al.  Calibration of probabilities: the state of the art to 1980 , 1982 .

[25]  A. Tversky,et al.  On the psychology of prediction , 1973 .

[26]  Thomas D. Hardiman,et al.  A Comparative Analysis of Sagat and Sart for Evaluations of Situation Awareness , 1998 .

[27]  Christopher D Wickens,et al.  Effects of conflict alerting system reliability and task difficulty on pilots' conflict detection with cockpit display of traffic information , 2007, Ergonomics.

[28]  Young Woo Sohn,et al.  Roles of working memory capacity and long-term working memory skill in complex task performance , 2003, Memory & cognition.

[29]  Edwin Hutchins.  How a Cockpit Remembers Its Speeds , 1995 .

[30]  Sidney W. A. Dekker,et al.  MABA-MABA or Abracadabra? Progress on Human–Automation Co-ordination , 2002, Cognition, Technology & Work.

[31]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[32]  Leo van Breda,et al.  Capability Prediction: An Effective Way to Improve Navigational Performance , 2000, Journal of Navigation.

[33]  P M Salmon,et al.  Representing situation awareness in collaborative systems: A case study in the energy distribution domain , 2008, Ergonomics.

[34]  F. McKenna,et al.  Skilled anticipation in real-world tasks: measurement of attentional demands in the domain of tennis. , 2001, Journal of experimental psychology. Applied.

[35]  David N. Hogg,et al.  Development of a situation awareness measure to evaluate advanced alarm systems in nuclear power plant control rooms , 1995 .

[36]  R. Jensen.  Pilot Judgment: Training and Evaluation , 1982 .

[37]  Charmine E. J. Härtel,et al.  Defining aircrew coordination: Searching mishaps for meaning , 1989 .

[38]  Francis T. Durso,et al.  Comprehension and Situation Awareness , 2008 .

[39]  Danko Nikolić,et al.  SITUATION AWARENESS AS A PREDICTOR OF PERFORMANCE IN EN ROUTE AIR TRAFFIC CONTROLLERS , 1998 .

[40]  Mica R. Endsley,et al.  Designing for Situation Awareness : An Approach to User-Centered Design , 2003 .

[41]  F. Lichacz.  Augmenting understanding of the relationship between situation awareness and confidence using calibration analysis , 2008, Ergonomics.

[42]  Jens Alfredson,et al.  Differences in Situational Awareness and How to Manage Them in Development of Complex Systems , 2007 .

[43]  Mica R. Endsley,et al.  A Survey of Situation Awareness Requirements in Air-to-Air Combat Fighters , 1993 .

[44]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[45]  N A Stanton,et al.  Distributed situation awareness in dynamic systems: theoretical development and application of an ergonomics methodology , 2006, Ergonomics.

[46]  Brad Cain,et al.  Calibration of Confidence in Situation Awareness Queries , 2003 .

[47]  Christopher D. Wickens,et al.  Exploring the Concept of Team Situation Awareness in a Simulated Air Combat Environment , 2009 .

[48]  B. Fischhoff,et al.  Subjective Confidence in Forecasts. , 1982 .

[49]  Christopher D. Wickens,et al.  Situation Awareness: Review of Mica Endsley's 1995 Articles on Situation Awareness Theory and Measurement , 2008, Hum. Factors.

[50]  Erik Hollnagel,et al.  Human factors and folk models , 2004, Cognition, Technology & Work.

[51]  Christopher D. Wickens,et al.  Workload and Reliability of Predictor Displays in Aircraft Traffic Avoidance , 2000 .

[52]  Christopher D. Wickens,et al.  The Future of Air Traffic Control: Human Operators and Automation , 1998 .