Level of automation and failure frequency effects on simulated lunar lander performance

A human-in-the-loop experiment was conducted at the NASA Ames Research Center Vertical Motion Simulator, where instrument-rated pilots completed a simulated terminal descent phase of a lunar landing. Ten pilots participated in a 2 x 2 mixed design experiment, with level of automation as the within-subjects factor and failure frequency as the between-subjects factor. The two evaluated levels of automation were high (fully automated landing) and low (manual controlled landing). During test trials, participants were exposed to either a high number of failures (75% failure frequency) or low number of failures (25% failure frequency). In order to investigate the pilots' sensitivity to changes in levels of automation and failure frequency, the dependent measure selected for this experiment was accuracy of failure diagnosis, from which D Prime and Decision Criterion were derived. For each of the dependent measures, no significant difference was found for level of automation and no significant interaction was detected between level of automation and failure frequency. A significant effect was identified for failure frequency suggesting failure frequency has a significant effect on pilots' sensitivity to failure detection and diagnosis. Participants were more likely to correctly identify and diagnose failures if they experienced the higher levels of failures, regardless of level of automation.

[1]  Thomas B. Sheridan,et al.  Human and Computer Control of Undersea Teleoperators , 1978 .

[2]  Christopher D. Wickens,et al.  Situation Awareness: Review of Mica Endsley's 1995 Articles on Situation Awareness Theory and Measurement , 2008, Hum. Factors.

[3]  Karl D. Bilimoria,et al.  Effects of Control Power and Inceptor Sensitivity on Lunar Lander Handling Qualities , 2011 .

[4]  Mica R. Endsley,et al.  Automation and situation awareness. , 1996 .

[5]  Mica R. Endsley,et al.  The Application of Human Factors to the Development of Expert Systems for Advanced Cockpits , 1987 .

[6]  Mustapha Mouloua,et al.  Automation and Human Performance : Theory and Applications , 1996 .

[7]  Alex M. Andrew,et al.  Humans and Automation: System Design and Research Issues , 2003 .

[8]  Raja Parasuraman,et al.  Performance Consequences of Automation-Induced 'Complacency' , 1993 .

[9]  Mica R. Endsley,et al.  The Out-of-the-Loop Performance Problem and Level of Control in Automation , 1995, Hum. Factors.

[10]  Joseph B. Lyons,et al.  Human–Human Reliance in the Context of Automation , 2012, Hum. Factors.

[11]  Earl L. Wiener,et al.  Human factors of advanced technology (glass cockpit) transport aircraft , 1989 .

[12]  M R Endsley,et al.  Level of automation effects on performance, situation awareness and workload in a dynamic control task. , 1999, Ergonomics.

[13]  Raja Parasuraman,et al.  Humans and Automation: Use, Misuse, Disuse, Abuse , 1997, Hum. Factors.

[14]  Michael Cobb.  Security solutions , 1997 .

[15]  Christopher D. Wickens,et al.  Automation Reliability in Unmanned Aerial Vehicle Control: A Reliance-Compliance Model of Automation Dependence in High Workload , 2006, Hum. Factors.

[16]  Raja Parasuraman,et al.  Human-Automation Interaction , 2005 .

[17]  G. Jamieson,et al.  CONSIDERING SUBJECTIVE TRUST AND MONITORING BEHAVIOR IN ASSESSING AUTOMATION-INDUCED “COMPLACENCY” , 2004 .

[18]  John M. Lockhart,et al.  Automation and Supervisory Control: A Perspective on Human Performance, Training, and Performance Aiding , 1993 .

[19]  Christopher D. Wickens,et al.  On the Independence of Compliance and Reliance: Are Automation False Alarms Worse Than Misses? , 2007, Hum. Factors.

[20]  Kenneth Y. Goldberg,et al.  Automation , 2018, AWS® Certified Advanced Networking Official Study Guide.

[21]  Vyacheslav P. Tuzlukov,et al.  Signal detection theory , 2001 .

[22]  Raja Parasuraman,et al.  The Effects of Level of Automation on the Out-of-the-Loop Unfamiliarity in a Complex Dynamic Fault-Management Task during Simulated Spaceflight Operations , 2001 .

[23]  M. Endsley.  Automation and situation awareness. , 1996 .

[24]  Charles M. Oman,et al.  Pilot Performance, Workload, and Situation Awareness During Lunar Landing Mode Transitions , 2013 .

[25]  D. Woods,et al.  Automation Surprises , 2001 .

[26]  Christopher D. Wickens,et al.  The Transfer of Failure-Detection Skills between Monitoring and Controlling Dynamic Systems , 1982 .

[27]  David A Simmon.  BOEING 757 CFIT ACCIDENT AT CALI, COLOMBIA, BECOMES FOCUS OF LESSONS LEARNED , 1998 .

[28]  Victor A. Riley,et al.  A General Model of Mixed-Initiative Human-Machine Systems , 1989 .