The Effects of Level of Automation on the Out-of-the-Loop Unfamiliarity in a Complex Dynamic Fault-Management Task during Simulated Spaceflight Operations

Out-of-the-loop unfamiliarity (OOTLUF) and information sampling strategies were examined after operator use of intelligent fault management (FM) support in interaction with a generic, autonomous, atmospheric control system. A simulated model-based reasoning agent provided fault diagnosis and recovery at three different levels of automation (LOA). In about 10% of the experimental trials FM support was withdrawn simulating a catastrophic failure. This allowed investigating OOTLUF potentially developed during reliable FM automation and its variation as a function of LOA. Dependent measures were the percentage of time the system was out-of-target, time elapsed until a repair was initiated, alarm reaction time, accuracy of manual tank level recordings, information sampling behavior, and subjective ratings of workload, tension, fatigue, self-confidence and trust in automation. It was found that automated FM support improved overall system performance and reduced subjective operator workload. OOTLUF occurred in terms of longer times until repair initiation only at medium LOA and not as expected at high LOA. A differential effect of LOA on information sampling strategies was observed suggesting that offloading operators from recovery implementation at high LOA during reliable automation enabled engagement in fault assessment activities, thereby maintaining situation awareness. It is concluded that for avoiding OOTLUF in complex, dynamic tasks, involving intelligent FM support, the important factor is not simply the LOA, but the ability of a particular LOA to support the human operator's information sampling and higher-level reasoning activities.

[1]  J Sauer,et al.  Performance evaluation in analogue space environments: adaptation during an 8-month Antarctic wintering-over expedition. , 1999, Aviation, space, and environmental medicine.

[2]  Jürgen S. Sauer,et al.  Effects of Sleep Deprivation and User Interface on Complex Performance: A Multilevel Analysis of Compensatory Control , 1998, Hum. Factors.

[3]  Renwick E. Curry,et al.  Flight-deck automation: promises and problems , 1980 .

[4]  Charles E. Billings,et al.  Aviation Automation: The Search for A Human-centered Approach , 1996 .

[5]  J Sauer,et al.  Maintenance of complex performance during a 135-day spaceflight simulation. , 1999, Aviation, space, and environmental medicine.

[6]  J Sauer,et al.  Effects of training on short- and long-term skill retention in a complex multiple-task environment , 2000, Ergonomics.

[7]  N Moray,et al.  Trust, control strategies and allocation of function in human-machine systems. , 1992, Ergonomics.

[8]  Mica R. Endsley,et al.  The Out-of-the-Loop Performance Problem and Level of Control in Automation , 1995, Hum. Factors.

[9]  M R Endsley,et al.  Level of automation effects on performance, situation awareness and workload in a dynamic control task. , 1999, Ergonomics.

[10]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[11]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[12]  Christopher D. Wickens,et al.  Failure Detection in Dynamic Systems , 1981 .

[13]  DAVID MILLER,et al.  Information and Communication , 1966, Nature.