Modern Flight Deck Automation: Pilots’ Mental Model and Monitoring Patterns and Performance

This paper describes how substantial empirical evidence from surveys and simulator studies indicates that “glass cockpit” pilots sometimes lose track of the status and behavior of automated flight deck systems and, as a result, experience “automation surprises.” A number of related factors are assumed to contribute to these problems, including the nature of current automation feedback and gaps and misconceptions in pilots' understanding of the automation. To date, most research on pilot-automation interaction has focused on subjective accounts and on performance outcome measures. Little is known about underlying processes, including how pilots monitor the automation and at what stages their information processing tends to break down. To fill this gap, a simulator study was conducted where twenty 747-400 pilots flew a routine one-hour flight on a fixed-base 747-400 simulator. Several scenario events were introduced to assess pilots’ monitoring behavior and their awareness of automation status and behavior. Throughout the scenario, behavioral and performance data as well as eye fixations were recorded. After the scenario was complete, pilots' mental model of the automation was probed based on a predefined set of questions. Overall, the findings from this research confirm that pilots experience considerable problems with monitoring the automation on modern glass cockpit aircraft. There is considerable diversity across pilots in terms of the frequency, duration, and pattern of scanning automation indications. Also, during the debriefing, pilots revealed significant gaps in their understanding of some of the automation features. The results from this study – both in terms of process and outcome measures – will be discussed in terms of their implications for improving training and design for effective pilot-automation collaboration.