Development and Validation of a Multilevel Model for Predicting Workload Under Routine and Nonroutine Conditions in an Air Traffic Management Center

Objective: The aim of this study was to develop a model capable of predicting variability in the mental workload experienced by frontline operators under routine and nonroutine conditions. Background: Excess workload is a risk that needs to be managed in safety-critical industries. Predictive models are needed to manage this risk effectively yet are difficult to develop. Much of the difficulty stems from the fact that workload prediction is a multilevel problem. Method: A multilevel workload model was developed in Study 1 with data collected from an en route air traffic management center. Dynamic density metrics were used to predict variability in workload within and between work units while controlling for variability among raters. The model was cross-validated in Studies 2 and 3 with the use of a high-fidelity simulator. Results: Reported workload generally remained within the bounds of the 90% prediction interval in Studies 2 and 3. Workload crossed the upper bound of the prediction interval only under nonroutine conditions. Qualitative analyses suggest that nonroutine events caused workload to cross the upper bound of the prediction interval because the controllers could not manage their workload strategically. Conclusion: The model performed well under both routine and nonroutine conditions and over different patterns of workload variation. Application: Workload prediction models can be used to support both strategic and tactical workload management. Strategic uses include the analysis of historical and projected workflows and the assessment of staffing needs. Tactical uses include the dynamic reallocation of resources to meet changes in demand.

[1]  R M Rose,et al.  Objective job difficulty, behavioural response, and sector characteristics in air route traffic control centres. , 1978, Ergonomics.

[2]  Shayne Loft,et al.  Modeling and Predicting Mental Workload in En Route Air Traffic Control: Critical Review and Broader Implications , 2007, Hum. Factors.

[3]  Kevin M. Corker,et al.  AN ARCHITECTURE AND MODEL FOR COGNITIVE ENGINEERING SIMULATION ANALYSIS: APPLICATION TO ADVANCED AVIATION AUTOMATION , 1993 .

[4]  G. Innes,et al.  Prospective time study derivation of emergency physician workload predictors. , 2005, CJEM.

[5]  Daniel Gopher,et al.  Workload: An examination of the concept. , 1986 .

[6]  Raja Parasuraman,et al.  Workload Modeling and Workload Management: Recent Theoretical Developments , 2005 .

[7]  P. Kopardekar.  1 MEASUREMENT AND PREDICTION OF DYNAMIC DENSITY , 2004 .

[8]  Anthony S. Bryk,et al.  A Multilevel Model of the Social Distribution of High School Achievement. , 1989 .

[9]  Stephen W. Raudenbush,et al.  Efficiency and Robustness of Alternative Estimators for Two- and Three-level Models: The Case of NAEP , 2001 .

[10]  Shayne Loft,et al.  An analysis of relational complexity in an air traffic control conflict detection task , 2006, Ergonomics.

[11]  Arnab Majumdar,et al.  Factors Affecting Air Traffic Controller Workload: Multivariate Analysis Based on Simulation Modeling of Controller Workload , 2002 .

[12]  E PaterdeI.,et al.  Psychological and physiological reactions to high workloads: implications for well-being , 2010 .

[13]  R Weston,et al.  HUMAN FACTORS IN AIR TRAFFIC CONTROL , 1982 .

[14]  D. Gianola,et al.  Mixed effects linear models with t-distributions for quantitative genetic analysis: a Bayesian approach , 1999, Genetics Selection Evolution.

[15]  Richard W. Pew,et al.  Human-system integration in the system development process : a new look , 2007 .

[16]  Mary M. Omodei,et al.  How Expertise Is Applied in Real-World Dynamic Environments: Head-Mounted Video and Cued Recall as a Methodology for Studying Routines of Decision Making , 2014 .

[17]  J. C. Spearandio.  Variation of operator's strategies and regulating effects on workload. , 1971, Ergonomics.

[18]  Mark S. Young,et al.  Fundamental examination of mental workload in the rail industry , 2005 .

[19]  Andrew Neal,et al.  An examination of the relationship between workload and fatigue within and across consecutive days of work: is the relationship static or dynamic? , 2009, Journal of occupational health psychology.

[20]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[21]  K. Ronald Laughery,et al.  The integrated performance modeling environment—simulating human-system performance , 1997, WSC '97.

[22]  Andrew Neal,et al.  An Evidence Accumulation Model for Conflict Detection Performance in a Simulated Air Traffic Control Task , 2009, Hum. Factors.

[23]  L J Muzzin,et al.  Workload, Data Gathering, and Quality of Community Pharmacists' Advice , 1995, Medical care.

[24]  Glen A. Gilbert,et al.  Air traffic control , 1973, Nature.

[25]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[26]  Scott M. Galster,et al.  Air Traffic Controller Performance and Workload Under Mature Free Flight: Conflict Detection and Resolution of Aircraft Self-Separation , 2001 .

[27]  Carol A. Manning,et al.  INVESTIGATING THE VALIDITY OF PERFORMANCE AND OBJECTIVE WORKLOAD EVALUATION RESEARCH (POWER) , 2001 .

[28]  D. C. Howell.  Statistical methods for psychology, 3rd ed. , 1992 .

[29]  Tyler H. Shaw,et al.  Effects of warned and unwarned demand transitions on vigilance performance and stress , 2008, Anxiety, stress, and coping.

[30]  David Dahn,et al.  The Integrated Performance Modeling Environment a Tool for Simulating Human-System Performance , 1997 .

[31]  Shayne Loft,et al.  A theory and model of conflict detection in air traffic control: incorporating environmental constraints. , 2009, Journal of experimental psychology. Applied.

[32]  M. Gauthier,et al.  Validation of Workload Algorithms Within the Integrated Performance Modeling Environment , 2007 .

[33]  A. Diez-Roux.  Multilevel analysis in public health research. , 2000, Annual review of public health.

[34]  Tom Kontogiannis,et al.  Managing emergencies and abnormal situations in air traffic control (part I): taskwork strategies. , 2010, Applied ergonomics.

[35]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[36]  Neville Moray,et al.  Models and Measures of Mental Workload , 1979 .

[37]  Silke L. Schneider.  Anti-Immigrant Attitudes in Europe: Outgroup Size and Perceived Ethnic Threat , 2007 .

[38]  G. Matthews,et al.  Task-induced fatigue states and simulated driving performance , 2002, The Quarterly journal of experimental psychology. A, Human experimental psychology.