Vigilance and Automation Dependence in Operation of Multiple Unmanned Aerial Systems (UAS): A Simulation Study

Objective: This simulation study investigated factors influencing sustained performance and fatigue during operation of multiple Unmanned Aerial Systems (UAS). The study tested effects of time-on-task and automation reliability on accuracy in surveillance tasks and dependence on automation. It also investigated the role of trait and state individual difference factors. Background: Warm’s resource model of vigilance has been highly influential in human factors, but further tests of its applicability to complex, real-world tasks requiring sustained attention are necessary. Multi-UAS operation differs from standard vigilance paradigms in that the operator must switch attention between multiple subtasks, with support from automation. Method: 131 participants performed surveillance tasks requiring signal discrimination and symbol counting with a multi-UAS simulation configured to impose low cognitive demands, for 2 hr. Automation reliability was manipulated between-groups. Five Factor Model personality traits were measured prior to performance. Subjective states were assessed with the Dundee Stress State Questionnaire. Results: Performance accuracy on the more demanding surveillance task showed a vigilance decrement, especially when automation reliability was low. Dependence on automation on this task declined over time. State but not trait factors predicted performance. High distress was associated with poorer performance in more demanding task conditions. Conclusions: Vigilance decrement may be an operational issue for multi-UAS surveillance missions. Warm’s resource theory may require modification to incorporate changes in information processing and task strategy associated with multitasking in low-workload, fatiguing environments. Application: Interface design and operator evaluation for multi-UAS operations should address issues including motivation, stress, and sustaining attention to automation.

[1]  Anthony P Tvaryanas,et al.  Fatigue in pilots of remotely piloted aircraft before and after shift work adjustment. , 2009, Aviation, space, and environmental medicine.

[2]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[3]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[4]  Joel S. Warm,et al.  Vigilance, Workload, and Stress , 2018 .

[5]  Heath A. Ruff,et al.  Automation-Level Transference Effects in Simulated Multiple Unmanned Aerial Vehicle Control , 2011 .

[6]  Gerald Matthews,et al.  Use of EEG Workload Indices for Diagnostic Monitoring of Vigilance Decrement , 2014, Hum. Factors.

[7]  Joel S. Warm,et al.  Vigilance Requires Hard Mental Work and Is Stressful , 2008, Hum. Factors.

[8]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[9]  K. Hendy,et al.  Measuring Subjective Workload: When Is One Scale Better Than Many? , 1993 .

[10]  James L. Szalma,et al.  Individual differences in human–technology interaction: incorporating variation in human characteristics into human factors and ergonomics research and design , 2009 .

[11]  Tyler H. Shaw,et al.  Individual differences in vigilance: Personality, ability and states of stress , 2010 .

[12]  Wendy A. Rogers,et al.  The Effect of Incorrect Reliability Information on Expectations, Perceptions, and Use of Automation , 2016, Hum. Factors.

[13]  R. McCrae,et al.  Self‐Regulation and the Five‐Factor Model of Personality Traits , 2010 .

[14]  Fiona M Donald.  The classification of vigilance tasks in the real world , 2008, Ergonomics.

[15]  Catherine Neubauer,et al.  Fatigue and Voluntary Utilization of Automation in Simulated Driving , 2012, Hum. Factors.

[16]  G. Robert J. Hockey,et al.  Performance in a Complex Multiple-Task Environment during a Laboratory-Based Simulation of Occasional Night Work , 2003, Hum. Factors.

[17]  Tyler H. Shaw,et al.  Predicting vigilance: A fresh look at an old problem , 2009, Ergonomics.

[18]  Petra Holtzmann.  Sustained Attention In Human Performance , 2016 .

[19]  Robert W Proctor,et al.  Cumulative knowledge and progress in human factors. , 2010, Annual review of psychology.

[20]  Mary L. Cummings,et al.  Boredom and Distraction in Multiple Unmanned Vehicle Supervisory Control , 2013, Interact. Comput..

[21]  Daniel G Bobrow,et al.  On data-limited and resource-limited processes , 1975, Cognitive Psychology.

[22]  B. Bergum,et al.  END-SPURT IN VIGILANCE. , 1963, Journal of experimental psychology.

[23]  Joel S. Warm,et al.  Detection-Action Linkage in Vigilance: Effects on Workload and Stress , 2007 .

[24]  Christopher D. Wickens,et al.  The benefits of imperfect diagnostic automation: a synthesis of the literature , 2007 .

[25]  Alexis R. Neigel,et al.  A Critical Examination of the Research and Theoretical Underpinnings Discussed in Thomson, Besner, and Smilek (2016) , 2017, Psychological review.

[26]  W. T. Nelson,et al.  Target Acquisition With UAVs: Vigilance Displays and Advanced Cuing Interfaces , 2005, Hum. Factors.

[27]  R. DeShon,et al.  Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. , 2002, Psychological methods.

[28]  G. R. J. Hockey.  Compensatory control in the regulation of human performance under stress and high workload: A cognitive-energetical framework , 1997, Biological Psychology.

[29]  Grant S. Taylor,et al.  Individual differences in response to automation: the five factor model of personality. , 2011, Journal of experimental psychology. Applied.

[30]  W. N. Dember,et al.  Tests of vigilance taxonomy. , 1998 .

[31]  Stephen Rice,et al.  Using System-Wide Trust Theory to Make Predictions About Dependence on Four Diagnostic Aids , 2010, The Journal of general psychology.

[32]  Joel S. Warm,et al.  Cerebral Hemodynamic Indices of Operator Fatigue in Vigilance , 2017 .

[33]  Derek Besner,et al.  A critical examination of the evidence for sensitivity loss in modern vigilance tasks. , 2015, Psychological review.

[34]  Lauren Reinerman-Jones,et al.  Metrics for individual differences in EEG response to cognitive workload: Optimizing performance prediction , 2017 .

[35]  Joel S. Warm,et al.  Task Engagement and Attentional Resources , 2017, Hum. Factors.

[36]  Edward M. Hitchcock,et al.  Active and passive fatigue in simulated driving: discriminating styles of workload regulation and their safety impacts. , 2013, Journal of experimental psychology. Applied.

[37]  Joseph B. Lyons,et al.  Being Transparent about Transparency: A Model for Human-Robot Interaction , 2013, AAAI Spring Symposium: Trust and Autonomous Systems.

[38]  Gerald Matthews,et al.  Dynamic relationships between stress states and working memory , 2010 .

[39]  William N. Dember,et al.  VIGILANCE AND WORKLOAD IN AUTOMATED SYSTEMS. , 1996 .

[40]  James L. Szalma,et al.  On the Application of Motivation Theory to Human Factors/Ergonomics , 2014, Hum. Factors.

[41]  P. Hancock,et al.  In search of vigilance: the problem of iatrogenically created psychological phenomena. , 2013, The American psychologist.

[42]  Joel S. Warm,et al.  The Effects of Signal Salience and Caffeine on Performance, Workload, and Stress in an Abbreviated Vigilance Task , 2000, Hum. Factors.

[43]  Peter A. Hancock,et al.  ACTIVE AND PASSIVE FATIGUE STATES , 2001 .

[44]  A. R. Panganiban,et al.  PROFILING TASK STRESS WITH THE DUNDEE STRESS STATE QUESTIONNAIRE , 2013 .

[45]  Peter A. Hancock,et al.  Human-Centered Design of Unmanned Aerial Vehicles , 2003 .

[46]  Joel S. Warm,et al.  The Psychophysics of Vigilance , 1980 .

[47]  Judi E. See,et al.  Meta-analysis of the sensitivity decrement in vigilance. , 1995 .

[48]  Joel S. Warm,et al.  The Cambridge Handbook of Applied Perception Research: Vigilance: A Perceptual Challenge , 2015 .

[49]  G. Matthews,et al.  Task-induced fatigue states and simulated driving performance , 2002, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[50]  William T. Thompson,et al.  Effects of Shift Work and Sustained Operations: Operator Performance in Remotely Piloted Aircraft (OP-REPAIR) , 2006 .

[51]  Amber Salinas,et al.  Facets of Occupational Burnout Among U.S. Air Force Active Duty and National Guard/Reserve MQ-1 Predator and MQ-9 Reaper Operators , 2011 .

[52]  Raja Parasuraman,et al.  Humans and Automation: Use, Misuse, Disuse, Abuse , 1997, Hum. Factors.

[53]  Rebecca A. Grier,et al.  Fundamental dimensions of subjective state in performance settings: task engagement, distress, and worry. , 2002, Emotion.

[54]  R R Mackie,et al.  Vigilance Research—Are We Ready for Countermeasures? , 1987, Human factors.

[55]  Raja Parasuraman,et al.  Complacency and Bias in Human Use of Automation: An Attentional Integration , 2010, Hum. Factors.

[56]  Kent McDonald,et al.  Psychological Health Screening of Remotely Piloted Aircraft (RPA) Operators and Supporting Units , 2011 .

[57]  Joel S. Warm,et al.  Predicting battlefield vigilance: a multivariate approach to assessment of attentional resources , 2014, Ergonomics.

[58]  J. Schooler,et al.  Vigilance impossible: Diligence, distraction, and daydreaming all lead to failures in a practical monitoring task , 2015, Consciousness and Cognition.

[59]  Joel S. Warm,et al.  Training Techniques for Visual Search in Complex Task Environments , 2017, Hum. Factors.

[60]  B. Oken,et al.  Vigilance, alertness, or sustained attention: physiological basis and measurement , 2006, Clinical Neurophysiology.

[61]  Heath A. Ruff,et al.  The Impact of Automation Reliability and Operator Fatigue on Performance and Reliance , 2016 .

[62]  J. Eastwood,et al.  Personality and boredom proneness in the prediction of creativity and curiosity , 2016 .

[63]  M. Eysenck,et al.  New perspectives in attentional control theory , 2011 .

[64]  Raja Parasuraman,et al.  Adaptable and Adaptive Automation for Supervisory Control of Multiple Autonomous Vehicles , 2012 .

[65]  Gerard Saucier,et al.  Orthogonal Markers for Orthogonal Factors: The Case of the Big Five , 2002 .

[66]  M. Russo,et al.  Visual neglect: occurrence and patterns in pilots in a simulated overnight flight. , 2004, Aviation, space, and environmental medicine.

[67]  Brad E. Sheese,et al.  The Cambridge Handbook of Personality Psychology , 2020 .

[68]  Gerald Matthews,et al.  Dangerous intersections? A review of studies of fatigue and distraction in the automated vehicle. , 2019, Accident; analysis and prevention.

[69]  Daniel Goldberg,et al.  Testing Adaptive Levels of Automation (ALOA) for UAV Supervisory Control , 2007 .

[70]  Robert R. McCrae,et al.  The Cambridge Handbook of Personality Psychology: The Five-Factor Model of personality traits: consensus and controversy , 2009 .

[71]  D. Washburn,et al.  Task engagement, cerebral blood flow velocity, and diagnostic monitoring for sustained attention. , 2010, Journal of experimental psychology. Applied.

[72]  Christopher D. Wickens,et al.  Humans: Still Vital After All These Years of Automation , 2008, Hum. Factors.