Does What We See Shape History? Examining Workload History as a Function of Performance and Ambient/Focal Visual Attention

Changes in task demands can have delayed adverse impacts on performance. This phenomenon, known as the workload history effect, is especially of concern in dynamic work domains where operators manage fluctuating task demands. The existing workload history literature does not depict a consistent picture regarding how these effects manifest, prompting research to consider measures that are informative on the operator's process. One promising measure is visual attention patterns, due to its informativeness on various cognitive processes. To explore its ability to explain workload history effects, participants completed a task in an unmanned aerial vehicle command and control testbed where workload transitioned gradually and suddenly. The participants’ performance and visual attention patterns were studied over time to identify workload history effects. The eye-tracking analysis consisted of using a recently developed eye-tracking metric called coefficient K, as it indicates whether visual attention is more focal or ambient. The performance results found workload history effects, but it depended on the workload level, time elapsed, and performance measure. The eye-tracking analysis suggested performance suffered when focal attention was deployed during low workload, which was an unexpected finding. When synthesizing these results, they suggest unexpected visual attention patterns can impact performance immediately over time. Further research is needed; however, this work shows the value of including a real-time visual attention measure, such as coefficient K, as a means to understand how the operator manages varying task demands in complex work environments.

[1]  Joseph T. Coyne,et al.  Research Considerations for Managing Future Unmanned Systems , 2015, AAAI Spring Symposia.

[2]  Peter A. Hancock,et al.  The Effect of Prior Task Loading on Mental Workload , 2011, Hum. Factors.

[3]  Tyler H. Shaw,et al.  Effects of warned and unwarned demand transitions on vigilance performance and stress , 2008, Anxiety, stress, and coping.

[4]  R R Rosa,et al.  Demand transitions and sustained attention. , 1993, The Journal of general psychology.

[5]  T. Gog,et al.  In the eyes of the beholder: How experts and novices interpret dynamic stimuli , 2010 .

[6]  C. Erkelens,et al.  Coarse-to-fine eye movement strategy in visual search , 2007, Vision Research.

[7]  C. Wickens,et al.  Workload transition: Implications for individual and team performance. , 1993 .

[8]  D. E. Irwin,et al.  Eye movements and scene perception: Memory for things observed , 2002, Perception & psychophysics.

[9]  Robert A Goldberg,et al.  Memory overload or expectancy effect? 'Hysteresis' revisited. , 1980, Ergonomics.

[10]  J. Warm,et al.  Effects of shifts in the rate of repetitive stimulation on sustained attention , 1975 .

[11]  Krzysztof Krejtz,et al.  Using Microsaccades to Estimate Task Difficulty During Visual Search of Layered Surfaces , 2020, IEEE Transactions on Visualization and Computer Graphics.

[12]  Amanda D. Angie,et al.  The effects of workload history on dual task performance , 2005 .

[13]  Christopher D. Wickens,et al.  Eye-tracking and Individual Differences in off-Normal Event Detection when Flying with a Synthetic Vision System Display , 2004 .

[14]  Christopher D. Wickens,et al.  Transitions Between Low and High Levels of Mental Workload can Improve Multitasking Performance , 2020, IISE transactions on occupational ergonomics and human factors.

[15]  F. Previc The neuropsychology of 3-D space. , 1998, Psychological bulletin.

[16]  Melissa Patricia Coral Analyzing Cognitive Workload through Eye-Related Measurements: A Meta-Analysis , 2016 .

[17]  Gregory L Feitshans,et al.  Vigilant Spirit Control Station (VSCS) "The Face of COUNTER" , 2008 .

[18]  David B. Kaber,et al.  The Underpinnings of Workload in Unmanned Vehicle Systems , 2018, IEEE Transactions on Human-Machine Systems.

[19]  G. T. Buswell How People Look At Pictures: A Study Of The Psychology Of Perception In Art , 2012 .

[20]  M. Scerbo,et al.  Changes in stress and subjective workload over time following a workload transition , 2015 .

[21]  Philip S. E. Farrell,et al.  The Hysteresis Effect , 1999, Hum. Factors.

[22]  J. Helmert,et al.  Visual Fixation Durations and Saccade Amplitudes: Shifting Relationship in a Variety of Conditions , 2008 .

[23]  Keith S. Karn,et al.  Commentary on Section 4. Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. , 2003 .

[24]  G. R. J. Hockey Compensatory control in the regulation of human performance under stress and high workload: A cognitive-energetical framework , 1997, Biological Psychology.

[25]  Chang S. Nam,et al.  Neural Correlates of Workload Transition in Multitasking: An ACT-R Model of Hysteresis Effect , 2019, Front. Hum. Neurosci..

[26]  Linden J. Ball,et al.  Eye tracking in HCI and usability research. , 2006 .

[27]  Andrew T. Duchowski,et al.  Gaze transitions when learning with multimedia , 2016 .

[28]  Bing Pan,et al.  The determinants of web page viewing behavior: an eye-tracking study , 2004, ETRA.

[29]  Michela Terenzi,et al.  A Random Glance at the Flight Deck: Pilots' Scanning Strategies and the Real-Time Assessment of Mental Workload , 2007 .

[30]  Peter A. Hancock,et al.  Workload and Performance: Associations, Insensitivities, and Dissociations , 2018, Hum. Factors.

[31]  Daqing He,et al.  Searching, browsing, and clicking in a search session: changes in user behavior by task and over time , 2014, SIGIR.

[32]  P. Hancock,et al.  A Dynamic Model of Stress and Sustained Attention , 1989, Human factors.

[33]  Wonjoon Kim,et al.  Behavioral and Neural Correlates of Hysteresis Effects during Multitasking , 2018 .

[34]  Lauren Reinerman-Jones,et al.  Impact of three task demand factors on simulated unmanned system intelligence, surveillance, and reconnaissance operations , 2017, Ergonomics.

[35]  Andrew T. Duchowski,et al.  Discerning Ambient/Focal Attention with Coefficient K , 2016, ACM Trans. Appl. Percept..

[36]  M L Matthews The Influence of Visual Workload History on Visual Performance , 1986, Human factors.

[37]  Tamsyn Edwards,et al.  The Relationship between Workload and Performance in Air Traffic Control , 2017, H-WORKLOAD.

[38]  S. B. Hutton,et al.  Eye Tracking Methodology , 2019, Eye Movement Research.

[39]  Andrew T. Duchowski,et al.  Entropy-based statistical analysis of eye movement transitions , 2014, ETRA.

[40]  Huib de Ridder,et al.  Hysteresis in Mental Workload and Task Performance , 2016, Hum. Factors.

[41]  Majid Fallahi,et al.  Analysis of the mental workload of city traffic control operators while monitoring traffic density: A field study , 2016 .

[42]  B. Velichkovsky,et al.  Two Visual Systems and Their Eye Movements: Evidence from Static and Dynamic Scene Perception , 2005 .

[43]  Amanda D. Angie,et al.  Extraversion and task performance: A fresh look through the workload history lens , 2006 .

[44]  J. Hittner,et al.  Effects of neuroticism and workload history on performance , 2004 .

[45]  Sara L. Riggs,et al.  Effects of Workload and Workload Transitions on Attention Allocation in a Dual-Task Environment: Evidence From Eye Tracking Metrics , 2020 .

[46]  Joseph H. Goldberg,et al.  Computer interface evaluation using eye movements: methods and constructs , 1999 .

[47]  Michael C. Dorneich,et al.  Towards a Characterization of Adaptive Systems: a Framework for Researchers and System Designers , 2017 .

[48]  James C. Christensen,et al.  The Effects of Workload Transitions in a Multitasking Environment , 2014 .

[49]  Andrew T Duchowski,et al.  A breadth-first survey of eye-tracking applications , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[50]  Joseph H. Goldberg,et al.  Identifying fixations and saccades in eye-tracking protocols , 2000, ETRA.

[51]  P A Hancock,et al.  Influence of task demand characteristics on workload and performance. , 1995, The International journal of aviation psychology.

[52]  Brian W. Moroney,et al.  Effects of Demand Transitions on Vigilance Performance and Perceived Workload , 1995 .

[53]  Majid Fallahi,et al.  Assessment of operators’ mental workload using physiological and subjective measures in cement, city traffic and power plant control centers , 2016, Health promotion perspectives.

[54]  Andrew T. Duchowski,et al.  Dynamics of emotional facial expression recognition in individuals with social anxiety , 2018, ETRA.

[55]  Ing,et al.  Review and reappraisal of adaptive interfaces : toward biologically inspired paradigms , 2002 .

[56]  Daniël Lakens,et al.  Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs , 2013, Front. Psychol..

[57]  Luz-Eugenia Cox-Fuenzalida,et al.  Effect of Workload History on Task Performance , 2007, Hum. Factors.

[58]  R. Cumming Human information processing under varying task demand. , 1973, Ergonomics.

[59]  Edward Cutrell,et al.  What are you looking for?: an eye-tracking study of information usage in web search , 2007, CHI.