Towards continuous and real-time attention monitoring at work: reaction time versus brain response

Abstract Continuous and objective measurement of the user attention state still represents a major challenge in the ergonomics research. Recently available wearable electroencephalography (EEG) opens new opportunities for objective and continuous evaluation of operators’ attention, which may provide a new paradigm in ergonomics. In this study, wearable EEG was recorded during simulated assembly operation, with the aim to analyse P300 event-related potential component, which provides reliable information on attention processing. In parallel, reaction times (RTs) were recorded and the correlation between these two attention-related modalities was investigated. Negative correlation between P300 amplitudes and RTs has been observed on the group level (p < .001). However, on the individual level, the obtained correlations were not consistent. As a result, we propose the P300 amplitude for accurate attention monitoring in ergonomics research. On the other hand, no significant correlation between RTs and P300 latency was found on group, neither on individual level. Practitioner Summary: Ergonomic studies of assembly operations mainly investigated physical aspects, while mental states of the assemblers were not sufficiently addressed. Presented study aims at attention tracking, using realistic workplace replica. It is shown that drops in attention could be successfully traced only by direct brainwave observation, using wireless electroencephalographic measurements.

[1]  John J. Foxe,et al.  Optimal sustained attention is linked to the spectral content of background EEG activity: greater ongoing tonic alpha (∼10 Hz) power supports successful phasic goal activation , 2007 .

[2]  J. Polich,et al.  Cognitive and biological determinants of P300: an integrative review , 1995, Biological Psychology.

[3]  Fuvro Yevloe,et al.  Frontal midline theta rhythm and eyeblinking activity during a VDT task and a video game : useful tools for psychophysiology in ergonomics , 2012 .

[4]  S. Kelly,et al.  A supramodal accumulation-to-bound signal that determines perceptual decisions in humans , 2012, Nature Neuroscience.

[5]  Marina Schmid,et al.  An Introduction To The Event Related Potential Technique , 2016 .

[6]  Neville A Stanton,et al.  Predicting pilot error: testing a new methodology and a multi-methods and analysts approach. , 2009, Applied ergonomics.

[7]  Raja Parasuraman,et al.  Event-Related Potentials (ERPs) in Neuroergonomics , 2006, Neuroergonomics.

[8]  Andry Rakotonirainy,et al.  Real-time performance modelling of a Sustained Attention to Response Task , 2010, Ergonomics.

[9]  Raja Parasuraman,et al.  Neuroergonomics: Research and practice , 2003 .

[10]  T Marek,et al.  Quo vadis, neuroergonomics? , 2007, Ergonomics.

[11]  Sven Hoffmann,et al.  Towards the measurement of event-related EEG activity in real-life working environments. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[12]  C G Drury,et al.  Information processing in assembly tasks--a case study. , 1988, Applied ergonomics.

[13]  T W Picton,et al.  The P300 Wave of the Human Event‐Related Potential , 1992, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[14]  V. Balasubramanian,et al.  Assessment of early onset of driver fatigue using multimodal fatigue measures in a static simulator. , 2014, Applied ergonomics.

[15]  I. Robertson,et al.  `Oops!': Performance correlates of everyday attentional failures in traumatic brain injured and normal subjects , 1997, Neuropsychologia.

[16]  Hagen C. Flehmig,et al.  Assessing intraindividual variability in sustained attention: reliability, relation to speed and accuracy, and practice effects , 2007 .

[17]  W. Klimesch,et al.  Induced alpha band power changes in the human EEG and attention , 1998, Neuroscience Letters.

[18]  Jens Rasmussen,et al.  Cognitive Systems Engineering , 2022 .

[19]  Stephen J. Vodanovich,et al.  Can Accidents and Industrial Mishaps Be Predicted? Further Investigation into the Relationship Between Cognitive Failure and Reports of Accidents , 2003 .

[20]  F. Yamada.  Frontal midline theta rhythm and eyeblinking activity during a VDT task and a video game: useful tools for psychophysiology in ergonomics. , 1998, Ergonomics.

[21]  T. Åkerstedt,et al.  Sleepiness on the job: continuously measured EEG changes in train drivers. , 1987, Electroencephalography and clinical neurophysiology.

[22]  Ranjana K. Mehta,et al.  Neuroergonomics: a review of applications to physical and cognitive work , 2013, Front. Hum. Neurosci..

[23]  C M Haslegrave,et al.  Seated work postures for manual, visual and combined tasks. , 1999, Ergonomics.

[24]  Rachael Gordon,et al.  The contribution of human factors to accidents in the offshore oil industry , 1998 .

[25]  Raja Parasuraman,et al.  Neuroergonomics: The Brain at Work , 2006 .

[26]  R Kakigi,et al.  [Event-related brain potentials]. , 1997, Nihon rinsho. Japanese journal of clinical medicine.

[27]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[28]  M M Mesulam,et al.  An electrophysiological index of stimulus unfamiliarity. , 2000, Psychophysiology.

[29]  Andreas Bulling,et al.  Cognition-Aware Computing , 2014, IEEE Pervasive Computing.

[30]  Timothy A. Salthouse,et al.  Interpreting Reaction Time Measures in Between-Group Comparisons , 2002, Journal of clinical and experimental neuropsychology.

[31]  B. K. N. Rao,et al.  An Engineer’s View of Human Error , 1993 .

[32]  Colin G. Drury,et al.  Operator-specific model: An assembly time prediction model , 1997 .

[33]  Stephen Darling,et al.  Intraindividual reaction time variability affects P300 amplitude rather than latency , 2014, Front. Hum. Neurosci..

[34]  W. Arthur,et al.  Prediction of Vehicular Accident Involvement: A Meta-Analysis , 1991 .

[35]  Sotiris Makris,et al.  Automotive assembly technologies review: challenges and outlook for a flexible and adaptive approach , 2010 .

[36]  Steven J. Landry,et al.  Introduction to Human Factors and Ergonomics for Engineers , 2007 .

[37]  H. Gray,et al.  P300 as an index of attention to self-relevant stimuli , 2004 .

[38]  Julien Penders,et al.  Wearable, Wireless EEG Solutions in Daily Life Applications: What are we Missing? , 2015, IEEE Journal of Biomedical and Health Informatics.

[39]  Sabine Van Huffel,et al.  Removal of Muscle Artifacts from EEG Recordings of Spoken Language Production , 2010, Neuroinformatics.

[40]  Cynthia D. Fisher,et al.  Effects of external and internal interruptions on boredom at work: two studies , 1998 .

[41]  Lawrence W. Barsalou,et al.  Mind wandering and attention during focused meditation: A fine-grained temporal analysis of fluctuating cognitive states , 2012, NeuroImage.

[42]  J. Polich,et al.  Bifurcated P300 Peaks: P3a and P3b Revisited? , 1988, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[43]  Margot J. Taylor,et al.  Guidelines for using human event-related potentials to study cognition: recording standards and publication criteria. , 2000, Psychophysiology.

[44]  Sabine Van Huffel,et al.  Spatially constrained ICA algorithm with an application in EEG processing , 2011, Signal Process..

[45]  Stephen Fairclough,et al.  Physiological data must remain confidential , 2014, Nature.

[46]  Branislav Jeremic,et al.  A Tool for Neuroergonomic Study of Repetitive Operational Tasks , 2014, ECCE.

[47]  Tom Eichele,et al.  Semi-automatic identification of independent components representing EEG artifact , 2009, Clinical Neurophysiology.

[48]  Erratum to: Removal of Muscle Artifacts from EEG Recordings of Spoken Language Production , 2010, Neuroinformatics.

[49]  S. Sternberg.  Memory-scanning: mental processes revealed by reaction-time experiments. , 1969, American scientist.

[50]  Johannes Hewig,et al.  Skull thickness and magnitude of EEG alpha activity , 2008, Clinical Neurophysiology.

[51]  G. Woodman,et al.  Event-related potential studies of attention , 2000, Trends in Cognitive Sciences.

[52]  Mark W. Scerbo,et al.  Effects of a Psychophysiological System for Adaptive Automation on Performance, Workload, and the Event-Related Potential P300 Component , 2003, Hum. Factors.

[53]  Raja Parasuraman,et al.  Event-related brain potentials and human factors research. , 1990 .

[54]  Frank Biocca,et al.  Comparative effectiveness of augmented reality in object assembly , 2003, CHI '03.

[55]  S. Debener,et al.  How about taking a low-cost, small, and wireless EEG for a walk? , 2012, Psychophysiology.

[56]  J. Shaoul.  Human Error , 1973, Nature.

[57]  Peter Nickel,et al.  Sensitivity of candidate markers of psychophysiological strain to cyclical changes in manual control load during simulated process control. , 2009, Applied ergonomics.

[58]  S. Debener,et al.  Towards a truly mobile auditory brain-computer interface: exploring the P300 to take away. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[59]  Neville A Stanton,et al.  Back to the future: Brake reaction times for manual and automated vehicles , 2007, Ergonomics.

[60]  Atsuo Murata,et al.  Evaluation of mental fatigue using feature parameter extracted from event-related potential , 2002 .

[61]  Adam Hamrol,et al.  Impact of selected work condition factors on quality of manual assembly process , 2011 .

[62]  Darryl G. Humphrey,et al.  Toward a Psychophysiological Assessment of Dynamic Changes in Mental Workload , 1994, Human factors.

[63]  Dick de Waard,et al.  Monitoring drivers' mental workload in driving simulators using physiological measures. , 2010, Accident; analysis and prevention.

[64]  Piotr Jaskowski,et al.  Evidence for an Integrative Role of P3b in Linking Reaction to Perception , 2005 .

[65]  R. Baumeister,et al.  Social exclusion impairs self-regulation. , 2005, Journal of personality and social psychology.

[66]  Monique H W Frings-Dresen,et al.  Effects of job rotation on musculoskeletal complaints and related work exposures: a systematic literature review , 2015, Ergonomics.

[67]  S. Simpson,et al.  Minor injuries, cognitive failures and accidents at work: incidence and associated features. , 2005, Occupational medicine.

[68]  S. MacDonald,et al.  Intra-individual variability in behavior: links to brain structure, neurotransmission and neuronal activity , 2006, Trends in Neurosciences.

[69]  J Hohnsbein,et al.  Performance differences in reaction tasks are reflected in event-related brain potentials (ERPs). , 1998, Ergonomics.

[70]  T. Sejnowski,et al.  Estimating alertness from the EEG power spectrum , 1997, IEEE Transactions on Biomedical Engineering.

[71]  Maarten De Vos,et al.  Benefits of Instructed Responding in Manual Assembly Tasks: An ERP Approach , 2016, Front. Hum. Neurosci..

[72]  J. Polich.  Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[73]  Kenneth Kreutz-Delgado,et al.  Hierarchical Event Descriptor (HED) tags for analysis of event-related EEG studies , 2013, 2013 IEEE Global Conference on Signal and Information Processing.