Proactive mental fatigue detection of traffic control operators using bagged trees and gaze-bin analysis

Abstract Most of existing eye movement-based fatigue detectors utilize statistical analysis of fixations, saccades, and blinks as inputs. Nevertheless, these parameters require long recording time and heavily depend on eye trackers. In an effort to facilitate proactive detection of mental fatigue, we introduced a complemental fatigue indicator, named gaze-bin analysis, which simply presents the eye-tracking data with histograms. A method which engaged the gaze-bin analysis as inputs of semisupervised bagged trees was developed. A case study in a vessel traffic service center demonstrated that this approach can alleviate the burden of manual labeling as well as improve the performance of fatigue detection model. In addition, the results show that the approach can achieve an excellent accuracy of 89%, which outperformed other methods. In general, this study provided a complemental indicator for detecting mental fatigue as well as enabled the application of a low sampling rate eye tracker in the traffic control center.

[1]  K. Müller,et al.  The effect of an irregular shift system on sleepiness at work in train drivers and railway traffic controllers , 2002, Journal of sleep research.

[2]  Ulf Ahlstrom,et al.  Using eye movement activity as a correlate of cognitive workload , 2006 .

[3]  John D. Lee,et al.  Real-Time Detection of Driver Cognitive Distraction Using Support Vector Machines , 2007, IEEE Transactions on Intelligent Transportation Systems.

[4]  Li Pheng Khoo,et al.  Hybrid data-driven vigilance model in traffic control center using eye-tracking data and context data , 2019, Adv. Eng. Informatics.

[5]  Mark Harrison,et al.  Adaptive binning: An improved binning method for metabolomics data using the undecimated wavelet transform , 2007 .

[6]  Saso Dzeroski,et al.  A framework for redescription set construction , 2017, Expert Syst. Appl..

[7]  Michael B. McCamy,et al.  Saccadic Eye Movement Metrics Reflect Surgical Residents' Fatigue , 2014, Annals of surgery.

[8]  Hans-Werner Gellersen,et al.  Correlation between gaze and hovers during decision-making interaction , 2018, ETRA.

[9]  Indu P. Bodala,et al.  EEG and Eye Tracking Demonstrate Vigilance Enhancement with Challenge Integration , 2016, Front. Hum. Neurosci..

[10]  G. De’ath.  Boosted trees for ecological modeling and prediction. , 2007, Ecology.

[11]  Carsten Finke,et al.  Dynamics of saccade parameters in multiple sclerosis patients with fatigue , 2012, Journal of Neurology.

[12]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[13]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[14]  Darren Gergle,et al.  An Eye For Design: Gaze Visualizations for Remote Collaborative Work , 2018, CHI.

[15]  Thomas G. Dietterich.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.

[16]  Oleg V. Komogortsev,et al.  User Eye Fatigue Detection via Eye Movement Behavior , 2015, CHI Extended Abstracts.

[17]  Nadine Sarter,et al.  Using Eye Tracking to Detect the Effects of Clutter on Visual Search in Real Time , 2017, IEEE Transactions on Human-Machine Systems.

[18]  Changmin Kim,et al.  Classification of major construction materials in construction environments using ensemble classifiers , 2014, Adv. Eng. Informatics.

[19]  Christopher Kennard,et al.  Eye Movements Discriminate Fatigue Due to Chronotypical Factors and Time Spent on Task – A Double Dissociation , 2014, PloS one.

[20]  K. Pearson.  Contributions to the Mathematical Theory of Evolution. II. Skew Variation in Homogeneous Material , 1895 .

[21]  Tim Horberry,et al.  REVIEW OF FATIGUE DETECTION AND PREDICTION TECHNOLOGIES , 2000 .

[22]  Dewang Chen,et al.  Data-driven train operation models based on data mining and driving experience for the diesel-electric locomotive , 2016, Adv. Eng. Informatics.

[23]  S. Hursh,et al.  Fatigue Risk Management in the Workplace , 2012, Journal of occupational and environmental medicine.

[24]  Ching-Hung Lee,et al.  A User Requirement-driven Approach Incorporating TRIZ and QFD for Designing a Smart Vessel Alarm System to Reduce Alarm Fatigue , 2019, Journal of Navigation.

[25]  Indu P. Bodala,et al.  Measuring vigilance decrement using computer vision assisted eye tracking in dynamic naturalistic environments , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[26]  David Zadok,et al.  Ocular parameters as an objective tool for the assessment of truck drivers fatigue. , 2009, Accident; analysis and prevention.

[27]  R. Schleicher,et al.  Blinks and saccades as indicators of fatigue in sleepiness warners: looking tired? , 2022 .

[28]  Wei Tech Ang,et al.  EEG-Based Human Factors Evaluation of Conflict Resolution Aid and Tactile User Interface in Future Air Traffic Control Systems , 2017 .

[29]  S. Raffaele,et al.  Two-dimensional data binning for the analysis of genome architecture in filamentous plant pathogens and other eukaryotes. , 2014, Methods in molecular biology.

[30]  Qiang Ji,et al.  Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance , 2002, Real Time Imaging.

[31]  S Saito,et al.  Does fatigue exist in a quantitative measurement of eye movements? , 1992, Ergonomics.

[32]  Bart Roets,et al.  Shift work, fatigue, and human error: An empirical analysis of railway traffic control , 2019 .

[33]  Gesa Praetorius,et al.  Modelling Vessel Traffic Service to understand resilience in everyday operations , 2015, Reliab. Eng. Syst. Saf..

[34]  Yuying Jiang,et al.  Driver Sleepiness Detection System Based on Eye Movements Variables , 2013 .

[35]  Marcus Nyström,et al.  An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data , 2010, Behavior research methods.

[36]  Marc Lavielle,et al.  Automatic data binning for improved visual diagnosis of pharmacometric models , 2011, Journal of Pharmacokinetics and Pharmacodynamics.

[37]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[38]  Masatomo Kobayashi,et al.  Detecting Mental Fatigue from Eye-Tracking Data Gathered While Watching Video , 2017, AIME.

[39]  Fan Li,et al.  Investigation on the Correlation between Eye Movement and Reaction Time under Mental Fatigue Influence , 2018, 2018 International Conference on Cyberworlds (CW).

[40]  Pernilla Qvarfordt,et al.  Gaze patterns during remote presentations while listening and speaking , 2018, ETRA.

[41]  Wei Liu,et al.  Detecting driving fatigue with multimodal deep learning , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[42]  J. C. Miller,et al.  Electrooculographic and performance indices of fatigue during simulated flight , 1996, Biological Psychology.

[43]  Leandro L. Di Stasi,et al.  Towards a driver fatigue test based on the saccadic main sequence: A partial validation by subjective report data , 2012 .

[44]  Raja Parasuraman,et al.  The Role of the Air Traffic Controller in Future Air Traffic Management: An Empirical Study of Active Control versus Passive Monitoring , 2001, Hum. Factors.

[45]  Samit Bhattacharya,et al.  Fixation-indices based correlation between text and image visual features of webpages , 2018, ETRA.

[46]  Daniel W. Repperger,et al.  Evaluation of Eye Metrics as a Detector of Fatigue , 2011 .

[47]  Jochen Teizer,et al.  Real-time construction worker posture analysis for ergonomics training , 2012, Adv. Eng. Informatics.

[48]  Kristian S. Gould,et al.  Improving the saccade peak velocity measurement for detecting fatigue , 2010, Journal of Neuroscience Methods.