Eye blink detection for different driver states in conditionally automated driving and manual driving using EOG and a driver camera

In this article, we examine the performance of different eye blink detection algorithms under various constraints. The goal of the present study was to evaluate the performance of an electrooculogram- and camera-based blink detection process in both manually and conditionally automated driving phases. A further comparison between alert and drowsy drivers was performed in order to evaluate the impact of drowsiness on the performance of blink detection algorithms in both driving modes. Data snippets from 14 monotonous manually driven sessions (mean 2 h 46 min) and 16 monotonous conditionally automated driven sessions (mean 2 h 45 min) were used. In addition to comparing two data-sampling frequencies for the electrooculogram measures (50 vs. 25 Hz) and four different signal-processing algorithms for the camera videos, we compared the blink detection performance of 24 reference groups. The analysis of the videos was based on very detailed definitions of eyelid closure events. The correct detection rates for the alert and manual driving phases (maximum 94%) decreased significantly in the drowsy (minus 2% or more) and conditionally automated (minus 9% or more) phases. Blinking behavior is therefore significantly impacted by drowsiness as well as by automated driving, resulting in less accurate blink detection.

[1]  Wanda Benesova,et al.  Eye blink detection based on motion vectors analysis , 2016, Comput. Vis. Image Underst..

[2]  D. Schroeder,et al.  Blink Rate: A Possible Measure of Fatigue , 1994, Human factors.

[3]  Parisa Ebrahim,et al.  Driver drowsiness monitoring using eye movement features derived from electrooculography , 2016 .

[4]  P Troiano,et al.  Eye complaints in the office environment: precorneal tear film integrity influenced by eye blinking efficiency , 2004, Occupational and Environmental Medicine.

[5]  Margrit Betke,et al.  Real Time Eye Tracking and Blink Detection with USB Cameras , 2005 .

[6]  Ragunathan Rajkumar,et al.  Towards a viable autonomous driving research platform , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[7]  Drew Dawson,et al.  Look before you (s)leep: evaluating the use of fatigue detection technologies within a fatigue risk management system for the road transport industry. , 2014, Sleep medicine reviews.

[8]  B. Smolka,et al.  A novel approach to the eye movement analysis using a high speed camera , 2012, 2012 2nd International Conference on Advances in Computational Tools for Engineering Applications (ACTEA).

[9]  J. Stern,et al.  Pattern of eyelid motion predictive of decision errors during drowsiness: oculomotor indices of altered states. , 1986, The International journal of neuroscience.

[10]  L. T,et al.  Electrooculographic and performance indices of fatigue during simulated flight , 2022 .

[11]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[12]  Shin Yamamoto,et al.  Driver blink measurement by the motion picture processing and its application to drowsiness detection , 2002, Proceedings. The IEEE 5th International Conference on Intelligent Transportation Systems.

[13]  Margrit Betke,et al.  Blink and wink detection for mouse pointer control , 2010, PETRA '10.

[14]  Chabane Djeraba,et al.  Drowsy driver detection system using eye blink patterns , 2010, 2010 International Conference on Machine and Web Intelligence.

[15]  Bin Yang,et al.  Camera-based drowsiness reference for driver state classification under real driving conditions , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[16]  John-John Cabibihan,et al.  A Non Intrusive Human Fatigue Monitoring System , 2012 .

[17]  Ross M. Henderson,et al.  Effect of Visual Display Unit Use on Blink Rate and Tear Stability , 1991, Optometry and vision science : official publication of the American Academy of Optometry.

[18]  T. Åkerstedt,et al.  Subjective and objective sleepiness in the active individual. , 1990, The International journal of neuroscience.

[19]  Alice Caplier,et al.  Fusing bio-inspired vision data for simplified high level scene interpretation: Application to face motion analysis , 2010, Comput. Vis. Image Underst..

[20]  Wolfgang Rosenstiel,et al.  Eyes wide open? eyelid location and eye aperture estimation for pervasive eye tracking in real-world scenarios , 2016, UbiComp Adjunct.

[21]  Antoine Picot,et al.  On-Line Detection of Drowsiness Using Brain and Visual Information , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[22]  Miguel Ángel Sotelo,et al.  Real-time system for monitoring driver vigilance , 2004, Proceedings of the IEEE International Symposium on Industrial Electronics, 2005. ISIE 2005..

[23]  Kenneth Sundaraj,et al.  Detecting Driver Drowsiness Based on Sensors: A Review , 2012, Sensors.

[24]  Keiichi Uchimura,et al.  Driver inattention monitoring system for intelligent vehicles: A review , 2009 .

[25]  T. Jung,et al.  Task performance and eye activity: predicting behavior relating to cognitive workload. , 2007, Aviation, space, and environmental medicine.

[26]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[27]  H. Collewijn,et al.  Human eye movements associated with blinks and prolonged eyelid closure. , 1985, Journal of neurophysiology.

[28]  Antoine Picot,et al.  Comparison between EOG and high frame rate camera for drowsiness detection , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[29]  Jürgen Schmidt,et al.  Driver drowsiness and behavior detection in prolonged conditionally automated drives , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[30]  Lin Sun,et al.  Eyeblink-based Anti-Spoofing in Face Recognition from a Generic Webcamera , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[31]  Bruno Jammes,et al.  Automatic EOG analysis: A first step toward automatic drowsiness scoring during wake-sleep transitions , 2008 .

[32]  Azriel Rosenfeld,et al.  A method of detecting and tracking irises and eyelids in video , 2002, Pattern Recognit..

[33]  Inho Choi,et al.  Eye Detection and Eye Blink Detection Using AdaBoost Learning and Grouping , 2011, 2011 Proceedings of 20th International Conference on Computer Communications and Networks (ICCCN).

[34]  Silvia Conforto,et al.  An adaptive blink detector to initialize and update a view-basedremote eye gaze tracking system in a natural scenario , 2009, Pattern Recognit. Lett..

[35]  Alejandro F. Frangi,et al.  Automatic Assessment of Eye Blinking Patterns through Statistical Shape Models , 2009, ICVS.

[36]  Jeronimo Dzaack,et al.  A data-driven algorithm for offline pupil signal preprocessing and eyeblink detection in low-speed eye-tracking protocols , 2011, Behavior research methods.

[37]  Jing Xiao,et al.  Automatic recognition of eye blinking in spontaneously occurring behavior , 2002, Object recognition supported by user interaction for service robots.

[38]  Kang Ryoung Park,et al.  Blink detection robust to various facial poses , 2010, Journal of Neuroscience Methods.

[39]  K. Nakamura,et al.  Development of non-contact real-time blink detection system for doze alarm , 2004, SICE 2004 Annual Conference.

[40]  Xianta Jiang,et al.  Capturing and Evaluating Blinks from Video-based Eyetrackers Developed a Data-driven Eye-blink Detection Algorithm for 50-hz Eyetracking Proto- Cols, Where the Pupil Diameter Was Used for Correcting , 2022 .

[41]  G. Sjøgaard,et al.  Eye blink frequency during different computer tasks quantified by electrooculography , 2006, European Journal of Applied Physiology.

[42]  J. Stern,et al.  The endogenous eyeblink. , 1984, Psychophysiology.

[43]  Margrit Betke,et al.  Communication via eye blinks - detection and duration analysis in real time , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[44]  Chak-Kuen Wong,et al.  An Efficient Method for Weighted Sampling Without Replacement , 1980, SIAM J. Comput..

[45]  Bogdan Smolka,et al.  Eye blink detection using Local Binary Patterns , 2014, 2014 International Conference on Multimedia Computing and Systems (ICMCS).

[46]  Luis Miguel Bergasa,et al.  Vision-based drowsiness detector for real driving conditions , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[47]  Dan Witzner Hansen,et al.  Eye tracking in the wild , 2005, Comput. Vis. Image Underst..

[48]  Shuyan Hu,et al.  Driver drowsiness detection with eyelid related parameters by Support Vector Machine , 2009, Expert Syst. Appl..

[49]  Horst Bischof,et al.  Eye Blink Based Fatigue Detection for Prevention of Computer Vision Syndrome , 2009, MVA.

[50]  Y. V. Venkatesh,et al.  Blink detection and eye tracking for eye localization , 2003, TENCON 2003. Conference on Convergent Technologies for Asia-Pacific Region.

[51]  Zhaohui Wu,et al.  Liveness Detection for Face Recognition , 2008 .

[52]  Wolfgang Rosenstiel,et al.  ExCuSe: Robust Pupil Detection in Real-World Scenarios , 2015, CAIP.