Eye blink completeness detection

Abstract Computer users often complain about eye discomfort caused by dry eye syndrome. This is sometimes caused and accompanied by incomplete blinks. There are several algorithms for eye blink detection, but none which would distinguish complete blinks from the incomplete ones. We introduce the first method which detects blink completeness. Blinks differ in speed and duration similar to speech, therefore Recurrent Neural Network (RNN) is used as a classifier due to its suitability for sequence-based features. We show that using unidirectional RNN with time shifting achieves higher performance compared to a bidirectional RNN, which is a suitable choice in this kind of problem where the feature pattern is not yet observed for the initial frames. We report the best results (increase by almost 8%) on the most challenging dataset: Researcher’s night. We formulate a new important problem and state an initial benchmark for further research.

[1]  M. Collins,et al.  Blinking Patterns in Soft Contact Lens Wearers Can Be Altered with Training , 1987, American journal of optometry and physiological optics.

[2]  Andrej Fogelton,et al.  Eye Blink Detection Using Variance of Motion Vectors , 2014, ECCV Workshops.

[3]  H T Siegelmann,et al.  Dating and Context of Three Middle Stone Age Sites with Bone Points in the Upper Semliki Valley, Zaire , 2007 .

[4]  Daniela S Nosch,et al.  Blink Animation Software to Improve Blinking and Dry Eye Symptoms , 2015, Optometry and vision science : official publication of the American Academy of Optometry.

[5]  Mubarak Shah,et al.  UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild , 2012, ArXiv.

[6]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Bogdan J. Matuszewski,et al.  Online Eye Status Detection in the Wild with Convolutional Neural Networks , 2017, VISIGRAPP.

[8]  K. Tsubota,et al.  TFOS DEWS II Definition and Classification Report. , 2017, The ocular surface.

[9]  Marc Argilés,et al.  Blink Rate and Incomplete Blinks in Six Different Controlled Hard-Copy and Electronic Reading Conditions. , 2015, Investigative ophthalmology & visual science.

[10]  Bogdan Smolka,et al.  Silesian Deception Database: Presentation and Analysis , 2015, WMDD@ICMI.

[11]  Filipe Neves dos Santos,et al.  EyeLSD a Robust Approach for Eye Localization and State Detection , 2017, Journal of Signal Processing Systems.

[12]  Meritxell Vilaseca,et al.  Blink Rate, Blink Amplitude, and Tear Film Integrity during Dynamic Visual Display Terminal Tasks , 2011, Current eye research.

[13]  A. Bentivoglio,et al.  Analysis of blink rate patterns in normal subjects , 1997, Movement disorders : official journal of the Movement Disorder Society.

[14]  Yan Yang,et al.  Driver Drowsiness Detection Based on Novel Eye Openness Recognition Method and Unsupervised Feature Learning , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[15]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[16]  Li Fei-Fei,et al.  End-to-End Learning of Action Detection from Frame Glimpses in Videos , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Chabane Djeraba,et al.  Drowsy driver detection system using eye blink patterns , 2010, 2010 International Conference on Machine and Web Intelligence.

[18]  A J Bron,et al.  Functional aspects of the tear film lipid layer. , 2004, Experimental eye research.

[19]  Xiaoyang Tan,et al.  Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients , 2014, Pattern Recognit..

[20]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[21]  R. Bedi,et al.  Nonobvious Obstructive Meibomian Gland Dysfunction , 2010, Cornea.

[22]  D R Korb,et al.  Tear Film Lipid Layer Thickness as a Function of Blinking , 1994, Cornea.

[23]  C. Blehm,et al.  Computer vision syndrome: a review. , 2005, Survey of ophthalmology.

[24]  J. Stern,et al.  The endogenous eyeblink. , 1984, Psychophysiology.

[25]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[26]  Margrit Betke,et al.  Communication via eye blinks and eyebrow raises: video-based human-computer interfaces , 2003, Universal Access in the Information Society.

[27]  Lei Zhao,et al.  Eye state recognition based on deep integrated neural network and transfer learning , 2017, Multimedia Tools and Applications.

[28]  Horst Bischof,et al.  Eye Blink Based Fatigue Detection for Prevention of Computer Vision Syndrome , 2009, MVA.

[29]  Unfolded recurrent neural networks for speech recognition , 2014, INTERSPEECH.

[30]  Mariusz Szwoch,et al.  Eye Blink Based Detection of Liveness in Biometric Authentication Systems Using Conditional Random Fields , 2012, ICCVG.

[31]  Navdeep Jaitly,et al.  Towards End-To-End Speech Recognition with Recurrent Neural Networks , 2014, ICML.

[32]  Mark Rosenfield,et al.  Blink Rate, Incomplete Blinks and Computer Vision Syndrome , 2013, Optometry and vision science : official publication of the American Academy of Optometry.

[33]  Haroon Idrees,et al.  The THUMOS challenge on action recognition for videos "in the wild" , 2016, Comput. Vis. Image Underst..

[34]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[35]  Mei Wang,et al.  Blink detection using Adaboost and contour circle for fatigue recognition , 2017, Comput. Electr. Eng..

[36]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[37]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[38]  Lakhmi C. Jain,et al.  Recurrent Neural Networks: Design and Applications , 1999 .

[39]  Wanda Benesova,et al.  Eye blink detection based on motion vectors analysis , 2016, Comput. Vis. Image Underst..

[40]  Mark Rosenfield,et al.  Blink Patterns: Reading from a Computer Screen versus Hard Copy , 2014, Optometry and vision science : official publication of the American Academy of Optometry.

[41]  Sepp Hochreiter,et al.  The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions , 1998, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[42]  Lin Sun,et al.  Eyeblink-based Anti-Spoofing in Face Recognition from a Generic Webcamera , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[43]  Bogdan Smolka,et al.  Blink Detection Based on the Weighted Gradient Descriptor , 2013, CORES.

[44]  Surya Ganguli,et al.  Exact solutions to the nonlinear dynamics of learning in deep linear neural networks , 2013, ICLR.

[45]  Thiago Santini,et al.  Brightness- and motion-based blink detection for head-mounted eye trackers , 2016, UbiComp Adjunct.

[46]  Shanshan Yu,et al.  A novel blink detection system for user monitoring , 2013, 2013 1st IEEE Workshop on User-Centered Computer Vision (UCCV).