EyeLSD a Robust Approach for Eye Localization and State Detection

Improving the safety of public roads and industrial factories requires more reliable and robust computer vision-based approaches for monitoring the eye state (open or closed) of human operators. Getting this information in real time when humans are driving cars or using hazardous machinery will help to prevent accidents and deaths. This paper proposes a new framework called EyeLSD to localize the eyes and detect their states without face detection step. For EyeLSD aims, two novel descriptors are proposed: enhanced Pyramidal Local Binary Pattern Histogram (ePLBPH) and Multi-Three-Patch LBP histogram (Multi-TPLBP). The performance of EyeLSD with ePLBPH and Multi-TPLBP is evaluated and compared against other approaches. For this evaluation three independent and public datasets were used: BioID, CAS-PEAL-R1 and ZJU datasets. The set EyeLSD, ePLBPH and Multi-TPLBP have a greater performance when compared against the state-of-the-art algorithms. The proposed approach is very stable under large range of eye appearances caused by expression, rotation, lighting, head pose, and occlusion.

[1]  Wenhui Li,et al.  Eye Location via a Novel Integral Projection Function and Radial Symmetry Transform , 2011 .

[2]  Matti Pietikäinen,et al.  Multi-scale Binary Patterns for Texture Analysis , 2003, SCIA.

[3]  Patrick J. Flynn,et al.  Eyebrow segmentation using active shape models , 2013, Defense, Security, and Sensing.

[4]  Arturo de la Escalera,et al.  Real-Time Warning System for Driver Drowsiness Detection Using Visual Information , 2010, J. Intell. Robotic Syst..

[5]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Francisco Javier Díaz Pernas,et al.  Real-time vision-based eye state detection for driver alertness monitoring , 2013, Pattern Analysis and Applications.

[7]  Lin Sun,et al.  Eyeblink-based Anti-Spoofing in Face Recognition from a Generic Webcamera , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[8]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[9]  Azriel Rosenfeld,et al.  A method of detecting and tracking irises and eyelids in video , 2002, Pattern Recognit..

[10]  Matti Pietikäinen,et al.  Texture classification by center-symmetric auto-correlation, using Kullback discrimination of distributions , 1995, Pattern Recognit. Lett..

[11]  Filipe Neves dos Santos,et al.  Automatic Eye Localization; Multi-block LBP vs. Pyramidal LBP Three-Levels Image Decomposition for Eye Visual Appearance Description , 2015, IbPRIA.

[12]  Luc Van Gool,et al.  Fast PRISM: Branch and Bound Hough Transform for Object Class Detection , 2011, International Journal of Computer Vision.

[13]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[14]  Lin Mei,et al.  A face recognition algorithm based on ASM and Gabor features of key points , 2013, International Conference on Graphic and Image Processing.

[15]  Seong G. Kong,et al.  Visual Analysis of Eye State and Head Pose for Driver Alertness Monitoring , 2013, IEEE Transactions on Intelligent Transportation Systems.

[16]  Wen Gao,et al.  Histogram of Gabor Phase Patterns (HGPP): A Novel Object Representation Approach for Face Recognition , 2007, IEEE Transactions on Image Processing.

[17]  Mahrokh G. Shayesteh,et al.  Efficient algorithms for detection of face, eye and eye state , 2013, IET Comput. Vis..

[18]  Brahim Boulebtateche,et al.  Gabor/PCA/SVM-Based Face Detection for Driver's Monitoring , 2013 .

[19]  Wen Gao,et al.  The CAS-PEAL Large-Scale Chinese Face Database and Baseline Evaluations , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[20]  Matti Pietikäinen,et al.  Rotation-Invariant Image and Video Description With Local Binary Pattern Features , 2012, IEEE Transactions on Image Processing.

[21]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[22]  Pong C. Yuen,et al.  Variance projection function and its application to eye detection for human face recognition , 1998, Pattern Recognit. Lett..

[23]  Matti Pietikäinen,et al.  Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2000, ECCV.

[24]  Stan Z. Li,et al.  A robust eye localization method for low quality face images , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[25]  Dongkyung Nam,et al.  Precise eye localization with improved SDM , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[26]  Tal Hassner,et al.  Effective face frontalization in unconstrained images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Matti Pietikäinen,et al.  Contextual Analysis of Textured Scene Images , 2006, BMVC.

[28]  Karl Ricanek,et al.  LBP-based periocular recognition on challenging face datasets , 2013, EURASIP J. Image Video Process..

[29]  Yan Ren,et al.  A Novel Eye Localization Method With Rotation Invariance , 2014, IEEE Transactions on Image Processing.

[30]  Zhi-Hua Zhou,et al.  Projection functions for eye detection , 2004, Pattern Recognit..

[31]  M. Hassaballah,et al.  Efficient eye detection method based on grey intensity variance and independent components analysis , 2010 .

[32]  Zhi-Hua Zhou,et al.  Enhanced Pictorial Structures for precise eye localization under incontrolled conditions , 2009, CVPR.

[33]  Tong Heng Lee,et al.  Geometrical interpretation and architecture selection of MLP , 2005, IEEE Transactions on Neural Networks.

[34]  Quen-Zong Wu,et al.  An Eye State Recognition Method for Drowsiness Detection , 2010, 2010 IEEE 71st Vehicular Technology Conference.

[35]  Zhi-Hua Zhou,et al.  A literature survey on robust and efficient eye localization in real-life scenarios , 2013, Pattern Recognit..

[36]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[37]  Filipe Neves dos Santos,et al.  Online Vision-Based Eye Detection: LBP/SVM vs LBP/LSTM-RNN , 2015 .

[38]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[39]  Zhe L. Lin,et al.  Nonparametric Context Modeling of Local Appearance for Pose- and Expression-Robust Facial Landmark Localization , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Rui Yang,et al.  Eye localization based on correlation filter bank , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[41]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[42]  Xueming Qian,et al.  PLBP: An effective local binary patterns texture descriptor with pyramid representation , 2011, Pattern Recognit..

[43]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[44]  Qiang Ji,et al.  Learning the Deep Features for Eye Detection in Uncontrolled Conditions , 2014, 2014 22nd International Conference on Pattern Recognition.

[45]  Xiaoyang Tan,et al.  Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients , 2014, Pattern Recognit..

[46]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Ying Zheng,et al.  Eye states detection by boosting Local Binary Pattern Histogram features , 2008, 2008 15th IEEE International Conference on Image Processing.

[48]  Qiong Wang,et al.  Eye Detection in Facial Images With Unconstrained Background , 2006 .

[49]  Bernt Schiele,et al.  Robust Object Detection with Interleaved Categorization and Segmentation , 2008, International Journal of Computer Vision.

[50]  Hao Ge,et al.  Visual Objects Location Based on Hand Eye Coordination , 2014 .

[51]  Xuelong Li,et al.  Gabor-Based Region Covariance Matrices for Face Recognition , 2008, IEEE Transactions on Circuits and Systems for Video Technology.