Eye and mouth state detection algorithm based on contour feature extraction

Abstract. Eye and mouth state analysis is an important step in fatigue detection. An algorithm that analyzes the state of the eye and mouth by extracting contour features is proposed. First, the face area is detected in the acquired image database. Then, the eyes are located by an EyeMap algorithm through a clustering method to extract the sclera-fitting eye contour and calculate the contour aspect ratio. In addition, an effective algorithm is proposed to solve the problem of contour fitting when the human eye is affected by strabismus. Meanwhile, the value of chromatism s is defined in the RGB space, and the mouth is accurately located through lip segmentation. Based on the color difference of the lip, skin, and internal mouth, the internal mouth contour can be fitted to analyze the opening state of mouth; at the same time, another unique and effective yawning judgment mechanism is considered to determine whether the driver is tired. This paper is based on the three different databases to evaluate the performance of the proposed algorithm, and it does not need training with high calculation efficiency.

[1]  Shesheng Gao,et al.  Image Segmentation-Based Multi-Focus Image Fusion Through Multi-Scale Convolutional Neural Network , 2017, IEEE Access.

[2]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[3]  Feng You,et al.  Exploring to direct the reaction pathway for hydrogenation of levulinic acid into γ-valerolactone for future Clean-Energy Vehicles over a magnetic Cu-Ni catalyst , 2017 .

[4]  Mona Omidyeganeh,et al.  Intelligent driver drowsiness detection through fusion of yawning and eye closure , 2011, 2011 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems Proceedings.

[5]  Fan Zhang,et al.  Smartphone-Based Fatigue Detection System Using Progressive Locating Method , 2016 .

[6]  Anil K. Jain,et al.  Face Detection in Color Images , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Tieniu Tan,et al.  Efficient and robust segmentation of noisy iris images for non-cooperative iris recognition , 2010, Image Vis. Comput..

[8]  Qing Liu,et al.  Human fatigue expression recognition through image-based dynamic multi-information and bimodal deep learning , 2016, J. Electronic Imaging.

[9]  Alireza Behrad,et al.  Lip contour extraction using RGB color space and fuzzy c-means clustering , 2010, 2010 IEEE 9th International Conference on Cyberntic Intelligent Systems.

[10]  Haibin Yu,et al.  Fast Robust Face Detection under a Skin Color Model with Geometry Constraints , 2009, 2009 International Conference on Computational Intelligence and Security.

[11]  Raul E. Sanchez-Yanez,et al.  Multilevel Color Transfer on Images for Providing an Artistic Sight of the World , 2017, IEEE Access.

[12]  Jie Lin,et al.  Towards Detection of Bus Driver Fatigue Based on Robust Visual Analysis of Eye State , 2017, IEEE Transactions on Intelligent Transportation Systems.

[13]  Edward J. Delp,et al.  Video-based real-time surveillance of vehicles , 2013, J. Electronic Imaging.

[14]  Kimihiko Nakano,et al.  The Effect of a Haptic Guidance Steering System on Fatigue-Related Driver Behavior , 2017, IEEE Transactions on Human-Machine Systems.

[15]  Mahrokh G. Shayesteh,et al.  Efficient algorithms for detection of face, eye and eye state , 2013, IET Comput. Vis..

[16]  Feng You,et al.  Study on Self-Tuning Tyre Friction Control for Developing Main-Servo Loop Integrated Chassis Control System , 2017, IEEE Access.

[17]  Fang Chen,et al.  Vehicle Detection Method for Intelligent Vehicle at Night Time Based on Video and Laser Information , 2018, Int. J. Pattern Recognit. Artif. Intell..

[18]  Jean-Luc Dugelay,et al.  Combining Edge Detection and Region Segmentation for Lip Contour Extraction , 2010, AMDO.

[19]  M. Kalaiselvi Geetha,et al.  Driver fatigue monitoring system based on eye state analysis , 2014, 2014 International Conference on Circuits, Power and Computing Technologies [ICCPCT-2014].

[20]  Abhinandan Jain,et al.  Fatigue detection and estimation using auto-regression analysis in EEG , 2016, 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[21]  Ali Mollahosseini,et al.  Accurate Extraction of Iris and Eye Corners for Facial Biometrics , .

[22]  S. P. Narote,et al.  Lane departure warning system based on Hough transform and Euclidean distance , 2015, 2015 Third International Conference on Image Information Processing (ICIIP).

[23]  Mouli Ramasamy,et al.  Real-time monitoring of drowsiness through wireless nanosensor systems , 2016, SPIE Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring.

[24]  Xin Song,et al.  The anti-fatigue driving system design based on the eye blink detect , 2017, International Conference on Electronics and Information Engineering.

[25]  Vinodkrishnan Kulathumani,et al.  Multiview fusion for activity recognition using deep neural networks , 2016, J. Electronic Imaging.

[26]  Ling Huang,et al.  Monitoring drivers’ sleepy status at night based on machine vision , 2016, Multimedia Tools and Applications.

[27]  R.A. Zoroofi,et al.  Open/Closed Eye Analysis for Drowsiness Detection , 2008, 2008 First Workshops on Image Processing Theory, Tools and Applications.

[28]  Khashayar Yaghmaie,et al.  Automatic combined lip segmentation in color images , 2011, 2011 IEEE 3rd International Conference on Communication Software and Networks.

[29]  George K. I. Mann,et al.  Hybrid Vision Based Reach-to-Grasp Task Planning Method for Trans-Humeral Prostheses , 2017, IEEE Access.

[30]  Yufeng Lu,et al.  Recognition of driver eyes' states based on variance projections function , 2010, 2010 3rd International Congress on Image and Signal Processing.

[31]  Bao-Liang Lu,et al.  Driving fatigue detection with fusion of EEG and forehead EOG , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).