Fatigue detection with 3D facial features based on binocular stereo vision

Fatigue may lead to potential accidents, but its diagnosis is difficult so it is easy to be delayed or missed. In this paper, a novel 3D facial-image-based fatigue detection method is presented. There are three steps involved: First, 3D surface curvature-based methods are combined with some 2D image-based methods to locate the position of facial fatigue feature points on the facial 3D image-based model, which is reconstructed based on a pair of binocular facial images. Secondly, facial fatigue features, such as eye blink, gaze direction, mouth morphology, and facial expressions are extracted from the facial 3D image-based model by the corresponding extraction methods. Finally, together with N-times sampling data, the attribute value of each facial fatigue feature is calculated, and then integrated together to recognize fatigue by linear discriminant analysis (LDA) algorithm. The following experimental results show that compared with only 2D image-based method, the accurate location of facial feature points, attribute value's calculation, and fatigue's fusion detection are obviously improved using the method presented in this paper.

[1]  Chandan Chakraborty,et al.  Application of Higher Order cumulant Features for Cardiac Health Diagnosis using ECG signals , 2013, Int. J. Neural Syst..

[2]  Panos Liatsis,et al.  Improving fusion with optimal weight selection in Face Recognition , 2012, Integr. Comput. Aided Eng..

[3]  Gernot R. Müller-Putz,et al.  A Single-Switch BCI Based on Passive and imagined movements: toward Restoring Communication in Minimally Conscious patients , 2013, Int. J. Neural Syst..

[4]  Rongrong Fu,et al.  Detection of Driving fatigue by using Noncontact EMG and ECG signals Measurement System , 2014, Int. J. Neural Syst..

[5]  Xingyu Wang,et al.  Aggregation of Sparse Linear Discriminant analyses for Event-Related potential Classification in Brain-Computer Interface , 2014, Int. J. Neural Syst..

[6]  Zhiwei Zhu,et al.  Real-time nonintrusive monitoring and prediction of driver fatigue , 2004, IEEE Transactions on Vehicular Technology.

[7]  Yonghui Zhang,et al.  Fatigue Detection Based on Regional Local Binary Patterns Histogram and Support Vector Machine , 2012, 2012 International Conference on Computer Science and Electronics Engineering.

[8]  Minho Lee,et al.  Action-perception cycle learning for incremental emotion recognition in a movie clip using 3D fuzzy GIST based on visual and EEG signals , 2014, Integr. Comput. Aided Eng..

[9]  William P. Marnane,et al.  Robust neonatal EEG seizure Detection through Adaptive Background Modeling , 2013, Int. J. Neural Syst..

[10]  Joaquim Ciurana,et al.  Improvement of surface roughness models for face milling operations through dimensionality reduction , 2012, Integr. Comput. Aided Eng..

[11]  Yan Zhang,et al.  Objective Evaluation of Driver Fatigue by Using Spontaneous Pupillary Fluctuation , 2011, 2011 5th International Conference on Bioinformatics and Biomedical Engineering.

[12]  Eui Chul Lee,et al.  A new objective visual fatigue measurement system by using a remote infrared camera , 2011, 2011 Eighth International Joint Conference on Computer Science and Software Engineering (JCSSE).

[13]  Cheng-Chi Tai,et al.  AN IMPROVED AND PORTABLE EYE-BLINK DURATION DETECTION SYSTEM TO WARN OF DRIVER FATIGUE , 2013 .

[14]  Amit Sethi,et al.  Drowsy driver detection using representation learning , 2014, 2014 IEEE International Advance Computing Conference (IACC).

[15]  Norbert Pfeifer,et al.  Motion estimation and segmentation in depth and intensity videos , 2014, Integr. Comput. Aided Eng..

[16]  Kwanghoon Sohn,et al.  Stereoscopic visual fatigue measurement based on fusional response curve and eye-blinks , 2011, 2011 17th International Conference on Digital Signal Processing (DSP).

[17]  Lijun Yin,et al.  Viewing direction estimation based on 3D eyeball construction for HRI , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[18]  Hadi Abdullah,et al.  Vision based composite approach for lethargy detection , 2014, 2014 IEEE 10th International Colloquium on Signal Processing and its Applications.

[19]  S JayasenanJ.,et al.  Driver Drowsiness Detection System , 2014 .

[20]  Kwanghoon Sohn,et al.  Smart stereo camera system based on visual fatigue factors , 2012, IEEE international Symposium on Broadband Multimedia Systems and Broadcasting.

[21]  Pedro J. García-Laencina,et al.  Efficient Automatic Selection and Combination of EEG Features in Least Squares Classifiers for Motor Imagery Brain-Computer Interfaces , 2013, Int. J. Neural Syst..

[22]  Mohammad H. Mahoor,et al.  Face recognition based on 3D ridge images obtained from range data , 2009, Pattern Recognit..

[23]  Jin Zhou,et al.  Fatigue Detection Based on Infrared Video Pupillography , 2010, 2010 4th International Conference on Bioinformatics and Biomedical Engineering.

[24]  Xiao Fan,et al.  Nonintrusive Driver Fatigue Detection , 2008, 2008 IEEE International Conference on Networking, Sensing and Control.

[25]  Ho-Hyun Park,et al.  Corner classification using Harris algorithm , 2011 .

[26]  Cristiano Premebida,et al.  Pedestrian detection in far infrared images , 2013, Integr. Comput. Aided Eng..

[27]  Wei Wang,et al.  Driver gaze tracker using deformable template matching , 2011, Proceedings of 2011 IEEE International Conference on Vehicular Electronics and Safety.

[28]  Ji-Wei Wu,et al.  A hybrid linear text segmentation algorithm using hierarchical agglomerative clustering and discrete particle swarm optimization , 2014, Integr. Comput. Aided Eng..

[29]  Ling Zhang,et al.  Yawning Detection Based on Mouth Feature Points Curve Fitting , 2012 .

[30]  Carlos González,et al.  A transferable belief model applied to LIDAR perception for autonomous vehicles , 2013, Integr. Comput. Aided Eng..

[31]  Witold Pedrycz,et al.  Face recognition: A study in information fusion using fuzzy integral , 2005, Pattern Recognit. Lett..

[32]  Silvia Conforto,et al.  An adaptive blink detector to initialize and update a view-basedremote eye gaze tracking system in a natural scenario , 2009, Pattern Recognit. Lett..

[33]  Gianluigi Ferrari,et al.  Improved ultra wideband-based tracking of twin-receiver automated guided vehicles , 2012, Integr. Comput. Aided Eng..

[34]  Jie Li,et al.  Evaluation and Application of a Hybrid Brain Computer Interface for Real Wheelchair Parallel Control with Multi-Degree of Freedom , 2014, Int. J. Neural Syst..

[35]  Qiang Ji,et al.  Facial Feature Tracking Under Varying Facial Expressions and Face Poses Based on Restricted Boltzmann Machines , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Rong Hu,et al.  The driver fatigue monitoring system based on face recognition technology , 2013, 2013 Fourth International Conference on Intelligent Control and Information Processing (ICICIP).

[37]  Weiwei Liu,et al.  Driver fatigue detection through pupil detection and yawing analysis , 2010, 2010 International Conference on Bioinformatics and Biomedical Technology.

[38]  Edwige E. Pissaloux,et al.  On the Design of a Low Cost Gaze Tracker for Interaction , 2012 .

[39]  Tian Yun,et al.  A new algorithm detects pilot fatigue based on machine vision , 2014, CCDC 2014.

[40]  Aude Billard,et al.  A wearable gaze tracking system for children in unconstrained environments , 2011, Comput. Vis. Image Underst..