Yawn analysis with mouth occlusion detection

Abstract One of the most common signs of tiredness or fatigue is yawning. Naturally, identification of fatigued individuals would be helped if yawning is detected. Existing techniques for yawn detection are centred on measuring the mouth opening. This approach, however, may fail if the mouth is occluded by the hand, as it is frequently the case. The work presented in this paper focuses on a technique to detect yawning whilst also allowing for cases of occlusion. For measuring the mouth opening, a new technique which applies adaptive colour region is introduced. For detecting yawning whilst the mouth is occluded, local binary pattern (LBP) features are used to also identify facial distortions during yawning. In this research, the Strathclyde Facial Fatigue (SFF) database which contains genuine video footage of fatigued individuals is used for training, testing and evaluation of the system.

[1]  Zhenlong Li,et al.  Yawning detection for monitoring driver fatigue based on two cameras , 2009, 2009 12th International IEEE Conference on Intelligent Transportation Systems.

[2]  Yang Ying,et al.  The Monitoring Method of Driver's Fatigue Based on Neural Network , 2007, 2007 International Conference on Mechatronics and Automation.

[3]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Jing Xu,et al.  Lip Detection and Tracking Using Variance Based Haar-Like Features and Kalman filter , 2010, 2010 Fifth International Conference on Frontier of Computer Science and Technology.

[5]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[6]  N Mabbott.  ARRB Pro-Active Fatigue Management System , 2003 .

[7]  Mona Omidyeganeh,et al.  Intelligent driver drowsiness detection through fusion of yawning and eye closure , 2011, 2011 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems Proceedings.

[8]  Miguel Torres-Torriti,et al.  Driver alert state and fatigue detection by salient points analysis , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[9]  John D. Fernandez,et al.  Facial feature detection using Haar classifiers , 2006 .

[10]  Max Hirshkowitz,et al.  Practice parameters for the role of actigraphy in the study of sleep and circadian rhythms: an update for 2002. , 2003, Sleep.

[11]  Oscar Déniz-Suárez,et al.  ENCARA2: Real-time detection of multiple faces at different resolutions in video streams , 2007, J. Vis. Commun. Image Represent..

[12]  Masrullizam Mat Ibrahim,et al.  Mouth covered detection for yawn , 2013, 2013 IEEE International Conference on Signal and Image Processing Applications.

[13]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[14]  E. R. Davies,et al.  Machine vision - theory, algorithms, practicalities , 2004 .

[15]  R Resendes,et al.  SAVING LIVES THROUGH ADVANCED SAFETY TECHNOLOGY: INTELLIGENT VEHICLE INITIATIVE 2002 ANNUAL REPORT , 2003 .

[16]  Yao WenJuan,et al.  A real-time lip localization and tacking for lip reading , 2010, 2010 3rd International Conference on Advanced Computer Theory and Engineering(ICACTE).

[17]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Natasha Merat,et al.  Fatigue and road safety: a critical analysis of recent evidence , 2011 .

[19]  Driss Aboutajdine,et al.  Driver's Fatigue and Drowsiness Detection to Reduce Traffic Accidents on Road , 2011, CAIP.

[20]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Xiao Fan,et al.  Yawning Detection for Monitoring Driver Fatigue , 2007, 2007 International Conference on Machine Learning and Cybernetics.

[22]  Zhang Jin-Yu,et al.  Edge detection of images based on improved Sobel operator and genetic algorithms , 2009, 2009 International Conference on Image Analysis and Signal Processing.

[23]  Lu Yufeng,et al.  Detecting Driver Yawning in Successive Images , 2007, 2007 1st International Conference on Bioinformatics and Biomedical Engineering.

[24]  Xiao Fan,et al.  Dynamic Human Fatigue Detection Using Feature-Level Fusion , 2008, ICISP.

[25]  Aurobinda Routray,et al.  A non-rigid motion estimation algorithm for yawn detection in human drivers , 2009, Int. J. Comput. Vis. Robotics.

[26]  Kenji Ishida,et al.  Facial Expression Measurement for Detecting Driver Drowsiness , 2011, HCI.

[27]  S. Anumas,et al.  Driver fatigue monitoring system using video face images & physiological information , 2012, The 4th 2011 Biomedical Engineering International Conference.

[28]  Gwen Littlewort,et al.  Drowsy Driver Detection Through Facial Movement Analysis , 2007, ICCV-HCI.

[29]  Hernán García,et al.  Driving Fatigue Detection Using Active Shape Models , 2010, ISVC.