Fast Computation of PERCLOS and Saccadic Ratio

This thesis describes the development of fast algorithms for the computation of PERcentage CLOSure of eyes (PERCLOS) and Saccadic Ratio (SR). PERCLOS and SR are two ocular parameters reported to be measures of alertness levels in human beings. PERCLOS is the percentage of time in which at least 80% of the eyelid remains closed over the pupil. Saccades are fast and simultaneous movement of both the eyes in the same direction. SR is the ratio of peak saccadic velocity to the saccadic duration. This thesis addresses the issues of image based estimation of PERCLOS and SR, prevailing in the literature such as illumination variation, poor illumination conditions, head rotations etc. In this work, algorithms for real-time PERCLOS computation has been developed and implemented on an embedded platform. The platform has been used as a case study for assessment of loss of attention in automotive drivers. The SR estimation has been carried out offline as real-time implementation requires high frame rates of processing which is difficult to achieve due to hardware limitations. The accuracy in estimation of the loss of attention using PERCLOS and SR has been validated using brain signals, which are reported to be an authentic cue for estimating the state of alertness in human beings. The major contributions of this thesis include database creation, design and implementation of fast algorithms for estimating PERCLOS and SR on embedded computing platforms.

[1]  Aurobinda Routray,et al.  A Vision-Based System for Monitoring the Loss of Attention in Automotive Drivers , 2013, IEEE Transactions on Intelligent Transportation Systems.

[2]  A. J. Van Opstal,et al.  Skewness of saccadic velocity profiles: A unifying parameter for normal and slow saccades , 1987, Vision Research.

[3]  Xie Bin,et al.  A PERCLOS-Based Driver Fatigue Recognition Application for Smart Vehicle Space , 2010, 2010 Third International Symposium on Information Processing.

[4]  Aurobinda Routray,et al.  A real time algorithm for detection of spectacles leading to eye detection , 2012, 2012 4th International Conference on Intelligent Human Computer Interaction (IHCI).

[5]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[6]  A. G. Ramakrishnan,et al.  Eye detection using color cues and projection functions , 2002, Proceedings. International Conference on Image Processing.

[7]  Aurobinda Routray,et al.  Analysis of training parameters for classifiers based on Haar-like features to detect human faces , 2011, 2011 International Conference on Image Information Processing.

[8]  Mubarak Shah,et al.  Determining driver visual attention with one camera , 2003, IEEE Trans. Intell. Transp. Syst..

[9]  Kazunori Shidoji,et al.  Detecting drowsiness while driving by measuring eye movement - a pilot study , 2002, Proceedings. The IEEE 5th International Conference on Intelligent Transportation Systems.

[10]  Yasuyuki Saito,et al.  Estimation of eyeglassless facial images using principal component analysis , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[11]  Rahul Banerjee,et al.  Detection of fatigue of vehicular driver using skin conductance and oximetry pulse: a neural network approach , 2009, iiWAS.

[12]  B. Menser,et al.  Face detection in color images using principal components analysis , 1999 .

[13]  M. I. Dessouky,et al.  High performance face recognition using PCA and ZM on fused LWIR and VISIBLE images on the wavelet domain , 2009, 2009 International Conference on Computer Engineering & Systems.

[14]  O. Nakamura,et al.  On the isolation of spectacles and the extraction of faces for personal identification , 2004, Canadian Conference on Electrical and Computer Engineering 2004 (IEEE Cat. No.04CH37513).

[15]  Aurobinda Routray,et al.  Video & EOG based investigation of pure saccades in human subjects , 2012, 2012 4th International Conference on Intelligent Human Computer Interaction (IHCI).

[16]  D. Dijk,et al.  Dose-response relationship for light intensity and ocular and electroencephalographic correlates of human alertness , 2000, Behavioural Brain Research.

[17]  William M. Wells,et al.  Efficient Synthesis of Gaussian Filters by Cascaded Uniform Filters , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Harry Shum,et al.  Automatic eyeglasses removal from face images , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Klaus D. Tönnies,et al.  Feasibility of Hough-transform-based iris localisation for real-time-application , 2002, Object recognition supported by user interaction for service robots.

[20]  Aurobinda Routray,et al.  A video database of human faces under near Infra-Red illumination for human computer interaction applications , 2012, 2012 4th International Conference on Intelligent Human Computer Interaction (IHCI).

[21]  Riad I. Hammoud,et al.  An improved likelihood model for eye tracking , 2007, Comput. Vis. Image Underst..

[22]  M. A. Frens,et al.  Recording eye movements with video-oculography and scleral search coils: a direct comparison of two methods , 2002, Journal of Neuroscience Methods.

[23]  Darrell S Bowman,et al.  PERCLOS+: Development of a Robust Field Measure of Driver Drowsiness , 2008 .

[24]  Yeong-Taeg Kim,et al.  Contrast enhancement using brightness preserving bi-histogram equalization , 1997 .

[25]  Riad I. Hammoud,et al.  On Driver Eye Closure Recognition for Commercial Vehicles , 2008 .

[26]  H. Deubel,et al.  Saccade target selection and object recognition: Evidence for a common attentional mechanism , 1996, Vision Research.

[27]  Karel J. Zuiderveld,et al.  Contrast Limited Adaptive Histogram Equalization , 1994, Graphics Gems.

[28]  Fionn Murtagh,et al.  Gray and color image contrast enhancement by the curvelet transform , 2003, IEEE Trans. Image Process..

[29]  Mohamed Rizon,et al.  Iris detection using intensity and edge information , 2003, Pattern Recognit..

[30]  Aurobinda Routray,et al.  A novel drowsiness detection scheme based on speech analysis with validation using simultaneous EEG recordings , 2010, 2010 IEEE International Conference on Automation Science and Engineering.

[31]  Naif Alajlan,et al.  Real-time iris detection , 2010, Artificial Life and Robotics.

[32]  Cataldo Guaragnella,et al.  A visual approach for driver inattention detection , 2007, Pattern Recognit..

[33]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Jian Yang,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Qiang Ji,et al.  Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance , 2002, Real Time Imaging.

[36]  Brojeshwar Bhowmick,et al.  Detection and classification of eye state in IR camera for driver drowsiness identification , 2009, 2009 IEEE International Conference on Signal and Image Processing Applications.

[37]  Gunilla Borgefors,et al.  Distance transformations in digital images , 1986, Comput. Vis. Graph. Image Process..

[38]  Ying Li,et al.  Eye detection by using fuzzy template matching and feature-parameter-based judgement , 2001, Pattern Recognit. Lett..

[39]  Ralph Gross,et al.  An Image Preprocessing Algorithm for Illumination Invariant Face Recognition , 2003, AVBPA.

[40]  I. Pitas,et al.  An Eye Detection Algorithm Using Pixel to Edge Information , 2005 .

[41]  R J Fairbanks,et al.  RESEARCH ON VEHICLE-BASED DRIVER STATUS/PERFORMANCE MONITORING; DEVELOPMENT, VALIDATION, AND REFINEMENT OF ALGORITHMS FOR DETECTION OF DRIVER DROWSINESS. FINAL REPORT , 1994 .

[42]  Aaron Steinfeld,et al.  Identification of An “Appropriate” Drowsy Driver Detection Interface for Commercial Vehicle Operations , 2003 .

[43]  Miguel Ángel Sotelo,et al.  Real-time system for monitoring driver vigilance , 2004, Proceedings of the IEEE International Symposium on Industrial Electronics, 2005. ISIE 2005..

[44]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Sudeep Sarkar,et al.  Designing A.ne Transformations based Face Recognition Algorithms , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[46]  Paolo Inchingolo,et al.  On the Identification and Analysis of Saccadic Eye Movements-A Quantitative Study of the Processing Procedures , 1985, IEEE Transactions on Biomedical Engineering.

[47]  Aurobinda Routray,et al.  Estimation of Saccadic Ratio from eye image sequences to detect human alertness , 2012, 2012 4th International Conference on Intelligent Human Computer Interaction (IHCI).

[48]  T. Triggs,et al.  Time of day variations in driving performance. , 1997, Accident; analysis and prevention.

[49]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Researchers , 2007 .

[50]  Gupta Supratim,et al.  A New Method for Edge Extraction in Images using Local Form Factors , 2011 .

[51]  Anjith George,et al.  An on-board vision based system for drowsiness detection in automotive drivers , 2013 .

[52]  Hugo Proença,et al.  A robust eye-corner detection method for real-world data , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[53]  Yasuyuki Saito,et al.  Extraction of a symmetric object for eyeglass face analysis using active contour model , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[54]  H. Collewijn,et al.  Precise recording of human eye movements , 1975, Vision Research.

[55]  Pong C. Yuen,et al.  Multi-cues eye detection on gray intensity image , 2001, Pattern Recognit..

[56]  Dan Simon,et al.  Optimal State Estimation: Kalman, H∞, and Nonlinear Approaches , 2006 .

[57]  Hong Yan,et al.  Unified formulation of a class of image thresholding techniques , 1996, Pattern Recognit..

[58]  Zhi-Hua Zhou,et al.  Projection functions for eye detection , 2004, Pattern Recognit..

[59]  Guangda Su,et al.  Eyeglasses removal from facial images , 2005, Pattern Recognit. Lett..

[60]  Carlos Hitoshi Morimoto,et al.  Pupil detection and tracking using multiple light sources , 2000, Image Vis. Comput..

[61]  T. Åkerstedt,et al.  Impaired alertness and performance driving home from the night shift: a driving simulator study , 2005, Journal of sleep research.

[62]  D. Dinges,et al.  EVALUATION OF TECHNIQUES FOR OCULAR MEASUREMENT AS AN INDEX OF FATIGUE AND THE BASIS FOR ALERTNESS MANAGEMENT , 1998 .

[63]  M. Fitton,et al.  Methods of measuring eye movements. , 1966, American Orthoptic Journal.

[64]  Yoshinobu Ebisawa.  Improved video-based eye-gaze detection method , 1998, IEEE Trans. Instrum. Meas..

[65]  Djoko Purwanto,et al.  Relation between eye movement and fatigue: Classification of morning and afternoon measurement based on Fuzzy rule , 2009, International Conference on Instrumentation, Communication, Information Technology, and Biomedical Engineering 2009.

[66]  J. V. Van Gisbergen,et al.  Skewness of saccadic velocity profiles: a unifying parameter for normal and slow saccades. , 1987, Vision research.

[67]  D. Simon.  Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches , 2006 .

[68]  Ioannis Pitas,et al.  A novel eye detection algorithm utilizing edge-related geometrical information , 2006, 2006 14th European Signal Processing Conference.

[69]  J. Alex Stark,et al.  Adaptive image contrast enhancement using generalizations of histogram equalization , 2000, IEEE Trans. Image Process..

[70]  Zhiwei Zhu,et al.  Real time and non-intrusive driver fatigue monitoring , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[71]  Huabiao Qin,et al.  Drivers drowsiness detection in embedded system , 2007, 2007 IEEE International Conference on Vehicular Electronics and Safety.

[72]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[73]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[74]  Y. Uchikawa,et al.  Relation between human alertness, velocity wave profile of saccade, and performance of visual activities , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[75]  Tomaso A. Poggio,et al.  Example-Based Learning for View-Based Human Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[76]  Liying Lang,et al.  The Study of Driver Fatigue Monitor Algorithm Combined PERCLOS and AECS , 2008, 2008 International Conference on Computer Science and Software Engineering.

[77]  Gwen Littlewort,et al.  Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction. , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[78]  John D. Austin,et al.  Adaptive histogram equalization and its variations , 1987 .

[79]  Maria L. Thomas,et al.  Caffeine reversal of sleep deprivation effects on alertness and mood , 1993, Psychopharmacology.

[80]  Shengcai Liao,et al.  Illumination Invariant Face Recognition Using Near-Infrared Images , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[81]  M. Matousek,et al.  A method for assessing alertness fluctuations from EEG spectra. , 1983, Electroencephalography and clinical neurophysiology.

[82]  Xiaoyi Jiang,et al.  Towards Detection of Glasses in Facial Images , 2000, Pattern Analysis & Applications.

[83]  Theo Gevers,et al.  Accurate eye center location and tracking using isophote curvature , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[84]  John A. Stern,et al.  Blinks, saccades, and fixation pauses during vigilance task performance. II., Gender and time of day. , 1996 .

[85]  Azriel Rosenfeld,et al.  Eye detection in a face image using linear and nonlinear filters , 2001, Pattern Recognit..

[86]  KimLee-Sup,et al.  An advanced contrast enhancement using partially overlapped sub-block histogram equalization , 2001 .

[87]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[88]  Anirban Mukherjee,et al.  A New Method for Edge Extraction in Images using Local Form Factors , 2011 .

[89]  Y. Chan,et al.  A Kalman Filter Based Tracking Scheme with Input Estimation , 1979, IEEE Transactions on Aerospace and Electronic Systems.

[90]  Harry Wechsler,et al.  Eye Detection Using Optimal Wavelet Packets and Radial Basis Functions (RBFs) , 1999, Int. J. Pattern Recognit. Artif. Intell..

[91]  William A. Schaudt,et al.  Advances in Drowsy Driver Assistance Systems Through Data Fusion , 2012 .

[92]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[93]  Qiang Ji,et al.  Automatic Eye Detection and Its Validation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[94]  Jing-Yu Yang,et al.  Face detection using template matching and skin-color information , 2007, Neurocomputing.