Real-time eye blink detection with GPU-based SIFT tracking

This paper reports on the implementation of a GPU-based, real-time eye blink detector on very low contrast images acquired under near-infrared illumination. This detector is part of a multi-sensor data acquisition and analysis system for driver performance assessment and training. Eye blinks are detected inside regions of interest that are aligned with the subject's eyes at initialization. Alignment is maintained through time by tracking SIFT feature points that are used to estimate the affine transformation between the initial face pose and the pose in subsequent frames. The GPU implementation of the SIFT feature point extraction algorithm ensures real-time processing. An eye blink detection rate of 97% is obtained on a video dataset of 33,000 frames showing 237 blinks from 22 subjects.

[1]  Ioannis Pitas,et al.  A Fully Automatic Approach to Facial Feature Detection and Tracking , 1997, AVBPA.

[2]  Zhiwei Zhu,et al.  Combining Kalman filtering and mean shift for real time eye tracking under active IR illumination , 2002, Object recognition supported by user interaction for service robots.

[3]  Glenn F. Wilson,et al.  An Analysis of Mental Workload in Pilots During Flight Using Multiple Psychophysiological Measures , 2002 .

[4]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[5]  Jing Xiao,et al.  Automatic recognition of eye blinking in spontaneously occurring behavior , 2002, Object recognition supported by user interaction for service robots.

[6]  Y. V. Venkatesh,et al.  Blink detection and eye tracking for eye localization , 2003, TENCON 2003. Conference on Convergent Technologies for Asia-Pacific Region.

[7]  Fourth Canadian Conference on Computer and Robot Vision (CRV 2007), 28-30 May 2007, Montreal, Quebec, Canada , 2007, CRV.

[8]  Andrew Calway,et al.  Using affine correspondence to estimate 3-D facial pose , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[9]  D. Schroeder,et al.  Blink Rate: A Possible Measure of Fatigue , 1994, Human factors.

[10]  Gwen Littlewort,et al.  A Prototype for Automatic Recognition of Spontaneous Facial Actions , 2002, NIPS.

[11]  Qiang Ji,et al.  Automatic Eye Detection and Its Validation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[12]  Steve Mann,et al.  OpenVIDIA: parallel GPU computer vision , 2005, MULTIMEDIA '05.

[13]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[14]  Dick de Waard,et al.  The measurement of drivers' mental workload , 1996 .

[15]  Irfan A. Essa,et al.  Detecting and tracking eyes by using their physiological properties, dynamics, and appearance , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[16]  A Williamson,et al.  Review of on-road driver fatigue monitoring devices , 2005 .

[17]  Maja Pantic,et al.  Spontaneous vs. posed facial behavior: automatic analysis of brow actions , 2006, ICMI '06.