Simultaneous Eye Tracking and Blink Detection with Interactive Particle Filters

We present a system that simultaneously tracks eyes and detects eye blinks. Two interactive particle filters are used for this purpose, one for the closed eyes and the other one for the open eyes. Each particle filter is used to track the eye locations as well as the scales of the eye subjects. The set of particles that gives higher confidence is defined as the primary set and the other one is defined as the secondary set. The eye location is estimated by the primary particle filter, and whether the eye status is open or closed is also decided by the label of the primary particle filter. When a new frame comes, the secondary particle filter is reinitialized according to the estimates from the primary particle filter. We use autoregression models for describing the state transition and a classification-based model for measuring the observation. Tensor subspace analysis is used for feature extraction which is followed by a logistic regression model to give the posterior estimation. The performance is carefully evaluated from two aspects: the blink detection rate and the tracking accuracy. The blink detection rate is evaluated using videos from varying scenarios, and the tracking accuracy is given by comparing with the benchmark data obtained using the Vicon motion capturing system. The setup for obtaining benchmark data for tracking accuracy evaluation is presented and experimental results are shown. Extensive experimental evaluations validate the capability of the algorithm.

[1]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Margrit Betke,et al.  Communication via eye blinks and eyebrow raises: video-based human-computer interfaces , 2003, Universal Access in the Information Society.

[3]  Rashid Ansari,et al.  Kernel particle filter for visual tracking , 2005, IEEE Signal Processing Letters.

[4]  Jouko Lampinen,et al.  Rao-Blackwellized particle filter for multiple target tracking , 2007, Inf. Fusion.

[5]  Anton van den Hengel,et al.  Augmented particle filtering for efficient visual tracking , 2005, IEEE International Conference on Image Processing 2005.

[6]  Margrit Betke,et al.  Real Time Eye Tracking and Blink Detection with USB Cameras , 2005 .

[7]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[8]  Mubarak Shah,et al.  Monitoring head/eye motion for driver alertness with one camera , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[9]  Stephen M. Krone,et al.  Markov Chain Monte Carlo in small worlds , 2006, Stat. Comput..

[10]  Jiawei Han,et al.  Subspace Learning Based on Tensor Analysis , 2005 .

[11]  M. Pitt,et al.  Filtering via Simulation: Auxiliary Particle Filters , 1999 .

[12]  Kiyoshi Nishiyama,et al.  Fast and effective generation of the proposal distribution for particle filters , 2005, Signal Process..

[13]  Margrit Betke,et al.  Communication via eye blinks - detection and duration analysis in real time , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[14]  Patrick Pérez,et al.  Color-Based Probabilistic Tracking , 2002, ECCV.

[15]  Yoshinobu Ebisawa,et al.  Quick menu selection using eye blink for eye-slaved nonverbal communicator with video-based eye-gaze detection , 1997, Proceedings of the 19th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 'Magnificent Milestones and Emerging Opportunities in Medical Engineering' (Cat. No.97CH36136).

[16]  Soon Ki Jung,et al.  Particle filter with analytical inference for human body tracking , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[17]  Jean-Yves Tourneret,et al.  A Rao-Blackwellized particle filter for INS/GPS integration , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[18]  Mohan M. Trivedi,et al.  Robust Facial Landmark Detection for Intelligent Vehicle System , 2005, AMFG.

[19]  Rashid Ansari,et al.  Kernel particle filter: iterative sampling for efficient visual tracking , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[20]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[21]  P. Fearnhead,et al.  An improved particle filter for non-linear problems , 1999 .

[22]  K. Heine,et al.  Unified framework for sampling/importance resampling algorithms , 2005, 2005 7th International Conference on Information Fusion.

[23]  Jun S. Liu,et al.  Blind Deconvolution via Sequential Imputations , 1995 .

[24]  James J. Little,et al.  A Boosted Particle Filter: Multitarget Detection and Tracking , 2004, ECCV.

[25]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[26]  Baoxin Li,et al.  Head Tracking Using Particle Filter with Intensity Gradient and Color Histogram , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[27]  Jing Xiao,et al.  Automatic recognition of eye blinking in spontaneously occurring behavior , 2002, Object recognition supported by user interaction for service robots.

[28]  Paul Blenkhorn,et al.  Blink detection for real-time eye tracking , 2002, J. Netw. Comput. Appl..

[29]  Fredrik Gustafsson,et al.  Particle filters for positioning, navigation, and tracking , 2002, IEEE Trans. Signal Process..

[30]  Sangjin Hong,et al.  Performance and complexity analysis of adaptive particle filtering for tracking applications , 2002, Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers, 2002..

[31]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[32]  Shin Yamamoto,et al.  Detection of consciousness degradation and concentration of a driver for friendly information service , 2001, IVEC2001. Proceedings of the IEEE International Vehicle Electronics Conference 2001. IVEC 2001 (Cat. No.01EX522).

[33]  Rickard Karlsson,et al.  Particle filtering for positioning and tracking applications , 2005 .

[34]  Deng Cai,et al.  Tensor Subspace Analysis , 2005, NIPS.

[35]  Mohan M. Trivedi,et al.  A Binary Tree for Probability Learning in Eye Detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[36]  F. Dellaert,et al.  A Rao-Blackwellized particle filter for EigenTracking , 2004, CVPR 2004.

[37]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[38]  Yong Rui,et al.  Better proposal distributions: object tracking using unscented particle filter , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[39]  Søren Hein,et al.  On the Estimation of Rigid Body Rotation from Noisy Data , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Jun S. Liu,et al.  Sequential Monte Carlo methods for dynamic systems , 1997 .

[41]  Fredrik Gustafsson,et al.  On Resampling Algorithms for Particle Filters , 2006, 2006 IEEE Nonlinear Statistical Signal Processing Workshop.

[42]  J. L. Roux.  An Introduction to the Kalman Filter , 2003 .

[43]  Larry S. Davis,et al.  Fast multiple object tracking via a hierarchical particle filter , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[44]  Mohan M. Trivedi,et al.  Facial Action Coding Using Multiple Visual Cues and a Hierarchy of Particle Filters , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[45]  Demetri Terzopoulos,et al.  Multilinear Analysis of Image Ensembles: TensorFaces , 2002, ECCV.

[46]  Andrew Calway,et al.  Real-Time Camera Tracking Using a Particle Filter , 2005, BMVC.

[47]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.