Appearance-Based Gaze Estimation Using Visual Saliency

We propose a gaze sensing method using visual saliency maps that does not need explicit personal calibration. Our goal is to create a gaze estimator using only the eye images captured from a person watching a video clip. Our method treats the saliency maps of the video frames as the probability distributions of the gaze points. We aggregate the saliency maps based on the similarity in eye images to efficiently identify the gaze points from the saliency maps. We establish a mapping between the eye images to the gaze points by using Gaussian process regression. In addition, we use a feedback loop from the gaze estimator to refine the gaze probability maps to improve the accuracy of the gaze estimation. The experimental results show that the proposed method works well with different people and video clips and achieves a 3.5-degree accuracy, which is sufficient for estimating a user's attention on a display.

[1]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[2]  Dan Witzner Hansen,et al.  Homography normalization for robust gaze estimation in uncalibrated setups , 2010, ETRA.

[3]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[4]  Claudio M. Privitera,et al.  Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Bernhard Schölkopf,et al.  How to Find Interesting Locations in Video: A Spatiotemporal Interest Point Detector Learned from Human Eye Movements , 2007, DAGM-Symposium.

[6]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[7]  Asha Iyer,et al.  Components of bottom-up gaze allocation in natural images , 2005, Vision Research.

[8]  Derrick J. Parkhurst,et al.  Modeling the role of salience in the allocation of overt visual attention , 2002, Vision Research.

[9]  Naoki Tanaka,et al.  One-point calibration gaze tracking based on eyeball kinematics using stereo cameras , 2008, ETRA.

[10]  Yoichi Sato,et al.  Calibration-free gaze sensing using saliency maps , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Hirotake Yamazoe,et al.  Remote gaze estimation with a single camera based on facial-feature tracking without special calibration actions , 2008, ETRA.

[12]  Moshe Eizenman,et al.  General theory of remote gaze estimation using the pupil center and corneal reflections , 2006, IEEE Transactions on Biomedical Engineering.

[13]  Takehiko Ohno,et al.  One-point calibration gaze tracking method , 2006, ETRA.

[14]  Eric Horvitz,et al.  Models of attention in computing and communication , 2003, Commun. ACM.

[15]  Qiang Ji,et al.  In the Eye of the Beholder: A Survey of Models for Eyes and Gaze , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  John K. Tsotsos,et al.  Saliency, attention, and visual search: an information theoretic approach. , 2009, Journal of vision.

[17]  Christof Koch,et al.  Learning a saliency map using fixated locations in natural scenes. , 2011, Journal of vision.

[18]  Takahiro Okabe,et al.  A Head Pose-free Approach for Appearance-based Gaze Estimation , 2011, BMVC.

[19]  Robert J. K. Jacob,et al.  Eye tracking in advanced interface design , 1995 .

[20]  Jeffrey S. Shell,et al.  Designing for augmented attention: Towards a framework for attentive user interfaces , 2006, Comput. Hum. Behav..

[21]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[22]  Rafael Cabeza,et al.  A Novel Gaze Estimation System With One Calibration Point , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Andrew Blake,et al.  Sparse and Semi-supervised Visual Mapping with the S^3GP , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[24]  Qiang Ji,et al.  Probabilistic gaze estimation without active personal calibration , 2011, CVPR 2011.

[25]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[26]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[27]  S. Sundararajan,et al.  Predictive Approaches for Choosing Hyperparameters in Gaussian Processes , 1999, Neural Computation.

[28]  Yoichi Sato,et al.  An Incremental Learning Method for Unconstrained Gaze Estimation , 2008, ECCV.

[29]  Christof Koch,et al.  Predicting human gaze using low-level saliency combined with face detection , 2007, NIPS.

[30]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[31]  Moshe Eizenman,et al.  Remote point-of-gaze estimation requiring a single-point calibration for applications with infants , 2008, ETRA.

[32]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[33]  George Loizou,et al.  Computer vision and pattern recognition , 2007, Int. J. Comput. Math..

[34]  Bernhard Schölkopf,et al.  A Nonparametric Approach to Bottom-Up Visual Saliency , 2006, NIPS.

[35]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[36]  Alexander C. Schütz,et al.  Eye movements and perception: a selective review. , 2011, Journal of vision.