Appearance-Based Gaze Estimation via Evaluation-Guided Asymmetric Regression

Eye gaze estimation has been increasingly demanded by recent intelligent systems to accomplish a range of interaction-related tasks, by using simple eye images as input. However, learning the highly complex regression between eye images and gaze directions is nontrivial, and thus the problem is yet to be solved efficiently. In this paper, we propose the Asymmetric Regression-Evaluation Network (ARE-Net), and try to improve the gaze estimation performance to its full extent. At the core of our method is the notion of “two eye asymmetry” observed during gaze estimation for the left and right eyes. Inspired by this, we design the multi-stream ARE-Net; one asymmetric regression network (AR-Net) predicts 3D gaze directions for both eyes with a novel asymmetric strategy, and the evaluation network (E-Net) adaptively adjusts the strategy by evaluating the two eyes in terms of their performance during optimization. By training the whole network, our method achieves promising results and surpasses the state-of-the-art methods on multiple public datasets.

[1]  Shumeet Baluja,et al.  Non-Intrusive Gaze Tracking Using Artificial Neural Networks , 1993, NIPS.

[2]  Takahiro Okabe,et al.  Head pose-free appearance-based gaze sensing via eye image synthesis , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[3]  Jeffrey F. Cohn,et al.  Person-Independent 3D Gaze Estimation Using Face Frontalization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[4]  David R. Bull,et al.  Gaze Location Prediction for Broadcast Football Video , 2013, IEEE Transactions on Image Processing.

[5]  Narendra Ahuja,et al.  Appearance-based eye gaze estimation , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[6]  Atsushi Nakazawa,et al.  Point of Gaze Estimation through Corneal Surface Reflection in an Active Illumination Environment , 2012, ECCV.

[7]  Mario Fritz,et al.  Appearance-based gaze estimation in the wild , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Wojciech Matusik,et al.  Eye Tracking for Everyone , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Qiang Ji,et al.  In the Eye of the Beholder: A Survey of Models for Eyes and Gaze , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Timo Schneider,et al.  Manifold Alignment for Person Independent Appearance-Based Gaze Estimation , 2014, 2014 22nd International Conference on Pattern Recognition.

[11]  Jean-Marc Odobez,et al.  EYEDIAP: a database for the development and evaluation of gaze estimation algorithms from RGB and RGB-D cameras , 2014, ETRA.

[12]  Qiang Ji,et al.  Real Time Eye Gaze Tracking with 3D Deformable Eye-Face Model , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Yusuke Sugano,et al.  Everyday Eye Contact Detection Using Unsupervised Gaze Target Discovery , 2017, UIST.

[14]  Yoichi Sato,et al.  Appearance-Based Gaze Estimation With Online Calibration From Mouse Operations , 2015, IEEE Transactions on Human-Machine Systems.

[15]  Wangjiang Zhu,et al.  Monocular Free-Head 3D Gaze Tracking with Deep Learning and Geometry Constraints , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[17]  Jean-Marc Odobez,et al.  Geometric Generative Gaze Estimation (G3E) for Remote RGB-D Cameras , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Yusuke Sugano,et al.  AggreGaze: Collective Estimation of Audience Attention on Public Displays , 2016, UIST.

[19]  Peter Robinson,et al.  Learning an appearance-based gaze estimator from one million synthesised images , 2016, ETRA.

[20]  Rongrong Ji,et al.  Toward Statistical Modeling of Saccadic Eye-Movement and Visual Saliency , 2014, IEEE Transactions on Image Processing.

[21]  Zicheng Liu,et al.  Eye gaze tracking using an RGBD camera: a comparison with a RGB solution , 2014, UbiComp Adjunct.

[22]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[23]  Yoichi Sato,et al.  Appearance-Based Gaze Estimation via Uncalibrated Gaze Pattern Recovery , 2017, IEEE Transactions on Image Processing.

[24]  Yoichi Sato,et al.  Learning-by-Synthesis for Appearance-Based 3D Gaze Estimation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Takahiro Okabe,et al.  Adaptive Linear Regression for Appearance-Based Gaze Estimation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Andrew Blake,et al.  Sparse and Semi-supervised Visual Mapping with the S^3GP , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[27]  Zhiwei Zhu,et al.  Novel Eye Gaze Tracking Techniques Under Natural Head Movement , 2007, IEEE Transactions on Biomedical Engineering.

[28]  Carlos Hitoshi Morimoto,et al.  Eye gaze tracking techniques for interactive applications , 2005, Comput. Vis. Image Underst..

[29]  Moshe Eizenman,et al.  General theory of remote gaze estimation using the pupil center and corneal reflections , 2006, IEEE Transactions on Biomedical Engineering.

[30]  Nicu Sebe,et al.  Combining Head Pose and Eye Location Information for Gaze Estimation , 2012, IEEE Transactions on Image Processing.

[31]  Mario Fritz,et al.  It’s Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[32]  Li-Qun Xu,et al.  A Novel Approach to Real-time Non-intrusive Gaze Finding , 1998, BMVC.

[33]  Andrew T Duchowski,et al.  A breadth-first survey of eye-tracking applications , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[34]  Yoichi Sato,et al.  Appearance-Based Gaze Estimation Using Visual Saliency , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.