Eye-Gaze Estimation using a Deep Capsule-based Regression Network

Eye-gaze information is used in a variety of user platforms, such as driver monitoring systems and head-mounted interfaces. In order to estimate human eye-gaze, many solutions have been proposed, using different devices and techniques. However, achieving such estimation using only cheap devices like RGB cameras would enable gaze interactions on mobile devices and therefore generalise this kind of interaction. It could also enable behavior studies based on gaze and made on every day devices. We propose in this paper a new method for eye-gaze estimation using a new deep learning architecture based on the Capsule Neural Network. Capsule Networks have shown great results so far on classification tasks, but only a few works use them for regression tasks.By taking advantage of the Capsule Network architecture and its ability to reconstruct images, we are able to recreate simplified eye images and then estimate human gaze from them. Experiments are performed on two representative datasets for the task of eye-gaze estimation. Encouraging results are obtained for both the estimation and the reconstruction.

[1]  Mitsuru Ishizuka,et al.  Cascading Hand and Eye Movement for Augmented Reality Videoconferencing , 2007, 2007 IEEE Symposium on 3D User Interfaces.

[2]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Hazem Wannous,et al.  3D human motion analysis framework for shape similarity and retrieval , 2014, Image Vis. Comput..

[4]  Wangjiang Zhu,et al.  Monocular Free-Head 3D Gaze Tracking with Deep Learning and Geometry Constraints , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[5]  Qiang Ji,et al.  In the Eye of the Beholder: A Survey of Models for Eyes and Gaze , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Armando Barreto,et al.  Integrated electromyogram and eye-gaze tracking cursor control system for computer users with motor disabilities. , 2008, Journal of rehabilitation research and development.

[7]  Konstantinos N. Plataniotis,et al.  Brain Tumor Type Classification via Capsule Networks , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[8]  Jan Kautz,et al.  Light-Weight Head Pose Invariant Gaze Tracking , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Jean-Marc Odobez,et al.  Gaze Estimation in the 3D Space Using RGB-D Sensors , 2015, International Journal of Computer Vision.

[10]  Otmar Hilliges,et al.  Deep Pictorial Gaze Estimation , 2018, ECCV.

[11]  Wojciech Matusik,et al.  Eye Tracking for Everyone , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Sergio Escalera,et al.  Recurrent CNN for 3D Gaze Estimation using Appearance and Shape Cues , 2018, BMVC.

[13]  Mario Fritz,et al.  Appearance-based gaze estimation in the wild , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  David Filliat,et al.  3D Hand Gesture Recognition Using a Depth and Skeletal Dataset , 2017, 3DOR@Eurographics.

[15]  Stefanos Zafeiriou,et al.  300 Faces In-The-Wild Challenge: database and results , 2016, Image Vis. Comput..

[16]  Hazem Wannous,et al.  3D Hand Gesture Recognition by Analysing Set-of-Joints Trajectories , 2016, UHA3DS@ICPR.

[17]  Jian-Gang Wang,et al.  Eye gaze estimation from a single image of one eye , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[18]  H. Engeland,et al.  Gaze behavior of children with pervasive developmental disorder toward human faces: a fixation time study. , 2002, Journal of child psychology and psychiatry, and allied disciplines.

[19]  Seyed-Ahmad Ahmadi,et al.  DeepVOG: Open-source pupil segmentation and gaze estimation in neuroscience using deep learning , 2019, Journal of Neuroscience Methods.

[20]  Steven K. Feiner,et al.  Gaze locking: passive eye contact detection for human-object interaction , 2013, UIST.

[21]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[22]  Takahiro Okabe,et al.  Learning gaze biases with head motion for head pose-free gaze estimation , 2014, Image Vis. Comput..

[23]  Zicheng Liu,et al.  Real time gaze estimation with a consumer depth camera , 2015, Inf. Sci..

[24]  Mario Fritz,et al.  MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  T. Loetscher,et al.  Eye Movements During Everyday Behavior Predict Personality Traits , 2018, Front. Hum. Neurosci..

[26]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.