EllSeg: An Ellipse Segmentation Framework for Robust Gaze Tracking

Ellipse fitting, an essential component in pupil or iris tracking based video oculography, is performed on previously segmented eye parts generated using various computer vision techniques. Several factors, such as occlusions due to eyelid shape, camera position or eyelashes, frequently break ellipse fitting algorithms that rely on well-defined pupil or iris edge segments. In this work, we propose training a convolutional neural network to directly segment entire elliptical structures and demonstrate that such a framework is robust to occlusions and offers superior pupil and iris tracking performance (at least 10% and 24% increase in pupil and iris center detection rate respectively within a two-pixel error margin) compared to using standard eye parts segmentation for multiple publicly available synthetic segmentation datasets.

[1]  Thiago Santini,et al.  PuRe: Robust pupil detection for real-time pervasive eye tracking , 2017, Comput. Vis. Image Underst..

[2]  Qiang Ji,et al.  In the Eye of the Beholder: A Survey of Models for Eyes and Gaze , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Marcus Nyström,et al.  The influence of calibration method and eye physiology on eyetracking data quality , 2013, Behavior research methods.

[4]  Andreas Bulling,et al.  Pupil: an open source platform for pervasive eye tracking and mobile gaze-based interaction , 2014, UbiComp Adjunct.

[5]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[6]  Joohwan Kim,et al.  NVGaze: An Anatomically-Informed Dataset for Low-Latency, Near-Eye Gaze Estimation , 2019, CHI.

[7]  Thiago Santini,et al.  ElSe: ellipse selection for robust pupil detection in real-world environments , 2015, ETRA.

[8]  Bingrong Hong,et al.  Accurate Pupil Features Extraction Based on New Projection Function , 2012, Comput. Informatics.

[9]  Reynold Bailey,et al.  RIT-Eyes: realistically rendered eye images for eye-tracking applications , 2020, ETRA Adjunct.

[10]  Jeff B. Pelz,et al.  Binocular eye tracking calibration during a virtual ball catching task using head mounted display , 2016, SAP.

[11]  Wolfgang Rosenstiel,et al.  PupilNet v2.0: Convolutional Neural Networks for CPU based real time Robust Pupil Detection , 2017, ArXiv.

[12]  Gregory Hughes,et al.  OpenEDS: Open Eye Dataset , 2019, ArXiv.

[13]  Hiok Chai Quek,et al.  ElliFit: An unconstrained, non-iterative, least squares based geometric Ellipse Fitting method , 2013, Pattern Recognit..

[14]  Wolfgang Rosenstiel,et al.  500, 000 Images Closer to Eyelid and Pupil Segmentation , 2019, CAIP.

[15]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Larry N Thibos,et al.  Optical models of the human eye , 2016, Clinical & experimental optometry.

[17]  Dieter Schmalstieg,et al.  Hybrid Eye Tracking: Combining Iris Contour and Corneal Imaging , 2015, ICAT-EGVE.

[18]  Otmar Hilliges,et al.  Deep Pictorial Gaze Estimation , 2018, ECCV.

[19]  Zhengyang Wu,et al.  EyeNet: A Multi-Task Network for Off-Axis Eye Gaze Estimation and User Understanding , 2019, ArXiv.

[20]  Sébastien Ourselin,et al.  Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations , 2017, DLMIA/ML-CDS@MICCAI.

[21]  J. Pelz,et al.  Motion tracking of iris features to detect small eye movements , 2019, Journal of eye movement research.

[22]  Andrew T. Duchowski,et al.  Eye Tracking Methodology: Theory and Practice , 2003, Springer London.

[23]  Gudrun Klinker,et al.  Interaction-free calibration for optical see-through head-mounted displays based on 3D Eye localization , 2014, 2014 IEEE Symposium on 3D User Interfaces (3DUI).

[24]  C. McAlinden,et al.  Centration axis in refractive surgery , 2015, Eye and Vision.

[25]  Neil Dodgson,et al.  A fully-automatic , temporal approach to single camera , glint-free 3 D eye model fitting , 2013 .

[26]  Enkelejda Kasneci,et al.  Tiny convolution, decision tree, and binary neuronal networks for robust and real time pupil outline estimation , 2020, ETRA Short Papers.

[27]  Zhen He,et al.  Numerical Coordinate Regression with Convolutional Neural Networks , 2018, ArXiv.

[28]  Neil A. Dodgson,et al.  Robust real-time pupil tracking in highly off-axis images , 2012, ETRA.

[29]  Yoshua Bengio,et al.  The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[30]  Seyed-Ahmad Ahmadi,et al.  DeepVOG: Open-source pupil segmentation and gaze estimation in neuroscience using deep learning , 2019, Journal of Neuroscience Methods.

[31]  Jeff B. Pelz,et al.  Compensating for eye tracker camera movement , 2006, ETRA.

[32]  Jose Dolz,et al.  Boundary loss for highly unbalanced segmentation , 2018, MIDL.

[33]  Derek Bradley,et al.  Lightweight eye capture using a parametric model , 2016, ACM Trans. Graph..

[34]  Thiago Santini,et al.  Get a grip: slippage-robust and glint-free gaze estimation for real-time pervasive head-mounted eye tracking , 2019, ETRA.

[35]  Wolfgang Rosenstiel,et al.  ExCuSe: Robust Pupil Detection in Real-World Scenarios , 2015, CAIP.

[36]  Antje Winkel,et al.  Eye Tracking Methodology Theory And Practice , 2016 .

[37]  Thiago Santini,et al.  PuReST: robust pupil tracking for real-time pervasive eye tracking , 2018, ETRA.

[38]  Gordon D. Love,et al.  Chromablur , 2017, ACM Trans. Graph..

[39]  Reynold J. Bailey,et al.  RIT-Eyes: Rendering of near-eye images for eye-tracking applications , 2020, ACM Symposium on Applied Perception.

[40]  Enkelejda Kasneci,et al.  Neural networks for optical vector and eye ball parameter estimation , 2020, ETRA Short Papers.

[41]  Enkelejda Kasneci,et al.  Pupil detection for head-mounted eye tracking in the wild: an evaluation of the state of the art , 2016, Machine Vision and Applications.

[42]  William H. Sanders,et al.  Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , 2014 .

[43]  Thiago Santini,et al.  EyeRecToo: Open-source Software for Real-time Pervasive Head-mounted Eye Tracking , 2017, VISIGRAPP.

[44]  Reynold J. Bailey,et al.  RITnet: Real-time Semantic Segmentation of the Eye for Gaze Tracking , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[45]  Andreas Bulling,et al.  EyeTab: model-based gaze estimation on unmodified tablet computers , 2014, ETRA.

[46]  Cordelia Schmid,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[47]  Qian Chen,et al.  Conic-based algorithm for visual line estimation from one image , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[48]  Joohwan Kim,et al.  Towards foveated rendering for gaze-tracked virtual reality , 2016, ACM Trans. Graph..

[49]  Yusuke Sugano,et al.  Labelled pupils in the wild: a dataset for studying pupil detection in unconstrained environments , 2015, ETRA.

[50]  Beno Benhabib,et al.  Three-dimensional location estimation of circular features for machine vision , 1992, IEEE Trans. Robotics Autom..

[51]  Gjergji Kasneci,et al.  PupilNet: Convolutional Neural Networks for Robust Pupil Detection , 2016, ArXiv.

[52]  Thomas Haslwanter,et al.  Measuring torsional eye movements by tracking stable iris features , 2010, Journal of Neuroscience Methods.

[53]  Yiguang Liu,et al.  A Geometry-Appearance-Based Pupil Detection Method for Near-Infrared Head-Mounted Cameras , 2018, IEEE Access.

[54]  Francesco Visin,et al.  A guide to convolution arithmetic for deep learning , 2016, ArXiv.

[55]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[56]  Dmytro Mishkin,et al.  Kornia: an Open Source Differentiable Computer Vision Library for PyTorch , 2019, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[57]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.