CBF: circular binary features for robust and real-time pupil center detection

Modern eye tracking systems rely on fast and robust pupil detection, and several algorithms have been proposed for eye tracking under real world conditions. In this work, we propose a novel binary feature selection approach that is trained by computing conditional distributions. These features are scalable and rotatable, allowing for distinct image resolutions, and consist of simple intensity comparisons, making the approach robust to different illumination conditions as well as rapid illumination changes. The proposed method was evaluated on multiple publicly available data sets, considerably outperforming state-of-the-art methods, and being real-time capable for very high frame rates. Moreover, our method is designed to be able to sustain pupil center estimation even when typical edge-detection-based approaches fail - e.g., when the pupil outline is not visible due to occlusions from reflections or eye lids / lashes. As a consequece, it does not attempt to provide an estimate for the pupil outline. Nevertheless, the pupil center suffices for gaze estimation - e.g., by regressing the relationship between pupil center and gaze point during calibration.

[1]  Anatole Lécuyer,et al.  Using an Eye-Tracking System to Improve Camera Motions and Depth-of-Field Blur Effects in Virtual Environments , 2008, 2008 IEEE Virtual Reality Conference.

[2]  Susan K. Schnipke,et al.  Trials and tribulations of using an eye-tracking system , 2000, CHI Extended Abstracts.

[3]  Yusuke Sugano,et al.  InvisibleEye: Mobile Eye Tracking Using Multiple Low-Resolution Cameras and Learning-Based Gaze Estimation , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[4]  Wolfgang Rosenstiel,et al.  Online Classification of Eye Tracking Data for Automated Analysis of Traffic Hazard Perception , 2013, ICANN.

[5]  Thomas C. Kübler,et al.  Online Recognition of Fixations, Saccades and Smooth Pursuits for Automated Analysis of Traffic Hazard Perception , 2015 .

[6]  Vincent Lepetit,et al.  Fast Keypoint Recognition Using Random Ferns , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Wolfgang Rosenstiel,et al.  The art of pervasive eye tracking: unconstrained eye tracking in the Austrian Gallery Belvedere , 2018, PETMEI@ETRA.

[8]  Andreas Bulling,et al.  EyeTab: model-based gaze estimation on unmodified tablet computers , 2014, ETRA.

[9]  W. Rosenstiel,et al.  Binocular Glaucomatous Visual Field Loss and Its Impact on Visual Exploration - A Supermarket Study , 2014, PloS one.

[10]  Francesc Moreno-Noguer,et al.  Efficient rotation invariant object detection using boosted Random Ferns , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Päivi Majaranta,et al.  Eye Tracking and Eye-Based Human–Computer Interaction , 2014 .

[12]  M. Magnor,et al.  Real-Time Gaze Tracking with a Consumer-Grade Video Camera , 2010 .

[13]  Theo Gevers,et al.  Accurate Eye Center Location through Invariant Isocentric Patterns , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  S T Moore,et al.  Robust pupil center detection using a curvature algorithm. , 1999, Computer methods and programs in biomedicine.

[15]  Raimund Dachselt,et al.  Look & touch: gaze-supported target acquisition , 2012, CHI.

[16]  Kin-Man Lam,et al.  Simultaneous Hallucination and Recognition of Low-Resolution Faces Based on Singular Value Decomposition , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Andreas Bulling,et al.  Pupil: an open source platform for pervasive eye tracking and mobile gaze-based interaction , 2014, UbiComp Adjunct.

[18]  Dario Cazzato,et al.  Randomized circle detection with isophotes curvature analysis , 2015, Pattern Recognit..

[19]  Hans-Werner Gellersen,et al.  Cross-device gaze-supported point-to-point content transfer , 2014, ETRA.

[20]  Heiner Bubb,et al.  Dikablis (digital wireless gaze tracking system) – operation mode and evaluation of the human machine interaction , 2006 .

[21]  Nikola Kasabov,et al.  Artificial Neural Networks: Methods and Applications in Bio-/Neuroinformatics , 2014 .

[22]  Neil A. Dodgson,et al.  Robust real-time pupil tracking in highly off-axis images , 2012, ETRA.

[23]  W. Rosenstiel,et al.  Homonymous Visual Field Loss and Its Impact on Visual Exploration: A Supermarket Study. , 2014, Translational vision science & technology.

[24]  Fengliang Xu,et al.  Real-time eye detection and tracking for driver observation under various light conditions , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[25]  Rafael Cabeza,et al.  Robust algorithm for pupil-glint vector detection in a video-oculography eyetracking system , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[26]  Anand K. Gramopadhye,et al.  3D eye movement analysis for VR visual inspection training , 2002, ETRA.

[27]  Antonio García Dopico,et al.  A Precise Eye-Gaze Detection and Tracking System , 2003, WSCG.

[28]  M Haubner,et al.  VR interaction techniques for medical imaging applications. , 1998, Computer methods and programs in biomedicine.

[29]  Zahra Hakimi,et al.  SET: a pupil detection method using sinusoidal approximation , 2015, Front. Neuroeng..

[30]  Dongheng Li,et al.  Starburst: A hybrid algorithm for video-based eye tracking combining feature-based and model-based approaches , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[31]  Yusuke Sugano,et al.  Self-Calibrating Head-Mounted Eye Trackers Using Egocentric Visual Saliency , 2015, UIST.

[32]  Francesc Moreno-Noguer,et al.  Boosted Random Ferns for Object Detection , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Wolfgang Rosenstiel,et al.  Driver-Activity Recognition in the Context of Conditionally Autonomous Driving , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[34]  Manfred Tscheligi,et al.  Eye Tracking in the Car: Challenges in a Dual-Task Scenario on a Test Track , 2014, AutomotiveUI.

[35]  Anand K. Gramopadhye,et al.  Binocular eye tracking in VR for visual inspection training , 2001, VRST '01.

[36]  Enkelejda Kasneci,et al.  Towards the automated recognition of assistance need for drivers with impaired visual field , 2013 .

[37]  Lun Yu,et al.  A robust and accurate detection of pupil images , 2010, 2010 3rd International Conference on Biomedical Engineering and Informatics.

[38]  Wolfgang Rosenstiel,et al.  ExCuSe: Robust Pupil Detection in Real-World Scenarios , 2015, CAIP.

[39]  Thiago Santini,et al.  ElSe: ellipse selection for robust pupil detection in real-world environments , 2015, ETRA.

[40]  Kin-Man Lam,et al.  Facial-feature detection and localization based on a hierarchical scheme , 2014, Inf. Sci..

[41]  W. Rosenstiel,et al.  Driving with Binocular Visual Field Loss? A Study on a Supervised On-Road Parcours with Simultaneous Eye and Head Tracking , 2014, PloS one.

[42]  Kin-Man Lam,et al.  A novel face-hallucination scheme based on singular value decomposition , 2013, Pattern Recognit..

[43]  Keiichi Abe,et al.  Topological structural analysis of digitized binary images by border following , 1985, Comput. Vis. Graph. Image Process..

[44]  Thiago Santini,et al.  PuRe: Robust pupil detection for real-time pervasive eye tracking , 2017, Comput. Vis. Image Underst..

[45]  Andrew T Duchowski,et al.  A breadth-first survey of eye-tracking applications , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[46]  O.K. Tonguz,et al.  A High Speed Eye Tracking System with Robust Pupil Center Estimation Algorithm , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[47]  Yusuke Sugano,et al.  Labelled pupils in the wild: a dataset for studying pupil detection in unconstrained environments , 2015, ETRA.

[48]  Gjergji Kasneci,et al.  PupilNet: Convolutional Neural Networks for Robust Pupil Detection , 2016, ArXiv.

[49]  Mario Fritz,et al.  Appearance-based gaze estimation in the wild , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).