SmartEye: An Accurate Infrared Eye Tracking System for Smartphones

The capability to estimate where a user is looking on a screen is known as gaze estimation or eye tracking. It has been used in medical applications including assessment of mood and learning disorders, and brain injury diagnosis. If accurate eye tracking could be integrated into commodity smartphones these diagnostics could be broadly deployed at very low cost. The highest accuracy and most robust eye tracking methods employ infrared cameras and illumination which are not yet available on all standard smartphones. In this paper, we present an accurate infrared eye tracking system on a smartphone, named SmartEye, on an industrial prototype phone equipped with an infrared camera and illumination. The system is accurate in the presence of head pose variation and device movements in the user’s hands, and requires only a one-time calibration routine to measure specific parameters of the user’s eye. Our system achieves a gaze estimation bias of 0.57° at a 20cm distance from the user, 5 times better than state-of-the art mobile device eye-tracking systems that do not use infrared illumination. Our system also allows for free head movements at distances between 20-40cm with a moderate increase in average gaze bias (to ~1°), and can operate at 12fps. This enhanced accuracy and increased mobility can expand significantly the range of eye-tracking applications that can be supported by smartphones.

[1]  Pierre Dillenbourg,et al.  3D model-based gaze estimation in natural reading: a systematic error correction procedure based on annotated texts , 2014, ETRA.

[2]  Qiong Huang,et al.  TabletGaze: dataset and analysis for unconstrained appearance-based gaze estimation in mobile tablets , 2017, Machine Vision and Applications.

[3]  Takahiro Okabe,et al.  Learning gaze biases with head motion for head pose-free gaze estimation , 2014, Image Vis. Comput..

[4]  Qiang Ji,et al.  In the Eye of the Beholder: A Survey of Models for Eyes and Gaze , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  M. Hautzinger,et al.  Attentional Processing of Food Pictures in Individuals with Anorexia Nervosa—An Eye-Tracking Study , 2011, Biological Psychiatry.

[6]  Marc L. Resnick,et al.  Ergonomics Society Annual Meeting , 2013 .

[7]  M. Calkins,et al.  Eye movement dysfunction in schizophrenia: a heritable characteristic for enhancing phenotype definition. , 2000, American journal of medical genetics.

[8]  P. Hancock,et al.  Looking at movies and cartoons: eye-tracking evidence from Williams syndrome and autism. , 2009, Journal of intellectual disability research : JIDR.

[9]  Oleg V. Komogortsev,et al.  Usability evaluation of eye tracking on an unmodified common tablet , 2013, CHI Extended Abstracts.

[10]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..

[11]  M. Eizenman,et al.  Attentional biases to body shape images in adolescents with anorexia nervosa: An exploratory eye-tracking study , 2014, Psychiatry Research.

[12]  Hiroshi Sato,et al.  MobiGaze: development of a gaze interface for handheld mobile devices , 2010, CHI EA '10.

[13]  Jonathan Chung.  A novel test of implicit memory; an eye tracking study , 2014 .

[14]  Wojciech Matusik,et al.  Eye Tracking for Everyone , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Moshe Eizenman,et al.  Accurate Model-Based Point of Gaze Estimation on Mobile Devices , 2018, Vision.

[16]  R. Asarnow,et al.  Exploratory Eye Movements to Pictures in Childhood-Onset Schizophrenia and Attention-Deficit/Hyperactivity Disorder (ADHD) , 1999, Journal of abnormal child psychology.

[17]  S. Valdois,et al.  The eye movements of dyslexic children during reading and visual search: Impact of the visual attention span , 2007, Vision Research.

[18]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[19]  Stephen J. Payne,et al.  Skim reading by satisficing: evidence from eye tracking , 2011, CHI.

[20]  Hong Va Leong,et al.  ScreenGlint: Practical, In-situ Gaze Estimation on Smartphones , 2017, CHI.

[21]  Rafael Cabeza,et al.  Evaluation of pupil center-eye corner vector for gaze estimation using a web cam , 2012, ETRA '12.

[22]  Guillaume Hervet,et al.  Is Banner Blindness Genuine? Eye Tracking Internet Text Advertising , 2011 .

[23]  Dan Witzner Hansen,et al.  Eye tracking in the wild , 2005, Comput. Vis. Image Underst..

[24]  Khushali Deulkar,et al.  Google Project Tango - A Convenient 3D Modeling Device , 2014 .

[25]  Moshe Eizenman,et al.  An Automatic Personal Calibration Procedure for Advanced Gaze Estimation Systems , 2010, IEEE Transactions on Biomedical Engineering.

[26]  Qiang Ji,et al.  3D gaze estimation with a single camera without IR illumination , 2008, 2008 19th International Conference on Pattern Recognition.

[27]  Jian-Gang Wang,et al.  Gaze determination via images of irises , 2001, Image Vis. Comput..

[28]  Roel Vertegaal,et al.  Designing attentive cell phone using wearable eyecontact sensors , 2002, CHI Extended Abstracts.

[29]  Kai Kunze,et al.  Where Are You Looking At? - Feature-Based Eye Tracking on Unmodified Tablets , 2013, 2013 2nd IAPR Asian Conference on Pattern Recognition.

[30]  Moshe Eizenman,et al.  Remote point-of-gaze estimation requiring a single-point calibration for applications with infants , 2008, ETRA.

[31]  Oleg V. Komogortsev,et al.  Eye tracking on unmodified common tablets: challenges and solutions , 2012, ETRA '12.

[32]  Carlos Hitoshi Morimoto,et al.  Eye gaze tracking techniques for interactive applications , 2005, Comput. Vis. Image Underst..

[33]  Qiong Huang,et al.  TabletGaze: Unconstrained Appearance-based Gaze Estimation in Mobile Tablets , 2015 .

[34]  Moshe Eizenman,et al.  General theory of remote gaze estimation using the pupil center and corneal reflections , 2006, IEEE Transactions on Biomedical Engineering.

[35]  Martin Böhme,et al.  Report on New Approaches to Eye Tracking. Summary of new algorithms. , 2009 .

[36]  Moshe Eizenman,et al.  A naturalistic visual scanning approach to assess selective attention in major depressive disorder , 2003, Psychiatry Research.

[37]  Oliver Hohlfeld,et al.  On the Applicability of Computer Vision based Gaze Tracking in Mobile Scenarios , 2015, MobileHCI.

[38]  Andreas Bulling,et al.  EyeTab: model-based gaze estimation on unmodified tablet computers , 2014, ETRA.

[39]  Kuo-Chin Fan,et al.  An adaptive eye gaze tracker system in the integrated cloud computing and mobile device , 2011, 2011 International Conference on Machine Learning and Cybernetics.

[40]  Jian-Gang Wang,et al.  Estimating the eye gaze from one eye , 2005, Comput. Vis. Image Underst..

[41]  Alan Cowey,et al.  The Basis of a Method of Perimetry with Monkeys , 1963 .