CamType: assistive text entry using gaze with an off-the-shelf webcam

As modern assistive technology advances, eye-based text entry systems have been developed to help a subset of physically challenged people to improve their communication ability. However, speed of text entry in early eye-typing system tends to be relatively slow due to dwell time. Recently, dwell-free methods have been proposed which outperform the dwell-based systems in terms of speed and resilience, but the extra eye-tracking device is still an indispensable equipment. In this article, we propose a prototype of eye-typing system using an off-the-shelf webcam without the extra eye tracker, in which the appearance-based method is proposed to estimate people’s gaze coordinates on the screen based on the frontal face images captured by the webcam. We also investigate some critical issues of the appearance-based method, which helps to improve the estimation accuracy and reduce computing complexity in practice. The performance evaluation shows that eye typing with webcam using the proposed method is comparable to the eye tracker under a small degree of head movement.

[1]  Malek Adjouadi,et al.  Remote Eye Gaze Tracking System as a Computer Interface for Persons with Severe Motor Disability , 2004, ICCHP.

[2]  Sung-Jea Ko,et al.  A novel iris center localization based on circle fitting using radially sampled features , 2015, 2015 International Symposium on Consumer Electronics (ISCE).

[3]  David R. Shanks,et al.  A simple algorithm for the offline recalibration of eye-tracking data through best-fitting linear transformation , 2015, Behavior research methods.

[4]  Qiang Ji,et al.  Probabilistic gaze estimation without active personal calibration , 2011, CVPR 2011.

[5]  Andrew Blake,et al.  Sparse and Semi-supervised Visual Mapping with the S^3GP , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[6]  Takahiro Okabe,et al.  Inferring human gaze from appearance via adaptive linear regression , 2011, 2011 International Conference on Computer Vision.

[7]  Jian-Gang Wang,et al.  Estimating the eye gaze from one eye , 2005, Comput. Vis. Image Underst..

[8]  Bu-Sung Lee,et al.  A robust recognition approach in eye-based dwell-free typing , 2015, 2015 IEEE International Conference on Progress in Informatics and Computing (PIC).

[9]  I. Scott MacKenzie,et al.  Eye typing using word and letter prediction and a fixation algorithm , 2008, ETRA.

[10]  Yoshinobu Ebisawa,et al.  Effectiveness of pupil area detection technique using two light sources and image difference method , 1993, Proceedings of the 15th Annual International Conference of the IEEE Engineering in Medicine and Biology Societ.

[11]  Nikos Fakotakis,et al.  Precise localization of eye centers in low resolution color images , 2015, Image Vis. Comput..

[12]  Carlos Hitoshi Morimoto,et al.  Eye gaze tracking techniques for interactive applications , 2005, Comput. Vis. Image Underst..

[13]  Sabrina Hoppe,et al.  Eype-Using Eye-Traces for Eye-Typing , 2013 .

[14]  Kari-Jouko Räihä,et al.  An exploratory study of eye typing fundamentals: dwell time, text entry rate, errors, and workload , 2012, CHI.

[15]  Juan J. Cerrolaza,et al.  Taxonomic study of polynomial regressions applied to the calibration of video-oculographic systems , 2008, ETRA.

[16]  Thorsten O. Zander,et al.  Combining Eye Gaze Input With a Brain–Computer Interface for Touchless Human–Computer Interaction , 2010, Int. J. Hum. Comput. Interact..

[17]  Per Ola Kristensson,et al.  The potential of dwell-free eye-typing for fast assistive gaze communication , 2012, ETRA.

[18]  Qiang Ji,et al.  In the Eye of the Beholder: A Survey of Models for Eyes and Gaze , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Bu-Sung Lee,et al.  Robust Eye-Based Dwell-Free Typing , 2016, Int. J. Hum. Comput. Interact..

[20]  Oleg Spakov,et al.  Fast gaze typing with an adjustable dwell time , 2009, CHI.

[21]  Anthony J. Hornof,et al.  Easy post-hoc spatial recalibration of eye tracking data , 2014, ETRA.

[22]  Kentaro Kotani,et al.  Design of Eye-Typing Interface Using Saccadic Latency of Eye Movement , 2010, Int. J. Hum. Comput. Interact..

[23]  E. B. Huey.  The Psychology And Pedagogy Of Reading , 1908 .

[24]  Jeffrey B. Mulligan,et al.  Implicit Calibration of a Remote Gaze Tracker , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[25]  Päivi Majaranta,et al.  Twenty years of eye typing: systems and design issues , 2002, ETRA.

[26]  Rafael Cabeza,et al.  Gaze Tracking System Model Based on Physical Parameters , 2007, Int. J. Pattern Recognit. Artif. Intell..

[27]  Antonio Nardone,et al.  Eye tracking communication devices in amyotrophic lateral sclerosis: Impact on disability and quality of life , 2013, Amyotrophic lateral sclerosis & frontotemporal degeneration.

[28]  Narendra Ahuja,et al.  Appearance-based eye gaze estimation , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[29]  Yoichi Sato,et al.  An Incremental Learning Method for Unconstrained Gaze Estimation , 2008, ECCV.

[30]  Chonho Lee,et al.  GazeTry: Swipe Text Typing Using Gaze , 2015, OZCHI.

[31]  Takahiro Okabe,et al.  Adaptive Linear Regression for Appearance-Based Gaze Estimation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Maria da Graça Campos Pimentel,et al.  Filteryedping: Design Challenges and User Performance of Dwell-Free Eye Typing , 2015, TACC.

[33]  Rafael Cabeza,et al.  Evaluation of pupil center-eye corner vector for gaze estimation using a web cam , 2012, ETRA '12.

[34]  Mario Fritz,et al.  Appearance-based gaze estimation in the wild , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Shumin Zhai,et al.  SHARK2: a large vocabulary shorthand writing system for pen-based computers , 2004, UIST '04.

[36]  Zhi-Hua Zhou,et al.  Projection functions for eye detection , 2004, Pattern Recognit..

[37]  David J. Ward,et al.  Artificial intelligence: Fast hands-free writing by gaze direction , 2002, Nature.

[38]  Zhiwei Zhu,et al.  Robust real-time eye detection and tracking under variable lighting conditions and various face orientations , 2005, Comput. Vis. Image Underst..

[39]  Jiahuan Zhou,et al.  Learning Reconstruction-Based Remote Gaze Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  A. Bujnowski,et al.  Eye mouse for disabled , 2008, 2008 Conference on Human System Interactions.

[41]  Gwo-Dong Chen,et al.  AN EYE TRACKING SYSTEM AND ITS APPLICATION IN AIDS FOR PEOPLE WITH SEVERE DISABILITIES , 2006 .

[42]  Theo Gevers,et al.  Accurate Eye Center Location through Invariant Isocentric Patterns , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  R. Spataro,et al.  The eye‐tracking computer device for communication in amyotrophic lateral sclerosis , 2014, Acta neurologica Scandinavica.

[44]  Atsuo Murata,et al.  Eye‐gaze input versus mouse: Cursor control as a function of age , 2006, Int. J. Hum. Comput. Interact..

[45]  Wojciech Matusik,et al.  Eye Tracking for Everyone , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Sayan Sarcar,et al.  EyeK: an efficient dwell-free eye gaze-based text entry system , 2013, APCHI.

[47]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[48]  Peter Robinson,et al.  OpenFace: An open source facial behavior analysis toolkit , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[49]  Tommy Strandvall,et al.  Eye Tracking in Human-Computer Interaction and Usability Research , 2009, INTERACT.

[50]  Maria da Graça Campos Pimentel,et al.  Filteryedping: A Dwell-Free Eye Typing Technique , 2015, CHI Extended Abstracts.

[51]  Takehiko Ohno,et al.  EyePrint: Using Passive Eye Trace From Reading to Enhance Document Access and Comprehension , 2007, Int. J. Hum. Comput. Interact..

[52]  Qiang Ji,et al.  Automatic Eye Detection and Its Validation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[53]  Mark Davies.  The Corpus of Contemporary American English (COCA) , 2012 .

[54]  Anke Huckauf,et al.  Alternatives to single character entry and dwell time selection on eye typing , 2010, ETRA.