A robust eye gaze tracking method based on a virtual eyeball model

Gaze positions can provide important cues for natural computer interfaces. In this paper, we describe a new gaze estimation method based on a three dimensional analysis of the human eye which can be used in head-mounted display (HMD) environments. This paper presents four advantages over previous works. First, in order to obtain accurate gaze positions, we used a virtual eyeball model based on the 3D characteristics of the human eyeball. Second, we calculated the 3D position of the virtual eyeball and gaze vector by using a camera and three collimated IR-LEDs. Third, three reference frames (the camera, the monitor and the eye reference frames) were unified, which simplified the complex 3D converting calculations and allowed for calculation of the 3D eye position and gaze position on a HMD monitor. Fourth, a simple user-dependent calibration method was proposed by gazing at one position based on Kappa compensation. Experimental results showed that the eye gaze estimation error of the proposed method was lower than 1°.

[1]  Zhiwei Zhu,et al.  Eye and gaze tracking for interactive graphic display , 2002, SMARTGRAPH '02.

[2]  Kang Ryoung Park,et al.  Robust Fake Iris Detection Based on Variation of the Reflectance Ratio Between the IRIS and the Sclera , 2006, 2006 Biometrics Symposium: Special Session on Research at the Biometric Consortium Conference.

[3]  Anneli Folkesson,et al.  Numerical methods for engineers , 2007 .

[4]  Arie E. Kaufman,et al.  An eye tracking computer user interface , 1993, Proceedings of 1993 IEEE Research Properties in Virtual Reality Symposium.

[5]  Hidekazu Yoshikawa,et al.  Construction of the measurement system and its experimental study for diagnosing cerebral functional disorders using eye-sensing HMD , 2002, Proceedings of the 41st SICE Annual Conference. SICE 2002..

[6]  Worthy N. Martin,et al.  Human-computer interaction using eye-gaze input , 1989, IEEE Trans. Syst. Man Cybern..

[7]  Rangachar Kasturi,et al.  Machine vision , 1995 .

[8]  Sheng-Wen Shih,et al.  A novel approach to 3-D gaze tracking using stereo cameras , 2004, IEEE Trans. Syst. Man Cybern. Part B.

[9]  Carlos Hitoshi Morimoto,et al.  A computer vision framework for eye gaze tracking , 2003, 16th Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 2003).

[10]  J. Eskridge,et al.  The Hirschberg test: a double-masked clinical evaluation. , 1988, American journal of optometry and physiological optics.

[11]  Kang Ryoung Park,et al.  Intelligent Process Control Via Gaze Detection Technology. , 2000 .

[12]  Yoshinobu Ebisawa,et al.  Improved video-based eye-gaze detection method , 1994, Conference Proceedings. 10th Anniversary. IMTC/94. Advanced Technologies in I & M. 1994 IEEE Instrumentation and Measurement Technolgy Conference (Cat. No.94CH3424-9).

[13]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[14]  Moshe Eizenman,et al.  General theory of remote gaze estimation using the pupil center and corneal reflections , 2006, IEEE Transactions on Biomedical Engineering.

[15]  Naoki Mukawa,et al.  FreeGaze: a gaze tracking system for everyday gaze interaction , 2002, ETRA.

[16]  Alex Pentland,et al.  Visually Controlled Graphics , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Light, Color and Vision.Yves Le Grand , 1960 .

[18]  Saeed Mubarak,et al.  Fundamentals of Computer Vision , 1997 .

[19]  Jie Zhu,et al.  Subpixel eye gaze tracking , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[20]  Roberto Cipolla,et al.  Fast visual tracking by temporal consensus , 1996, Image Vis. Comput..

[21]  Kang Ryoung Park,et al.  A Study on Eye Gaze Estimation Method Based on Cornea Model of Human Eye , 2007, MIRAGE.

[22]  Kang Ryoung Park,et al.  3D First Person Shooting Game by Using Eye Gaze Tracking , 2005 .

[23]  Kang Ryoung Park,et al.  A Study on Fake Iris Detection based on the Reflectance of the Iris to the Sclera for Iris Recognition , 2005 .

[24]  Kang Ryoung Park,et al.  Vision-based gaze estimation method for stereoscopic display systems , 2007, IFAC HMS.

[25]  F. Schaeffel.  Kappa and Hirschberg Ratio Measured with an Automated Video Gaze Tracker , 2002, Optometry and vision science : official publication of the American Academy of Optometry.

[26]  Kentaro Toyama,et al.  Gaze Awareness for Video-Conferencing: A Software Approach , 2000, IEEE Multim..

[27]  Kang Ryoung Park,et al.  Fake Iris Detection by Using Purkinje Image , 2006, ICB.

[28]  Kang Ryoung Park.  Robust Gaze Estimation for Human Computer Interaction , 2006, PRICAI.

[29]  Junseok Park,et al.  Robust Gaze Tracking Method for Stereoscopic Virtual Reality Systems , 2007, HCI.

[30]  Emilio L. Zapata,et al.  Lower order circle and ellipse Hough transform , 1997, Pattern Recognit..

[31]  Shumin Zhai,et al.  Keeping an eye for HCI , 1999, XII Brazilian Symposium on Computer Graphics and Image Processing (Cat. No.PR00481).

[32]  George E. Park,et al.  THE CENTER OF OCULAR ROTATION IN THE HORIZONTAL PLANE , 1933 .

[33]  Janne Heikkilä,et al.  A four-step camera calibration procedure with implicit image correction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Myung Jin Chung,et al.  Non-contact eye gaze tracking system by mapping of corneal reflections , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[35]  H. Anton.  Elementary Linear Algebra , 1970 .

[36]  Chern-Sheng Lin.  An eye behavior measuring device for VR system , 2002 .

[37]  Hidekazu Yoshikawa,et al.  A new adaptive CAI system based on bio-informatic sensing: study on real-time method of analyzing ocular information by using eye-sensing HMD and method of adaptive CAI system configuration , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[38]  Michael J. Jones,et al.  Gaze estimation using morphable models , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[39]  Alberto Del Bimbo,et al.  Interacting through eyes , 1997, Robotics Auton. Syst..

[40]  Shumeet Baluja,et al.  Non-Intrusive Gaze Tracking Using Artificial Neural Networks , 1993, NIPS.

[41]  Stephen J. Maybank,et al.  On plane-based camera calibration: A general algorithm, singularities, applications , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[42]  Alain Finkel,et al.  CapRe: a Gaze Tracking System in Man-machine Interaction , 1998, J. Adv. Comput. Intell. Intell. Informatics.

[43]  R. C. Gonzalez,et al.  Unit 1. Introduction to Digital Image Processing , 2002 .

[44]  Alexander Zelinsky,et al.  3-D facial pose and gaze point estimation using a robust real-time tracking paradigm , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[45]  Jian-Gang Wang,et al.  Pose determination of human faces by using vanishing points , 2001, Pattern Recognit..

[46]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[47]  Margrit Betke,et al.  Gaze detection via self-organizing gray-scale units , 1999, Proceedings International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems. In Conjunction with ICCV'99 (Cat. No.PR00378).

[48]  Y. Matsumotot,et al.  Development of intelligent wheelchair system with face and gaze based interface , 2001, Proceedings 10th IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2001 (Cat. No.01TH8591).

[49]  George C. Stockman,et al.  Controlling a computer via facial aspect , 1995, IEEE Trans. Syst. Man Cybern..

[50]  M. Luckiesh,et al.  Color and vision. , 1915 .

[51]  Kang Ryoung Park,et al.  A Study on Manipulating Method of 3D Game in HMD Environment by using Eye Tracking , 2008 .

[52]  Kang Ryoung Park,et al.  Gaze position detection by computing the three dimensional facial positions and motions , 2002, Pattern Recognit..

[53]  Paolo Dario,et al.  Prototype of a vision-based gaze-driven man-machine interface , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[54]  Qiang Ji,et al.  3D Face pose estimation and tracking from a monocular camera , 2002, Image Vis. Comput..

[55]  Yoshinobu Ebisawa,et al.  Noncontact video-based eye-gaze detection method allowing large head displacements , 1996, Proceedings of 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[56]  James Gips,et al.  Direct Control of the Computer Through Electrodes Placed Around the Eyes , 1993, HCI.

[57]  Zhiwei Zhu,et al.  Novel Eye Gaze Tracking Techniques Under Natural Head Movement , 2007, IEEE Transactions on Biomedical Engineering.

[58]  Jian-Gang Wang,et al.  Study on eye gaze estimation , 2002, IEEE Trans. Syst. Man Cybern. Part B.