A general method for the point of regard estimation in 3D space

A novel approach to 3D gaze estimation for wearable multi-camera devices is proposed and its effectiveness is demonstrated both theoretically and empirically. The proposed approach, firmly grounded on the geometry of the multiple views, introduces a calibration procedure that is efficient, accurate, highly innovative but also practical and easy. Thus, it can run online with little intervention from the user. The overall gaze estimation model is general, as no particular complex model of the human eye is assumed in this work. This is made possible by a novel approach, that can be sketched as follows: each eye is imaged by a camera; two conics are fitted to the imaged pupils and a calibration sequence, consisting in the subject gazing a known 3D point, while moving his/her head, provides information to 1) estimate the optical axis in 3D world; 2) compute the geometry of the multi-camera system; 3) estimate the Point of Regard in 3D world. The resultant model is being used effectively to study visual attention by means of gaze estimation experiments, involving people performing natural tasks in wide-field, unstructured scenarios.

[1]  R. Carpenter,et al.  Movements of the Eyes , 1978 .

[2]  Moshe Eizenman,et al.  General theory of remote gaze estimation using the pupil center and corneal reflections , 2006, IEEE Transactions on Biomedical Engineering.

[3]  Qiang Ji,et al.  In the Eye of the Beholder: A Survey of Models for Eyes and Gaze , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  E. H. Thompson.  Some observations on analytical aerial triangulation , 1959 .

[5]  Alberto Del Bimbo,et al.  Visual capture and understanding of hand pointing actions in a 3-D environment , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[6]  Rafael Cabeza,et al.  A Novel Gaze Estimation System With One Calibration Point , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Richard I. Hartley,et al.  Lines and Points in Three Views and the Trifocal Tensor , 1997, International Journal of Computer Vision.

[8]  O. Faugeras,et al.  The Geometry of Multiple Images , 1999 .

[9]  David A. Atchison,et al.  Optics of the Human Eye , 2023 .

[10]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  Michael Werman,et al.  Trilinearity of three perspective views and its associated tensor , 1995, Proceedings of IEEE International Conference on Computer Vision.

[12]  D. Simons,et al.  CHAPTER 13 – Change Blindness , 2005 .

[13]  Ronald A. Rensink,et al.  On the Failure to Detect Changes in Scenes Across Brief Interruptions , 2000 .

[14]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[15]  Ron Goldman,et al.  Elimination and resultants. 1. Elimination and bivariate resultants , 1995, IEEE Computer Graphics and Applications.

[16]  Peter D. Lawrence,et al.  A single camera eye-gaze tracking system with free head motion , 2006, ETRA.

[17]  K. Preston White,et al.  Spatially dynamic calibration of an eye-tracking system , 1993, IEEE Trans. Syst. Man Cybern..

[18]  Daphna Weinshall,et al.  A computer vision system for on-screen item selection by finger pointing , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[19]  Zhiwei Zhu,et al.  Novel Eye Gaze Tracking Techniques Under Natural Head Movement , 2007, IEEE Transactions on Biomedical Engineering.

[20]  Ron Goldman,et al.  Elimination and resultants.2. Multivariate resultants , 1995, IEEE Computer Graphics and Applications.

[21]  Qiang Ji,et al.  Special issue: eye detection and tracking , 2005, Comput. Vis. Image Underst..

[22]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[23]  Sheng-Wen Shih,et al.  A novel approach to 3-D gaze tracking using stereo cameras , 2004, IEEE Trans. Syst. Man Cybern. Part B.

[24]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[25]  Moshe Eizenman,et al.  General theory of remote gaze estimation using the pupil center and corneal reflections , 2006, IEEE Transactions on Biomedical Engineering.

[26]  Olivier D. Faugeras,et al.  Motion of points and lines in the uncalibrated case , 2004, International Journal of Computer Vision.

[27]  Quang-Tuan Luong.  Matrice fondamentale et autocalibration en vision par ordinateur , 1992 .

[28]  R H S Carpenter,et al.  Supplementary Eye Field: Keeping an Eye on Eye Movement , 2004, Current Biology.