Low-latency video tracking of horizontal, vertical, and torsional eye movements as a basis for 3dof realtime motion control of a head-mounted camera

A mobile head-mounted camera system was developed that is continuously aligned with the users direction of view. The camera looks where the eyes look. The eye movements are evaluated by an eye tracking system and used as signals to drive camera actuators for pan, tilt, and roll. The pupil detection is not sufficient to track the third degree of freedom, ocular torsion around the axis of gaze. Therefore, artificial markers are applied to the sclera of the eye to allow evaluation of all three components of eye orientation. A three-dimensional geometric model of the eyeball is required to detect these markers and calculate the angle of ocular torsion. A short calibration procedure, where the user looks at given targets, is performed to adjust the parameters to the user, and for each session. The data processing is done close-to-realtime on standard PC/laptop hardware within one to three milliseconds, making a minor contribution to the total reaction time of the system (about 36 ms). A camera guided in this way mimics the natural exploration of a visual scene, based on the sensorimotor output of a biological system for the control of eye movements, which has evolved over millions of years.

[1]  Andrew W. Fitzgibbon,et al.  Direct least squares fitting of ellipses , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[2]  Andrew W. Fitzgibbon,et al.  Direct Least Square Fitting of Ellipses , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Klaus Bartl,et al.  Eye movement driven head-mounted camera: it looks where the eyes look , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[4]  Klaus Bartl,et al.  A pivotable head mounted camera system that is aligned by three-dimensional eye movements , 2006, ETRA.

[5]  Rolf Eckmiller,et al.  Video-oculographic measurement of 3-dimensional eye rotations , 1990, Journal of Neuroscience Methods.

[6]  S T Moore,et al.  Robust pupil center detection using a curvature algorithm. , 1999, Computer methods and programs in biomedicine.

[7]  S. Glasauer,et al.  Comparison of human ocular torsion patterns during natural and galvanic vestibular stimulation. , 2002, Journal of neurophysiology.

[8]  T. Haslwanter.  Mathematics of three-dimensional eye rotations , 1995, Vision Research.

[9]  Juno Kim,et al.  A simple pupil-independent method for recording eye movements in rodents using video , 2004, Journal of Neuroscience Methods.

[10]  E. Edelman,et al.  OCULAR TORSION ON EARTH AND IN WEIGHTLESSNESS * , 1981, Annals of the New York Academy of Sciences.

[11]  J. Pelz,et al.  Oculomotor behavior and perceptual strategies in complex tasks , 2001, Vision Research.

[12]  T. Haslwanter,et al.  A theoretical analysis of three-dimensional eye position measurement using polar cross-correlation , 1995, IEEE Transactions on Biomedical Engineering.

[13]  Klaus Bartl,et al.  Gaze-aligned head-mounted camera with pan, tilt, and roll motion control for medical documentation and teaching applications , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[14]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[15]  Werner Haustein,et al.  Considerations on Listing's Law and the primary position by means of a matrix description of eye position control , 1989, Biological Cybernetics.

[16]  D Tweed,et al.  Implications of rotational kinematics for the oculomotor system in three dimensions. , 1987, Journal of neurophysiology.