Active estimation of distance in a robotic system that replicates human eye movement

In a moving agent, the different apparent motion of objects located at various distances provides an important source of depth information. While motion parallax is evident for large translations of the agent, a small parallax also occurs in most head/eye systems during rotations of the cameras. A similar parallax is also present in the human eye, so that a redirection of gaze shifts the projection of an object on the retina by an amount that depends not only on the amplitude of the rotation, but also on the distance of the object with respect to the observer. This study examines the accuracy of distance estimation on the basis of the parallax produced by camera rotations. Sequences of human eye movements were used to control the motion of a pan/tilt system specifically designed to reproduce the oculomotor parallax present in the human eye. We show that the oculomotor strategies by which humans scan visual scenes produce parallaxes that provide accurate estimation of distance. This information simplifies challenging visual tasks such as image segmentation and figure/ground segregation.

[1]  S. McKee,et al.  Spatial configurations for visual hyperacuity , 1977, Vision Research.

[2]  Douglas B Tweed,et al.  Influence of eye position on stereo matching , 2003, Strabismus.

[3]  J. M. Foley,et al.  Effects of voluntary eye movement and convergence on the binocular appreciation of depth , 1972 .

[4]  Amar Mitiche,et al.  Stereokinematic analysis of visual data in active convergent stereoscopy , 1998, Robotics Auton. Syst..

[5]  Mandyam V. Srinivasan,et al.  Active vision in honeybees: Task-oriented suppression of an innate behaviour , 1994, Vision Research.

[6]  H D Crane,et al.  Generation-V dual-Purkinje-image eyetracker. , 1985, Applied optics.

[7]  Olivier D. Faugeras,et al.  The geometry of multiple images - the laws that govern the formation of multiple images of a scene and some of their applications , 2001 .

[8]  Alex Pentland,et al.  A New Sense for Depth of Field , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  A. A. Skavenski,et al.  Miniature eye movement. , 1973, Science.

[10]  Dana H. Ballard,et al.  Animate Vision , 1991, Artif. Intell..

[11]  J. Douglas Crawford,et al.  The motor side of depth vision , 2001, Nature.

[12]  B. G. Cumming,et al.  An unexpected specialization for horizontal disparity in primate primary visual cortex , 2002, Nature.

[13]  Yiannis Aloimonos,et al.  Active vision , 2004, International Journal of Computer Vision.

[14]  Michele Rucci,et al.  Decorrelation of neural activity during fixational instability: Possible implications for the refinement of V1 receptive fields , 2004, Visual Neuroscience.

[15]  S Thorpe,et al.  Neural processing of stereopsis as a function of viewing distance in primate visual cortical area V1. , 1993, Journal of neurophysiology.

[16]  Jan-Olof Eklundh,et al.  Real-Time Epipolar Geometry Estimation of Binocular Stereo Heads , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Terry E. Weymouth,et al.  Robust monocular depth perception using feature pairs and approximate motion , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[18]  Mark Nawrot,et al.  Eye movements provide the extra-retinal signal required for the perception of depth from motion parallax , 2003, Vision Research.

[19]  Patrick Cavanagh,et al.  Stabilizing the visual world: A jitter aftereffect reveals motion-based correction , 1998 .

[20]  Thomas S. Huang,et al.  Motion and Structure from Image Sequences , 1992 .

[21]  D. Noton,et al.  Eye movements and visual perception. , 1971, Scientific American.

[22]  Ray A. Jarvis,et al.  A Perspective on Range Finding Techniques for Computer Vision , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Yeung Sam Hung,et al.  A Kalman Filter Approach to Direct Depth Estimation Incorporating Surface Structure , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Michael F. Land,et al.  Fast-focus telephoto eye , 1995, Nature.

[25]  Dennis M. Levi,et al.  Peripheral positional acuity: Retinal and cortical constraints on 2-dot separation discrimination under photopic and scotopic conditions , 1989, Vision Research.

[26]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Thomas S. Huang,et al.  Motion and structure from feature correspondences: a review , 1994, Proc. IEEE.

[28]  Giulio Sandini,et al.  Active Tracking Strategy for Monocular Depth Inference over Multiple Frames , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Patrick Cavanagh,et al.  A jitter after-effect reveals motion-based stabilization of vision , 1998, Nature.

[30]  Hermann von Helmholtz,et al.  Treatise on Physiological Optics , 1962 .

[31]  Subhasis Chaudhuri,et al.  Depth estimation and image restoration using defocused stereo pairs , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Marian Stamp Dawkins,et al.  Pattern recognition and active vision in chickens , 2000, Nature.

[33]  Antonio Torralba,et al.  Depth Estimation from Image Structure , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  A. Mitiche.  Computational Analysis of Visual Motion , 1994, Advances in Computer Vision and Machine Intelligence.

[35]  Ian P. Howard,et al.  Seeing in Depth , 2008 .

[36]  Alex Pentland,et al.  Fractal-Based Description of Natural Scenes , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Hiroshi Ono,et al.  The rhino-optical phenomenon: Ocular parallax and the visible field beyond the nose , 1986, Vision Research.

[38]  J. Gibson,et al.  Motion parallax as a determinant of perceived depth. , 1959, Journal of experimental psychology.

[39]  Gerald M. Edelman,et al.  Robust localization of auditory and visual targets in a robotic barn owl , 2000, Robotics Auton. Syst..

[40]  R. D. Watkins,et al.  THE OPTICAL SYSTEM OF THE EYE , 1970 .

[41]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[42]  L. Stark,et al.  Most naturally occurring human saccades have magnitudes of 15 degrees or less. , 1975, Investigative ophthalmology.

[43]  Alan C. Bovik,et al.  Shape from Texture Using Local Spectral Moments , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  David R. Williams,et al.  Seeing through the photoreceptor mosaic , 1986, Trends in Neurosciences.

[45]  J. Findlay.  Active vision: Visual activity in everyday life , 1998, Current Biology.

[46]  M. Rucci,et al.  Contributions of fixational eye movements to the discrimination of briefly presented stimuli. , 2003, Journal of vision.

[47]  Mohan M. Trivedi,et al.  High-Speed Extraction of 3D Structure of Selectable Quality Using a Translating Camera , 1996, Comput. Vis. Image Underst..

[48]  John Oliensis,et al.  A Critique of Structure-from-Motion Algorithms , 2000, Comput. Vis. Image Underst..

[49]  Dietrich Paulus,et al.  Segmentation of lines and arcs and its application for depth recovery , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[50]  M. Fahle,et al.  Psychophysical measurement of eye drifts and tremor by dichoptic or monocular vernier acuity , 1991, Vision Research.

[51]  Bernard Heit,et al.  Visual depth perception based on optical blur , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[52]  Roger A. Samy,et al.  Pyramidal robust estimation of the depth-from-motion , 1994, Other Conferences.

[53]  G Ishai,et al.  Visual stability and space perception in monocular vision: mathematical model. , 1980, Journal of the Optical Society of America.

[54]  G. Edelman,et al.  Modeling LGN Responses during Free-Viewing: A Possible Role of Microscopic Eye Movements in the Refinement of Cortical Orientation Selectivity , 2000, The Journal of Neuroscience.

[55]  Alan C. Bovik,et al.  FOVEA: a foveated vergent active stereo vision system for dynamic three-dimensional scene recovery , 1998, IEEE Trans. Robotics Autom..

[56]  Yiannis Aloimonos,et al.  Estimating the heading direction using normal flow , 1994, International Journal of Computer Vision.

[57]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[58]  Geoffrey P. Bingham,et al.  Optical flow from eye movement with head immobilized: “Ocular occlusion” beyond the nose , 1993, Vision Research.

[59]  Ze-Nian Li,et al.  A survey of motion-parallax-based 3-D reconstruction algorithms , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[60]  Sukhan Lee,et al.  Depth from magnification and blurring , 1997, Proceedings of International Conference on Robotics and Automation.

[61]  Gerald M. Edelman,et al.  Adaptation of orienting behavior: from the barn owl to a robotic system , 1999, IEEE Trans. Robotics Autom..

[62]  Nicholas Ayache,et al.  Artificial vision for mobile robots - stereo vision and multisensory perception , 1991 .

[63]  John D. Pettigrew,et al.  Convergence of specialised behaviour, eye movements and visual optics in the sandlance (Teleostei) and the chameleon (Reptilia) , 1999, Current Biology.