Integrating Visual Information Across Camera Movements with a Visual-Motor Calibration Map

Facing the competing demands for wider field of view and higher spatial resolution, computer vision will evolve toward greater use of foveal sensors and frequent camera movements. Integration of visual information across movements becomes a fundamental problem. We show that integration is possible using a biologically-inspired representation we call the visual-motor calibration map. The map is a memory-based model of the relationship between camera movements and corresponding pixel locations before and after any movement. The map constitutes a self-calibration that can compensate for non-uniform sampling, lens distortion, mechanical misalignments, and arbitrary pixel reordering. Integration takes place entirely in a retinotopic frame, using a short-term, predictive visual memory.

[1]  G. Stratton.  Vision without inversion of the retinal image. , 1897 .

[2]  M. Hayhoe,et al.  Integration of Form across Saccadic Eye Movements , 1991, Perception.

[3]  A. L. I︠A︡rbus.  Eye Movements and Vision , 1967 .

[4]  Bartlett W. Mel.  Connectionist Robot Motion Planning: A Neurally-Inspired Approach to Visually-Guided Reaching , 1990 .

[5]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[6]  Peter Louise Prokopowicz.  The development of perceptual integration in visual robots , 1994 .

[7]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[8]  D. Felleman,et al.  Probing the Primate Visual Cortex: Pathways and Perspectives , 1991 .

[9]  J. Feldman.  Four frames suffice: A provisional model of vision and space , 1985, Behavioral and Brain Sciences.

[10]  H E M.  Journal of Neurophysiology , 1938, Nature.

[11]  J. O'Regan,et al.  Integrating visual information from successive fixations:Does trans-saccadic fusion exist? , 1983, Vision Research.

[12]  Leonard Matin,et al.  Visual Localization and Eye Movements , 1982 .

[13]  Bruce Bridgeman,et al.  Failure to detect displacement of the visual world during saccadic eye movements , 1975, Vision Research.

[14]  Christopher G. Atkeson,et al.  Using Local Models to Control Movement , 1989, NIPS.

[15]  D L Sparks,et al.  Spatial localization of saccade targets. II. Activity of superior colliculus neurons preceding compensatory saccades. , 1983, Journal of neurophysiology.

[16]  O'Regan Jk,et al.  Integrating visual information from successive fixations: does trans-saccadic fusion exist? , 1983 .

[17]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[18]  J R Duhamel,et al.  The updating of the representation of visual space in parietal cortex by intended eye movements. , 1992, Science.

[19]  W. E. Collins,et al.  Integrating pictorial information across eye movements. , 1984, Journal of experimental psychology. General.