Testing a Biologically-Based System for Extracting Depth from Brief Monocular 2-D Video Sequences

Knowledge of the 3-D layout in front of a moving robot or vehicle is essential for obstacle avoidance and navigation. Currently the most common methods for acquiring that information rely on ‘active’ technologies which project light into the world (e.g., LIDAR). Some passive (non-emitting) systems use stereo cameras but only a relatively small number of techniques attempt to solve the 3-D layout problem using the information from a single video camera. A single camera offers many advantages such as lighter weight and fewer video streams to process. The visual motion occurring in brief monocular video sequences contains information regarding the movement of the camera and the structure of the scene. Extracting that information is difficult however because it relies on accurate estimates of the image motion velocities (optical flow) and knowledge of the camera motion, especially the heading direction. We have solved these two problems and can now obtain image flow and heading direction using mechanisms based on the properties of motion sensitive neurones in the brain. This allows us to recover depth information from monocular video sequences and here we report on a series of tests that assess the accuracy of this novel approach to 3-D depth recovery.

[1]  Richard J Krauzlis,et al.  Vector subtraction using visual and extraretinal motion signals: a new look at efference copy and corollary discharge theories. , 2008, Journal of vision.

[2]  Smith-Kettlewell,et al.  BIOLOGICAL IMAGE MOTION PROCESSING : A REVIEW , 2012 .

[3]  R. Shapley,et al.  “On the Visually Perceived Direction of Motion” by Hans Wallach: 60 Years Later , 1996 .

[4]  C. Busettini,et al.  Radial optic flow induces vergence eye movements with ultra-short latencies , 1997, Nature.

[5]  J. Perrone.  A neural-based code for computing image velocity from small sets of middle temporal (MT/V5) neuron inputs. , 2012, Journal of vision.

[6]  D Regan,et al.  How do we avoid confounding the direction we are looking and the direction we are moving? , 1982, Science.

[7]  K. Nakayama,et al.  Optical Velocity Patterns, Velocity-Sensitive Neurons, and Space Perception: A Hypothesis , 1974, Perception.

[8]  J A Perrone,et al.  Model for the computation of self-motion in biological systems. , 1992, Journal of the Optical Society of America. A, Optics and image science.

[9]  J H Rieger,et al.  Processing differential image motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[10]  F. Fraundorfer,et al.  Visual Odometry : Part II: Matching, Robustness, Optimization, and Applications , 2012, IEEE Robotics & Automation Magazine.

[11]  J. Perrone,et al.  A model of self-motion estimation within primate extrastriate visual cortex , 1994, Vision Research.

[12]  Claude L. Fennema,et al.  Velocity determination in scenes containing several moving objects , 1979 .

[13]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[14]  C. Koch,et al.  The analysis of visual motion: from computational theory to neuronal mechanisms. , 1986, Annual review of neuroscience.

[15]  R. Hetherington.  The Perception of the Visual World , 1952 .

[16]  John A. Perrone,et al.  A visual motion sensor based on the properties of V1 and MT neurons , 2004, Vision Research.

[17]  Michael J. Cree,et al.  Estimating heading direction from monocular video sequences using biologically-based sensors , 2016, 2016 International Conference on Image and Vision Computing New Zealand (IVCNZ).

[18]  Hiroshi Ono,et al.  Depth and Motion in Historical Descriptions of Motion Parallax , 2005, Perception.

[19]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[20]  John A Perrone,et al.  Visual-vestibular estimation of the body's curvilinear motion through the world: A computational model. , 2018, Journal of vision.

[21]  Andrea J. van Doorn,et al.  Invariant Properties of the Motion Parallax Field due to the Movement of Rigid Bodies Relative to an Observer , 1975 .