Bio-inspired motion estimation { From modelling to evaluation, can biology be a source of inspiration?

We propose a bio-inspired approach to motion estimation based on recent neuroscience ndings concerning the motion pathway. Our goal is to identify the key biological features in order to reach a good compromise be- tween bio-inspiration and computational eciency. Here we choose the neural eld formalism which provides a sound mathematical framework to describe the model at a macroscopic scale. Within this framework we dene the cortical ac- tivity as coupled integro-dierenti al equations and we prove the well-posedness of the model. We show how our model performs on some classical computer vision videos, and we compare its behaviour against the visual system on a simple classical video used in psychophysics. Following this idea, we propose a new benchmark to evaluate models against visual system performance. Baseline results are provided for both bio-inspired and computer vision models. Results conrm the good performance of recent computer vision approaches even on such synthetic stimuli, and also show that taking biology into account in mod- els can improve performance. As a whole, this article aords a considerable insight into how biology can bring new ideas in computer vision at dierent lev- els: modelling principles, mathematical formalism and evaluation methodology. Perspectives around this work are promising and cover the addition of delays to constrain propagation as well as the extension of our benchmark to better characterise the visual system performance.

[1]  E. Callaway,et al.  Multiple Circuits Relaying Primate Parallel Visual Pathways to the Middle Temporal Area , 2006, The Journal of Neuroscience.

[2]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[3]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[4]  L. Stone,et al.  From following edges to pursuing objects. , 2002, Journal of neurophysiology.

[5]  Olivier D. Faugeras,et al.  Local/Global Analysis of the Stationary Solutions of Some Neural Field Equations , 2009, SIAM J. Appl. Dyn. Syst..

[6]  Heiko Neumann,et al.  Disambiguating Visual Motion Through Contextual Feedback Modulation , 2004, Neural Computation.

[7]  D. Bradley,et al.  Structure and function of visual area MT. , 2005, Annual review of neuroscience.

[8]  W. Reichardt Autokorrelations-Auswertung als Funktionsprinzip des Zentralnervensystems , 1957 .

[9]  Pierre Kornprobst,et al.  Towards a bio-inspired evaluation methodology for motion estimation models , 2010 .

[10]  Georges-Henri Cottet,et al.  A Volterra type model for image processing , 1998, IEEE Trans. Image Process..

[11]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[12]  I. Vanzetta,et al.  Behavioral receptive field for ocular following in humans: dynamics of spatial summation and center-surround interactions. , 2006, Journal of neurophysiology.

[13]  U. Ilg,et al.  Initiation of smooth-pursuit eye movements by real and illusory contours , 2008, Vision Research.

[14]  Olivier D. Faugeras,et al.  Absolute Stability and Complete Synchronization in a Class of Neural Fields Models , 2008, SIAM J. Appl. Math..

[15]  P. Degond,et al.  The weighted particle method for convection-diffusion equations. II. The anisotropic case , 1989 .

[16]  Harry S. Orbach,et al.  Computing feature motion without feature detectors: A model for terminator motion without end-stopped cells 1 This research was first reported at the Annual Meeting of the Association for Research in Vision and Ophthalmology, May 1997. 1 , 1999, Vision Research.

[17]  Nicole C. Rust,et al.  Do We Know What the Early Visual System Does? , 2005, The Journal of Neuroscience.

[18]  B. Ermentrout Neural networks as spatio-temporal pattern-forming systems , 1998 .

[19]  Stephen Coombes,et al.  Waves, bumps, and patterns in neural field theories , 2005, Biological Cybernetics.

[20]  Hans Wallach Über visuell wahrgenommene Bewegungsrichtung , 1935 .

[21]  Gustavo Deco,et al.  The role of multi-area interactions for the computation of apparent motion , 2010, NeuroImage.

[22]  E. Callaway,et al.  Parallel processing strategies of the primate visual system , 2009, Nature Reviews Neuroscience.

[23]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[24]  K. Rockland,et al.  Feedback connections from area MT of the squirrel monkey to areas V1 and V2 , 2000, The Journal of comparative neurology.

[25]  Nava Rubin,et al.  The dynamics of bi-stable alternation in ambiguous motion displays: a fresh look at plaids , 2003, Vision Research.

[26]  S. Grossberg,et al.  Neural dynamics of form perception: boundary completion, illusory figures, and neon color spreading. , 1985, Psychological review.

[27]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[28]  R T Born,et al.  Visual processing: Parallel-er and Parallel-er , 2001, Current Biology.

[29]  Alan L. Yuille,et al.  A Winner-Take-All Mechanism Based on Presynaptic Inhibition Feedback , 1989, Neural Computation.

[30]  Nicholas J. Priebe,et al.  Tuning for Spatiotemporal Frequency and Speed in Directionally Selective Neurons of Macaque Striate Cortex , 2006, The Journal of Neuroscience.

[31]  S. Grossberg,et al.  Neural dynamics of motion processing and speed discrimination , 1998, Vision Research.

[32]  Eero P. Simoncelli,et al.  How MT cells analyze the motion of visual patterns , 2006, Nature Neuroscience.

[33]  Ellen C. Hildreth,et al.  Measurement of Visual Motion , 1984 .

[34]  K. Nakayama,et al.  The aperture problem—II. Spatial integration of velocity information along contours , 1988, Vision Research.

[35]  Christopher C. Pack,et al.  Integration of Contour and Terminator Signals in Visual Area MT of Alert Macaque , 2004, The Journal of Neuroscience.

[36]  H. Wilson,et al.  Perceived direction of moving two-dimensional patterns , 1990, Vision Research.

[37]  M. Shiffrar,et al.  Increased Motion Linking Across Edges with Decreased Luminance Contrast, Edge Width and Duration , 1996, Vision Research.

[38]  Joachim Weickert,et al.  Variational Optic Flow Computation with a Spatio-Temporal Smoothness Constraint , 2001, Journal of Mathematical Imaging and Vision.

[39]  R. Frostig,et al.  Cortical point-spread function and long-range lateral interactions revealed by real-time optical imaging of macaque monkey primary visual cortex , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[40]  Olivier Faugeras,et al.  Some theoretical and numerical results for delayed neural field equations , 2010 .

[41]  Hugh R. Wilson,et al.  Perceived direction of moving two-dimensional patterns depends on duration, contrast and eccentricity , 1992, Vision Research.

[42]  Terrence J. Sejnowski,et al.  Filter selection model for motion segmentation and velocity integration , 1994 .

[43]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[44]  J. Cowan,et al.  Excitatory and inhibitory interactions in localized populations of model neurons. , 1972, Biophysical journal.

[45]  Pierre Kornprobst,et al.  How do high-level specifications of the brain relate to variational approaches? , 2007, Journal of Physiology-Paris.

[46]  Guillaume S Masson,et al.  Object motion computation for the initiation of smooth pursuit eye movements in humans. , 2005, Journal of neurophysiology.

[47]  Michael J. Black,et al.  On the unification of line processes, outlier rejection, and robust statistics with applications in early vision , 1996, International Journal of Computer Vision.

[48]  Terrence J. Sejnowski,et al.  The Computational Brain , 1996, Artif. Intell..

[49]  Alfred M. Bruckstein,et al.  Over-Parameterized Variational Optical Flow , 2007, International Journal of Computer Vision.

[50]  Michael S. Landy,et al.  Theories for the Visual Perception of Local Velocity and Coherent Motion , 1991 .

[51]  Roderick Edwards,et al.  Approximation of Neural Network Dynamics by Reaction‐Diffusion Equations , 1996 .

[52]  Carlos R. Ponce,et al.  Temporal evolution of 2-dimensional direction signals used to guide eye movements. , 2005, Journal of neurophysiology.

[53]  J. M. Hupé,et al.  Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons , 1998, Nature.

[54]  A. Sillito,et al.  Always returning: feedback and sensory processing in visual cortex and thalamus , 2006, Trends in Neurosciences.

[55]  E. Castet,et al.  Temporal dynamics of motion integration for the initiation of tracking eye movements at ultra-short latencies , 2000, Visual Neuroscience.

[56]  E H Adelson,et al.  Adventures with Gelatinous Ellipses—Constraints on Models of Human Motion Analysis , 2000, Perception.

[57]  J. Bullier,et al.  Reaching beyond the classical receptive field of V1 neurons: horizontal or feedback axons? , 2003, Journal of Physiology-Paris.

[58]  Christopher C. Pack,et al.  Temporal dynamics of a neural solution to the aperture problem in visual area MT of macaque brain , 2001, Nature.

[59]  P. Kornprobst,et al.  Modelling the dynamics of motion integration with a new luminance-gated diffusion mechanism , 2010, Vision Research.

[60]  M. Giese Dynamic neural field theory for motion perception , 1998 .

[61]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Pierre Kornprobst,et al.  Can the Nonlocal Characterization of Sobolev Spaces by Bourgain et al. Be Useful for Solving Variational Problems? , 2009, SIAM J. Numer. Anal..

[63]  D. Bradley,et al.  Velocity computation in the primate visual system , 2008, Nature Reviews Neuroscience.

[64]  Eric Castet,et al.  The extrinsic/intrinsic classification of two-dimensional motion signals with barber-pole stimuli , 1999, Vision Research.

[65]  Pascal Mamassian,et al.  Bayesian modeling of dynamic motion integration , 2007, Journal of Physiology-Paris.

[66]  Joshua C. Brumberg,et al.  A quantitative population model of whisker barrels: Re-examining the Wilson-Cowan equations , 1996, Journal of Computational Neuroscience.

[67]  J. Movshon,et al.  Dynamics of motion signaling by neurons in macaque area MT , 2005, Nature Neuroscience.

[68]  E. Adelson,et al.  Slow and Smooth: A Bayesian theory for the combination of local motion signals in human vision , 1998 .

[69]  F. Kooi Local direction of edge motion causes and abolishes the barberpole illusion , 1993, Vision Research.

[70]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[71]  Michael J. Black,et al.  Secrets of optical flow estimation and their principles , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[72]  M. Shiffrar,et al.  Different motion sensitive units are involved in recovering the direction of moving lines , 1993, Vision Research.

[73]  Jean Lorenceau,et al.  A network view of the structure of center/surround modulations of V1 receptive field properties in visual and cortical spaces , 2001, Neurocomputing.

[74]  Eero P. Simoncelli,et al.  A model of neuronal responses in visual area MT , 1998, Vision Research.

[75]  H. Wilson,et al.  A psychophysically motivated model for two-dimensional motion perception , 1992, Visual Neuroscience.

[76]  M. Shiffrar,et al.  Perceived speed of moving lines depends on orientation, length, speed and luminance , 1993, Vision Research.

[77]  J. van Santen,et al.  Elaborated Reichardt detectors. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[78]  J. Cowan,et al.  A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue , 1973, Kybernetik.

[79]  Pierre Kornprobst,et al.  Action Recognition Using a Bio-Inspired Feedforward Spiking Network , 2009, International Journal of Computer Vision.

[80]  Jean-Michel Morel,et al.  Neighborhood filters and PDE’s , 2006, Numerische Mathematik.