Learning visual motion in recurrent neural networks

We present a dynamic nonlinear generative model for visual motion based on a latent representation of binary-gated Gaussian variables. Trained on sequences of images, the model learns to represent different movement directions in different variables. We use an online approximate inference scheme that can be mapped to the dynamics of networks of neurons. Probed with drifting grating stimuli and moving bars of light, neurons in the model show patterns of responses analogous to those of direction-selective simple cells in primary visual cortex. Most model neurons also show speed tuning and respond equally well to a range of motion directions and speeds aligned to the constraint line of their respective preferred speed. We show how these computations are enabled by a specific pattern of recurrent connections learned by the model.

[1]  G. Orban,et al.  Velocity sensitivity and direction selectivity of neurons in areas V1 and V2 of the monkey: influence of eccentricity. , 1986, Journal of neurophysiology.

[2]  W. Newsome,et al.  Motion selectivity in macaque visual cortex. II. Spatiotemporal range of directional interactions in MT and V1. , 1986, Journal of neurophysiology.

[3]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[4]  C. Koch,et al.  Recurrent excitation in neocortical circuits , 1995, Science.

[5]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[6]  K. I. Blum,et al.  Functional significance of long-term potentiation for sequence learning and prediction. , 1996, Cerebral cortex.

[7]  D. Fitzpatrick,et al.  Orientation Selectivity and the Arrangement of Horizontal Connections in Tree Shrew Striate Cortex , 1997, The Journal of Neuroscience.

[8]  M. Livingstone,et al.  Mechanisms of Direction Selectivity in Macaque V1 , 1998, Neuron.

[9]  Eero P. Simoncelli,et al.  A model of neuronal responses in visual area MT , 1998, Vision Research.

[10]  K. Jarrod Millman,et al.  Learning Sparse Codes with a Mixture-of-Gaussians Prior , 1999, NIPS.

[11]  A. Borst.  Seeing smells: imaging olfactory learning in bees , 1999, Nature Neuroscience.

[12]  Rajesh P. N. Rao,et al.  Predictive Sequence Learning in Recurrent Neocortical Circuits , 1999, NIPS.

[13]  Nando de Freitas,et al.  Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks , 2000, UAI.

[14]  Tom Minka,et al.  Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[15]  David Barber,et al.  Learning in Spiking Neural Assemblies , 2002, NIPS.

[16]  Terrence J. Sejnowski,et al.  Slow Feature Analysis: Unsupervised Learning of Invariances , 2002, Neural Computation.

[17]  Bruno A. Olshausen,et al.  Learning sparse, overcomplete representations of time-varying natural images , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[18]  R. Freeman,et al.  The Derivation of Direction Selectivity in the Striate Cortex , 2004, The Journal of Neuroscience.

[19]  F. Werblin,et al.  Directional Selectivity Is Formed at Multiple Levels by Laterally Offset Inhibition in the Rabbit Retina , 2005, Neuron.

[20]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[21]  D. Fitzpatrick,et al.  The development of direction selectivity in ferret visual cortex requires early visual experience , 2006, Nature Neuroscience.

[22]  Martin Rehn,et al.  A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields , 2007, Journal of Computational Neuroscience.

[23]  Bruno A. Olshausen,et al.  Learning Transformational Invariants from Natural Movies , 2008, NIPS.

[24]  Richard E. Turner,et al.  A Structured Model of Video Reproduces Primary Visual Cortical Organisation , 2009, PLoS Comput. Biol..

[25]  Jean-Pascal Pfister,et al.  Sequence learning with hidden units in spiking neural networks , 2011, NIPS.

[26]  Yoshua Bengio,et al.  Spike-and-Slab Sparse Coding for Unsupervised Feature Discovery , 2012, ArXiv.

[27]  Y. Dan,et al.  Activity Recall in Visual Cortical Ensemble , 2012, Nature Neuroscience.