Emergence of complex cell properties by learning to generalize in natural scenes

A fundamental function of the visual system is to encode the building blocks of natural scenes—edges, textures and shapes—that subserve visual tasks such as object recognition and scene understanding. Essential to this process is the formation of abstract representations that generalize from specific instances of visual input. A common view holds that neurons in the early visual system signal conjunctions of image features, but how these produce invariant representations is poorly understood. Here we propose that to generalize over similar images, higher-level visual neurons encode statistical variations that characterize local image regions. We present a model in which neural activity encodes the probability distribution most consistent with a given image. Trained on natural images, the model generalizes by learning a compact set of dictionary elements for image distributions typically encountered in natural scenes. Model neurons show a diverse range of properties observed in cortical cells. These results provide a new functional explanation for nonlinear effects in complex cells and offer insight into coding strategies in primary visual cortex (V1) and higher visual areas.

[1]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[2]  J. Movshon,et al.  Spatial summation in the receptive fields of simple cells in the cat's striate cortex. , 1978, The Journal of physiology.

[3]  J. Movshon,et al.  Receptive field organization of complex cells in the cat's striate cortex. , 1978, The Journal of physiology.

[4]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[5]  J. P. Jones,et al.  The two-dimensional spatial structure of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[6]  William Bialek,et al.  Real-time performance of a movement-sensitive neuron in the blowfly visual system: coding and information transfer in short spike sequences , 1988, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[7]  A. B. Bonds.  Role of Inhibition in the Specification of Orientation Selectivity of Cells in the Cat Striate Cortex , 1989, Visual Neuroscience.

[8]  D. Heeger.  Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[9]  C. Baker,et al.  Envelope-responsive neurons in areas 17 and 18 of cat. , 1994, Journal of neurophysiology.

[10]  Keiji Tanaka,et al.  Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. , 1994, Journal of neurophysiology.

[11]  Victor A. F. Lamme.  The neurophysiology of figure-ground segregation in primary visual cortex , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[12]  D. C. Essen,et al.  Neural responses to polar, hyperbolic, and Cartesian gratings in area V4 of the macaque monkey. , 1996, Journal of neurophysiology.

[13]  Eero P. Simoncelli,et al.  Computational models of cortical visual processing. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[14]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[15]  J. Movshon,et al.  Linearity and Normalization in Simple Cells of the Macaque Primary Visual Cortex , 1997, The Journal of Neuroscience.

[16]  V. Hateren,et al.  Processing of natural time series of intensities by the visual system of the blowfly , 1997, Vision Research.

[17]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[18]  J. H. Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998 .

[19]  D. Mumford,et al.  The role of the primary visual cortex in higher level vision , 1998, Vision Research.

[20]  C. Baker,et al.  Temporal and spatial response to second-order stimuli in cat area 18. , 1998, Journal of neurophysiology.

[21]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[22]  D. V. van Essen,et al.  Response profiles to texture border patterns in area V1 , 2000, Visual Neuroscience.

[23]  J. Hegdé,et al.  Selectivity for Complex Shapes in Primate Visual Area V2 , 2000, The Journal of Neuroscience.

[24]  C. Connor,et al.  Shape representation in area V4: position-specific tuning for boundary conformation. , 2001, Journal of neurophysiology.

[25]  Leslie G. Ungerleider,et al.  Contextual Modulation in Primary Visual Cortex of Macaques , 2001, The Journal of Neuroscience.

[26]  E J Chichilnisky,et al.  A simple white noise analysis of neuronal light responses , 2001, Network.

[27]  Eero P. Simoncelli,et al.  Natural signal statistics and sensory gain control , 2001, Nature Neuroscience.

[28]  Aapo Hyvärinen,et al.  A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images , 2001, Vision Research.

[29]  H. Jones,et al.  Spatial organization and magnitude of orientation contrast interactions in primate V1. , 2002, Journal of neurophysiology.

[30]  J. Movshon,et al.  Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. , 2002, Journal of neurophysiology.

[31]  Aapo Hyvärinen,et al.  Simple-Cell-Like Receptive Fields Maximize Temporal Coherence in Natural Video , 2003, Neural Computation.

[32]  M. Lewicki,et al.  Learning higher-order structures in natural images , 2003, Network.

[33]  Peter Dayan,et al.  Doubly Distributional Population Codes: Simultaneous Representation of Uncertainty and Multiplicity , 2003, Neural Computation.

[34]  Bruno A Olshausen,et al.  Sparse coding of sensory inputs , 2004, Current Opinion in Neurobiology.

[35]  Rajesh P. N. Rao.  Bayesian Computation in Recurrent Neural Circuits , 2004, Neural Computation.

[36]  Eero P. Simoncelli,et al.  To appear in: The New Cognitive Neurosciences, 3rd edition Editor: M. Gazzaniga. MIT Press, 2004. Characterization of Neural Responses with Stochastic Stimuli , 2022 .

[37]  Laurenz Wiskott,et al.  Slow feature analysis yields a rich repertoire of complex cell properties. , 2005, Journal of vision.

[38]  Michael S. Lewicki,et al.  A Hierarchical Bayesian Model for Learning Nonlinear Statistical Regularities in Nonstationary Natural Signals , 2005, Neural Computation.

[39]  Eero P. Simoncelli,et al.  Spatiotemporal Elements of Macaque V1 Receptive Fields , 2005, Neuron.

[40]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[41]  J. Gallant,et al.  Spectral receptive field properties explain shape selectivity in area V4. , 2006, Journal of neurophysiology.

[42]  Geoffrey E. Hinton,et al.  Topographic Product Models Applied to Natural Scene Statistics , 2006, Neural Computation.

[43]  Terrence J. Sejnowski,et al.  Soft Mixer Assignment in a Hierarchical Generative Model of Natural Scene Statistics , 2006, Neural Computation.

[44]  Geoffrey E. Hinton.  Learning multiple layers of representation , 2007, Trends in Cognitive Sciences.

[45]  T. Poggio,et al.  A model of V4 shape selectivity and invariance. , 2007, Journal of neurophysiology.

[46]  Anitha Pasupathy,et al.  Transformation of shape information in the ventral pathway , 2007, Current Opinion in Neurobiology.

[47]  C. Baker,et al.  Neuronal response to texture- and contrast-defined boundaries in early visual cortex , 2007, Visual Neuroscience.

[48]  Feng Qi Han,et al.  Excitatory and suppressive receptive field subunits in awake monkey primary visual cortex (V1) , 2007, Proceedings of the National Academy of Sciences.