A Canonical Neural Circuit for Cortical Nonlinear Operations

A few distinct cortical operations have been postulated over the past few years, suggested by experimental data on nonlinear neural response across different areas in the cortex. Among these, the energy model proposes the summation of quadrature pairs following a squaring nonlinearity in order to explain phase invariance of complex V1 cells. The divisive normalization model assumes a gain-controlling, divisive inhibition to explain sigmoid-like response profiles within a pool of neurons. A gaussian-like operation hypothesizes a bell-shaped response tuned to a specific, optimal pattern of activation of the presynaptic inputs. A max-like operation assumes the selection and transmission of the most active response among a set of neural inputs. We propose that these distinct neural operations can be computed by the same canonical circuitry, involving divisive normalization and polynomial nonlinearities, for different parameter values within the circuit. Hence, this canonical circuit may provide a unifying framework for several circuit models, such as the divisive normalization and the energy models. As a case in point, we consider a feedforward hierarchical model of the ventral pathway of the primate visual cortex, which is built on a combination of the gaussian-like and max-like operations. We show that when the two operations are approximated by the circuit proposed here, the model is capable of generating selective and invariant neural responses and performing object recognition, in good agreement with neurophysiological data.

[1]  K. Miller,et al.  Neural noise can explain expansive, power-law nonlinearities in neural response functions. , 2002, Journal of neurophysiology.

[2]  N. Logothetis,et al.  Shape representation in the inferior temporal cortex of monkeys , 1995, Current Biology.

[3]  C. Connor,et al.  Shape representation in area V4: position-specific tuning for boundary conformation. , 2001, Journal of neurophysiology.

[4]  Minjoon Kouh,et al.  Toward a more biologically plausible model of object recognition , 2007 .

[5]  F. Girosi,et al.  A Connection Between GRBF and MLP , 1992 .

[6]  Gabriel Kreiman,et al.  Neural coding: computational and biophysical perspectives , 2004, Physics of Life Reviews.

[7]  Eero P. Simoncelli,et al.  A model of neuronal responses in visual area MT , 1998, Vision Research.

[8]  J. Movshon,et al.  Linearity and Normalization in Simple Cells of the Macaque Primary Visual Cortex , 1997, The Journal of Neuroscience.

[9]  Martin A. Giese,et al.  Biophysiologically Plausible Implementations of the Maximum Operation , 2002, Neural Computation.

[10]  G. Sáry,et al.  A review on the inferior temporal cortex of the macaque , 2010, Brain Research Reviews.

[11]  Randy M Bruno,et al.  Feedforward Mechanisms of Excitatory and Inhibitory Cortical Receptive Fields , 2002, The Journal of Neuroscience.

[12]  D. Heeger Modeling simple-cell direction selectivity with normalized, half-squared, linear operators. , 1993, Journal of neurophysiology.

[13]  Eero P. Simoncelli,et al.  Spatiotemporal Elements of Macaque V1 Receptive Fields , 2005, Neuron.

[14]  Eero P. Simoncelli,et al.  Spike-triggered neural characterization. , 2006, Journal of vision.

[15]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[16]  Timothée Masquelier,et al.  Competitive STDP-Based Spike Pattern Learning , 2009, Neural Computation.

[17]  D. Fitzpatrick The functional organization of local circuits in visual cortex: insights from the study of tree shrew striate cortex. , 1996, Cerebral cortex.

[18]  D. Ferster,et al.  The contribution of noise to contrast invariance of orientation tuning in cat visual cortex. , 2000, Science.

[19]  S. Grossberg Contour Enhancement , Short Term Memory , and Constancies in Reverberating Neural Networks , 1973 .

[20]  T. Sejnowski,et al.  A selection model for motion processing in area MT of primates , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[21]  Gilles Laurent,et al.  Transformation of Olfactory Representations in the Drosophila Antennal Lobe , 2004, Science.

[22]  Kevan A. C. Martin,et al.  A Canonical Microcircuit for Neocortex , 1989, Neural Computation.

[23]  M. Sur,et al.  Invariant computations in local cortical networks with balanced excitation and inhibition , 2005, Nature Neuroscience.

[24]  Tomaso A. Poggio,et al.  Extensions of a Theory of Networks for Approximation and Learning , 1990, NIPS.

[25]  W. K. Taylor Cortico-thalamic organization and memory , 1964, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[26]  Alex R. Wade,et al.  Representation of Concurrent Stimuli by Population Activity in Visual Cortex , 2009, Neuron.

[27]  Minami Ito,et al.  Size and position invariance of neuronal responses in monkey inferotemporal cortex. , 1995, Journal of neurophysiology.

[28]  Carson C. Chow,et al.  Competitive dynamics in cortical responses to visual stimuli. , 2005, Journal of neurophysiology.

[29]  Tomaso Poggio,et al.  Generalization in vision and motor control , 2004, Nature.

[30]  C. Koch,et al.  Attention activates winner-take-all competition among visual filters , 1999, Nature Neuroscience.

[31]  Christof Koch,et al.  Shunting Inhibition Does Not Have a Divisive Effect on Firing Rates , 1997, Neural Computation.

[32]  Y. Frégnac,et al.  Visual input evokes transient and strong shunting inhibition in visual cortical neurons , 1998, Nature.

[33]  Bartlett W. Mel SEEMORE: Combining Color, Shape, and Texture Histogramming in a Neurally Inspired Approach to Visual Object Recognition , 1997, Neural Computation.

[34]  Werner Reichardt,et al.  Figure-ground discrimination by relative movement in the visual system of the fly , 2004, Biological Cybernetics.

[35]  David I. Perrett,et al.  Neurophysiology of shape processing , 1993, Image Vis. Comput..

[36]  J. Movshon,et al.  Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. , 2002, Journal of neurophysiology.

[37]  Alexander Borst,et al.  Mechanisms of dendritic integration underlying gain control in fly motion-sensitive interneurons , 1995, Journal of Computational Neuroscience.

[38]  S. Nelson ReportCortical Microcircuits : Diverse or Canonical ? neuron and another ? , 2002 .

[39]  H. Seung,et al.  Learning in Spiking Neural Networks by Reinforcement of Stochastic Synaptic Transmission , 2003, Neuron.

[40]  Takayuki Ito,et al.  Neocognitron: A neural network model for a mechanism of visual pattern recognition , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[41]  T J Sejnowski,et al.  Learning viewpoint-invariant face representations from visual experience in an attractor network. , 1998, Network.

[42]  L. Abbott,et al.  Divisive inhibition in recurrent networks , 2000, Network.

[43]  Tomaso Poggio,et al.  Intracellular measurements of spatial integration and the MAX operation in complex cells of the cat primary visual cortex. , 2004, Journal of neurophysiology.

[44]  Frances S. Chance,et al.  Complex cells as cortically amplified simple cells , 1999, Nature Neuroscience.

[45]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[46]  J. Rauschecker,et al.  Processing of complex sounds in the macaque nonprimary auditory cortex. , 1995, Science.

[47]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[48]  P. Lennie,et al.  Coding of image contrast in central visual pathways of the macaque monkey , 1990, Vision Research.

[49]  Y. Dan,et al.  Spike Timing-Dependent Plasticity of Neural Circuits , 2004, Neuron.

[50]  D. B. Bender,et al.  Visual properties of neurons in inferotemporal cortex of the Macaque. , 1972, Journal of neurophysiology.

[51]  D. Heeger,et al.  The Normalization Model of Attention , 2009, Neuron.

[52]  T. Poggio A theory of how the brain might work. , 1990, Cold Spring Harbor symposia on quantitative biology.

[53]  D. Ferster,et al.  Neural mechanisms of orientation selectivity in the visual cortex. , 2000, Annual review of neuroscience.

[54]  D. C. Essen,et al.  Neural responses to polar, hyperbolic, and Cartesian gratings in area V4 of the macaque monkey. , 1996, Journal of neurophysiology.

[55]  T. Poggio,et al.  A network that learns to recognize three-dimensional objects , 1990, Nature.

[56]  Alan L. Yuille,et al.  A Winner-Take-All Mechanism Based on Presynaptic Inhibition Feedback , 1989, Neural Computation.

[57]  K. H. Britten,et al.  Spatial Summation in the Receptive Fields of MT Neurons , 1999, The Journal of Neuroscience.

[58]  Eero P. Simoncelli,et al.  How MT cells analyze the motion of visual patterns , 2006, Nature Neuroscience.

[59]  I. Fujita,et al.  Neuronal mechanisms of selectivity for object features revealed by blocking inhibition in inferotemporal cortex , 2000, Nature Neuroscience.

[60]  J. O’Keefe Place units in the hippocampus of the freely moving rat , 1976, Experimental Neurology.

[61]  Tomaso A. Poggio,et al.  Biophysical Models of Neural Computation: Max and Tuning Circuits , 2006, WImBI.

[62]  D. Ringach,et al.  On the classification of simple and complex cells , 2002, Vision Research.

[63]  M. Giese,et al.  Norm-based face encoding by single neurons in the monkey inferotemporal cortex , 2006, Nature.

[64]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[65]  S. Thorpe,et al.  Taking the MAX from neuronal responses , 2003, Trends in Cognitive Sciences.

[66]  Richard Hans Robert Hahnloser,et al.  Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit , 2000, Nature.

[67]  Kim T. Blackwell,et al.  Analog Pattern Matching in a Model of Dendritic Spines , 1998, ICONIP.

[68]  T. Gawne,et al.  Responses of primate visual cortical V4 neurons to simultaneously presented stimuli. , 2002, Journal of neurophysiology.

[69]  Y. Frégnac,et al.  The “silent” surround of V1 receptive fields: theory and experiments , 2003, Journal of Physiology-Paris.

[70]  R. Desimone,et al.  Competitive Mechanisms Subserve Attention in Macaque Areas V2 and V4 , 1999, The Journal of Neuroscience.

[71]  Nicholas J. Priebe,et al.  The contribution of spike threshold to the dichotomy of cortical simple and complex cells , 2004, Nature Neuroscience.

[72]  Peter Földiák,et al.  Learning Invariance from Transformation Sequences , 1991, Neural Comput..

[73]  Bartlett W. Mel,et al.  Translation-Invariant Orientation Tuning in Visual “Complex” Cells Could Derive from Intradendritic Computations , 1998, The Journal of Neuroscience.

[74]  Vernon B Mountcastle,et al.  Introduction. Computation in cortical columns. , 2003, Cerebral cortex.

[75]  M. Carandini,et al.  Summation and division by neurons in primate visual cortex. , 1994, Science.

[76]  R. Douglas,et al.  Neuronal circuits of the neocortex. , 2004, Annual review of neuroscience.

[77]  Eero P. Simoncelli,et al.  Natural signal statistics and sensory gain control , 2001, Nature Neuroscience.

[78]  Thomas Serre,et al.  A Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in Primate Visual Cortex , 2005 .

[79]  Frances S. Chance,et al.  Gain Modulation from Background Synaptic Input , 2002, Neuron.

[80]  E. Oja Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[81]  K. Miller,et al.  Synaptic Economics: Competition and Cooperation in Synaptic Plasticity , 1996, Neuron.

[82]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.