Dynamics of Feature Categorization

In visual and auditory scenes, we are able to identify shared features among sensory objects and group them according to their similarity. This grouping is preattentive and fast and is thought of as an elementary form of categorization by which objects sharing similar features are clustered in some abstract perceptual space. It is unclear what neuronal mechanisms underlie this fast categorization. Here we propose a neuromechanistic model of fast feature categorization based on the framework of continuous attractor networks. The mechanism for category formation does not rely on learning and is based on biologically plausible assumptions, for example, the existence of populations of neurons tuned to feature values, feature-specific interactions, and subthreshold-evoked responses upon the presentation of single objects. When the network is presented with a sequence of stimuli characterized by some feature, the network sums the evoked responses and provides a running estimate of the distribution of features in the input stream. If the distribution of features is structured into different components or peaks (i.e., is multimodal), recurrent excitation amplifies the response of activated neurons, and categories are singled out as emerging localized patterns of elevated neuronal activity (bumps), centered at the centroid of each cluster. The emergence of bump states through sequential, subthreshold activation and the dependence on input statistics is a novel application of attractor networks. We show that the extraction and representation of multiple categories are facilitated by the rich attractor structure of the network, which can sustain multiple stable activity patterns for a robust range of connectivity parameters compatible with cortical physiology.

[1]  A. Zador,et al.  Balanced inhibition underlies tuning and sharpens spike timing in auditory cortex , 2003, Nature.

[2]  Xiao-Jing Wang,et al.  A Model of Visuospatial Working Memory in Prefrontal Cortex: Recurrent Network and Cellular Bistability , 1998, Journal of Computational Neuroscience.

[3]  D. Contreras,et al.  Balanced Excitation and Inhibition Determine Spike Timing during Frequency Adaptation , 2006, The Journal of Neuroscience.

[4]  Alessandro Treves,et al.  Attractor neural networks storing multiple space representations: A model for hippocampal place fields , 1998, cond-mat/9807101.

[5]  J. Beck Effect of orientation and of shape similarity on perceptual grouping , 1966 .

[6]  A. Koulakov,et al.  Model for a robust neural integrator , 2002, Nature Neuroscience.

[7]  A. Izenman Review Papers: Recent Developments in Nonparametric Density Estimation , 1991 .

[8]  Xiao-Jing Wang,et al.  A Common Cortical Circuit Mechanism for Perceptual Categorical Discrimination and Veridical Judgment , 2008, PLoS Comput. Biol..

[9]  P. Zweifel Advanced Mathematical Methods for Scientists and Engineers , 1980 .

[10]  B. Ermentrout Neural networks as spatio-temporal pattern-forming systems , 1998 .

[11]  D. Ariely Seeing Sets: Representation by Statistical Properties , 2001, Psychological science.

[12]  P. Roelfsema,et al.  Automatic spread of attentional response modulation along Gestalt criteria in primary visual cortex , 2011, Nature Neuroscience.

[13]  Carson C. Chow,et al.  Stationary Bumps in Networks of Spiking Neurons , 2001, Neural Computation.

[14]  A. Izenman Recent Developments in Nonparametric Density Estimation , 1991 .

[15]  Bin Li,et al.  A New Clustering Approach on the Basis of Dynamical Neural Field , 2011, Neural Computation.

[16]  Bruce L. McNaughton,et al.  A Model of the Neural Basis of the Rat's Sense of Direction , 1994, NIPS.

[17]  Misha Tsodyks,et al.  Short-Term Facilitation may Stabilize Parametric Working Memory Trace , 2011, Front. Comput. Neurosci..

[18]  Max Wertheimer,et al.  Untersuchungen zur Lehre von der Gestalt , .

[19]  Yoram Burakyy,et al.  Accurate Path Integration in Continuous Attractor Network Models of Grid Cells , 2009 .

[20]  Leon van Noorden,et al.  Minimum differences of level and frequency for perceptual fission of tone sequences ABAB , 1977 .

[21]  Nicolas Brunel,et al.  A Continuous Attractor Network Model Without Recurrent Excitation: Maintenance and Integration in the Head Direction Cell System , 2005, Journal of Computational Neuroscience.

[22]  K. Zhang,et al.  Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[23]  G. Lord,et al.  Waves and bumps in neuronal networks with axo-dendritic synaptic interactions , 2003 .

[24]  P. Dayan,et al.  Supporting Online Material Materials and Methods Som Text Figs. S1 to S9 References the Asynchronous State in Cortical Circuits , 2022 .

[25]  M. Wertheimer,et al.  A source book of Gestalt psychology. , 1939 .

[26]  M. Abramowitz,et al.  Handbook of Mathematical Functions With Formulas, Graphs and Mathematical Tables (National Bureau of Standards Applied Mathematics Series No. 55) , 1965 .

[27]  Haim Sompolinsky,et al.  Chaotic Balanced State in a Model of Cortical Circuits , 1998, Neural Computation.

[28]  David L. Sheinberg,et al.  Visual object recognition. , 1996, Annual review of neuroscience.

[29]  Steven H. Strogatz,et al.  Nonlinear Dynamics and Chaos , 2024 .

[30]  B L McNaughton,et al.  Path Integration and Cognitive Mapping in a Continuous Attractor Neural Network Model , 1997, The Journal of Neuroscience.

[31]  Peter V. E. McClintock,et al.  Pattern formation : an introduction to methods. , 2006 .

[32]  B. Mandelbrot,et al.  RANDOM WALK MODELS FOR THE SPIKE ACTIVITY OF A SINGLE NEURON. , 1964, Biophysical journal.

[33]  M. Golubitsky,et al.  Geometric visual hallucinations, Euclidean symmetry and the functional architecture of striate cortex. , 2001, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[34]  H. Sompolinsky,et al.  Theory of orientation tuning in visual cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Robert L. Goldstone The role of similarity in categorization: providing a groundwork , 1994, Cognition.

[36]  G. A. Miller,et al.  An experimental study of auditory patterns. , 1951, The American journal of psychology.

[37]  David J. Freedman,et al.  Categorical representation of visual stimuli in the primate prefrontal cortex. , 2001, Science.

[38]  H S Seung,et al.  How the brain keeps the eyes still. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[39]  N Brunel,et al.  Slow stochastic Hebbian learning of classes of stimuli in a recurrent neural network. , 1998, Network.

[40]  A David Redishyx,et al.  A coupled attractor model of the rodent head direction system , 1996 .

[41]  S. Nelson,et al.  An emergent model of orientation selectivity in cat visual cortical simple cells , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[42]  J. Kruschke,et al.  ALCOVE: an exemplar-based connectionist model of category learning. , 1992, Psychological review.

[43]  Xiao-Jing Wang,et al.  Angular Path Integration by Moving “Hill of Activity”: A Spiking Neuron Model without Recurrent Excitation of the Head-Direction System , 2005, The Journal of Neuroscience.

[44]  黄培基,et al.  REPRESENTATION OF STATISTICAL PROPERTIES OFWAVE GROUPS FOR SEA PROCESSES WITHDOUBLE-PEAKED SPECTRA , 1994 .

[45]  William K. Estes,et al.  Classification and cognition , 1994 .

[46]  K. Koffka Principles Of Gestalt Psychology , 1936 .

[47]  Nicole C. Rust,et al.  Selectivity and Tolerance (“Invariance”) Both Increase as Visual Information Propagates from Cortical Area V4 to IT , 2010, The Journal of Neuroscience.

[48]  Paul C Bressloff,et al.  Nonlocal Ginzburg-Landau equation for cortical pattern formation. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[49]  M. Wertheimer Untersuchungen zur Lehre von der Gestalt. II , 1923 .

[50]  F. Attneave,et al.  What Variables Produce Similarity Grouping , 1970 .

[51]  F. Ashby,et al.  Categorization as probability density estimation , 1995 .

[52]  D J Amit,et al.  Multiple-object working memory--a model for behavioral performance. , 2003, Cerebral cortex.

[53]  Daniel J. Amit,et al.  Modeling brain function: the world of attractor neural networks, 1st Edition , 1989 .

[54]  Boris S. Gutkin,et al.  Turning On and Off with Excitation: The Role of Spike-Timing Asynchrony and Synchrony in Sustained Neural Activity , 2001, Journal of Computational Neuroscience.

[55]  S. Wang,et al.  Graded bidirectional synaptic plasticity is composed of switch-like unitary events. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[56]  Gaute T. Einevoll,et al.  Generation and annihilation of localized persistent-activity states in a two-population neural-field model , 2013, Neural Networks.

[57]  H. Sompolinsky,et al.  13 Modeling Feature Selectivity in Local Cortical Circuits , 2022 .

[58]  Xiao-Jing Wang,et al.  Robust Spatial Working Memory through Homeostatic Synaptic Scaling in Heterogeneous Cortical Networks , 2003, Neuron.

[59]  Matteo Carandini,et al.  Coding of stimulus sequences by population responses in visual cortex , 2009, Nature Neuroscience.

[60]  Terrence J. Sejnowski,et al.  ASSOCIATIVE MEMORY AND HIPPOCAMPAL PLACE CELLS , 1995 .

[61]  D. McCormick,et al.  Neocortical Network Activity In Vivo Is Generated through a Dynamic Balance of Excitation and Inhibition , 2006, The Journal of Neuroscience.

[62]  John H. R. Maunsell,et al.  Visual processing in monkey extrastriate cortex. , 1987, Annual review of neuroscience.

[63]  Dario L. Ringach,et al.  Dynamics of orientation tuning in macaque primary visual cortex , 1997, Nature.

[64]  P. Tass Cortical pattern formation during visual hallucinations , 1995 .

[65]  Michael Okun,et al.  Instantaneous correlation of excitation and inhibition during ongoing and sensory-evoked activities , 2008, Nature Neuroscience.

[66]  R. Desimone,et al.  Stimulus-selective properties of inferior temporal neurons in the macaque , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[67]  C. Laing,et al.  Two-bump solutions of Amari-type models of neuronal pattern formation , 2003 .

[68]  P. Goldman-Rakic,et al.  Synaptic mechanisms and network dynamics underlying spatial working memory in a cortical network model. , 2000, Cerebral cortex.

[69]  S Fusi,et al.  Forming classes by stimulus frequency: Behavior and theory , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[70]  A. Bregman,et al.  Primary auditory stream segregation and perception of order in rapid sequences of tones. , 1971, Journal of experimental psychology.

[71]  Y. Dan,et al.  Contribution of individual spikes in burst-induced long-term synaptic modification. , 2006, Journal of neurophysiology.

[72]  J. Cowan,et al.  Large Scale Spatially Organized Activity in Neural Nets , 1980 .

[73]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[74]  A. Grinvald,et al.  Relationship between intrinsic connections and functional architecture revealed by optical imaging and in vivo targeted biocytin injections in primate striate cortex. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[75]  R. Nosofsky Attention, similarity, and the identification-categorization relationship. , 1986, Journal of experimental psychology. General.

[76]  Boris S. Gutkin,et al.  Multiple Bumps in a Neuronal Model of Working Memory , 2002, SIAM J. Appl. Math..

[77]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[78]  D. Heeger,et al.  A Hierarchy of Temporal Receptive Windows in Human Cortex , 2008, The Journal of Neuroscience.

[79]  Daniel D. Lee,et al.  Stability of the Memory of Eye Position in a Recurrent Network of Conductance-Based Model Neurons , 2000, Neuron.

[80]  Yoram Burak,et al.  Accurate Path Integration in Continuous Attractor Network Models of Grid Cells , 2008, PLoS Comput. Biol..

[81]  Henry Markram,et al.  Neural Networks with Dynamic Synapses , 1998, Neural Computation.

[82]  Yixin Guo,et al.  Existence and stability of standing pulses in neural networks , 2003 .

[83]  W. Newsome,et al.  The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding , 1998, The Journal of Neuroscience.

[84]  William R. Softky,et al.  The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[85]  Alex Roxin,et al.  How effective delays shape oscillatory dynamics in neuronal networks , 2009, 0905.0701.

[86]  D. McCormick,et al.  Turning on and off recurrent balanced cortical activity , 2003, Nature.

[87]  Jesper Tegnér,et al.  Mechanism for top-down control of working memory capacity , 2009, Proceedings of the National Academy of Sciences.

[88]  D. Amit,et al.  Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. , 1997, Cerebral cortex.

[89]  John Rinzel,et al.  Background-activity-dependent properties of a network model for working memory that incorporates cellular bistability , 2005, Biological Cybernetics.

[90]  K. Holyoak,et al.  Induction of category distributions: a framework for classification learning. , 1984, Journal of experimental psychology. Learning, memory, and cognition.

[91]  S. Hochstein,et al.  View from the Top Hierarchies and Reverse Hierarchies in the Visual System , 2002, Neuron.

[92]  T. Wiesel,et al.  Columnar specificity of intrinsic horizontal and corticocortical connections in cat visual cortex , 1989, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[93]  S. Amari Dynamics of pattern formation in lateral-inhibition type neural fields , 1977, Biological Cybernetics.

[94]  Hong-hua Xu,et al.  Nonlocal Ginzburg-Landau Equations* , 1990 .

[95]  T. Poggio,et al.  Object Selectivity of Local Field Potentials and Spikes in the Macaque Inferior Temporal Cortex , 2006, Neuron.

[96]  Terrence J. Sejnowski,et al.  Unsupervised Learning , 2018, Encyclopedia of GIS.

[97]  M. Tsodyks,et al.  Synaptic Theory of Working Memory , 2008, Science.

[98]  S. Amari,et al.  Existence and stability of local excitations in homogeneous neural fields , 1979, Journal of mathematical biology.

[99]  Xiao-Jing Wang,et al.  Similarity Effect and Optimal Control of Multiple-Choice Decision Making , 2008, Neuron.

[100]  M. Abramowitz,et al.  Handbook of Mathematical Functions With Formulas, Graphs and Mathematical Tables (National Bureau of Standards Applied Mathematics Series No. 55) , 1965 .

[101]  R. Nosofsky Attention, similarity, and the identification-categorization relationship. , 1986 .

[102]  Gianluigi Mongillo,et al.  Selective delay activity in the cortex: phenomena and interpretation. , 2003, Cerebral cortex.

[103]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[104]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[105]  Andreas Daffertshofer,et al.  Non-identical smoothing operators for estimating time-frequency interdependence in electrophysiological recordings , 2013, EURASIP J. Adv. Signal Process..

[106]  M. Hirsch,et al.  Differential Equations, Dynamical Systems, and Linear Algebra , 1974 .