Recruitment and Consolidation of Cell Assemblies for Words by Way of Hebbian Learning and Competition in a Multi-Layer Neural Network

Current cognitive theories postulate either localist representations of knowledge or fully overlapping, distributed ones. We use a connectionist model that closely replicates known anatomical properties of the cerebral cortex and neurophysiological principles to show that Hebbian learning in a multi-layer neural network leads to memory traces (cell assemblies) that are both distributed and anatomically distinct. Taking the example of word learning based on action-perception correlation, we document mechanisms underlying the emergence of these assemblies, especially (i) the recruitment of neurons and consolidation of connections defining the kernel of the assembly along with (ii) the pruning of the cell assembly’s halo (consisting of very weakly connected cells). We found that, whereas a learning rule mapping covariance led to significant overlap and merging of assemblies, a neurobiologically grounded synaptic plasticity rule with fixed LTP/LTD thresholds produced minimal overlap and prevented merging, exhibiting competitive learning behaviour. Our results are discussed in light of current theories of language and memory. As simulations with neurobiologically realistic neural networks demonstrate here spontaneous emergence of lexical representations that are both cortically dispersed and anatomically distinct, both localist and distributed cognitive accounts receive partial support.

[1]  Volkmar Glauche,et al.  Ventral and dorsal pathways for language , 2008, Proceedings of the National Academy of Sciences.

[2]  Satrajit S. Ghosh,et al.  Neural modeling and imaging of the cortical interactions underlying syllable production , 2006, Brain and Language.

[3]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[4]  B. Bergum,et al.  Attention and Performance VI , 1978 .

[5]  Alan Peters,et al.  Association and Auditory Cortices , 1985, Cerebral Cortex.

[6]  Friedemann Pulvermüller,et al.  A cell assembly model of language , 1991 .

[7]  Willem J. M. Levelt,et al.  A theory of lexical access in speech production , 1999, Behavioral and Brain Sciences.

[8]  Friedemann Pulvermüller,et al.  Motor cortex maps articulatory features of speech sounds , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[9]  S. Nelson,et al.  Homeostatic plasticity in the developing nervous system , 2004, Nature Reviews Neuroscience.

[10]  Leon N. Cooper,et al.  BCM network develops orientation selectivity and ocular dominance in natural scene environment , 1997, Vision Research.

[11]  G. Rizzolatti,et al.  Speech listening specifically modulates the excitability of tongue muscles: a TMS study , 2002, The European journal of neuroscience.

[12]  Max Coltheart,et al.  Access to the internal lexicon , 1977 .

[13]  Derek K. Jones,et al.  Perisylvian language networks of the human brain , 2005, Annals of neurology.

[14]  M. Bear,et al.  Homosynaptic long-term depression in area CA1 of hippocampus and effects of N-methyl-D-aspartate receptor blockade. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[15]  D. Buonomano,et al.  Cortical plasticity: from synapses to maps. , 1998, Annual review of neuroscience.

[16]  Friedemann Pulvermüller,et al.  A neuroanatomically grounded Hebbian-learning model of attention–language interactions in the human brain , 2008, The European journal of neuroscience.

[17]  Matthew A. Lambon Ralph,et al.  Lateralization of ventral and dorsal auditory-language pathways in the human brain , 2005, NeuroImage.

[18]  Gary S. Dell,et al.  Connectionist models of language production: lexical access and grammatical encoding , 1999, Cogn. Sci..

[19]  Lokendra Shastri,et al.  Biological Grounding of Recruitment Learning and Vicinal Algorithms in Long-Term Potentiation , 2001, Emergent Neural Computational Architectures Based on Neuroscience.

[20]  Eugene M. Izhikevich,et al.  Relating STDP to BCM , 2003, Neural Computation.

[21]  C. Shatz,et al.  Synaptic Activity and the Construction of Cortical Circuits , 1996, Science.

[22]  F. Crépel,et al.  Homo‐ and heterosynaptic changes in efficacy are expressed in prefrontal neurons: An in vitro study in the rat , 1992, Synapse.

[23]  Tadaharu Tsumoto,et al.  Long-term potentiation and long-term depression in the neocortex , 1992, Progress in Neurobiology.

[24]  F. Pulvermüller,et al.  Words in the brain's language , 1999, Behavioral and Brain Sciences.

[25]  Ad Aertsen,et al.  Editorial: Cell Assemblies , 2003 .

[26]  James L. McClelland,et al.  Distributed memory and the representation of general and specific information. , 1985, Journal of experimental psychology. General.

[27]  Teuvo Kohonen,et al.  The self-organizing feature maps , 1989 .

[28]  Lokendra Shastri,et al.  Semantic Networks: An Evidential Formalization and Its Connectionist Realization , 1988 .

[29]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[30]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[31]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[32]  F. T. Husain,et al.  Relating neuronal dynamics for auditory object processing to neuroimaging activity: a computational modeling and an fMRI study , 2004, NeuroImage.

[33]  M. Page,et al.  Connectionist modelling in psychology: A localist manifesto , 2000, Behavioral and Brain Sciences.

[34]  K. Miller,et al.  Synaptic Economics: Competition and Cooperation in Synaptic Plasticity , 1996, Neuron.

[35]  Friedemann Pulvermüller,et al.  A neuronal model of the language cortex , 2007, Neurocomputing.

[36]  James L. McClelland,et al.  Structure and deterioration of semantic memory: a neuropsychological and computational investigation. , 2004, Psychological review.

[37]  James L. McClelland,et al.  The TRACE model of speech perception , 1986, Cognitive Psychology.

[38]  Jerome A. Feldman,et al.  Dynamic connections in neural networks , 1990, Biological Cybernetics.

[39]  GrossbergS. Adaptive pattern classification and universal recoding , 1976 .

[40]  J. Donoghue,et al.  Learning-induced LTP in neocortex. , 2000, Science.

[41]  E. Yeterian,et al.  MRI-Based Topographic Parcellation of Human Cerebral White Matter and Nuclei II. Rationale and Applications with Systematics of Cerebral Connectivity , 1999, NeuroImage.

[42]  T. Paus,et al.  Modulation of Motor Excitability during Speech Perception: The Role of Broca's Area , 2004, Journal of Cognitive Neuroscience.

[43]  M. Bear,et al.  Mechanism for a sliding synaptic modification threshold , 1995, Neuron.

[44]  W. Press,et al.  Numerical Recipes in C++: The Art of Scientific Computing (2nd edn)1 Numerical Recipes Example Book (C++) (2nd edn)2 Numerical Recipes Multi-Language Code CD ROM with LINUX or UNIX Single-Screen License Revised Version3 , 2003 .

[45]  C. Shatz Impulse activity and the patterning of connections during cns development , 1990, Neuron.

[46]  S. Grossberg,et al.  Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors , 1976, Biological Cybernetics.

[47]  Kenneth C. Hill,et al.  The genesis of language , 1982 .

[48]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[49]  L. Garey Cortex: Statistics and Geometry of Neuronal Connectivity, 2nd edn. By V. BRAITENBERG and A. SCHÜZ. (Pp. xiii+249; 90 figures; ISBN 3 540 63816 4). Berlin: Springer. 1998. , 1999 .

[50]  Prof. Dr. Dr. Valentino Braitenberg,et al.  Cortex: Statistics and Geometry of Neuronal Connectivity , 1998, Springer Berlin Heidelberg.

[51]  Karl J. Friston,et al.  A neural mass model for MEG/EEG: coupling and neuronal dynamics , 2003, NeuroImage.

[52]  K. Harris Neural signatures of cell assembly organization , 2005, Nature Reviews Neuroscience.

[53]  M. Mishkin,et al.  Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex , 1999, Nature Neuroscience.

[54]  A. Artola,et al.  Synaptic Activity Modulates the Induction of Bidirectional Synaptic Changes in Adult Mouse Hippocampus , 2000, The Journal of Neuroscience.

[55]  T. Sejnowski,et al.  Associative long-term depression in the hippocampus induced by hebbian covariance , 1989, Nature.

[56]  P. Milner,et al.  Neural Representations: Some Old Problems Revisited , 1996, Journal of Cognitive Neuroscience.

[57]  E. Rolls,et al.  What and Where in Visual Working Memory: A Computational Neurodynamical Perspective for Integrating fMRI and Single-Neuron Data , 2004, Journal of Cognitive Neuroscience.

[58]  M. Bear,et al.  LTP and LTD An Embarrassment of Riches , 2004, Neuron.

[59]  J. Cowan,et al.  A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue , 1973, Kybernetik.

[60]  F. Pulvermüller,et al.  Language outside the focus of attention: The mismatch negativity as a tool for studying higher cognitive processes , 2006, Progress in Neurobiology.

[61]  M. Raijmakers Rethinking innateness: A connectionist perspective on development. , 1997 .

[62]  M. Bear,et al.  Metaplasticity: the plasticity of synaptic plasticity , 1996, Trends in Neurosciences.

[63]  Stephen Grossberg,et al.  Adaptive pattern classification and universal recoding: II. Feedback, expectation, olfaction, illusions , 1976, Biological Cybernetics.

[64]  T. Paus,et al.  Seeing and hearing speech excites the motor system involved in speech production , 2003, Neuropsychologia.

[65]  Alan C. Evans,et al.  PET studies of phonetic processing of speech: review, replication, and reanalysis. , 1996, Cerebral cortex.

[66]  James L. McClelland,et al.  Understanding normal and impaired word reading: computational principles in quasi-regular domains. , 1996, Psychological review.

[67]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[68]  W. Marslen-Wilson,et al.  The temporal structure of spoken language understanding , 1980, Cognition.

[69]  Roland Heim,et al.  Theoretical Approaches to Complex Systems , 1978 .

[70]  T. Sejnowski,et al.  Storing covariance with nonlinearly interacting neurons , 1977, Journal of mathematical biology.

[71]  G. Westermann,et al.  Broca's Area and Inflectional Morphology: Evidence from Broca's Aphasia and Computer Modeling , 2006, Cortex.

[72]  W. Singer,et al.  Different voltage-dependent thresholds for inducing long-term depression and long-term potentiation in slices of rat visual cortex , 1990, Nature.

[73]  G. Westermann,et al.  A new model of sensorimotor coupling in the development of speech , 2004, Brain and Language.

[74]  Mark S. Seidenberg,et al.  Impairments in verb morphology after brain injury: a connectionist model. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[75]  D. Norris Shortlist: a connectionist model of continuous speech recognition , 1994, Cognition.

[76]  Anil K. Seth,et al.  Distinguishing Causal Interactions in Neural Populations , 2007, Neural Computation.

[77]  J. Eggert,et al.  Unifying framework for neuronal assembly dynamics. , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[78]  W. Singer,et al.  Long-term depression of excitatory synaptic transmission and its relationship to long-term potentiation , 1993, Trends in Neurosciences.

[79]  T. Wiesel,et al.  Clustered intrinsic connections in cat visual cortex , 1983, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[80]  D. Pandya,et al.  Architecture and Connections of Cortical Association Areas , 1985 .

[81]  G S Dell,et al.  A spreading-activation theory of retrieval in sentence production. , 1986, Psychological review.

[82]  Moshe Abeles,et al.  Corticonics: Neural Circuits of Cerebral Cortex , 1991 .

[83]  B. Horwitz,et al.  Integrating electrophysiological and anatomical experimental data to create a large-scale model that simulates a delayed match-to-sample human brain imaging study. , 1998, Cerebral cortex.

[84]  C. Granger Investigating causal relations by econometric models and cross-spectral methods , 1969 .

[85]  Kenneth D. Miller,et al.  The Role of Constraints in Hebbian Learning , 1994, Neural Computation.

[86]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[87]  Paavo Alku,et al.  Memory Traces for Words as Revealed by the Mismatch Negativity , 2001, NeuroImage.

[88]  Jeffrey L. Krichmar,et al.  Spatial navigation and causal analysis in a brain-based device modeling cortical-hippocampal interactions , 2007, Neuroinformatics.

[89]  Gustavo Deco,et al.  Large-scale neural model for visual attention: integration of experimental single-cell and fMRI data. , 2002, Cerebral cortex.

[90]  Terrence J. Sejnowski,et al.  The Hebb Rule for Synaptic Plasticity: Algorithms and Implementations , 1989 .

[91]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[92]  D. Pandya,et al.  Comparative cytoarchitectonic analysis of the human and the macaque ventrolateral prefrontal cortex and corticocortical connection patterns in the monkey , 2002, The European journal of neuroscience.

[93]  R. Nicoll,et al.  Long-term potentiation--a decade of progress? , 1999, Science.

[94]  James L. McClelland,et al.  A distributed, developmental model of word recognition and naming. , 1989, Psychological review.

[95]  Joachim Diederich,et al.  Instruction and High-level Learning in Connectionist Networks , 1989 .

[96]  William D. Marslen-Wilson,et al.  A Connectionist Model of Phonological Representation in Speech Perception , 1995, Cogn. Sci..

[97]  M. Iacoboni,et al.  Listening to speech activates motor areas involved in speech production , 2004, Nature Neuroscience.

[98]  Terrence J. Sejnowski,et al.  The Variance of Covariance Rules for Associative Matrix Memories and Reinforcement Learning , 1993, Neural Computation.

[99]  Niraj S. Desai,et al.  Activity-dependent scaling of quantal amplitude in neocortical neurons , 1998, Nature.

[100]  M. Bear,et al.  Experience-dependent modification of synaptic plasticity in visual cortex , 1996, Nature.