Hebbian Crosstalk Prevents Nonlinear Unsupervised Learning

Learning is thought to occur by localized, activity-induced changes in the strength of synaptic connections between neurons. Recent work has shown that induction of change at one connection can affect changes at others (“crosstalk”). We studied the role of such crosstalk in nonlinear Hebbian learning using a neural network implementation of independent components analysis. We find that there is a sudden qualitative change in the performance of the network at a threshold crosstalk level, and discuss the implications of this for nonlinear learning from higher-order correlations in the neocortex.

[1]  Guo-Qiang Bi,et al.  Spatiotemporal specificity of synaptic plasticity: cellular rules and mechanisms , 2002, Biological Cybernetics.

[2]  J. Nadal,et al.  Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer Network 5 , 1994 .

[3]  J. Wickens.  Electrically coupled but chemically isolated synapses: Dendritic spines and calcium in a rule for synaptic modification , 1988, Progress in Neurobiology.

[4]  D. Ruderman,et al.  INDEPENDENT COMPONENT ANALYSIS OF NATURAL IMAGE SEQUENCES YIELDS SPATIOTEMPORAL FILTERS SIMILAR TO SIMPLE CELLS IN PRIMARY VISUAL CORTEX , 1998 .

[5]  G. Ellis‐Davies,et al.  Structural basis of long-term potentiation in single dendritic spines , 2004, Nature.

[6]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[7]  A. Polsky,et al.  Properties of basal dendrites of layer 5 pyramidal neurons: a direct patch-clamp recording study , 2007, Nature Neuroscience.

[8]  David Holcman,et al.  Dynamic regulation of spine–dendrite coupling in cultured hippocampal neurons , 2004, The European journal of neuroscience.

[9]  J. V. van Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[10]  Kingsley J. A. Cox,et al.  Synaptic Darwinism and neocortical function , 2002, Neurocomputing.

[11]  J. Connor,et al.  Dendritic spines as individual neuronal compartments for synaptic Ca2+ responses , 1991, Nature.

[12]  Aapo Hyvärinen,et al.  Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces , 2000, Neural Computation.

[13]  Florian Engert,et al.  Emergence of Input Specificity of LTP during Development of Retinotectal Connections In Vivo , 2001, Neuron.

[14]  Erkki Oja,et al.  Independent component analysis by general nonlinear Hebbian-like learning rules , 1998, Signal Process..

[15]  D. Madison,et al.  Locally distributed synaptic potentiation in the hippocampus. , 1994, Science.

[16]  Karel Svoboda,et al.  Locally dynamic synaptic learning rules in pyramidal neuron dendrites , 2007, Nature.

[17]  Shun-ichi Amari,et al.  Stability Analysis Of Adaptive Blind Source Separation , 1997 .

[18]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[19]  W. Levy,et al.  Synapses as associative memory elements in the hippocampal formation , 1979, Brain Research.

[20]  Nathan Intrator,et al.  Theory of Cortical Plasticity , 2004 .

[21]  P O Hoyer,et al.  Independent component analysis applied to feature extraction from colour and stereo images , 2000, Network.

[22]  J. Lisman,et al.  The molecular basis of CaMKII function in synaptic and behavioural memory , 2002, Nature Reviews Neuroscience.

[23]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[24]  勇一 作村,et al.  Biophysics of Computation , 2001 .

[25]  C. Koch,et al.  Linearized models of calcium dynamics: formal equivalence to the cable equation , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[26]  J DeFelipe,et al.  Estimation of the number of synapses in the cerebral cortex: methodological considerations. , 1999, Cerebral cortex.

[27]  R. Yuste,et al.  High Speed Two-Photon Imaging of Calcium Dynamics in Dendritic Spines: Consequences for Spine Calcium Kinetics and Buffer Capacity , 2007, PLoS ONE.

[28]  Y. Dan,et al.  Spike Timing-Dependent Plasticity of Neural Circuits , 2004, Neuron.

[29]  P. Schuster,et al.  Self-replication with errors. A model for polynucleotide replication. , 1982, Biophysical chemistry.

[30]  K. Svoboda,et al.  The Life Cycle of Ca2+ Ions in Dendritic Spines , 2002, Neuron.

[31]  Eugene M. Izhikevich,et al.  Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting , 2006 .

[32]  S. Laughlin,et al.  Predictive coding: a fresh view of inhibition in the retina , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[33]  Steven H. Strogatz,et al.  Nonlinear Dynamics and Chaos , 2024 .

[34]  J. Bolz,et al.  Non-Hebbian synapses in rat visual cortex. , 1990, Neuroreport.

[35]  F. Engert,et al.  Synapse specificity of long-term potentiation breaks down at short distances , 1997, Nature.

[36]  R. Guillery,et al.  Exploring the Thalamus , 2000 .

[37]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[38]  Joseph J. Atick,et al.  What Does the Retina Know about Natural Scenes? , 1992, Neural Computation.

[39]  J. Lisman,et al.  A mechanism for the Hebb and the anti-Hebb processes underlying learning and memory. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[40]  F. Botelho,et al.  Qualitative Behavior of Differential Equations Associated with Artificial Neural Networks , 2004 .

[41]  H. Markram,et al.  Physiology and anatomy of synaptic connections between thick tufted pyramidal neurones in the developing rat neocortex. , 1997, The Journal of physiology.

[42]  J. Lisman.  The CaM kinase II hypothesis for the storage of synaptic memory , 1994, Trends in Neurosciences.

[43]  Kingsley J. A. Cox,et al.  Implications of synaptic digitisation and error for neocortical function , 2000, Neurocomputing.

[44]  I. Leuthäusser,et al.  An exact correspondence between Eigen’s evolution model and a two‐dimensional Ising system , 1986 .

[45]  A. Aertsen,et al.  Synaptic plasticity in rat hippocampal slice cultures: local "Hebbian" conjunction of pre- and postsynaptic stimulation leads to distributed synaptic enhancement. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[46]  H. Markram,et al.  Spontaneous and evoked synaptic rewiring in the neonatal neocortex , 2006, Proceedings of the National Academy of Sciences.

[47]  P. Adams,et al.  A neurobiological perspective on building intelligent devices , 2007 .

[48]  P. Andersen,et al.  Specific long-lasting potentiation of synaptic transmission in hippocampal slices , 1977, Nature.

[49]  K. Svoboda,et al.  Structure and function of dendritic spines. , 2002, Annual review of physiology.

[50]  Bartlett W. Mel,et al.  Computational subunits in thin dendrites of pyramidal cells , 2004, Nature Neuroscience.

[51]  D. Ruderman,et al.  Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[52]  J. Hopfield,et al.  All-or-none potentiation at CA3-CA1 synapses. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[53]  Joseph J. Atick,et al.  Towards a Theory of Early Visual Processing , 1990, Neural Computation.

[54]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[55]  D. Chklovskii,et al.  Geometry and Structural Plasticity of Synaptic Connectivity , 2002, Neuron.

[56]  J. H. Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998 .

[57]  C. Koch,et al.  The function of dendritic spines: devices subserving biochemical rather than electrical compartmentalization , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[58]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[59]  Matthew E Larkum,et al.  Synaptic clustering by dendritic signalling mechanisms , 2008, Current Opinion in Neurobiology.

[60]  T. Soderling,et al.  Ca2+/calmodulin-kinase II enhances channel conductance of alpha-amino-3-hydroxy-5-methyl-4-isoxazolepropionate type glutamate receptors. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[61]  E. Oja,et al.  Independent Component Analysis , 2013 .

[62]  Russell M Taylor,et al.  Stepping into the Third Dimension , 2007, The Journal of Neuroscience.

[63]  R. Yuste,et al.  Mechanisms of Calcium Decay Kinetics in Hippocampal Spines: Role of Spine Calcium Pumps and Calcium Diffusion through the Spine Neck in Biochemical Compartmentalization , 2000, The Journal of Neuroscience.

[64]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[65]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[66]  Bernardo L Sabatini,et al.  Neuronal Activity Regulates Diffusion Across the Neck of Dendritic Spines , 2005, Science.

[67]  Bartlett W. Mel,et al.  Dendrites: bug or feature? , 2003, Current Opinion in Neurobiology.

[68]  B. Sakmann,et al.  Single Spine Ca2+ Signals Evoked by Coincident EPSPs and Backpropagating Action Potentials in Spiny Stellate Cells of Layer 4 in the Juvenile Rat Somatosensory Barrel Cortex , 2004, The Journal of Neuroscience.

[69]  P. Jonas,et al.  Kinetics of Mg2+ unblock of NMDA receptors: implications for spike‐timing dependent synaptic plasticity , 2004, The Journal of physiology.

[70]  R. Douglas,et al.  A Quantitative Map of the Circuit of Cat Primary Visual Cortex , 2004, The Journal of Neuroscience.

[71]  T. Bonhoeffer,et al.  Massive restructuring of neuronal circuits during functional reorganization of adult visual cortex , 2008, Nature Neuroscience.

[72]  H. Markram,et al.  The neocortical microcircuit as a tabula rasa. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[73]  P. De Koninck,et al.  Sensitivity of CaM kinase II to the frequency of Ca2+ oscillations. , 1998, Science.

[74]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[75]  S. Wang,et al.  Graded bidirectional synaptic plasticity is composed of switch-like unitary events. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[76]  S. Strogatz.  Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry and Engineering , 1995 .

[77]  Bruno A Olshausen,et al.  Sparse coding of sensory inputs , 2004, Current Opinion in Neurobiology.

[78]  E. White.  Cortical Circuits: Synaptic Organization of the Cerebral Cortex , 1989 .

[79]  Karel Svoboda,et al.  The Spread of Ras Activity Triggered by Activation of a Single Dendritic Spine , 2008, Science.

[80]  P. Castillo,et al.  Endocannabinoid-Mediated Metaplasticity in the Hippocampus , 2004, Neuron.

[81]  Kingsley Cox,et al.  Hebbian errors in learning: an analysis using the Oja model. , 2009, Journal of theoretical biology.

[82]  Bartlett W. Mel,et al.  Cortical rewiring and information storage , 2004, Nature.