Synergies Between Intrinsic and Synaptic Plasticity Mechanisms

We propose a model of intrinsic plasticity for a continuous activation model neuron based on information theory. We then show how intrinsic and synaptic plasticity mechanisms interact and allow the neuron to discover heavy-tailed directions in the input. We also demonstrate that intrinsic plasticity may be an alternative explanation for the sliding threshold postulated in the BCM theory of synaptic plasticity. We present a theoretical analysis of the interaction of intrinsic plasticity with different Hebbian learning rules for the case of clustered inputs. Finally, we perform experiments on the bars problem, a popular nonlinear independent component analysis problem.

[1]  S. Laughlin.  A Simple Coding Procedure Enhances a Neuron's Information Capacity , 1981, Zeitschrift fur Naturforschung. Section C, Biosciences.

[2]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[3]  D. Prince,et al.  Epileptogenesis in chronically injured cortex: in vitro studies. , 1993, Journal of neurophysiology.

[4]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[5]  Peter Dayan,et al.  Bee foraging in uncertain environments using predictive hebbian learning , 1995, Nature.

[6]  C. Pennartz.  The ascending neuromodulatory systems in learning by reinforcement: comparing computational conjectures with experimental findings , 1995, Brain Research Reviews.

[7]  E Marder,et al.  Memory from the dynamics of intrinsic membrane currents. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[8]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[9]  M. Bear,et al.  Experience-dependent modification of synaptic plasticity in visual cortex , 1996, Nature.

[10]  L. Abbott,et al.  Responses of neurons in primary and inferior temporal visual cortices to natural scenes , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[11]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[12]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[13]  Nathan Intrator,et al.  Receptive Field Formation in Natural Scene Environments: Comparison of Single-Cell Learning Rules , 1997, Neural Computation.

[14]  Niraj S. Desai,et al.  Activity-dependent scaling of quantal amplitude in neocortical neurons , 1998, Nature.

[15]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[16]  Erkki Oja,et al.  Independent component analysis by general nonlinear Hebbian-like learning rules , 1998, Signal Process..

[17]  K. Miller,et al.  Increased pyramidal excitability and NMDA conductance can explain posttraumatic epileptogenesis without disinhibition: a model. , 1999, Journal of neurophysiology.

[18]  Niraj S. Desai,et al.  Plasticity in the intrinsic excitability of cortical pyramidal neurons , 1999, Nature Neuroscience.

[19]  Christof Koch,et al.  Adaptive Neural Coding Dependent on the Time-Varying Statistics of the Somatic Input Current , 1999, Neural Computation.

[20]  Christof Koch,et al.  How voltage-dependent conductances can adapt to maximize the information encoded by neuronal firing rate , 1999, Nature Neuroscience.

[21]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[22]  S. Nelson,et al.  Hebb and homeostasis in neuronal plasticity , 2000, Current Opinion in Neurobiology.

[23]  D. Tolhurst,et al.  Characterizing the sparseness of neural codes , 2001, Network.

[24]  D. Debanne,et al.  Long-term plasticity of intrinsic excitability: learning rules and mechanisms. , 2003, Learning & memory.

[25]  D. Linden,et al.  The other side of the engram: experience-driven changes in neuronal intrinsic excitability , 2003, Nature Reviews Neuroscience.

[26]  C. Akerman,et al.  Visually Driven Regulation of Intrinsic Neuronal Excitability Improves Stimulus Detection In Vivo , 2003, Neuron.

[27]  Nils Bertschinger,et al.  Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks , 2004, Neural Computation.

[28]  Robert H. Cudmore,et al.  Long-term potentiation of intrinsic excitability in LV visual cortical neurons. , 2004, Journal of neurophysiology.

[29]  Jochen Triesch,et al.  Synergies between Intrinsic and Synaptic Plasticity in Individual Model Neurons , 2004, NIPS.

[30]  Mei Zhang,et al.  Calcium signal-dependent plasticity of neuronal excitability developed postnatally. , 2004, Journal of neurobiology.

[31]  S. Nelson,et al.  Homeostatic plasticity in the developing nervous system , 2004, Nature Reviews Neuroscience.

[32]  S. Nelson,et al.  Selective reconfiguration of layer 4 visual cortical circuitry by visual deprivation , 2004, Nature Neuroscience.

[33]  P. Földiák,et al.  Forming sparse representations by local anti-Hebbian learning , 1990, Biological Cybernetics.

[34]  W. Wadman,et al.  Homeostatic scaling of neuronal excitability by synaptic modulation of somatic hyperpolarization-activated Ih channels. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Nathan Intrator,et al.  Theory of Cortical Plasticity , 2004 .

[36]  T. Sejnowski,et al.  Homeostatic synaptic plasticity can explain post-traumatic epileptogenesis in chronically isolated neocortex. , 2005, Cerebral cortex.

[37]  Jochen Triesch,et al.  A Gradient Rule for the Plasticity of a Neuron's Intrinsic Excitability , 2005, ICANN.

[38]  Pieter R. Roelfsema,et al.  Attention-Gated Reinforcement Learning of Internal Representations for Classification , 2005, Neural Computation.

[39]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[40]  Denis Burdakov.  Gain Control by Concerted Changes in IA and I H Conductances , 2005, Neural Computation.

[41]  R. Desimone,et al.  Selectivity and sparseness in the responses of striate complex cells , 2005, Vision Research.

[42]  Mark C. W. van Rossum,et al.  Excitability changes that complement Hebbian learning , 2006, Network.

[43]  W. Wadman,et al.  Background activity regulates excitability of rat hippocampal CA1 pyramidal neurons by adaptation of a K+ conductance. , 2006, Journal of neurophysiology.

[44]  R. Malenka,et al.  Synaptic scaling mediated by glial TNF-α , 2006, Nature.

[45]  Jochen Triesch,et al.  Exploring the role of intrinsic plasticity for the learning of sensory representations , 2006, ESANN.

[46]  Cornelius Weber,et al.  A Sparse Generative Model of V1 Simple Cells with Intrinsic Plasticity , 2008, Neural Computation.

[47]  T. J. Sullivan,et al.  Sleeping Our Way to Weight Normalization and Stable Learning , 2008, Neural Computation.

[48]  D. Johnston,et al.  Active dendrites: colorful wings of the mysterious butterflies , 2008, Trends in Neurosciences.

[49]  Jochen J. Steil,et al.  Improving reservoirs using intrinsic plasticity , 2008, Neurocomputing.

[50]  Wulfram Gerstner,et al.  Phenomenological models of synaptic plasticity based on spike timing , 2008, Biological Cybernetics.