Does High Firing Irregularity Enhance Learning?

In this note, we demonstrate that the high firing irregularity produced by the leaky integrate-and-fire neuron with the partial somatic reset mechanism, which has been shown to be the most likely candidate to reflect the mechanism used in the brain for reproducing the highly irregular cortical neuron firing at high rates (Bugmann, Christodoulou, & Taylor, 1997; Christodoulou & Bugmann, 2001), enhances learning. More specifically, it enhances reward-modulated spike-timing-dependent plasticity with eligibility trace when used in spiking neural networks, as shown by the results when tested in the simple benchmark problem of XOR, as well as in a complex multiagent setting task.

[1]  Razvan V. Florian,et al.  Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity , 2007, Neural Computation.

[2]  Ron Meir,et al.  Reinforcement Learning, Spike-Time-Dependent Plasticity, and the BCM Rule , 2007, Neural Computation.

[3]  Jean-Pascal Pfister,et al.  Optimal Spike-Timing-Dependent Plasticity for Precise Action Potential Firing in Supervised Learning , 2005, Neural Computation.

[4]  William R. Softky,et al.  The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[5]  Guido Bugmann,et al.  Coefficient of variation vs. mean interspike interval curves: What do they tell us about the brain? , 2001, Neurocomputing.

[6]  Xiaohui Xie,et al.  Learning in neural networks by reinforcement of irregular spiking. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[7]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[8]  H. Seung,et al.  Learning in Spiking Neural Networks by Reinforcement of Stochastic Synaptic Transmission , 2003, Neuron.

[9]  E. Izhikevich.  Solving the distal reward problem through linkage of STDP and dopamine signaling , 2007, BMC Neuroscience.

[10]  Guido Bugmann,et al.  Distinguishing the Causes of Firing with the Membrane Potential Slope , 2012, Neural Computation.

[11]  Wulfram Gerstner,et al.  Spike-Based Reinforcement Learning in Continuous State and Action Space: When Policy Gradient Methods Fail , 2009, PLoS Comput. Biol..

[12]  Chris Christodoulou,et al.  Learning optimisation by high firing irregularity , 2012, Brain Research.

[13]  G Bugmann,et al.  Near Poisson-type firing produced by concurrent excitation and inhibition. , 2000, Bio Systems.

[14]  Vilfredo Pareto,et al.  Manuale di economia politica , 1965 .

[15]  M. Farries,et al.  Reinforcement learning with modulated spike timing dependent synaptic plasticity. , 2007, Journal of neurophysiology.

[16]  Chris Christodoulou,et al.  Multiagent Reinforcement Learning: Spiking and Nonspiking Agents in the Iterated Prisoner's Dilemma , 2011, IEEE Transactions on Neural Networks.

[17]  Christof Koch,et al.  Cortical Cells Should Fire Regularly, But Do Not , 1999, Neural Computation.

[18]  A. Rapoport,et al.  Prisoner's Dilemma: A Study in Conflict and Co-operation , 1970 .

[19]  Guido Bugmann,et al.  Role of Temporal Integration and Fluctuation Detection in the Highly Irregular Firing of a Leaky Integrator Neuron Model with Partial Reset , 1997, Neural Computation.

[20]  Robert A. Legenstein,et al.  A Learning Theory for Reward-Modulated Spike-Timing-Dependent Plasticity with Application to Biofeedback , 2008, PLoS Comput. Biol..

[21]  Chris Christodoulou,et al.  Coefficient of Variation ( CV ) vs Mean Interspike Interval ( ISI ) curves : what do they tell us about the brain ? , 2000 .

[22]  C. Christodoulou,et al.  Self-control with spiking and non-spiking neural networks playing games , 2010, Journal of Physiology-Paris.