Polychronization: Computation with Spikes

We present a minimal spiking network that can polychronize, that is, exhibit reproducible time-locked but not synchronous firing patterns with millisecond precision, as in synfire braids. The network consists of cortical spiking neurons with axonal conduction delays and spike-timing-dependent plasticity (STDP); a ready-to-use MATLAB code is included. It exhibits sleeplike oscillations, gamma (40 Hz) rhythms, conversion of firing rates to spike timings, and other interesting regimes. Due to the interplay between the delays and STDP, the spiking neurons spontaneously self-organize into groups and generate patterns of stereotypical polychronous activity. To our surprise, the number of coexisting polychronous groups far exceeds the number of neurons in the network, resulting in an unprecedented memory capacity of the system. We speculate on the significance of polychrony to the theory of neuronal group selection (TNGS, neural Darwinism), cognitive neural computations, binding and gamma rhythm, mechanisms of attention, and consciousness as attention to memories.

[1]  H. Swadlow,et al.  Systematic variations in the conduction velocity of slowly conducting axons in the rabbit corpus callosum. , 1974, Experimental neurology.

[2]  H. Swadlow,et al.  Observations on impulse conduction along central axons. , 1975, Proceedings of the National Academy of Sciences of the United States of America.

[3]  H. L. Bryant,et al.  Spike initiation by transmembrane current: a white‐noise analysis. , 1976, The Journal of physiology.

[4]  R. D. Driver,et al.  Ordinary and Delay Differential Equations , 1977 .

[5]  H. Swadlow,et al.  Modulation of impulse conduction along the axonal tree. , 1980, Annual review of biophysics and bioengineering.

[6]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[7]  D. Ferster,et al.  An intracellular analysis of geniculo‐cortical connectivity in area 17 of the cat. , 1983, The Journal of physiology.

[8]  H. Swadlow.  Physiological properties of individual cerebral axons studied in vivo for as long as one year. , 1985, Journal of neurophysiology.

[9]  B. Strehler,et al.  Evidence on precise time-coded symbols and memory of patterns in monkey cortical neuronal spike trains. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[10]  G. Edelman.  Neural Darwinism: The Theory Of Neuronal Group Selection , 1989 .

[11]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[12]  H. Swadlow.  Efferent neurons and suspected interneurons in binocular visual cortex of the awake rabbit: receptive fields and binocular properties. , 1988, Journal of neurophysiology.

[13]  A. Damasio.  Time-locked multiregional retroactivation: A systems-level proposal for the neural substrates of recall and recognition , 1989, Cognition.

[14]  Prof. Dr. Valentino Braitenberg,et al.  Anatomy of the Cortex , 1991, Studies of Brain Function.

[15]  T. M. Mayhew,et al.  Anatomy of the Cortex: Statistics and Geometry. , 1991 .

[16]  Moshe Abeles,et al.  Corticonics: Neural Circuits of Cerebral Cortex , 1991 .

[17]  H. Swadlow.  Monitoring the excitability of neocortical efferent neurons to direct activation by extracellular current pulses. , 1992, Journal of neurophysiology.

[18]  G. Edelman.  Neural Darwinism: Selection and reentrant signaling in higher brain function , 1993, Neuron.

[19]  H. Swadlow.  Efferent neurons and suspected interneurons in motor cortex of the awake rabbit: axonal properties, sensory receptive fields, and subthreshold synaptic inputs. , 1994, Journal of neurophysiology.

[20]  Michael N. Shadlen,et al.  Noise, neural codes and cortical organization , 1994, Current Opinion in Neurobiology.

[21]  W. Singer,et al.  Temporal Coding in the Brain , 1994, Research and Perspectives in Neurosciences.

[22]  E. Bienenstock.  A model of neocortex , 1995 .

[23]  T. Sejnowski,et al.  Reliability of spike timing in neocortical neurons. , 1995, Science.

[24]  J. J. Hopfield,et al.  Pattern recognition computation using action potential timing for stimulus representation , 1995, Nature.

[25]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[26]  Robert Miller,et al.  Neural assemblies and laminar interactions in the cerebral cortex , 1996, Biological Cybernetics.

[27]  Wulfram Gerstner,et al.  A neuronal learning rule for sub-millisecond temporal coding , 1996, Nature.

[28]  Robert Miller,et al.  Cortico-thalamic interplay and the security of operation of neural assemblies and temporal chains in the cerebral cortex , 1996, Biological Cybernetics.

[29]  H. Sompolinsky,et al.  Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity , 1996, Science.

[30]  A. Aertsen,et al.  Spike synchronization and rate modulation differentially involved in motor cortical function. , 1997, Science.

[31]  G D Lewen,et al.  Reproducibility and Variability in Neural Spike Trains , 1997, Science.

[32]  G. Gerstein,et al.  Repeated patterns of distributed synchrony in neuronal assemblies. , 1997, Journal of neurophysiology.

[33]  E. Izhikevich,et al.  Weakly connected neural networks , 1997 .

[34]  D. Amit,et al.  Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. , 1997, Cerebral cortex.

[35]  Günther Palm,et al.  Synaptic Delay Learning in Pulse-Coupled Neurons , 1998, Neural Computation.

[36]  W. Newsome,et al.  The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding , 1998, The Journal of Neuroscience.

[37]  Niraj S. Desai,et al.  Activity-dependent scaling of quantal amplitude in neocortical neurons , 1998, Nature.

[38]  E. Vaadia,et al.  Spatiotemporal structure of cortical activity: properties and behavioral relevance. , 1998, Journal of neurophysiology.

[39]  Jack D. Cowan,et al.  DYNAMICS OF SELF-ORGANIZED DELAY ADAPTATION , 1999 .

[40]  I. Tetko,et al.  Spatiotemporal activity patterns of rat cortical neurons predict responses in a conditioned task. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[41]  Ad Aertsen,et al.  Stable propagation of synchronous spiking in cortical neural networks , 1999, Nature.

[42]  B J Richmond,et al.  Stochastic nature of precisely timed spike patterns in visual system neuronal responses. , 1999, Journal of neurophysiology.

[43]  E. Gilson,et al.  Repeated Sequences , 1999 .

[44]  Michael N. Shadlen,et al.  Synchrony Unbound A Critical Evaluation of the Temporal Binding Hypothesis , 1999, Neuron.

[45]  R. Traub,et al.  Inhibition-based rhythms: experimental and mathematical observations on network dynamics. , 2000, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[46]  K. Morris,et al.  Repeated sequences of interspike intervals in baroresponsive respiratory related neuronal assemblies of the cat brain stem. , 2000, Journal of neurophysiology.

[47]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[48]  J. Milton,et al.  Multistability in recurrent neural loops arising from delay. , 2000, Journal of neurophysiology.

[49]  R N Lemon,et al.  Precise spatiotemporal repeating patterns in monkey primary and supplementary motor areas occur at chance levels. , 2000, Journal of neurophysiology.

[50]  Igor V. Tetko,et al.  A pattern grouping algorithm for analysis of spatiotemporal patterns in neuronal spike trains. 2. Application to simultaneous single unit recordings , 2001, Journal of Neuroscience Methods.

[51]  Igor V. Tetko,et al.  A pattern grouping algorithm for analysis of spatiotemporal patterns in neuronal spike trains. 1. Detection of repeated patterns , 2001, Journal of Neuroscience Methods.

[52]  R. Yuste,et al.  Dynamics of Spontaneous Activity in Neocortical Slices , 2001, Neuron.

[53]  G. Edelman,et al.  Degeneracy and complexity in biological systems , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[54]  Martin Schneider,et al.  Activity-Dependent Development of Axonal and Dendritic Delays, or, Why Synaptic Transmission Should Be Unreliable , 2002, Neural Computation.

[55]  R. Reid,et al.  Precise Firing Events Are Conserved across Neurons , 2002, The Journal of Neuroscience.

[56]  K. Svoboda,et al.  Long-term in vivo imaging of experience-dependent synaptic plasticity in adult cortex , 2002, Nature.

[57]  Niraj S. Desai,et al.  Critical periods for experience-dependent synaptic scaling in visual cortex , 2002, Nature Neuroscience.

[58]  N. Kasthuri,et al.  Long-term dendritic spine stability in the adult cortex , 2002, Nature.

[59]  M. Shadlen,et al.  Limits to the temporal fidelity of cortical spike rate signals , 2002, Nature Neuroscience.

[60]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[61]  W. M. Keck,et al.  Machine Psychology : Autonomous Behavior , Perceptual Categorization and Conditioning in a Brain-based Device , 2002 .

[62]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[63]  Marcus Pivato,et al.  Symmetry Groupoids and Patterns of Synchrony in Coupled Cell Networks , 2003, SIAM J. Appl. Dyn. Syst..

[64]  J. Moake,et al.  This article has been cited by other articles , 2003 .

[65]  A. Bellen,et al.  Numerical methods for delay differential equations , 2003 .

[66]  Dean V Buonomano,et al.  Timing of neural responses in cortical organotypic slices , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[67]  John M. Beggs,et al.  Neuronal Avalanches in Neocortical Circuits , 2003, The Journal of Neuroscience.

[68]  Idan Segev,et al.  On the Transmission of Rate Code in Long Feedforward Networks with Excitatory–Inhibitory Balance , 2003, The Journal of Neuroscience.

[69]  T. Tsumoto,et al.  Change of conduction velocity by regional myelination yields constant latency irrespective of distance between thalamus and cortex , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[70]  Yuji Ikegaya,et al.  Synfire Chains and Cortical Songs: Temporal Modules of Cortical Activity , 2004, Science.

[71]  G. Edelman,et al.  Spike-timing dynamics of neuronal groups. , 2004, Cerebral cortex.

[72]  G. Edelman,et al.  Spatiotemporal processing of whisker input supports texture discrimination by a brain-based device , 2004 .

[73]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[74]  G. Edelman.  Wider than the sky : the phenomenal gift of consciousness , 2004 .

[75]  John M. Beggs,et al.  Behavioral / Systems / Cognitive Neuronal Avalanches Are Diverse and Precise Activity Patterns That Are Stable for Many Hours in Cortical Slice Cultures , 2004 .

[76]  G. Edelman,et al.  Visual binding through reentrant connectivity and dynamic synchronization in a brain-based device. , 2004, Cerebral cortex.

[77]  E. Ben-Jacob,et al.  Manifestation of function-follow-form in cultured neuronal networks , 2005, Physical biology.

[78]  K. Harris.  Neural signatures of cell assembly organization , 2005, Nature Reviews Neuroscience.

[79]  Roman R Poznanski,et al.  fMRI models of dendritic and astrocytic networks. , 2006, Journal of integrative neuroscience.

[80]  K. Josić,et al.  Network architecture and spatio-temporally symmetric dynamics , 2006 .

[81]  Eugene M. Izhikevich,et al.  Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting , 2006 .

[82]  Pulin Gong,et al.  Dynamically maintained spike timing sequences in networks of pulse-coupled oscillators with delays. , 2007, Physical review letters.

[83]  Masato Okada,et al.  Sparse and Dense Encoding in Layered Associative Network of Spiking Neurons(Cross-disciplinary physics and related areas of science and technology) , 2007 .

[84]  Vicenç Gómez,et al.  Phase Transition and Hysteresis in an Ensemble of Stochastic Spiking Neurons , 2007, Neural Computation.

[85]  Shigeru Shinomoto,et al.  Reverberating activity in a neural network with distributed signal transmission delays. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[86]  Toshio Aoyagi,et al.  Synchrony-Induced Switching Behavior of Spike Pattern Attractors Created by Spike-Timing-Dependent Plasticity , 2007, Neural Computation.

[87]  Chris Eliasmith,et al.  Neural populations can induce reliable postsynaptic currents without observable spike rate changes or precise spike timing. , 2007, Cerebral cortex.

[88]  Steve M. Potter,et al.  Precisely timed spatiotemporal patterns of neural activity in dissociated cortical cultures , 2007, Neuroscience.

[89]  Kazuyuki Aihara,et al.  Variable Timescales of Repeated Spike Patterns in Synfire Chain with Mexican-Hat Connectivity , 2007, Neural Computation.

[90]  Steve M. Potter,et al.  Region-specific network plasticity in simulated and living cortical networks: comparison of the center of activity trajectory (CAT) with other statistics , 2007, Journal of neural engineering.

[91]  M. Torrent,et al.  Controlling the leader-laggard dynamics in delay-synchronized lasers. , 2007, Chaos.

[92]  Naoki Masuda,et al.  Formation of feedforward networks and frequency synchrony by spike-timing-dependent plasticity , 2007, Journal of Computational Neuroscience.

[93]  D. Debanne,et al.  Release-Dependent Variations in Synaptic Latency: A Putative Code for Short- and Long-Term Synaptic Dynamics , 2007, Neuron.

[94]  Kazuyuki Aihara,et al.  Dual coding hypotheses for neural information representation. , 2007, Mathematical biosciences.

[95]  Samy Bengio,et al.  Delay learning and polychronization for reservoir computing , 2008, Neurocomputing.

[96]  S. Shinomoto,et al.  Can distributed delays perfectly stabilize dynamical networks? , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[97]  Xi Shen,et al.  Oscillations and Spiking Pairs: Behavior of a Neuronal Model with STDP Learning , 2008, Neural Computation.

[98]  Tohru Ikeguchi,et al.  STDP Provides the Substrate for Igniting Synfire Chains by Spatiotemporal Input Patterns , 2008, Neural Computation.