Spiking neurons and the induction of finite state machines

We discuss in this short survey article some current mathematical models from neurophysiology for the computational units of biological neural systems: neurons and synapses. These models are contrasted with the computational units of common artificial neural network models, which reflect the state of knowledge in neurophysiology 50 years ago. We discuss the problem of carrying out computations in circuits consisting of biologically realistic computational units, focusing on the biologically particularly relevant case of computations on time series. Finite state machines are frequently used in computer science as models for computations on time series. One may argue that these models provide a reasonable common conceptual basis for analyzing computations in computers and biological neural systems, although the emphasis in biological neural systems is shifted more towards asynchronous computation on analog time series. In the second half of this article some new computer experiments and theoretical results are discussed, which address the question whether a biological neural system can, in principle, learn to behave like a given simple finite state machine.

[1]  Raymond L. Watrous,et al.  Induction of Finite-State Languages Using Second-Order Recurrent Networks , 1992, Neural Computation.

[2]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[3]  S. Strogatz.  Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry and Engineering , 1995 .

[4]  P. Auer,et al.  The p-Delta Learning Rule for Parallel Perceptrons , .

[5]  C. Lee Giles,et al.  Learning a class of large finite state machines with a recurrent neural network , 1995, Neural Networks.

[6]  H. Markram,et al.  Redistribution of synaptic efficacy between neocortical pyramidal neurons , 1996, Nature.

[7]  Edward A. Feigenbaum,et al.  Switching and Finite Automata Theory: Computer Science Series , 1990 .

[8]  H. Markram,et al.  Differential signaling via the same axon of neocortical pyramidal neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Wulfram Gerstner.  Populations of spiking neurons , 1999 .

[10]  D. Johnston,et al.  Foundations of Cellular Neurophysiology , 1994 .

[11]  J. Taylor,et al.  Switching and finite automata theory, 2nd ed. , 1980, Proceedings of the IEEE.

[12]  Sadri Hassani,et al.  Nonlinear Dynamics and Chaos , 2000 .

[13]  Vivien A. Casagrande,et al.  Biophysics of Computation: Information Processing in Single Neurons , 1999 .

[14]  H. Markram,et al.  Organizing principles for a diversity of GABAergic interneurons and synapses in the neocortex. , 2000, Science.

[15]  Eduardo D. Sontag,et al.  Neural Systems as Nonlinear Filters , 2000, Neural Computation.

[16]  Wolfgang Maass,et al.  E?cient Temporal Processing with Biolog-ically Realistic Dynamic Synapses , 2001 .

[17]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[18]  Wolfgang Maass,et al.  On the Computational Power of Winner-Take-All , 2000, Neural Computation.

[19]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[20]  L. Abbott,et al.  A Quantitative Description of Short-Term Plasticity at Excitatory Synapses in Layer 2/3 of Rat Primary Visual Cortex , 1997, The Journal of Neuroscience.

[21]  Wulfram Gerstner,et al.  Spiking neurons , 1999 .

[22]  L. Abbott,et al.  Synaptic Depression and Cortical Gain Control , 1997, Science.

[23]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[24]  C. Lee Giles,et al.  Learning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks , 1992, Neural Computation.

[25]  T. Kailath,et al.  Discrete Neural Computation: A Theoretical Foundation , 1995 .

[26]  W. Precht.  The synaptic organization of the brain G.M. Shepherd, Oxford University Press (1975). 364 pp., £3.80 (paperback) , 1976, Neuroscience.

[27]  Wolfgang Maass,et al.  Lower Bounds for the Computational Power of Networks of Spiking Neurons , 1996, Neural Computation.

[28]  C. Koch,et al.  Methods in Neuronal Modeling: From Ions to Networks , 1998 .

[29]  Anthony M. Zador,et al.  The basic unit of computation , 2000, Nature Neuroscience.

[30]  Christof Koch,et al.  Biophysics of Computation: Information Processing in Single Neurons (Computational Neuroscience Series) , 1998 .

[31]  Ah Chung Tsoi,et al.  A Simplified Gradient Algorithm for IIR Synapse Multilayer Perceptrons , 1993, Neural Computation.