Can't Get You Out of My Head: A Connectionist Model of Cyclic Rehearsal

Humans are able to perform a large variety of periodic activities in different modes, for instance cyclic rehearsal of phone numbers, humming a melody sniplet over and over again. These performances are, to a certain degree, robust against perturbations, and it often suffices to present a new pattern a few times only until it can be "picked up". From an abstract mathematical perspective, this implies that the brain, as a dynamical system, (1) hosts a very large number of cyclic attractors, such that (2) if the system is driven by external input with a cyclic motif, it can entrain to a closely corresponding attractor in a very short time. This chapter proposes a simple recurrent neural network architecture which displays these dynamical phenomena. The model builds on echo state networks (ESNs), which have recently become popular in machine learning and computational neuroscience.

[1]  Grosvenor W. Cooper,et al.  The Rhythmic Structure of Music , 1971 .

[2]  Peter M. Todd,et al.  A Connectionist Approach To Algorithmic Composition , 1989 .

[3]  C. Krumhansl.  Cognitive Foundations of Musical Pitch , 1990 .

[4]  Michael C. Mozer,et al.  Neural Network Music Composition by Prediction: Exploring the Benefits of Psychoacoustic Constraints and Multi-scale Processing , 1994, Connect. Sci..

[5]  John F. Kolen,et al.  Resonance and the Perception of Musical Meter , 1994, Connect. Sci..

[6]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[7]  Ronald J. Williams,et al.  Gradient-based learning algorithms for recurrent networks and their computational complexity , 1995 .

[8]  Frank Pasemann,et al.  Characterization of periodic attractors in neural ring networks , 1995, Neural Networks.

[9]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[10]  T. Gelder,et al.  Mind as Motion: Explorations in the Dynamics of Cognition , 1995 .

[11]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[12]  Bernard Widrow,et al.  Perceptrons, adalines, and backpropagation , 1998 .

[13]  T. Gelder,et al.  The dynamical hypothesis in cognitive science , 1998, Behavioral and Brain Sciences.

[14]  M. Thaut,et al.  The connection between rhythmicity and brain function , 1999, IEEE Engineering in Medicine and Biology Magazine.

[15]  Peter Cariani,et al.  Temporal Codes, Timing Nets, and Music Perception , 2001 .

[16]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[17]  Douglas Eck.  Finding downbeats with a relaxation oscillator , 2002, Psychological research.

[18]  Edward W. Large,et al.  Perceiving temporal regularity in music , 2002, Cogn. Sci..

[19]  Jürgen Schmidhuber,et al.  Finding temporal structure in music: blues improvisation with LSTM recurrent networks , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[20]  Mathias Quoy,et al.  Resonant spatiotemporal learning in large random recurrent networks , 2002, Biological Cybernetics.

[21]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[22]  G. Hickok,et al.  Auditory–Motor Interaction Revealed by fMRI: Speech, Music, and Working Memory in Area Spt , 2003 .

[23]  Caroline Palmer,et al.  Incremental planning in sequence production. , 2003, Psychological review.

[24]  A. Baddeley.  Working memory: looking back and looking forward , 2003, Nature Reviews Neuroscience.

[25]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[26]  D. Buonomano,et al.  The neural basis of temporal processing. , 2004, Annual review of neuroscience.

[27]  Haim Sompolinsky,et al.  Short-term memory in orthogonal neural networks. , 2004, Physical review letters.

[28]  Dean V Buonomano,et al.  A learning rule for the emergence of stable dynamics and timing in recurrent networks. , 2005, Journal of neurophysiology.

[29]  Stefan Schaal,et al.  Rapid synchronization and accurate phase-locking of rhythmic motor primitives , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  F. Gouyon.  A computational approach to rhythm description - Audio features for the computation of rhythm periodicity functions and their use in tempo induction and music content processing , 2005 .

[31]  Herbert Jaeger,et al.  A tutorial on training recurrent neural networks , covering BPPT , RTRL , EKF and the " echo state network " approach - Semantic Scholar , 2005 .

[32]  Nak Young Chong,et al.  Goal-directed imitation with self-adjusting adaptor based on a neural oscillator network , 2005, ICAR '05. Proceedings., 12th International Conference on Advanced Robotics, 2005..

[33]  Douglas Eck.  Identifying Metrical and Temporal Structure With an Autocorrelation Phase Matrix , 2006 .

[34]  Herbert Jaeger.  Generating exponentially many periodic attractors with linearly growing Echo State Networks , 2006 .

[35]  Wolfgang Maass,et al.  A model for the interaction of oscillations and pattern generation with real-time computing in generic neural microcircuit models , 2006, Neural Networks.

[36]  Zhaoping Li,et al.  Psychophysical Tests of the Hypothesis of a Bottom-Up Saliency Map in Primary Visual Cortex , 2007, PLoS Comput. Biol..

[37]  Eduardo D. Sontag,et al.  Computational Aspects of Feedback in Neural Circuits , 2006, PLoS Comput. Biol..