Sequential associative memory with nonuniformity of the layer sizes.

Sequence retrieval has a fundamental importance in information processing by the brain, and has extensively been studied in neural network models. Most of the previous sequential associative memory embedded sequences of memory patterns have nearly equal sizes. It was recently shown that local cortical networks display many diverse yet repeatable precise temporal sequences of neuronal activities, termed "neuronal avalanches." Interestingly, these avalanches displayed size and lifetime distributions that obey power laws. Inspired by these experimental findings, here we consider an associative memory model of binary neurons that stores sequences of memory patterns with highly variable sizes. Our analysis includes the case where the statistics of these size variations obey the above-mentioned power laws. We study the retrieval dynamics of such memory systems by analytically deriving the equations that govern the time evolution of macroscopic order parameters. We calculate the critical sequence length beyond which the network cannot retrieve memory sequences correctly. As an application of the analysis, we show how the present variability in sequential memory patterns degrades the power-law lifetime distribution of retrieved neural activities.

[1]  Shun-ichi Amari,et al.  Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements , 1972, IEEE Transactions on Computers.

[2]  G. Palm,et al.  On associative memory , 2004, Biological Cybernetics.

[3]  M. Morita.  Computational study on the neural mechanism of sequential pattern memory. , 1996, Brain research. Cognitive brain research.

[4]  J. Leo van Hemmen,et al.  Statistical Mechanics of Temporal Association in Neural Networks , 1990, NIPS.

[5]  M. Shiino,et al.  Self-consistent signal-to-noise analysis and its application to analogue neural networks with asymmetric connections , 1992 .

[6]  Shigeru Yamane,et al.  Neuronal Mechanisms Encoding Global-to-Fine Information in Inferior-Temporal Cortex , 2005, Journal of Computational Neuroscience.

[7]  John M. Beggs,et al.  Neuronal Avalanches in Neocortical Circuits , 2003, The Journal of Neuroscience.

[8]  M. Tsodyks,et al.  The Enhanced Storage Capacity in Neural Networks with Low Activity Level , 1988 .

[9]  Stanley,et al.  Self-organized branching processes: Mean-field theory for avalanches. , 1995, Physical review letters.

[10]  John M. Beggs,et al.  Behavioral / Systems / Cognitive Neuronal Avalanches Are Diverse and Precise Activity Patterns That Are Stable for Many Hours in Cortical Slice Cultures , 2004 .

[11]  Shiino,et al.  Retrieval of spatio-temporal sequence in asynchronous neural network. , 1990, Physical review. A, Atomic, molecular, and optical physics.

[12]  Yuji Ikegaya,et al.  Synfire Chains and Cortical Songs: Temporal Modules of Cortical Activity , 2004, Science.

[13]  T. E. Harris,et al.  The Theory of Branching Processes. , 1963 .

[14]  Ad Aertsen,et al.  Stable propagation of synchronous spiking in cortical neural networks , 1999, Nature.

[15]  Masato Okada,et al.  Notions of Associative Memory and Sparse Coding , 1996, Neural Networks.

[16]  W. Kinzel,et al.  Layered neural networks , 1989 .

[17]  E. Vaadia,et al.  Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. , 1993, Journal of neurophysiology.

[18]  Kanter,et al.  Temporal association in asymmetric neural networks. , 1986, Physical review letters.

[19]  Richard Hans Robert Hahnloser,et al.  An ultra-sparse code underliesthe generation of neural sequences in a songbird , 2002, Nature.

[20]  Shun-ichi Amari,et al.  Statistical neurodynamics of associative memory , 1988, Neural Networks.