A biologically inspired dual-network memory model for reduction of catastrophic forgetting

Neural networks encounter serious catastrophic forgetting when information is learned sequentially, which is unacceptable for both a model of human memory and practical engineering applications. In this study, we propose a novel biologically inspired dual-network memory model that can significantly reduce catastrophic forgetting. The proposed model consists of two distinct neural networks: hippocampal and neocortical networks. Information is first stored in the hippocampal network, and thereafter, it is transferred to the neocortical network. In the hippocampal network, chaotic behavior of neurons in the CA3 region of the hippocampus and neuronal turnover in the dentate gyrus region are introduced. Chaotic recall by CA3 enables retrieval of stored information in the hippocampal network. Thereafter, information retrieved from the hippocampal network is interleaved with previously stored information and consolidated by using pseudopatterns in the neocortical network. The computer simulation results show the effectiveness of the proposed dual-network memory model.

[1]  L’oubli catastrophique it,et al.  Avoiding catastrophic forgetting by coupling two reverberating neural networks , 2004 .

[2]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[3]  Motonobu Hattori,et al.  A Model of Hippocampal Learning with Neuronal Turnover in Dentate Gyrus , 2008 .

[4]  Masafumi Hagiwara,et al.  Chaotic bidirectional associative memory , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[5]  R. O’Reilly,et al.  Modeling hippocampal and neocortical contributions to recognition memory: a complementary-learning-systems approach. , 2003, Psychological review.

[6]  Motonobu Hattori,et al.  Avoiding Catastrophic Forgetting by a Biologically Inspired Dual-Network Memory Model , 2012, ICONIP.

[7]  Hatsuo Hayashi,et al.  Chaotic responses of the hippocampal CA3 region to a mossy fiber stimulation in vitro , 1995, Brain Research.

[8]  Robert M. French,et al.  Pseudo-recurrent Connectionist Networks: An Approach to the 'Sensitivity-Stability' Dilemma , 1997, Connect. Sci..

[9]  K. Aihara,et al.  Chaotic neural networks , 1990 .

[10]  Shun Ishizaki,et al.  Sequence disambiguation and pattern completion by cooperation between autoassociative and heteroassociative memories of functionally divided hippocampal CA3 , 2008, Neurocomputing.

[11]  Gerd Kempermann,et al.  Phenytoin inhibits expression of microtubule-associated protein 2 and influences cell-viability and neurite growth of cultured cerebellar granule cells , 1995, Brain Research.

[12]  H. Eichenbaum.  The Cognitive Neuroscience of Memory , 2002 .

[13]  Anthony V. Robins,et al.  Catastrophic Forgetting, Rehearsal and Pseudorehearsal , 1995, Connect. Sci..

[14]  Motonobu Hattori.  Dual-network memory model using a chaotic neural network , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[15]  S. Becker.  A computational principle for hippocampal learning and neurogenesis , 2005, Hippocampus.

[16]  Motonobu Hattori.  Extraction of patterns from a hippocampal network using chaotic recall , 2011 .

[17]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[18]  Motonobu Hattori,et al.  Performance analysis of associative memory using the maximal margin learning , 2006 .

[19]  M. Yeckel,et al.  Feedforward excitation of the hippocampus by afferents from the entorhinal cortex: redefinition of the role of the trisynaptic pathway. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[20]  R Ratcliff,et al.  Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. , 1990, Psychological review.