Associative Memory for Online Learning in Noisy Environments Using Self-Organizing Incremental Neural Network

Associative memory operating in a real environment must perform well in online incremental learning and be robust to noisy data because noisy associative patterns are presented sequentially in a real environment. We propose a novel associative memory that satisfies these requirements. Using the proposed method, new associative pairs that are presented sequentially can be learned accurately without forgetting previously learned patterns. The memory size of the proposed method increases adaptively with learning patterns. Therefore, it suffers neither redundancy nor insufficiency of memory size, even in an environment in which the maximum number of associative pairs to be presented is unknown before learning. Noisy inputs in real environments are classifiable into two types: noise-added original patterns and faultily presented random patterns. The proposed method deals with two types of noise. To our knowledge, no conventional associative memory addresses noise of both types. The proposed associative memory performs as a bidirectional one-to-many or many-to-one associative memory and deals not only with bipolar data, but also with real-valued data. Results demonstrate that the proposed method's features are important for application to an intelligent robot operating in a real environment. The originality of our work consists of two points: employing a growing self-organizing network for an associative memory, and discussing what features are necessary for an associative memory for an intelligent robot and proposing an associative memory that satisfies those requirements.

[1]  Peter Sussner,et al.  Gray-scale morphological associative memories , 2006, IEEE Transactions on Neural Networks.

[2]  A. Takanishi,et al.  New memory model for humanoid robots - introduction of co-associative memory using mutually coupled chaotic neural networks , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[3]  Bradley J. Rhodes,et al.  Margin notes: building a contextually aware associative memory , 2000, IUI '00.

[4]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[5]  Shen Furao,et al.  An incremental network for on-line unsupervised classification and topology learning , 2006, Neural Networks.

[6]  Robert M. French,et al.  Using Semi-Distributed Representations to Overcome Catastrophic Forgetting in Connectionist Networks , 1991 .

[7]  Shuzhi Sam Ge,et al.  Face recognition by applying wavelet subband representation and kernel associative memory , 2004, IEEE Transactions on Neural Networks.

[8]  Takashi Omori,et al.  Online map formation and path planning for mobile robot by associative memory with controllable attention , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[9]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[10]  Marc Mézard,et al.  Solvable models of working memories , 1986 .

[11]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[12]  Bernd Fritzke,et al.  A Self-Organizing Network that Can Follow Non-stationary Distributions , 1997, ICANN.

[13]  Aluizio F. R. Araújo,et al.  Two simple strategies to improve bidirectional associative memory's performance: unlearning and delta rule , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[14]  BART KOSKO,et al.  Bidirectional associative memories , 1988, IEEE Trans. Syst. Man Cybern..

[15]  Aluizio F. R. Araújo,et al.  Identification and control of dynamical systems using the self-organizing map , 2004, IEEE Transactions on Neural Networks.

[16]  Masafumi Hagiwara,et al.  Kohonen feature maps as a supervised learning machine , 1993, IEEE International Conference on Neural Networks.

[17]  Stephen R. Marsland,et al.  A self-organising network that grows when required , 2002, Neural Networks.

[18]  Iren Valova,et al.  A parallel growing architecture for self-organizing maps with unsupervised learning , 2005, Neurocomputing.

[19]  Alexander Bisler.  An associative memory for autonomous agents , 2004 .

[20]  Masato Okada,et al.  Sparsely Encoded Associative Memory Model with Forgetting Process , 2002 .

[21]  Heekuck Oh,et al.  Adaptation of the relaxation method for learning in bidirectional associative memory , 1994, IEEE Trans. Neural Networks.

[22]  Masayuki Morisawa,et al.  Sequential learning for associative memory using Kohonen feature map , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[23]  Mohamad H. Hassoun,et al.  Generalizations of the Hamming Associative Memory , 2004, Neural Processing Letters.

[24]  Helge J. Ritter,et al.  An instantaneous topological mapping model for correlated stimuli , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[25]  Shinji Doki,et al.  A New Associative Memory System for Supplemental Learning under Restriction of Memory Capacity , 1999 .

[26]  R. French.  Catastrophic forgetting in connectionist networks , 1999, Trends in Cognitive Sciences.

[27]  Masafumi Hagiwara.  Multidirectional associative memory , 1990 .