Bidirectional associative memory with learning capability using simultaneous perturbation

Bidirectional associative memory (BAM) is a typical recurrent network. It consists of two layers and can realize the hetero-associative memory in which recalled patterns are different from triggering patterns. Ordinarily, weights in the BAM are determined by Hebbian learning or the correlation learning for binary problems. In order to promote wider range of applications of the BAMs, it is crucial to invent new learning scheme which is applicable not only to the binary problems but also to analog ones. Moreover, hardware implementation of the BAMs with learning capability is intriguing. In this paper, a recursive learning scheme for the BAMs using the simultaneous perturbation is described. Moreover, its hardware realization using the FPGA is explained. Some results and the details of the realization are shown.

[1]  Johan A. K. Suykens,et al.  Artificial Neural Networks for Modeling and Control of Non-Linear Systems , 1995 .

[2]  Yutaka Maeda,et al.  A learning rule of neural networks via simultaneous perturbation and its hardware implementation , 1995, Neural Networks.

[3]  Gert Cauwenberghs,et al.  An analog VLSI recurrent neural network learning a continuous-time trajectory , 1996, IEEE Trans. Neural Networks.

[4]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[5]  Xue-Zhong He,et al.  Delay-independent stability in bidirectional associative memory networks , 1994, IEEE Trans. Neural Networks.

[6]  Yutaka Maeda,et al.  Learning Rules for Recurrent Neural Networks using Perturbation and their Application to Neuro-Control , 1993 .

[7]  Yutaka Maeda,et al.  FPGA implementation of a pulse density neural network with learning ability using simultaneous perturbation , 2003, IEEE Trans. Neural Networks.

[8]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[9]  BART KOSKO,et al.  Bidirectional associative memories , 1988, IEEE Trans. Syst. Man Cybern..

[10]  Bart Kosko,et al.  Neural networks and fuzzy systems , 1998 .

[11]  Laurene V. Fausett,et al.  Fundamentals Of Neural Networks , 1994 .

[12]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[13]  Johan A. K. Suykens,et al.  NLq theory: checking and imposing stability of recurrent neural networks for nonlinear modeling , 1997, IEEE Trans. Signal Process..

[14]  Jose B. Cruz,et al.  Guaranteed recall of all training pairs for bidirectional associative memory , 1991, IEEE Trans. Neural Networks.

[15]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[16]  Laurene V. Fausett,et al.  Fundamentals Of Neural Networks , 1993 .

[17]  Gert Cauwenberghs,et al.  A Fast Stochastic Error-Descent Algorithm for Supervised Learning and Optimization , 1992, NIPS.

[18]  Johan A. K. Suykens,et al.  Toward CNN chip-specific robustness , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[19]  Heekuck Oh,et al.  Adaptation of the relaxation method for learning in bidirectional associative memory , 1994, IEEE Trans. Neural Networks.

[20]  J. Spall.  A Stochastic Approximation Technique for Generating Maximum Likelihood Parameter Estimates , 1987, 1987 American Control Conference.

[21]  Yutaka Maeda,et al.  A Learning Rule of Neural Networks for Neuro-controller , 2000 .

[22]  Rui J. P. de Figueiredo,et al.  Learning rules for neuro-controller via simultaneous perturbation , 1997, IEEE Trans. Neural Networks.

[23]  Ron Meir,et al.  A Parallel Gradient Descent Method for Learning in Analog VLSI Neural Networks , 1992, NIPS.

[24]  Atsushi Nakazawa,et al.  Hardware Implementation of a Pulse Density Neural Network Using Simultaneous Perturbation Learning Rule , 1999 .

[25]  Johan A. K. Suykens,et al.  Artificial neural networks for modelling and control of non-linear systems , 1995 .