Analysis on Bidirectional Associative Memories with Multiplicative Weight Noise

In neural networks, network faults can be exhibited in different forms, such as node fault and weight fault. One kind of weight faults is due to the hardware or software precision. This kind of weight faults can be modelled as multiplicative weight noise. This paper analyzes the capacity of a bidirectional associative memory (BAM) affected by multiplicative weight noise. Assuming that weights are corrupted by multiplicative noise, we study how many number of pattern pairs can be stored as fixed points. Since capacity is not meaningful without considering the error correction capability, we also present the capacity of a BAM with multiplicative noise when there are some errors in the input pattern. Simulation results have been carried out to confirm our derivations.

[1]  Teuvo Kohonen,et al.  Correlation Matrix Memories , 1972, IEEE Transactions on Computers.

[2]  G. Vachtsevanos,et al.  Storage capacity of bidirectional associative memories , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[3]  S.-i. Amari.  Statistical neurodynamics of various versions of correlation associative memory , 1988, IEEE 1988 International Conference on Neural Networks.

[4]  E.E. Swartzlander,et al.  Digital neural network implementation , 1992, Eleventh Annual International Phoenix Conference on Computers and Communication [1992 Conference Proceedings].

[5]  G. Palm,et al.  On associative memory , 2004, Biological Cybernetics.

[6]  Julio Ortega Lopera,et al.  Assessing the Noise Immunity and Generalization of Radial Basis Function Networks , 2004, Neural Processing Letters.

[7]  Robert Hecht-Nielsen,et al.  A BAM with increased information storage capacity , 1988, IEEE 1988 International Conference on Neural Networks.

[8]  Ignacio Rojas,et al.  An Accurate Measure for Multilayer Perceptron Tolerance to Weight Deviations , 1999, Neural Processing Letters.

[9]  Jenq-Neng Hwang,et al.  Finite Precision Error Analysis of Neural Network Hardware Implementations , 1993, IEEE Trans. Computers.

[10]  Andrew Chi-Sing Leung,et al.  Noise-resistant fitting for spherical harmonics , 2006, IEEE Transactions on Visualization and Computer Graphics.

[11]  A. Sripad,et al.  Quantization errors in floating-point arithmetic , 1978 .

[12]  C. S. Leung.  Encoding method for bidirectional associative memory using projection on convex sets , 1993, IEEE Trans. Neural Networks.

[13]  Jose B. Cruz,et al.  Encoding strategy for maximum noise tolerance bidirectional associative memory , 2005, IEEE Transactions on Neural Networks.

[14]  Andrew Chi-Sing Leung,et al.  Stability and statistical properties of second-order bidirectional associative memory , 1997, IEEE Trans. Neural Networks.

[15]  Chi-Sing Leung.  Optimum learning for bidirectional associative memory in the sense of capacity , 1994 .

[16]  Burkhard Lenze,et al.  Improving Leung's bidirectional learning rule for associative memories , 2001, IEEE Trans. Neural Networks.

[17]  BART KOSKO,et al.  Bidirectional associative memories , 1988, IEEE Trans. Syst. Man Cybern..

[18]  Andrew Chi-Sing Leung,et al.  The Behavior of Forgetting Learning in Bidirectional Associative Memory , 1997, Neural Computation.

[19]  Jose B. Cruz,et al.  Two coding strategies for bidirectional associative memory , 1990, IEEE Trans. Neural Networks.

[20]  Veljko Milutinovic,et al.  Neural Networks: Concepts, Applications, and Implementations , 1991 .

[21]  ImplementationsJames B. BurrDepartment.  Digital Neural Network Implementations , 1995 .