Stochastic noise Process enhancement of Hopfield neural networks

Hopfield neural networks (HNN) are a class of densely connected single-layer nonlinear networks of perceptrons. The network's energy function is defined through a learning procedure so that its minima coincide with states from a predefined set. However, because of the network's nonlinearity, a number of undesirable local energy minima emerge from the learning procedure. This has shown to significantly effect the network's performance. In this brief, we present a stochastic process-enhanced binary HNN. Given a fixed network topology, the desired final distribution of states can be reached by modulating the network's stochastic process. We design this process, in a computationally efficient manner, by associating it with stability intervals of the nondesired stable states of the network. Our experimental simulations confirm the predicted improvement in performance.

[1]  Vladimir Pavlovic,et al.  Enhancement of Hopfield neural networks using stochastic noise processes , 2001, Neural Networks for Signal Processing XI: Proceedings of the 2001 IEEE Signal Processing Society Workshop (IEEE Cat. No.01TH8584).

[2]  Dan Schonfeld,et al.  On the hysteresis and robustness of Hopfield neural networks , 1993 .

[3]  S. Abe.  Global convergence and suppression of spurious states of the Hopfield neural networks , 1993 .

[4]  Satoshi Matsuda.  "Optimal" Hopfield network for combinatorial optimization with linear cost function , 1998, IEEE Trans. Neural Networks.

[5]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[6]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[7]  X. Mao,et al.  Stochastic Hopfield neural networks , 2003 .

[8]  Mahesan Niranjan,et al.  A theoretical investigation into the performance of the Hopfield model , 1990, IEEE Trans. Neural Networks.

[9]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[10]  Shun-ichi Amari,et al.  Statistical neurodynamics of associative memory , 1988, Neural Networks.

[11]  Takahiro Inoue,et al.  A New Stochastic Binary Neural Network Based on Hopfield Model and Its Application , 2002 .

[12]  Satoshi Matsuda,et al.  Theoretical considerations on the dynamics of hysteresis binary Hopfield networks for combinatorial optimization , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[13]  Demetri Psaltis,et al.  Linear and logarithmic capacities in associative neural networks , 1989, IEEE Trans. Inf. Theory.