The hysteretic Hopfield neural network

Several neuron activation functions have been proposed (e.g., linear, binary, sigmoid) for recurrent and multilayer artificial neural networks. In this paper we present a hysteretic neuron activation function for optimization and learning. We prove Lyapunov stability of a hysteretic Hopfield neural network, and then solve a combinatorial optimization problem (i.e., the N-queen problem) using this network. We demonstrate the advantages of hysteresis by showing increased frequency of convergence to a solution, when the parameters associated with the activation function are varied.

[1]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[2]  Ellis Horowitz,et al.  Divide-and-Conquer for Parallel Processing , 1983, IEEE Transactions on Computers.

[3]  Robert M. Haralick,et al.  Increasing Tree Search Efficiency for Constraint Satisfaction Problems , 1979, Artif. Intell..

[4]  D. Pozar,et al.  Electromagnetic concepts and applications , 1983, IEEE Antennas and Propagation Society Newsletter.

[5]  Laxmikant V. Kalé,et al.  An Almost Perfect Heuristic for the N Nonattacking Queens Problem , 1990, Inf. Process. Lett..

[6]  Edmond Schonberg,et al.  Programming with Sets: An Introduction to SETL , 1986 .

[7]  Yasuji Sawada,et al.  Associative memory network composed of neurons with hysteretic property , 1990, Neural Networks.

[8]  Udi Manber,et al.  DIB—a distributed implementation of backtracking , 1987, TOPL.

[9]  A. Visintin.  Differential models of hysteresis , 1994 .

[10]  Alan V. Oppenheim,et al.  Discrete-Time Signal Pro-cessing , 1989 .

[11]  Ellis Horowitz,et al.  Fundamentals of Computer Algorithms , 1978 .

[12]  Vito Volterra,et al.  Theory of Functionals and of Integral and Integro-Differential Equations , 2005 .

[13]  Robert W. Floyd,et al.  Nondeterministic Algorithms , 1967, JACM.

[14]  Daniel P. Friedman,et al.  Coordinated computing: tools and techniques for distributed software , 1984 .

[15]  Ashok K. Chandra,et al.  Independent Permutations, as Related to a Problem of Moser and a Theorem of Pólya , 1974, J. Comb. Theory, Ser. A.

[16]  Leon O. Chua,et al.  A Generalized Hysteresis Model , 1972 .

[17]  Y. Akiyama,et al.  Combinatorial optimization with Gaussian machines , 1989, International 1989 Joint Conference on Neural Networks.

[19]  Yoshiyasu Takefuji,et al.  Neural network parallel computing , 1992, The Kluwer international series in engineering and computer science.

[20]  J. Keeler,et al.  Noise in neural networks: thresholds, hysteresis, and neuromodulation of signal-to-noise. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[21]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[22]  R. M. Alexander,et al.  Optimization and gaits in the locomotion of vertebrates. , 1989, Physiological reviews.

[23]  T. Balendra.  Vibration of Buildings to Wind and Earthquake Loads , 1993 .

[24]  John J. Hopfield,et al.  Neurons, Dynamics and Computation , 1994 .

[25]  Niklaus Wirth,et al.  Algorithms + Data Structures = Programs , 1976 .

[26]  Janice M. Stone,et al.  Efficient Search Techniques - An Empirical Study of the N-Queens Problem , 1987, IBM J. Res. Dev..

[27]  J. Gaschnig.  Performance measurement and analysis of certain search algorithms. , 1979 .

[28]  Edward M. Reingold,et al.  Backtrack programming techniques , 1975, CACM.

[29]  E. Napieralska Juszczak,et al.  Modelling of losses due to eddy currents and hysteresis in converter transformer cores during failure , 1995 .

[30]  W. Brogan.  Modern Control Theory , 1971 .

[31]  G. Biorci,et al.  Hysteresis losses along open transformations , 1959 .

[32]  Moti Yung,et al.  Divide and Conquer under Global Constraints: A Solution to the N-Queens Problem , 1989, J. Parallel Distributed Comput..

[33]  R. Didday.  A model of visuomotor mechanisms in the frog optic tectum , 1976 .

[34]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.