Chaos in the discretized analog Hopfield neural network and potential applications to optimization

We consider the discretization of the analog Hopfield neural network (DAHNN) using Euler approximation. We suggest an alternative approach to chaotic simulated annealing using the discretizing time-step /spl Delta/t as the bifurcation parameter, because the DAHNN is chaotic when the time-step /spl Delta/t is chosen to be sufficiently large and stabilization is guaranteed when the time-step /spl Delta/t is small enough. It is not necessary to carefully choose other system parameters to assure minimization of Hopfield energy function and network convergence. We argue that this approach should find significant applications in solving combinatorial optimization problems with neural networks.

[1]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Lipo Wang.  Discrete-time convergence theory and updating rules for neural networks with energy functions , 1997, IEEE Trans. Neural Networks.

[3]  Pascal Koiran.  Dynamics of Discrete Time, Continuous State Hopfield Networks , 1994, Neural Computation.

[4]  Shigetoshi Nara,et al.  Memory search using complex dynamics in a recurrent neural network model , 1993, Neural Networks.

[5]  Kazuyuki Aihara,et al.  Chaotic simulated annealing by a neural network model with transient chaos , 1995, Neural Networks.

[6]  Lester Ingber,et al.  Simulated annealing: Practice versus theory , 1993 .

[7]  Marimuthu Palaniswami,et al.  A hybrid neural approach to combinatorial optimization , 1996, Comput. Oper. Res..

[8]  K. Aihara,et al.  Chaotic neural networks , 1990 .

[9]  Lipo Wang,et al.  Noise injection into inputs in sparsely connected Hopfield and winner-take-all neural networks , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[10]  Hiroshi Nozawa,et al.  A neural network model as a globally coupled map and applications based on chaos. , 1992, Chaos.

[11]  L. Wang,et al.  Oscillations and chaos in neural networks: an exactly solvable model. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[12]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[13]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[14]  Lipo Wang.  Suppressing chaos with hysteresis in a higher order neural network , 1996 .

[15]  S. Yoshizawa,et al.  An Active Pulse Transmission Line Simulating Nerve Axon , 1962, Proceedings of the IRE.

[16]  Marimuthu Palaniswami,et al.  Static and Dynamic Channel Assignment Using Neural Networks , 1997, IEEE J. Sel. Areas Commun..

[17]  Eric Goles Ch.,et al.  Energy Functions in Neural Networks with Continuous Local Functions , 1989, Complex Systems.

[18]  K. Aihara,et al.  12. Chaotic oscillations and bifurcations in squid giant axons , 1986 .

[19]  Lipo Wang,et al.  On the dynamics of discrete-time, continuous-state Hopfield neural networks , 1998 .

[20]  L. Wang,et al.  Synchronous neural networks of nonlinear threshold elements with hysteresis. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[21]  Wang,et al.  Chaos, multiplicity, crisis, and synchronicity in higher-order neural networks. , 1991, Physical review. A, Atomic, molecular, and optical physics.

[22]  H. Nozawa,et al.  Solution of the optimization problem using the neural network model as a globally coupled map , 1994 .

[23]  Lipo Wang,et al.  Chaotic simulated annealing with augmented Lagrange for solving combinatorial optimization problems , 2000, 2000 26th Annual Conference of the IEEE Industrial Electronics Society. IECON 2000. 2000 IEEE International Conference on Industrial Electronics, Control and Instrumentation. 21st Century Technologies.

[24]  Lipo Wang,et al.  Oscillatory and chaotic dynamics in neural networks under varying operating conditions , 1996, IEEE Trans. Neural Networks.

[25]  DeLiang Wang,et al.  Incremental learning of complex temporal patterns , 1996, IEEE Trans. Neural Networks.

[26]  Marimuthu Palaniswami,et al.  Traditional heuristic versus Hopfield neural network approaches to a car sequencing problem , 1996 .

[27]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.