Cooperative annealing Hopfield network for unconstrained binary quadratic programming problem

Abstract The updating rule of the original discrete Hopfield neural network (DHNN) is based on gradient descent dynamics, which always leads to the local minima problem. In this paper, by introducing the idea of the simulated annealing (SA) into the DHNN, we first propose an annealing HNN (AHNN) that permits temporary energy ascent to help the DHNN escape from local minima. Then, from a cooperative perspective, a population of the AHNN processes are simultaneously implemented and coupled by their acceptance probabilities, and thus a cooperative AHNN (CoAHNN) is proposed. The primary objective of the coupling in the CoAHNN is to create cooperative behavior via information exchange among neural networks. This objective helps in the decision of whether uphill moves will be accepted. In addition, coupling can provide information used online to guide the networks toward the global optimum. The CoAHNN is tested on 21 unconstrained binary quadratic programming problems (UBQP) with the size ranging from 3000 to 7000, and 48 maximum cut benchmark problems, a special case of the UBQP, with the size ranging from 512 to 3375. The UBQP consists in maximizing a quadratic 0–1 function. It is a well known NP-hard problem and is considered as a unified model for a variety of combinatorial optimization problems. Simulation results show that the CoAHNN is better than or competitive with other HNN based algorithms, metaheuristic algorithms and state-of-the-art algorithms.

[1]  Abraham Duarte,et al.  Hybridizing the cross-entropy method: An application to the max-cut problem , 2009, Comput. Oper. Res..

[2]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Minghe Sun,et al.  Tabu Machine: A New Neural Network Solution Approach for Combinatorial Optimization Problems , 2003, J. Heuristics.

[4]  Johan A. K. Suykens,et al.  Coupled Simulated Annealing , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[5]  Jing Liu,et al.  A Multiagent Evolutionary Algorithm for Combinatorial Optimization Problems , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Mark W. Lewis,et al.  Using xQx to model and solve the uncapacitated task allocation problem , 2005, Oper. Res. Lett..

[7]  P. V. Laarhoven,et al.  A quantitative analysis of the simulated annealing algorithm: A case study for the traveling salesman problem , 1988 .

[8]  Fred W. Glover,et al.  An Unconstrained Quadratic Binary Programming Approach to the Vertex Coloring Problem , 2005, Ann. Oper. Res..

[9]  Panos M. Pardalos,et al.  Randomized heuristics for the Max-Cut problem , 2002, Optim. Methods Softw..

[10]  Yunong Zhang,et al.  Competitive Hopfield Network Combined With Estimation of Distribution for Maximum Diversity Problems , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Gintaras Palubeckis,et al.  Iterated Tabu Search for the Unconstrained Binary Quadratic Optimization Problem , 2006, Informatica.

[12]  Mark W. Lewis,et al.  A note on xQx as a modelling and solution framework for the Linear Ordering Problem , 2009 .

[13]  Franz Rendl,et al.  A Spectral Bundle Method for Semidefinite Programming , 1999, SIAM J. Optim..

[14]  Jiahai Wang,et al.  Discrete Hopfield network combined with estimation of distribution for unconstrained binary quadratic programming problem , 2010, Expert Syst. Appl..

[15]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[16]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[17]  Xiaofei Huang The cooperative optimization metaheuristic: inspiration from nature and applications , 2006 .

[18]  Arun Jagota,et al.  Optimizing neural networks on SIMD parallel computers , 2005, Parallel Comput..

[19]  Fred W. Glover,et al.  Solving the maximum edge weight clique problem via unconstrained quadratic programming , 2007, Eur. J. Oper. Res..

[20]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[21]  Jiahai Wang,et al.  Hopfield Neural Network Based on Estimation of Distribution for Two-Page Crossing Number Problem , 2008, IEEE Transactions on Circuits and Systems II: Express Briefs.

[22]  Gintaras Palubeckis,et al.  Multistart Tabu Search Strategies for the Unconstrained Binary Quadratic Optimization Problem , 2004, Ann. Oper. Res..

[23]  Abraham Duarte,et al.  Advanced Scatter Search for the Max-Cut Problem , 2009, INFORMS J. Comput..

[24]  Eduard Babkin,et al.  The analysis of Tabu Machine parameters applied to discrete optimization problems , 2009, 2009 IEEE/ACS International Conference on Computer Systems and Applications.

[25]  Talal M. Alkhamis,et al.  Simulated annealing for the unconstrained quadratic pseudo-Boolean function , 1998, Eur. J. Oper. Res..

[26]  Carsten Peterson,et al.  Local routing algorithms based on Potts neural networks , 2000, IEEE Trans. Neural Networks Learn. Syst..

[27]  Tommy W. S. Chow,et al.  Self-Organizing and Self-Evolving Neurons: A New Neural Network for Optimization , 2007, IEEE Transactions on Neural Networks.

[28]  P. Merz,et al.  Memetic algorithms for the unconstrained binary quadratic programming problem. , 2004, Bio Systems.

[29]  Fred W. Glover,et al.  A unified modeling and solution framework for combinatorial optimization problems , 2004, OR Spectr..

[30]  Yin Zhang,et al.  Rank-Two Relaxation Heuristics for MAX-CUT and Other Binary Quadratic Programs , 2002, SIAM J. Optim..