Neural Networks for Combinatorial Optimization: A Review of More Than a Decade of Research

It has been over a decade since neural networks were first applied to solve combinatorial optimization problems. During this period, enthusiasm has been erratic as new approaches are developed and (sometimes years later) their limitations are realized. This article briefly summarizes the work that has been done and presents the current standing of neural networks for combinatorial optimization by considering each of the major classes of combinatorial optimization problems. Areas which have not yet been studied are identified for future research.

[1]  Tevfik Bultan,et al.  Circuit partitioning using parallel mean field annealing algorithms , 1991, Proceedings of the Third IEEE Symposium on Parallel and Distributed Processing.

[2]  Norio Shiratori,et al.  Neural networks for solving constrained Steiner tree problem , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[3]  C. H. Dagli,et al.  Possible applications of neural networks in manufacturing , 1989, International 1989 Joint Conference on Neural Networks.

[4]  Kotaro Hirasawa,et al.  Solving inequality constrained combinatorial optimization problems by the hopfield neural networks , 1992, Neural Networks.

[5]  H. Nozawa,et al.  Solution of the optimization problem using the neural network model as a globally coupled map , 1994 .

[6]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[7]  Arun Jagota,et al.  Approximating maximum clique with a Hopfield network , 1995, IEEE Trans. Neural Networks.

[8]  M. O. Berger,et al.  k-coloring vertices using a neural network with convergence to valid solutions , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[9]  David E. van den Bout,et al.  A traveling salesman objective function that works , 1988, IEEE 1988 International Conference on Neural Networks.

[10]  Paul Juell,et al.  Neural Networks for Selective Vehicle Routing Heuristics , 1990, INFORMS J. Comput..

[11]  Edward Tsang,et al.  Solving constraint satisfaction problems using neural networks , 1991 .

[12]  Yannis Tsividis,et al.  A reconfigurable VLSI neural network , 1992 .

[13]  Jean-Yves Potvin,et al.  Clustering for vehicle routing with a competitive neural network , 1995, Neurocomputing.

[14]  H. N. Schaller Problem solving by global optimization: the rolling-stone neural network , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[15]  B. Kamgar-Parsi Dynamical stability and parameter selection in neural optimization , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[16]  G. G. Coghill,et al.  Genetic breeding of control parameters for the Hopfield/Tank neural net , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[17]  Brian W. Kernighan,et al.  An efficient heuristic procedure for partitioning graphs , 1970, Bell Syst. Tech. J..

[18]  W.S. Wong,et al.  An elastic net solution to obstacle avoidance tour planning , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[19]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[20]  Amiram Gafni,et al.  Stable solutions vs. multiplicative utility solutions for the assignment problem , 1988 .

[21]  Hayakawa,et al.  Effects of the chaotic noise on the performance of a neural network model for optimization problems. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[22]  John N. Hooker,et al.  Testing heuristics: We have it all wrong , 1995, J. Heuristics.

[23]  Jean-Yves Potvin,et al.  THE TRAVELING SALESMAN PROBLEM: A NEURAL NETWORK PERSPECTIVE , 1993 .

[24]  Nesa L'abbe Wu,et al.  Linear programming and extensions , 1981 .

[25]  Yasuo Matsuyama,et al.  Self-organization via competition, cooperation and categorization applied to extended vehicle routing problems , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[26]  Michel Gendreau,et al.  A Hopfield-Tank Neural Network Model for the Generalized Traveling Salesman Problem , 1999 .

[27]  S. Grossberg,et al.  ART 2: self-organization of stable category recognition codes for analog input patterns. , 1987, Applied optics.

[28]  Akio Ogihara,et al.  Asymmetric Neural Network and Its Application to Knapsack Problem , 1995 .

[29]  H. Szu,et al.  Solving large-scale optimization problems by divide-and-conquer neural networks , 1989, International 1989 Joint Conference on Neural Networks.

[30]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[31]  Matevž Kovačič,et al.  Timetable construction with Markovian neural network , 1993 .

[32]  Marimuthu Palaniswami,et al.  Traditional heuristic versus Hopfield neural network approaches to a car sequencing problem , 1996 .

[33]  Yoshiyasu Takefuji,et al.  Integer linear programming neural networks for job-shop scheduling , 1988, IEEE 1988 International Conference on Neural Networks.

[34]  M. N. Murty,et al.  Connectionist approach for clustering , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[35]  Yoshiyasu Takefuji,et al.  Artificial neural networks for four-coloring map problems and K-colorability problems , 1991 .

[36]  Sun-Gi Hong,et al.  The minimum cost path finding algorithm using a Hopfield type neural network , 1995, Proceedings of 1995 IEEE International Conference on Fuzzy Systems..

[37]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[38]  M. Furst,et al.  Neural net implementation for assigning a product to a production line , 1989, International 1989 Joint Conference on Neural Networks.

[39]  Jarosław Arabas,et al.  A genetic approach to the Hopfield neural network in the optimization problems , 1994 .

[40]  Raymond Lister Annealing networks and fractal landscapes , 1993, IEEE International Conference on Neural Networks.

[41]  Naleih M. Botros,et al.  Hardware implementation of an artificial neural network using field programmable gate arrays (FPGA's) , 1994, IEEE Trans. Ind. Electron..

[42]  Mauricio G. C. Resende,et al.  Designing and reporting on computational experiments with heuristic methods , 1995, J. Heuristics.

[43]  Vangelis Th. Paschos,et al.  On the Approximation of NP-Complete Problems by Using the Boltzmann Machine Method: The Cases of Some Covering and Packing Problems , 1991, IEEE Trans. Computers.

[44]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[45]  Ramesh Sharda,et al.  Neural Networks for the MS/OR Analyst: An Application Bibliography , 1994 .

[46]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[47]  Fred W. Glover,et al.  Future paths for integer programming and links to artificial intelligence , 1986, Comput. Oper. Res..

[48]  Brian W. Kernighan,et al.  An Effective Heuristic Algorithm for the Traveling-Salesman Problem , 1973, Oper. Res..

[49]  Charles L. Britton,et al.  Neural network models for Linear Programming , 1989 .

[50]  武藤 佳恭 Neural network parallel computing , 1992 .

[51]  Carsten Peterson,et al.  Neural Networks for Optimization Problems with Inequality Constraints: The Knapsack Problem , 1993, Neural Computation.

[52]  David E. van den Bout,et al.  Graph partitioning using annealed neural networks , 1990, International 1989 Joint Conference on Neural Networks.

[53]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[54]  Badrinath Roysam,et al.  Hierarchically structured unit-simplex transformations for parallel distributed optimization problems , 1992, IEEE Trans. Neural Networks.

[55]  Qiwen Wang,et al.  Using artificial neural networks to solve the orienteering problem , 1995, Ann. Oper. Res..

[56]  Mathukumalli Vidyasagar,et al.  Location and stability of the high-gain equilibria of nonlinear neural networks , 1993, IEEE Trans. Neural Networks.

[57]  Laveen N. Kanal,et al.  Asymmetric mean-field neural networks for multiprocessor scheduling , 1992, Neural Networks.

[58]  Daniel L. Palumbo,et al.  Performance and fault-tolerance of neural networks for optimization , 1993, IEEE Trans. Neural Networks.

[59]  Jeong-Ho Park,et al.  Lower bounds of annealing schedule for Boltzmann and Cauchy machines , 1989, International 1989 Joint Conference on Neural Networks.

[60]  Jun Wang,et al.  A recurrent neural network for solving the shortest path problem , 1994, Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94.

[61]  J. Lo A new approach to global optimization and its applications to neural networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[62]  Y. Takefuji,et al.  Design of parallel distributed Cauchy machines , 1989, International 1989 Joint Conference on Neural Networks.

[63]  Pascal Van Hentenryck Constraint satisfaction in logic programming , 1989, Logic programming.

[64]  Marimuthu Palaniswami,et al.  Comparison of Heuristic Algorithms for the Degree Constrained Minimum Spanning Tree , 1996 .

[65]  Cihan H. Dagli,et al.  Genetic neuro-scheduler for job shop scheduling , 1993 .

[66]  Hassan Ghaziri,et al.  Supervision in the Self-Organizing Feature Map: Application to the Vehicle Routing Problem , 1996 .

[67]  Yoshiyasu Takefuji,et al.  A neural network parallel algorithm for channel assignment problems in cellular radio networks , 1992 .

[68]  Marimuthu Palaniswami,et al.  A hybrid neural approach to combinatorial optimization , 1996, Comput. Oper. Res..

[69]  Marimuthu Palaniswami,et al.  Static and Dynamic Channel Assignment Using Neural Networks , 1997, IEEE J. Sel. Areas Commun..

[70]  Mitsuo Gen,et al.  Neural network approach for general assignment problem , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[71]  Y. Takefuji,et al.  An O(1) approximate parallel algorithm for the n-task-n-person assignment problem , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[72]  Tamotsu Kasai,et al.  An efficient algorithm for the linear assignment problem , 1990 .

[73]  Sukhan Lee,et al.  Dual-mode dynamics neural network (D2NN) for knapsack packing problem , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[74]  Gilbert Laporte,et al.  Metaheuristics: A bibliography , 1996, Ann. Oper. Res..

[75]  Sreeram Viswanath Balakrishnan,et al.  Solving combinatorial optimization problems using neural networks with applications in speech recognition , 1992 .

[76]  Robert Hecht-Nielsen,et al.  Applications of counterpropagation networks , 1988, Neural Networks.

[77]  Kate Smith-Miles An argument for abandoning the travelling salesman problem as a neural-network benchmark , 1996, IEEE Trans. Neural Networks.

[78]  Chung-Kuan Cheng,et al.  Ratio cut partitioning for hierarchical designs , 1991, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[79]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[80]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[81]  Carsten Peterson,et al.  "Teachers and Classes" with Neural Networks , 1991, Int. J. Neural Syst..

[82]  David E. van den Bout,et al.  Encoding logical constraints into neural network cost functions , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[83]  Chee-Kit Looi,et al.  Neural network methods in combinatorial optimization , 1992, Comput. Oper. Res..

[84]  Richard W. Prager,et al.  Limitations of neural networks for solving traveling salesman problems , 1995, IEEE Trans. Neural Networks.

[85]  Shaw K. Chen,et al.  The comparative ability of self-organizing neural networks to define cluster structure , 1995 .

[86]  Nirwan Ansari,et al.  Scheduling computation tasks onto a multiprocessor system by mean field annealing of a Hopfield neural network , 1993 .

[87]  James R. Burns,et al.  An intelligent algorithm for mixed-integer programming models , 1992, Comput. Oper. Res..

[88]  Michel Verleysen,et al.  An analog VLSI implementation of Hopfield's neural network , 1989, IEEE Micro.

[89]  Thomas Jackson,et al.  Neural Computing - An Introduction , 1990 .

[90]  Yoshiyasu Takefuji,et al.  Constrained optimization with use of two-dimensional maximum neurons , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[91]  Y. Akiyama,et al.  Combinatorial optimization with Gaussian machines , 1989, International 1989 Joint Conference on Neural Networks.

[92]  Tomoharu Takada,et al.  A neural network system for solving an assortment problem in the steel industry , 1995, Ann. Oper. Res..

[93]  J.G. Delgado-Frias,et al.  Impact of energy function on a neural network model for optimization problems , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[94]  E. Kay,et al.  Graph Theory. An Algorithmic Approach , 1975 .

[95]  J. Ramanujam,et al.  Mapping combinatorial optimization problems onto neural networks , 1995, Inf. Sci..

[96]  C. P. Ravikumar,et al.  Heuristic and neural algorithms for mapping tasks to a reconfigurable array , 1995, Microprocess. Microprogramming.

[97]  Bing J. Sheu,et al.  Design of a neural-based A/D converter using modified Hopfield network , 1989 .

[98]  Kazuyuki Aihara,et al.  Chaotic simulated annealing by a neural network model with transient chaos , 1995, Neural Networks.

[99]  John J. Bartholdi,et al.  Spacefilling curves and the planar travelling salesman problem , 1989, JACM.

[100]  James P. Ignizio,et al.  A stochastic neural network for resource constrained scheduling , 1992, Comput. Oper. Res..

[101]  Tao Li,et al.  Design of Competition-Based Neural Networks for Combinatorial Optimization , 1990, Int. J. Neural Syst..

[102]  Rema Padman Choosing Solvers in Decision Support Systems: A Neural Network Application in Resource-Constrained Project Scheduling , 1993 .

[103]  Bozena Kaminska,et al.  Penalty formulation for 0-1 linear programming problem: a neural network approach , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[104]  Pong P. Chu A neural network for solving optimization problems with linear equality constraints , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[105]  Loren Paul Rees,et al.  Using Neural Networks to Determine Internally-Set Due-Date Assignments for Shop Scheduling* , 1994 .

[106]  D. Hammerstrom,et al.  A VLSI architecture for high-performance, low-cost, on-chip learning , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[107]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[108]  William B. Levy,et al.  Determination of parameters in a Hopfield/Tank computational network , 1988, IEEE 1988 International Conference on Neural Networks.

[109]  C. Gaspin Automatic translation of constraints for solving optimization problems by neural networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[110]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[111]  Kate A. Smith Solving the generalised quadratic assignment problem using a self-organising process , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[112]  Alan F. Murray,et al.  International Joint Conference on Neural Networks , 1993 .

[113]  Carsten Peterson,et al.  Neural Networks and NP-complete Optimization Problems; A Performance Study on the Graph Bisection Problem , 1988, Complex Syst..

[114]  D. R. Collins,et al.  Considerations for neural network hardware implementations , 1989, IEEE International Symposium on Circuits and Systems,.

[115]  H. Nonaka,et al.  Sub-optimal solution screening in optimization by neural networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[116]  G. W. Davis,et al.  Sensitivity analysis in neural net solutions , 1989, IEEE Trans. Syst. Man Cybern..

[117]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[118]  Tharam S. Dillon,et al.  Parallel distributed implementation of the shortest path algorithm , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[119]  K. Urahama,et al.  Neural algorithms for placement problems , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[120]  Kazuo Kyuma,et al.  10000 cell placement optimization using a self-organizing map , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[121]  Marimuthu Palaniswami,et al.  Neural versus traditional approaches to the location of interacting hub facilities , 1996 .

[122]  Jun Wang Analogue neural network for solving the assignment problem , 1992 .

[123]  Carsten Peterson,et al.  A New Method for Mapping Optimization Problems Onto Neural Networks , 1989, Int. J. Neural Syst..

[124]  Eitan Wacholder,et al.  A Neural Network-Based Optimization Algorithm for the Static Weapon-Target Assignment Problem , 1989, INFORMS J. Comput..

[125]  Masayoshi Inoue,et al.  Solving an Optimization Problem with a Chaos Neural Network , 1992 .

[126]  Ehl Emile Aarts,et al.  Statistical cooling : a general approach to combinatorial optimization problems , 1985 .

[127]  Hahn-Ming Lee,et al.  Neural network processing through energy minimization with learning ability to the multiconstraint zero-one knapsack problem , 1989, [Proceedings 1989] IEEE International Workshop on Tools for Artificial Intelligence.

[128]  D. Kunz,et al.  Channel assignment for cellular radio using neural networks , 1991 .

[129]  L. Raffo,et al.  Neural algorithms for cell placement in VLSI design , 1989, International 1989 Joint Conference on Neural Networks.

[130]  Sebastián Lozano,et al.  Machine Grouping in Cellular Manufacturing: A Self Organizing Neural Network , 1998 .

[131]  Mahesan Niranjan,et al.  A theoretical investigation into the performance of the Hopfield model , 1990, IEEE Trans. Neural Networks.

[132]  Ian F. Croall,et al.  Industrial Applications of Neural Networks , 1992, Research Reports ESPRIT.

[133]  J. D. Carothers,et al.  Graph color minimization using neural networks , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[134]  Yuichiro Anzai,et al.  Solving large scale puzzles with neural networks , 1989, [Proceedings 1989] IEEE International Workshop on Tools for Artificial Intelligence.

[135]  Sy-Yen Kuo,et al.  Neural networks for optimization problems in graph theory , 1994, Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94.

[136]  F. Glover,et al.  In Modern Heuristic Techniques for Combinatorial Problems , 1993 .

[137]  John B. Moore,et al.  Application of an Annealed Neural Network to a Timetabling Problem , 1996, INFORMS J. Comput..

[138]  Young Hoon Lee,et al.  A hybrid approach to sequencing jobs using heuristic rules and neural networks , 1995 .

[139]  J. Kruskal On the shortest spanning subtree of a graph and the traveling salesman problem , 1956 .

[140]  S.C.A. Thomopoulos,et al.  Neural network implementation of the shortest path algorithm for traffic routing in communication networks , 1989, International 1989 Joint Conference on Neural Networks.

[141]  Yoshiyasu Takefuji,et al.  Optimization neural networks for the segmentation of magnetic resonance images , 1992, IEEE Trans. Medical Imaging.

[142]  Fred W. Glover,et al.  A user's guide to tabu search , 1993, Ann. Oper. Res..

[143]  Bruce L. Golden,et al.  Solving vehicle routing problems using elastic nets , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[144]  K. Schulten,et al.  Kohonen's self-organizing maps: exploring their computational capabilities , 1988, IEEE 1988 International Conference on Neural Networks.

[145]  Mark D. Johnston,et al.  A discrete stochastic neural network algorithm for constraint satisfaction problems , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[146]  Richard Durbin,et al.  An analogue approach to the travelling salesman problem using an elastic net method , 1987, Nature.

[147]  X. Xu,et al.  Effective neural algorithms for the traveling salesman problem , 1991, Neural Networks.

[148]  Bruce L. Golden,et al.  A hierarchical strategy for solving traveling salesman problems using elastic nets , 1995, J. Heuristics.

[149]  J. D. Carothers,et al.  A Hopfield neural network solution to the TCM partitioning problem , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).