A Recurrent Neural Network for Solving Nonconvex Optimization Problems

An existing recurrent neural network for convex optimization is extended to solve nonconvex optimization problems. One of the prominent features of this neural network is the one-to-one correspondence between its equilibria and the Karush-Kuhn-Tucker (KKT) points of the nonconvex optimization problem. The conditions are derived under which the neural network (locally) converges to the KKT points. It is desired that the neural network is stable at minimum solutions, and unstable at maximum solutions or saddle solutions. It is found in the paper that most likely the neural network is unstable at the maximum solutions. Moreover, we found that if the derived conditions are not satisfied at minimum solutions, by transforming the original problem into an equivalent one with the p-power (or partial p-power) method, these conditions can be satisfied. As a result, the neural network will locally converge to a minimum solution. Finally, two illustrative examples are provided to demonstrate the performance of the recurrent neural network.

[1]  Youshen Xia,et al.  A recurrent neural network for nonlinear convex optimization subject to nonlinear inequality constraints , 2004, IEEE Trans. Circuits Syst. I Regul. Pap..

[2]  Edgar Sanchez-Sinencio,et al.  Nonlinear switched capacitor 'neural' networks for optimization problems , 1990 .

[3]  Duan Li,et al.  Convexification and existence of a saddle point in a pth-power reformulation for nonconvex constrained optimization☆ , 2001 .

[4]  Andreas Fischer,et al.  Modified Wilson's Method for Nonlinear Programs with Nonunique Multipliers , 1999, Math. Oper. Res..

[5]  Duan Li,et al.  Local Convexification of the Lagrangian Function in Nonconvex Optimization , 2000 .

[6]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[7]  Z. K. Xu.  Local Saddle Points and Convexification for Nonconvex Optimization Problems , 1997 .

[8]  Robert Michael Lewis,et al.  On the Local Convergence of Pattern Search , 2003, SIAM J. Optim..

[9]  Jun Wang,et al.  A recurrent neural network for solving nonlinear convex programs subject to linear constraints , 2005, IEEE Transactions on Neural Networks.

[10]  Jun Wang,et al.  A deterministic annealing neural network for convex programming , 1994, Neural Networks.

[11]  Jun Wang,et al.  A recurrent neural network with exponential convergence for solving convex quadratic program and related linear piecewise equations , 2004, Neural Networks.

[12]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[13]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[14]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[15]  David K. Smith,et al.  Mathematical Programming: Theory and Algorithms , 1986 .

[16]  F. Facchinei,et al.  A Simply Constrained Optimization Reformulation of KKT Systems Arising from Variational Inequalities , 1999 .