Finite-time recurrent neural networks for solving nonlinear optimization problems and their application

This paper focuses on finite-time recurrent neural networks with continuous but non-smooth activation function solving nonlinearly constrained optimization problems. Firstly, definition of finite-time stability and finite-time convergence criteria are reviewed. Secondly, a finite-time recurrent neural network is proposed to solve the nonlinear optimization problem. It is shown that the proposed recurrent neural network is globally finite-time stable under the condition that the Hessian matrix of the associated Lagrangian function is positive definite. Its output converges to a minimum solution globally and finite-time, which means that the actual minimum solution can be derived in finite-time period. In addition, our recurrent neural network is applied to a hydrothermal scheduling problem. Compared with other methods, a lower consumption scheme can be derived in finite-time interval. At last, numerical simulations demonstrate the superiority and effectiveness of our proposed neural networks by solving nonlinear optimization problems with inequality constraints.

[1]  Jun Wang,et al.  A projection neural network and its application to constrained optimization problems , 2002 .

[2]  Mauro Forti,et al.  Convergence of Neural Networks for Programming Problems via a Nonsmooth Łojasiewicz Inequality , 2006, IEEE Transactions on Neural Networks.

[3]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[4]  N. G. Parke,et al.  Ordinary Differential Equations. , 1958 .

[5]  Jun Wang,et al.  A recurrent neural network for solving nonlinear convex programs subject to linear constraints , 2005, IEEE Transactions on Neural Networks.

[6]  Guodong Zhang,et al.  Exponential synchronization of delayed memristor-based chaotic neural networks via periodically intermittent control , 2014, Neural Networks.

[7]  F. Facchinei,et al.  A Simply Constrained Optimization Reformulation of KKT Systems Arising from Variational Inequalities , 1999 .

[8]  Yan-Wu Wang,et al.  Global Synchronization of Complex Dynamical Networks Through Digital Communication With Limited Data Rate , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Xing-Bao Gao,et al.  A novel neural network for nonlinear convex programming , 2004, IEEE Trans. Neural Networks.

[10]  P. Luh,et al.  Nonlinear approximation method in Lagrangian relaxation-based algorithms for hydrothermal scheduling , 1995 .

[12]  X. Xia,et al.  Semi-global finite-time observers for nonlinear systems , 2008, Autom..

[13]  Xiaohua Xia,et al.  Finite time dual neural networks with a tunable activation function for solving quadratic programming problems and its application , 2014, Neurocomputing.

[14]  D. A. Beyer,et al.  Tabu learning: a neural network search method for solving nonconvex optimization problems , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[15]  Long Cheng,et al.  Recurrent Neural Network for Non-Smooth Convex Optimization Problems With Application to the Identification of Genetic Regulatory Networks , 2011, IEEE Transactions on Neural Networks.

[16]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[17]  Wei Bian,et al.  Subgradient-Based Neural Networks for Nonsmooth Nonconvex Optimization Problems , 2009, IEEE Transactions on Neural Networks.

[18]  Walter Engevald Lillo.  Solving constrained optimization problems with neural networks , 1992 .

[19]  Lorenz T. Biegler,et al.  Global and Local Convergence of Line Search Filter Methods for Nonlinear Programming , 2002 .

[20]  Xiaoping Wang,et al.  A Novel Design for Memristor-Based Logic Switch and Crossbar Circuits , 2015, IEEE Transactions on Circuits and Systems I: Regular Papers.

[21]  Abdesselam Bouzerdoum,et al.  Neural network for quadratic optimization with bound constraints , 1993, IEEE Trans. Neural Networks.

[22]  R. Naresh,et al.  Two-phase neural network based solution technique for short term hydrothermal scheduling , 1999 .

[23]  Guodong Zhang,et al.  Exponential Stabilization of Memristor-based Chaotic Neural Networks with Time-Varying Delays via Intermittent Control , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Youshen Xia,et al.  A recurrent neural network for nonlinear convex optimization subject to nonlinear inequality constraints , 2004, IEEE Trans. Circuits Syst. I Regul. Pap..

[25]  Kwong-Sak Leung,et al.  A Neural Network for Solving Nonlinear Programming Problems , 2002, Neural Computing & Applications.

[26]  D.A. Beyer,et al.  Tabu learning: a neural network search method for solving nonconvex optimization problems , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[27]  Yan-Wu Wang,et al.  Solving time-varying quadratic programs based on finite-time Zhang neural networks and their application to robot tracking , 2014, Neural Computing and Applications.

[28]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[29]  Dennis S. Bernstein,et al.  Finite-Time Stability of Continuous Autonomous Systems , 2000, SIAM J. Control. Optim..

[30]  Edgar Sanchez-Sinencio,et al.  Nonlinear switched capacitor 'neural' networks for optimization problems , 1990 .

[31]  Changyin Sun,et al.  Neural Networks for Nonconvex Nonlinear Programming Problems: A Switching Control Approach , 2005, ISNN.

[32]  Xiaohui Yuan,et al.  Hydrothermal scheduling using chaotic hybrid differential evolution , 2008 .

[33]  Long Cheng,et al.  Solving convex optimization problems using recurrent neural networks in finite time , 2009, 2009 International Joint Conference on Neural Networks.

[34]  Youshen Xia,et al.  An Extended Projection Neural Network for Constrained Optimization , 2004, Neural Computation.

[35]  Yuehua Huang,et al.  Finite-Time Stability and Its Application for Solving Time-Varying Sylvester Equation by Recurrent Neural Network , 2014, Neural Processing Letters.

[36]  Yangming Li,et al.  A class of finite-time dual neural networks for solving quadratic programming problems and its k-winners-take-all application , 2013, Neural Networks.

[37]  Lorenz T. Biegler,et al.  Line Search Filter Methods for Nonlinear Programming: Motivation and Global Convergence , 2005, SIAM J. Optim..

[38]  Shengwei Zhang,et al.  Lagrange programming neural networks , 1992 .

[39]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[40]  Jun Wang,et al.  A Novel Recurrent Neural Network for Solving Nonlinear Optimization Problems With Inequality Constraints , 2008, IEEE Transactions on Neural Networks.

[41]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[42]  Xiaolin Hu,et al.  Solving Pseudomonotone Variational Inequalities and Pseudoconvex Optimization Problems Using the Projection Neural Network , 2006, IEEE Transactions on Neural Networks.

[43]  P. Tseng,et al.  Modified Projection-Type Methods for Monotone Variational Inequalities , 1996 .

[44]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[45]  Stefen Hui,et al.  On solving constrained optimization problems with neural networks: a penalty method approach , 1993, IEEE Trans. Neural Networks.