Finite-Time Convergent Recurrent Neural Network With a Hard-Limiting Activation Function for Constrained Optimization With Piecewise-Linear Objective Functions

This paper presents a one-layer recurrent neural network for solving a class of constrained nonsmooth optimization problems with piecewise-linear objective functions. The proposed neural network is guaranteed to be globally convergent in finite time to the optimal solutions under a mild condition on a derived lower bound of a single gain parameter in the model. The number of neurons in the neural network is the same as the number of decision variables of the optimization problem. Compared with existing neural networks for optimization, the proposed neural network has a couple of salient features such as finite-time convergence and a low model complexity. Specific models for two important special cases, namely, linear programming and nonsmooth optimization, are also presented. In addition, applications to the shortest path problem and constrained least absolute deviation problem are discussed with simulation results to demonstrate the effectiveness and characteristics of the proposed neural network.

[1]  Joydeep Dutta,et al.  On saddle points and optima for non-smooth and non-convex programs , 1997 .

[2]  Jun Wang,et al.  A recurrent neural network for solving the shortest path problem , 1994, Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94.

[3]  Jun Wang,et al.  Primal and dual assignment networks , 1997, IEEE Trans. Neural Networks.

[4]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[5]  Jun Wang,et al.  A deterministic annealing neural network for convex programming , 1994, Neural Networks.

[6]  Youshen Xia.  A new neural network for solving linear programming problems and its application , 1996, IEEE Trans. Neural Networks.

[7]  Wei Bian,et al.  Subgradient-Based Neural Networks for Nonsmooth Convex Optimization Problems , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[8]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network for Non-smooth Convex Optimization Subject to Linear Equality Constraints , 2009, ICONIP.

[9]  Jun Wang.  Analysis and design of a recurrent neural network for linear programming , 1993 .

[10]  Jun Wang,et al.  Analysis and Design of a $k$ -Winners-Take-All Model With a Single State Variable and the Heaviside Step Activation Function , 2010, IEEE Transactions on Neural Networks.

[11]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[12]  Jun Wang,et al.  Grasping-force optimization for multifingered robotic hands using a recurrent neural network , 2004, IEEE Transactions on Robotics and Automation.

[13]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[14]  Jinde Cao,et al.  A Novel Recurrent Neural Network With One Neuron and Finite-Time Convergence for $k$-Winners-Take-All Operation , 2010, IEEE Transactions on Neural Networks.

[15]  Mohamed S. Kamel,et al.  Cooperative Recurrent Neural Networks for the Constrained $L_{1}$ Estimator , 2007, IEEE Transactions on Signal Processing.

[16]  Abdesselam Bouzerdoum,et al.  Neural network for quadratic optimization with bound constraints , 1993, IEEE Trans. Neural Networks.

[17]  Mauro Forti,et al.  Generalized neural network for nonsmooth nonlinear programming problems , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[18]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms, 3/E. , 2019 .

[19]  M. Forti,et al.  Generalized Lyapunov approach for convergence of neural networks with discontinuous or non-Lipschitz activations , 2006 .

[20]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network With a Discontinuous Hard-Limiting Activation Function for Quadratic Programming , 2008, IEEE Transactions on Neural Networks.

[21]  Zhishun Wang,et al.  Constrained Least Absolute Deviation Neural Networks , 2008, IEEE Transactions on Neural Networks.

[22]  Jun Wang.  Primal and dual neural networks for shortest-path routing , 1998, IEEE Trans. Syst. Man Cybern. Part A.

[23]  Shubao Liu,et al.  A Simplified Dual Neural Network for Quadratic Programming With Its KWTA Application , 2006, IEEE Transactions on Neural Networks.

[24]  Xiaolin Hu,et al.  An Improved Dual Neural Network for Solving a Class of Quadratic Programming Problems and Its $k$-Winners-Take-All Application , 2008, IEEE Transactions on Neural Networks.

[25]  Katta G. Murty,et al.  Nonlinear Programming Theory and Algorithms , 2007, Technometrics.

[26]  Shengwei Zhang,et al.  Lagrange programming neural networks , 1992 .

[27]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network with a Discontinuous Activation Function for Linear Programming , 2008, Neural Computation.

[28]  Youshen Xia,et al.  A recurrent neural network for nonlinear convex optimization subject to nonlinear inequality constraints , 2004, IEEE Trans. Circuits Syst. I Regul. Pap..

[29]  Edwin K. P. Chong,et al.  An analysis of a class of neural networks for solving linear programming problems , 1999, IEEE Trans. Autom. Control..

[30]  J. Aubin,et al.  Differential inclusions set-valued maps and viability theory , 1984 .

[31]  Jun Wang,et al.  A discrete-time recurrent neural network for shortest-path routing , 2000, IEEE Trans. Autom. Control..

[32]  Michael A. Shanblatt,et al.  Linear and quadratic programming neural network analysis , 1992, IEEE Trans. Neural Networks.

[33]  Tianping Chen,et al.  Dynamical Behaviors of Delayed Neural Network Systems with Discontinuous Activation Functions , 2006, Neural Computation.

[34]  Shiji Song,et al.  A Neural Network Model for Non-smooth Optimization over a Compact Convex Subset , 2006, ISNN.

[35]  Jun Wang,et al.  A projection neural network and its application to constrained optimization problems , 2002 .

[36]  Mauro Forti,et al.  Convergence of Neural Networks for Programming Problems via a Nonsmooth Łojasiewicz Inequality , 2006, IEEE Transactions on Neural Networks.

[37]  Mauro Forti,et al.  Robustness of convergence in finite time for linear programming neural networks , 2006, Int. J. Circuit Theory Appl..

[38]  Youshen Xia,et al.  A new neural network for solving linear and quadratic programming problems , 1996, IEEE Trans. Neural Networks.

[39]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[40]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[41]  Aleksej F. Filippov,et al.  Differential Equations with Discontinuous Righthand Sides , 1988, Mathematics and Its Applications.