Recurrent neural networks with fixed time convergence for linear and quadratic programming

In this paper, a new class of recurrent neural networks which solve linear and quadratic programs are presented. Their design is considered as a sliding mode control problem, where the network structure is based on the Karush-Kuhn-Tucker (KKT) optimality conditions with the KKT multipliers considered as control inputs to be implemented with fixed time stabilizing terms, instead of common used activation functions. Thus, the main feature of the proposed network is its fixed convergence time to the solution. That means, there is time independent to the initial conditions in which the network converges to the optimization solution. Simulations show the feasibility of the current approach.

[1]  W. Karush.  Minima of Functions of Several Variables with Inequalities as Side Conditions , 2014 .

[2]  Edwin K. P. Chong,et al.  An analysis of a class of neural networks for solving linear programming problems , 1999, IEEE Trans. Autom. Control..

[3]  J. Aubin,et al.  Differential inclusions set-valued maps and viability theory , 1984 .

[4]  R. Brockett.  Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems , 1991 .

[5]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[6]  Leonid Fridman,et al.  Uniform Second-Order Sliding Mode Observer for mechanical systems , 2010, 2010 11th International Workshop on Variable Structure Systems (VSS).

[7]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[8]  Aleksej F. Filippov,et al.  Differential Equations with Discontinuous Righthand Sides , 1988, Mathematics and Its Applications.

[9]  Qingshan Liu,et al.  A One-Layer Dual Recurrent Neural Network with a Heaviside Step Activation Function for Linear Programming with Its Linear Assignment Application , 2011, ICANN.

[10]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[11]  A. F. Filippov.  Equations with the Right-Hand Side Continuous in x and Discontinuous in t , 1988 .

[12]  Long Cheng,et al.  Solving convex optimization problems using recurrent neural networks in finite time , 2009, 2009 International Joint Conference on Neural Networks.

[13]  Mauro Forti,et al.  Generalized neural network for nonsmooth nonlinear programming problems , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[14]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[15]  Jun Wang.  Analysis and design of a recurrent neural network for linear programming , 1993 .

[16]  Sarah K. Spurgeon,et al.  Sliding Mode Control , 1998 .

[17]  G. Lin.  Nonlinear Programming without Computation , 2022 .

[18]  Andrey Polyakov,et al.  Nonlinear Feedback Design for Fixed-Time Stabilization of Linear Control Systems , 2012, IEEE Transactions on Automatic Control.

[19]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network for Pseudoconvex Optimization Subject to Linear Equality Constraints , 2011, IEEE Transactions on Neural Networks.

[20]  A. Levant.  Sliding order and sliding accuracy in sliding mode control , 1993 .

[21]  G. Wilson.  Quadratic programming analogs , 1986 .

[22]  Hao-Chi Chang,et al.  Sliding mode control on electro-mechanical systems , 1999 .

[23]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[24]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network With a Discontinuous Hard-Limiting Activation Function for Quadratic Programming , 2008, IEEE Transactions on Neural Networks.

[25]  Vadim I. Utkin,et al.  Sliding Modes in Control and Optimization , 1992, Communications and Control Engineering Series.

[26]  Jinde Cao,et al.  A Novel Recurrent Neural Network with Finite-Time Convergence for Linear Programming , 2010, Neural Computation.

[27]  V. Utkin,et al.  Sliding mode control in dynamic systems , 1992 .

[28]  Qingshan Liu,et al.  A one-layer recurrent neural network for constrained pseudoconvex optimization and its application for dynamic portfolio optimization , 2012, Neural Networks.

[29]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network with a Discontinuous Activation Function for Linear Programming , 2008, Neural Computation.

[30]  Edgar Sanchez-Sinencio,et al.  Nonlinear switched capacitor 'neural' networks for optimization problems , 1990 .

[31]  Jun Wang,et al.  A general methodology for designing globally convergent optimization neural networks , 1998, IEEE Trans. Neural Networks.

[32]  Qingshan Liu,et al.  Finite-Time Convergent Recurrent Neural Network With a Hard-Limiting Activation Function for Constrained Optimization With Piecewise-Linear Objective Functions , 2011, IEEE Transactions on Neural Networks.