A Neurodynamic Optimization Approach to Bilevel Quadratic Programming

This paper presents a neurodynamic optimization approach to bilevel quadratic programming (BQP). Based on the Karush–Kuhn–Tucker (KKT) theorem, the BQP problem is reduced to a one-level mathematical program subject to complementarity constraints (MPCC). It is proved that the global solution of the MPCC is the minimal one of the optimal solutions to multiple convex optimization subproblems. A recurrent neural network is developed for solving these convex optimization subproblems. From any initial state, the state of the proposed neural network is convergent to an equilibrium point of the neural network, which is just the optimal solution of the convex optimization subproblem. Compared with existing recurrent neural networks for BQP, the proposed neural network is guaranteed for delivering the exact optimal solutions to any convex BQP problems. Moreover, it is proved that the proposed neural network for bilevel linear programming is convergent to an equilibrium point in finite time. Finally, three numerical examples are elaborated to substantiate the efficacy of the proposed approach.

[1]  Seul Jung,et al.  Hardware Implementation of a Real-Time Neural Network Controller With a DSP and an FPGA for Nonlinear Systems , 2007, IEEE Transactions on Industrial Electronics.

[2]  Jane J. Ye,et al.  Optimality conditions for bilevel programming problems , 1995 .

[3]  Jun Wang,et al.  A One-Layer Recurrent Neural Network for Constrained Nonsmooth Optimization , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[4]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[5]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[6]  Herminia I. Calvete,et al.  On linear bilevel problems with multiple objectives at the lower level , 2011 .

[7]  Jane J. Ye,et al.  Exact Penalization and Necessary Optimality Conditions for Generalized Bilevel Programming Problems , 1997, SIAM J. Optim..

[8]  Jonathan F. BARD,et al.  Convex two-level optimization , 1988, Math. Program..

[9]  Youshen Xia,et al.  A new neural network for solving linear and quadratic programming problems , 1996, IEEE Trans. Neural Networks.

[10]  Robert J. Vanderbei,et al.  Interior-Point Algorithms, Penalty Methods and Equilibrium Problems , 2006, Comput. Optim. Appl..

[11]  Eitaro Aiyoshi,et al.  HIERARCHICAL DECENTRALIZED SYSTEM AND ITS NEW SOLUTION BY A BARRIER METHOD. , 1980 .

[12]  Athanasios Migdalas,et al.  Bilevel programming in traffic planning: Models, methods and challenge , 1995, J. Glob. Optim..

[13]  Robert G. Jeroslow,et al.  The polynomial hierarchy and a simple model for competitive analysis , 1985, Math. Program..

[14]  Jonathan F. Bard,et al.  An explicit solution to the multi-level programming problem , 1982, Comput. Oper. Res..

[15]  L. N. Vicente,et al.  Descent approaches for quadratic bilevel programming , 1994 .

[16]  Baoding Liu,et al.  Stackelberg-Nash equilibrium for multilevel programming with multiple followers using genetic algorithms , 1998 .

[17]  Wilfred Candler,et al.  A linear two-level programming problem, , 1982, Comput. Oper. Res..

[18]  Chuandong Li,et al.  Neural network for solving convex quadratic bilevel programming problems , 2014, Neural Networks.

[19]  Jonathan F. Bard,et al.  Practical Bilevel Optimization: Algorithms and Applications , 1998 .

[20]  Yibing Lv,et al.  Neural Network Approach for Semivectorial Bilevel Programming Problem , 2012, 2012 4th International Conference on Intelligent Human-Machine Systems and Cybernetics.

[21]  Stephan Dempe,et al.  Feasible direction method for bilevel programming problem , 2012 .

[22]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[23]  Mauro Forti,et al.  Generalized neural network for nonsmooth nonlinear programming problems , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[24]  Patrice Marcotte,et al.  A Trust-Region Method for Nonlinear Bilevel Programming: Algorithm and Computational Experience , 2005, Comput. Optim. Appl..

[25]  Wei Bian,et al.  Neural Network for Solving Constrained Convex Optimization Problems With Global Attractivity , 2013, IEEE Transactions on Circuits and Systems I: Regular Papers.

[26]  Wang Guang-min,et al.  Genetic Algorithms for Solving Linear Bilevel Programming , 2005 .

[27]  Jun Wang,et al.  Analysis and Design of a $k$ -Winners-Take-All Model With a Single State Variable and the Heaviside Step Activation Function , 2010, IEEE Transactions on Neural Networks.

[28]  Ue-Pyng Wen,et al.  A neural network approach to multiobjective and multilevel programming problems , 2004 .

[29]  Stefan Scholtes,et al.  Convergence Properties of a Regularization Scheme for Mathematical Programs with Complementarity Constraints , 2000, SIAM J. Optim..

[30]  Jean-Pierre Dussault,et al.  A New Regularization Scheme for Mathematical Programs with Complementarity Constraints , 2009, SIAM J. Optim..

[31]  Ue-Pyng Wen,et al.  A hybrid neural network approach to bilevel programming problems , 2007, Appl. Math. Lett..

[32]  Qingshan Liu,et al.  A One-Layer Recurrent Neural Network With a Discontinuous Hard-Limiting Activation Function for Quadratic Programming , 2008, IEEE Transactions on Neural Networks.

[33]  Xing-Bao Gao,et al.  A novel neural network for nonlinear convex programming , 2004, IEEE Trans. Neural Networks.

[34]  Mihai Anitescu,et al.  On Using the Elastic Mode in Nonlinear Programming Approaches to Mathematical Programs with Complementarity Constraints , 2005, SIAM J. Optim..

[35]  Zhong Chen,et al.  A neural network for solving a convex quadratic bilevel programming problem , 2010, J. Comput. Appl. Math..

[36]  Le Dung Muu,et al.  A Global Optimization Method for Solving Convex Quadratic Bilevel Programming Problems , 2003, J. Glob. Optim..

[37]  Long Cheng,et al.  Recurrent Neural Network for Non-Smooth Convex Optimization Problems With Application to the Identification of Genetic Regulatory Networks , 2011, IEEE Transactions on Neural Networks.

[38]  Li-Zhi Liao,et al.  A New One-Layer Neural Network for Linear and Quadratic Programming , 2010, IEEE Transactions on Neural Networks.

[39]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[40]  R. J. Kuo,et al.  Application of particle swarm optimization algorithm for solving bi-level linear programming problem , 2009, Comput. Math. Appl..

[41]  Xiao-Jun Zeng,et al.  Short-Term and Midterm Load Forecasting Using a Bilevel Optimization Model , 2009, IEEE Transactions on Power Systems.

[42]  Sitian Qin,et al.  Dynamical Analysis of Neural Networks of Subgradient System , 2010, IEEE Transactions on Automatic Control.

[43]  Xiaojun Chen,et al.  A Smoothing Method for a Mathematical Program with P-Matrix Linear Complementarity Constraints , 2004, Comput. Optim. Appl..

[44]  Kwong-Sak Leung,et al.  A new gradient-based neural network for solving linear and quadratic programming problems , 2001, IEEE Trans. Neural Networks.

[45]  Chuandong Li,et al.  A Recurrent Neural Network for Solving Bilevel Linear Programming Problem , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Patrice Marcotte,et al.  An overview of bilevel optimization , 2007, Ann. Oper. Res..

[47]  Tiesong Hu,et al.  A neural network approach for solving nonlinear bilevel programming problem , 2008, Comput. Math. Appl..

[48]  Wei Bian,et al.  Subgradient-Based Neural Networks for Nonsmooth Convex Optimization Problems , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[49]  Stephan Dempe,et al.  Optimality Conditions for a Simple Convex Bilevel Programming Problem , 2010 .

[50]  Charles E. Blair,et al.  Computational Difficulties of Bilevel Linear Programming , 1990, Oper. Res..

[51]  Sitian Qin,et al.  A Two-Layer Recurrent Neural Network for Nonsmooth Convex Optimization Problems , 2015, IEEE Transactions on Neural Networks and Learning Systems.