An alternating structured trust region algorithm for separable optimization problems with nonconvex constraints

In this paper, we propose a structured trust-region algorithm combining with filter technique to minimize the sum of two general functions with general constraints. Specifically, the new iterates are generated in the Gauss-Seidel type iterative procedure, whose sizes are controlled by a trust-region type parameter. The entries in the filter are a pair: one resulting from feasibility; the other resulting from optimality. The global convergence of the proposed algorithm is proved under some suitable assumptions. Some preliminary numerical results show that our algorithm is potentially efficient for solving general nonconvex optimization problems with separable structure.

[1]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[2]  Wenyu Sun,et al.  Global convergence of a filter-trust-region algorithm for solving nonsmooth equations , 2010, Int. J. Comput. Math..

[3]  Hanif D. Sherali,et al.  Solutions and optimality criteria for nonconvex constrained global optimization problems with connections between canonical and Lagrangian duality , 2009, J. Glob. Optim..

[4]  Nicholas I. M. Gould,et al.  Lancelot: A FORTRAN Package for Large-Scale Nonlinear Optimization (Release A) , 1992 .

[5]  Andreas Griewank,et al.  On the unconstrained optimization of partially separable functions , 1982 .

[6]  Hyunsoo Kim,et al.  Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method , 2008, SIAM J. Matrix Anal. Appl..

[7]  Sven Leyffer,et al.  On the Global Convergence of a Filter--SQP Algorithm , 2002, SIAM J. Optim..

[8]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[9]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[10]  Robert R. Meyer,et al.  A variable-penalty alternating directions method for convex optimization , 1998, Math. Program..

[11]  Wenyu Sun,et al.  On filter-successive linearization methods for nonlinear semidefinite programming , 2009 .

[12]  Antoine Soubeyran,et al.  Learning how to Play Nash, Potential Games and Alternating Minimization Method for Structured Nonconvex Problems on Riemannian Manifolds , 2013 .

[13]  Yi Xu,et al.  A filter successive linear programming method for nonlinear semidefinite programming problems , 2012 .

[14]  Nicholas I. M. Gould,et al.  Convergence Properties of Minimization Algorithms for Convex Constraints Using a Structured Trust Region , 1996, SIAM J. Optim..

[15]  M. El-Alem.  A global convergence theory for the Celis-Dennis-Tapia trust-region algorithm for constrained optimization , 1991 .

[16]  D. Gao.  Duality Principles in Nonconvex Systems: Theory, Methods and Applications , 2000 .

[17]  Georgios B. Giannakis,et al.  Consensus-Based Distributed Support Vector Machines , 2010, J. Mach. Learn. Res..

[18]  Marc Teboulle,et al.  A proximal-based decomposition method for convex minimization problems , 1994, Math. Program..

[19]  Yan Zhang,et al.  A Nonmonotone Filter Barzilai-Borwein Method for Optimization , 2010, Asia Pac. J. Oper. Res..

[20]  Richard H. Byrd,et al.  A Trust Region Algorithm for Nonlinearly Constrained Optimization , 1987 .

[21]  Nicholas I. M. Gould,et al.  A Filter-Trust-Region Method for Unconstrained Optimization , 2005, SIAM J. Optim..

[22]  Nicholas I. M. Gould,et al.  Global Convergence of a Trust-Region SQP-Filter Algorithm for General Nonlinear Programming , 2002, SIAM J. Optim..

[23]  William W. Hager,et al.  Self-adaptive inexact proximal point methods , 2008, Comput. Optim. Appl..

[24]  Masao Fukushima,et al.  Application of the alternating direction method of multipliers to separable convex programming problems , 1992, Comput. Optim. Appl..

[25]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[26]  Qian Zhou,et al.  An ODE-based trust region method for unconstrained optimization problems , 2009, J. Comput. Appl. Math..

[27]  Ya-Xiang Yuan,et al.  A trust region algorithm for equality constrained optimization , 1990, Math. Program..

[28]  Wenyu Sun,et al.  A modified trust region method with Beale’s PCG technique for optimization , 2008, Comput. Optim. Appl..

[29]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[30]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[31]  Bingsheng He,et al.  A new inexact alternating directions method for monotone variational inequalities , 2002, Math. Program..

[32]  P. Toint,et al.  On Large Scale Nonlinear Least Squares Calculations , 1987 .

[33]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .

[34]  Haimin Wang,et al.  CORONAL IMPLOSION AND PARTICLE ACCELERATION IN THE WAKE OF A FILAMENT ERUPTION , 2009, 0908.1137.

[35]  J. J. Moré,et al.  On the identification of active constraints , 1988 .

[36]  M. J. D. Powell,et al.  Nonlinear optimization, 1981 , 1982 .

[37]  Sven Leyffer,et al.  Nonlinear programming without a penalty function , 2002, Math. Program..

[38]  José M. Bioucas-Dias,et al.  Restoration of Poissonian Images Using Alternating Direction Optimization , 2010, IEEE Transactions on Image Processing.

[39]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[40]  D. Gao.  Duality Principles in Nonconvex Systems , 2000 .

[41]  M. Fortin,et al.  Augmented Lagrangian methods : applications to the numerical solution of boundary-value problems , 1983 .

[42]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[43]  Yan Chen,et al.  When Does Learning in Games Generate Convergence to Nash Equilibria? The Role of Supermodularity in an Experimental Setting ⁄ , 2004 .

[44]  P. Toint.  Global Convergence of a a of Trust-Region Methods for Nonconvex Minimization in Hilbert Space , 1988 .

[45]  J. C. Heideman,et al.  Sequential gradient-restoration algorithm for the minimization of constrained functions—Ordinary and conjugate gradient versions , 1969 .

[46]  Roger Fletcher,et al.  On the global convergence of an SLP–filter algorithm that takes EQP steps , 2003, Math. Program..

[47]  E. Haber,et al.  On optimization techniques for solving nonlinear inverse problems , 2000 .

[48]  Paul Tseng,et al.  Alternating Projection-Proximal Methods for Convex Programming and Variational Inequalities , 1997, SIAM J. Optim..

[49]  A. K. Aggarwal,et al.  A class of quadratically convergent algorithms for constrained function minimization , 1975 .