Non-monotone algorithm for minimization on arbitrary domains with applications to large-scale orthogonal Procrustes problem

Abstract This paper concerns a non-monotone algorithm for minimizing differentiable functions on closed sets. A general numerical scheme is proposed which combines a regularization/trust-region framework with a non-monotone strategy. Global convergence to stationary points is proved under usual assumptions. Numerical experiments for a particular version of the general algorithm are reported. In addition, a promising numerical scheme for medium/large-scale orthogonal Procrustes problem is also proposed and numerically illustrated.

[1]  Nicholas J. Higham,et al.  The symmetric procrustes problem , 1988 .

[2]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[3]  Faezeh Toutounian,et al.  The block least squares method for solving nonsymmetric linear systems with multiple right-hand sides , 2006, Appl. Math. Comput..

[4]  John C. Gower,et al.  Orthogonal and projection procrustes analysis , 1995 .

[5]  Wenyu Sun,et al.  Nonmonotone adaptive trust-region method for unconstrained optimization problems , 2005, Appl. Math. Comput..

[6]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[7]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[8]  Philippe L. Toint,et al.  Non-monotone trust-region algorithms for nonlinear optimization subject to convex constraints , 1997, Math. Program..

[9]  T. Bell,et al.  Global Positioning System-Based Attitude Determination and the Orthogonal Procrustes Problem , 2003 .

[10]  Wotao Yin,et al.  A feasible method for optimization with orthogonality constraints , 2013, Math. Program..

[11]  Arkadi Nemirovski,et al.  Sums of random symmetric matrices and quadratic optimization under orthogonality constraints , 2007, Math. Program..

[12]  Zhenyue Zhang,et al.  Successive projection method for solving the unbalanced Procrustes problem , 2006 .

[13]  Wenyu Sun,et al.  Global convergence of nonmonotone descent methods for unconstrained optimization problems , 2002 .

[14]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[15]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[16]  Jiangtao Mo,et al.  A nonmonotone trust region method based on nonincreasing technique of weighted average of the successive function values , 2007 .

[17]  A. Shapiro,et al.  First-order conditions for isolated locally optimal solutions , 1993 .

[18]  Zhensheng Yu.  Solving bound constrained optimization via a new nonmonotone spectral projected gradient method , 2008 .

[19]  Nicholas I. M. Gould,et al.  CUTEst: a Constrained and Unconstrained Testing Environment with safe threads for mathematical optimization , 2013, Computational Optimization and Applications.

[20]  Nickolay T. Trendafilov,et al.  On the l1 Procrustes problem , 2003, Future Gener. Comput. Syst..

[21]  Boying Wu,et al.  A new modified nonmonotone adaptive trust region method for unconstrained optimization , 2012, Comput. Optim. Appl..

[22]  Gene H. Golub,et al.  Matrix computations , 1983 .

[23]  Galahad , 1906 .

[24]  Tamás Rapcsák,et al.  On minimization on Stiefel manifolds , 2002, Eur. J. Oper. Res..

[25]  N. Trendafilov,et al.  The Orthogonally Constrained Regression Revisited , 2001 .

[26]  Nicholas I. M. Gould,et al.  GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization , 2003, TOMS.

[27]  Yuhong Dai.  On the Nonmonotone Line Search , 2002 .

[28]  Lothar Reichel,et al.  Restarted block Lanczos bidiagonalization methods , 2007, Numerical Algorithms.

[29]  Adam W. Bojanczyk,et al.  The Procrustes Problem for Orthogonal Stiefel Matrices , 1999, SIAM J. Sci. Comput..

[30]  William W. Hager,et al.  A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization , 2004, SIAM J. Optim..

[31]  Philippe L. Toint,et al.  An Assessment of Nonmonotone Linesearch Techniques for Unconstrained Optimization , 1996, SIAM J. Sci. Comput..

[32]  Qunyan Zhou,et al.  Nonmonotone adaptive trust region method with line search based on new diagonal updating , 2015 .

[33]  J. M. Martínez,et al.  Inexact spectral projected gradient methods on convex sets , 2003 .

[34]  Fermín S. Viloche Bazán,et al.  Nonmonotone algorithm for minimization on closed sets with applications to minimization on Stiefel manifolds , 2012, J. Comput. Appl. Math..

[35]  L. Grippo,et al.  A class of nonmonotone stabilization methods in unconstrained optimization , 1991 .

[36]  Wenyu Sun,et al.  Nonmonotone trust region method for solving optimization problems , 2004, Appl. Math. Comput..

[37]  Marcos Raydan,et al.  The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem , 1997, SIAM J. Optim..

[38]  Lothar Reichel,et al.  An implicitly restarted block Lanczos bidiagonalization method using Leja shifts , 2012 .