Convergence analysis of Riemannian trust-region methods

This document is an expanded version of [ABG05], with a detailed convergence analysis. A general scheme for trust-region methods on Riemannian manifolds is proposed and analyzed. Among the various approaches available to (approximately) solve the trust-region subproblems, particular attention is paid to the truncated conjugate-gradient technique. The method is illustrated on problems from numerical linear algebra.

[1]  I. Holopainen.  Riemannian Geometry , 1927, Nature.

[2]  J. Nash.  The imbedding problem for Riemannian manifolds , 1956 .

[3]  D. Luenberger.  The Gradient Projection Method Along Geodesics , 1972 .

[4]  W. Boothby.  An introduction to differentiable manifolds and Riemannian geometry , 1975 .

[5]  M. Powell.  CONVERGENCE PROPERTIES OF A CLASS OF MINIMIZATION ALGORITHMS , 1975 .

[6]  Philippe L. Toint,et al.  Towards an efficient sparsity exploiting newton method for minimization , 1981 .

[7]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[8]  D. Gabay.  Minimizing a differentiable function over a differential manifold , 1982 .

[9]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[10]  Gene H. Golub,et al.  Matrix computations , 1983 .

[11]  Ahmed H. Sameh,et al.  Trace Minimization Algorithm for the Generalized Eigenvalue Problem , 1982, PPSC.

[12]  C. Udriste,et al.  Convex Functions and Optimization Methods on Riemannian Manifolds , 1994 .

[13]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[14]  A. Bloch.  Hamiltonian and Gradient Flows, Algorithms and Control , 1995 .

[15]  Gene H. Golub,et al.  Matrix Computations, Third Edition , 1996 .

[16]  U. Helmke,et al.  Optimization and Dynamical Systems , 1994, Proceedings of the IEEE.

[17]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[18]  G. W. Stewart,et al.  Matrix algorithms , 1998 .

[19]  Craig T. Lawrence,et al.  A Computationally Efficient Feasible Sequential Quadratic Programming Algorithm , 2000, SIAM J. Optim..

[20]  James Lam,et al.  An approximate approach to H2 optimal model reduction , 1999, IEEE Trans. Autom. Control..

[21]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[22]  A. Iserles,et al.  Methods for the approximation of the matrix exponential in a Lie‐algebraic setting , 1999, math/9904122.

[23]  Nicholas I. M. Gould,et al.  Solving the Trust-Region Subproblem using the Lanczos Method , 1999, SIAM J. Optim..

[24]  B. Owren,et al.  The Newton Iteration on Lie Groups , 2000 .

[25]  Zhaojun Bai,et al.  Nonlinear Eigenvalue Problems , 2000, Templates for the Solution of Algebraic Eigenvalue Problems.

[26]  J. Vandewalle,et al.  New Lyapunov functions for the continuous-time QR algorithm , 2000 .

[27]  A. Sameh,et al.  The trace minimization method for the symmetric generalized eigenvalue problem , 2000 .

[28]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[29]  William W. Hager,et al.  Minimizing a Quadratic Over a Sphere , 2001, SIAM J. Optim..

[30]  Robert E. Mahony,et al.  The Geometry of the Newton Method on Non-Compact Lie Groups , 2002, J. Glob. Optim..

[31]  John B. Moore,et al.  Quadratically convergent algorithms for optimal dextrous hand grasping , 2002, IEEE Trans. Robotics Autom..

[32]  P. Priouret,et al.  Newton's method on Riemannian manifolds: covariant alpha theory , 2002, math/0209096.

[33]  R. Adler,et al.  Newton's method on Riemannian manifolds and a geometric model for the human spine , 2002 .

[34]  Jonathan H. Manton,et al.  Optimization algorithms exploiting unitary constraints , 2002, IEEE Trans. Signal Process..

[35]  K. Huper,et al.  Newton-like methods for numerical optimization on manifolds , 2004, Conference Record of the Thirty-Eighth Asilomar Conference on Signals, Systems and Computers, 2004..

[36]  P. Absil,et al.  Riemannian Geometry of Grassmann Manifolds with a View on Algorithmic Computation , 2004 .

[37]  S. Shankar Sastry,et al.  Optimization Criteria and Geometric Algorithms for Motion and Structure Estimation , 2001, International Journal of Computer Vision.

[38]  P. Absil,et al.  Trust-region methods on Riemannian manifolds with applications in numerical linear algebra , 2004 .

[39]  William W. Hager,et al.  Global convergence of SSM for minimizing a quadratic over a sphere , 2004, Math. Comput..

[40]  Pierre-Antoine Absil,et al.  Adaptive Model Trust Region Methods for Generalized Eigenvalue Problems , 2005, International Conference on Computational Science.

[41]  Shotaro Akaho,et al.  Learning algorithms utilizing quasi-geodesic flows on the Stiefel manifold , 2005, Neurocomputing.

[42]  S.T. Smith,et al.  Covariance, subspace, and intrinsic Crame/spl acute/r-Rao bounds , 2005, IEEE Transactions on Signal Processing.

[43]  Jean-Pierre Dedieu,et al.  Symplectic methods for the approximation of the exponential map and the Newton iteration on Riemannian submanifolds , 2005, J. Complex..

[44]  João M. F. Xavier,et al.  Intrinsic variance lower bound (IVLB): an extension of the Cramer-Rao bound to Riemannian manifolds , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[45]  Jérome M. B. Walmag,et al.  A Note on Trust-Region Radius Update , 2005, SIAM J. Optim..

[46]  P. Absil,et al.  A truncated-CG style method for symmetric generalized eigenvalue problems , 2006 .

[47]  Yaguang Yang.  Globally Convergent Optimization Algorithms on Riemannian Manifolds: Uniform Framework for Unconstrained and Constrained Optimization , 2007 .

[48]  Pierre-Antoine Absil,et al.  Trust-Region Methods on Riemannian Manifolds , 2007, Found. Comput. Math..