On the globalization of Riemannian Newton method

In the present paper, in order to fnd a singularity of a vector field defined on Riemannian manifolds, we present a new globalization strategy of Newton method and establish its global convergence with superlinear rate. In particular, this globalization generalizes for a general retraction the existing damped Newton's method. The presented global convergence analysis does not require any hypotesesis on singularity of the vector field. We applied the proposed method to solve the truncated singular value problem on the product of two Stiefel manifolds, the dextrous hand grasping problem on the cone of symmetric positive definite matrices and the Rayleigh quotient on the sphere. Moreover, some academic problems are solved. Numerical experiments are presented showing that the proposed algorithm has better robustness compared with the aforementioned method.

[1]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[2]  Wen Huang,et al.  Solving PhaseLift by Low-rank Riemannian Optimization Methods , 2017, ICCS.

[3]  Benedikt Wirth,et al.  Optimization Methods on Riemannian Manifolds and Their Application to Shape Space , 2012, SIAM J. Optim..

[4]  J. Moser,et al.  A NEW TECHNIQUE FOR THE CONSTRUCTION OF SOLUTIONS OF NONLINEAR DIFFERENTIAL EQUATIONS. , 1961, Proceedings of the National Academy of Sciences of the United States of America.

[5]  Bart Vandereycken,et al.  Low-Rank Matrix Completion by Riemannian Optimization , 2013, SIAM J. Optim..

[6]  Jonathan H. Manton,et al.  Optimization algorithms exploiting unitary constraints , 2002, IEEE Trans. Signal Process..

[7]  Jérôme Malick,et al.  Projection-like Retractions on Matrix Manifolds , 2012, SIAM J. Optim..

[8]  I. Holopainen.  Riemannian Geometry , 1927, Nature.

[9]  Pierre-Antoine Absil,et al.  Robust Low-Rank Matrix Completion by Riemannian Optimization , 2016, SIAM J. Sci. Comput..

[10]  Uwe Helmke,et al.  Newton's algorithm in Euclidean Jordan algebras, with applications to robotics , 2002, Commun. Inf. Syst..

[11]  Matthew M. Lin,et al.  Riemannian inexact Newton method for structured inverse eigenvalue and singular value problems , 2018, BIT Numerical Mathematics.

[12]  Pierre-Antoine Absil,et al.  Trust-Region Methods on Riemannian Manifolds , 2007, Found. Comput. Math..

[13]  O. Burdakov.  Some globally convergent modifications of Newton's method for solving systems of nonlinear equations , 1980 .

[14]  Wotao Yin,et al.  A feasible method for optimization with orthogonality constraints , 2013, Math. Program..

[15]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[16]  D. Gabay.  Minimizing a differentiable function over a differential manifold , 1982 .

[17]  Wen Huang,et al.  Riemannian Optimization for Registration of Curves in Elastic Shape Analysis , 2015, Journal of Mathematical Imaging and Vision.

[18]  John B. Moore,et al.  Quadratically convergent algorithms for optimal dextrous hand grasping , 2002, IEEE Trans. Robotics Autom..

[19]  Xiaojing Zhu,et al.  A Riemannian conjugate gradient method for optimization on the Stiefel manifold , 2016, Computational Optimization and Applications.

[20]  J. Nash.  The imbedding problem for Riemannian manifolds , 1956 .

[21]  Wen Huang,et al.  Intrinsic representation of tangent vectors and vector transports on matrix manifolds , 2017, Numerische Mathematik.

[22]  D. Luenberger.  The Gradient Projection Method Along Geodesics , 1972 .

[23]  O. P. Ferreira,et al.  Local convergence of Newton's method under a majorant condition in Riemannian manifolds , 2012 .

[24]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[25]  Steven Thomas Smith,et al.  Optimization Techniques on Riemannian Manifolds , 2014, ArXiv.

[26]  Chong Li,et al.  Kantorovich's theorems for Newton's method for mappings and optimization problems on Lie groups , 2011 .

[27]  I. Argyros,et al.  Newton’s method for approximating zeros of vector fields on Riemannian manifolds , 2009 .

[28]  Sergio Amat,et al.  Third-order methods on Riemannian manifolds under Kantorovich conditions , 2014, J. Comput. Appl. Math..

[29]  Hiroyuki Sato,et al.  A Riemannian Optimization Approach to the Matrix Singular Value Decomposition , 2013, SIAM J. Optim..

[30]  Ben Jeuris,et al.  A survey and comparison of contemporary algorithms for computing the matrix geometric mean , 2012 .

[31]  Yin Zhang,et al.  An Efficient Gauss-Newton Algorithm for Symmetric Low-Rank Product Matrix Approximations , 2015, SIAM J. Optim..

[32]  R. Adler,et al.  Newton's method on Riemannian manifolds and a geometric model for the human spine , 2002 .

[33]  U. Helmke,et al.  Nonsmooth Riemannian Optimization with Applications to Sphere Packing and Grasping , 2007 .

[34]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[35]  Volker Schulz,et al.  A Riemannian View on Shape Optimization , 2012, Foundations of Computational Mathematics.

[36]  Yin Zhang,et al.  Accelerating Convergence by Augmented Rayleigh-Ritz Projections For Large-Scale Eigenpair Computation , 2017, SIAM J. Matrix Anal. Appl..

[37]  Jinyun Yuan,et al.  Damped Newton’s method on Riemannian manifolds , 2018, J. Glob. Optim..

[38]  Chong Li,et al.  Newton's method on Riemannian manifolds: Smale's point estimate theory under the γ-condition , 2006 .

[39]  Chong Li,et al.  Smale's point estimate theory for Newton's method on Lie groups , 2009, J. Complex..

[40]  Yin Zhang,et al.  Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm , 2012, Mathematical Programming Computation.

[41]  A. F. Izmailov,et al.  Newton-Type Methods for Optimization and Variational Problems , 2014 .

[42]  Suvrit Sra,et al.  Conic Geometric Optimization on the Manifold of Positive Definite Matrices , 2013, SIAM J. Optim..

[43]  Wen Huang,et al.  A Riemannian symmetric rank-one trust-region method , 2014, Mathematical Programming.

[44]  Hiroyuki Sato,et al.  A matrix-free implementation of Riemannian Newton’s method on the Stiefel manifold , 2017, Optim. Lett..

[45]  Orizon Pereira Ferreira,et al.  Kantorovich's Theorem on Newton's Method in Riemannian Manifolds , 2002, J. Complex..

[46]  Bruno Iannazzo,et al.  Geometric means of structured matrices , 2014 .

[47]  Jinyun Yuan,et al.  On the Superlinear Convergence of Newton’s Method on Riemannian Manifolds , 2016, Journal of Optimization Theory and Applications.

[48]  Pierre-Antoine Absil,et al.  A Riemannian subgradient algorithm for economic dispatch with valve-point effect , 2014, J. Comput. Appl. Math..

[49]  Xin Zhang,et al.  Gradient Type Optimization Methods For Electronic Structure Calculations , 2013, SIAM J. Sci. Comput..