A Broyden Class of Quasi-Newton Methods for Riemannian Optimization

This paper develops and analyzes a generalization of the Broyden class of quasi-Newton methods to the problem of minimizing a smooth objective function $f$ on a Riemannian manifold. A condition on vector transport and retraction that guarantees convergence and facilitates efficient computation is derived. Experimental evidence is presented demonstrating the value of the extension to the Riemannian Broyden class through superior performance for some problems compared to existing Riemannian BFGS methods, in particular those that depend on differentiated retraction.

[1]  William C. Davidon,et al.  Optimally conditioned optimization algorithms without line searches , 1975, Math. Program..

[2]  Marie Faerber,et al.  Recent Advances In Optimization And Its Applications In Engineering , 2016 .

[3]  J. Nocedal,et al.  Global Convergence of a Class of Quasi-newton Methods on Convex Problems, Siam Some Global Convergence Properties of a Variable Metric Algorithm for Minimization without Exact Line Searches, Nonlinear Programming, Edited , 1996 .

[4]  Bamdev Mishra,et al.  Low-rank optimization for distance matrix completion , 2011, IEEE Conference on Decision and Control and European Control Conference.

[5]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[6]  Pierre-Antoine Absil,et al.  An Efficient BFGS Algorithm for Riemannian Optimization , 2010 .

[7]  Pierre-Antoine Absil,et al.  Trust-Region Methods on Riemannian Manifolds , 2007, Found. Comput. Math..

[8]  J. Manton,et al.  An improved BFGS-on-manifold algorithm for computing weighted low rank approximations , 2006 .

[9]  P. Toint,et al.  Local convergence analysis for partitioned quasi-Newton updates , 1982 .

[10]  R. Adler,et al.  Newton's method on Riemannian manifolds and a geometric model for the human spine , 2002 .

[11]  W. Boothby.  An introduction to differentiable manifolds and Riemannian geometry , 1975 .

[12]  K. Hüper,et al.  On FastICA Algorithms and Some Generalisations , 2011 .

[13]  Wen Huang,et al.  Optimization algorithms on Riemannian manifolds with applications , 2013 .

[14]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[15]  Benedikt Wirth,et al.  Optimization Methods on Riemannian Manifolds and Their Application to Shape Space , 2012, SIAM J. Optim..

[16]  Bart Vandereycken.  Low-Rank Matrix Completion by Riemannian Optimization , 2012, SIAM J. Optim..

[17]  Bart Vandereycken,et al.  Low-Rank Matrix Completion by Riemannian Optimization , 2013, SIAM J. Optim..

[18]  Oliver Sander,et al.  Geodesic finite elements for Cosserat rods , 2009 .

[19]  K. Hüper,et al.  Properties of the BFGS method on Riemannian manifolds , 2012 .

[20]  Bruno Alfano,et al.  Descent Algorithms on Oblique Manifold for Source-Adaptive ICA Contrast , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[21]  Pierre-Antoine Absil,et al.  RTRMC: A Riemannian trust-region method for low-rank matrix completion , 2011, NIPS.

[22]  Steven Thomas Smith,et al.  Optimization Techniques on Riemannian Manifolds , 2014, ArXiv.

[23]  D. Gabay.  Minimizing a differentiable function over a differential manifold , 1982 .

[24]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[25]  P. Absil,et al.  An implicit trust-region method on Riemannian manifolds , 2008 .

[26]  Dario Bini,et al.  Computing the Karcher mean of symmetric positive definite matrices , 2013 .

[27]  René Vidal,et al.  On the Convergence of Gradient Descent for Finding the Riemannian Center of Mass , 2011, SIAM J. Control. Optim..

[28]  Jorge Nocedal,et al.  On the Behavior of Broyden's Class of Quasi-Newton Methods , 1992, SIAM J. Optim..

[29]  Chuanhai Liu,et al.  Statistical Quasi-Newton: A New Look at Least Change , 2007, SIAM J. Optim..

[30]  R. Tewarson,et al.  Quasi-Newton Algorithms with Updates from the Preconvex Part of Broyden's Family , 1988 .

[31]  Pierre-Antoine Absil,et al.  A Riemannian Dennis-Moré Condition , 2012, High-Performance Scientific Computing.

[32]  Shirley Dex,et al.  JR 旅客販売総合システム（マルス）における運用及び管理について , 1991 .

[33]  D. Luenberger.  The Gradient Projection Method Along Geodesics , 1972 .

[34]  Berkant Savas,et al.  Quasi-Newton Methods on Grassmannians and Multilinear Approximations of Tensors , 2009, SIAM J. Sci. Comput..

[35]  Chunhong Qi.  Numerical Optimization Methods on Riemannian Manifolds , 2011 .

[36]  Olgica Milenkovic,et al.  A Geometric Approach to Low-Rank Matrix Completion , 2010, IEEE Transactions on Information Theory.

[37]  Wen Huang,et al.  A Riemannian symmetric rank-one trust-region method , 2014, Mathematical Programming.

[38]  Pierre-Antoine Absil,et al.  Riemannian BFGS Algorithm with Applications , 2010 .