Line Search Algorithms for Locally Lipschitz Functions on Riemannian Manifolds

This paper presents line search algorithms for finding extrema of locally Lipschitz functions defined on Riemannian manifolds. To this end we generalize the so-called Wolfe conditions for nonsmooth functions on Riemannian manifolds. Using $\varepsilon$-subgradient-oriented descent directions and the Wolfe conditions, we propose a nonsmooth Riemannian line search algorithm and establish the convergence of our algorithm to a stationary point. Moreover, we extend the classical BFGS algorithm to nonsmooth functions on Riemannian manifolds. Numerical experiments illustrate the effectiveness and efficiency of the proposed algorithm.

[1]  Robert Mifflin,et al.  An Algorithm for Constrained Optimization with Semismooth Functions , 1977, Math. Oper. Res..

[2]  S. Lang.  Fundamentals of differential geometry , 1998 .

[3]  Rohollah Yousefpour.  Combination of steepest descent and BFGS methods for nonconvex nonsmooth optimization , 2015, Numerical Algorithms.

[4]  U. Helmke,et al.  Nonsmooth Riemannian Optimization with Applications to Sphere Packing and Grasping , 2007 .

[5]  Dominikus Noll,et al.  Convergence of Non-smooth Descent Methods Using the Kurdyka–Łojasiewicz Inequality , 2014, J. Optim. Theory Appl..

[6]  Daniel Azagra Rueda,et al.  Applications of proximal calculus to fixed point theory on Riemannian manifolds , 2007 .

[7]  Wen Huang,et al.  Optimization algorithms on Riemannian manifolds with applications , 2013 .

[8]  Pei Yean Lee.  Geometric optimization for computer vision , 2005 .

[9]  Chong Li,et al.  Mathematics 4-1-2009 Weak Sharp Minima on Riemannian Manifolds , 2014 .

[10]  I. Holopainen.  Riemannian Geometry , 1927, Nature.

[11]  P. Grohs,et al.  Nonsmooth trust region algorithms for locally Lipschitz functions on Riemannian manifolds , 2016 .

[12]  D. Gabay.  Minimizing a differentiable function over a differential manifold , 1982 .

[13]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[14]  Benedikt Wirth,et al.  Optimization Methods on Riemannian Manifolds and Their Application to Shape Space , 2012, SIAM J. Optim..

[15]  M. R. Pouryayevali,et al.  On the metric projection onto prox-regular subsets of Riemannian manifolds , 2012 .

[16]  Wen Huang,et al.  A Broyden Class of Quasi-Newton Methods for Riemannian Optimization , 2015, SIAM J. Optim..

[17]  Felipe Cucker,et al.  Newton’s Method , 1998 .

[18]  Adrian S. Lewis,et al.  Nonsmooth optimization via quasi-Newton methods , 2012, Mathematical Programming.

[19]  Wen Huang,et al.  A Riemannian BFGS Method for Nonconvex Optimization Problems , 2015, ENUMATH.

[20]  Steven Thomas Smith,et al.  Optimization Techniques on Riemannian Manifolds , 2014, ArXiv.

[21]  M. R. Pouryayevali,et al.  Euler Characteristic of Epi-Lipschitz Subsets of Riemannian Manifolds , 2013 .

[22]  S. V. N. Vishwanathan,et al.  A Quasi-Newton Approach to Nonsmooth Convex Optimization Problems in Machine Learning , 2008, J. Mach. Learn. Res..

[23]  Pierre-Antoine Absil,et al.  Oriented Bounding Box Computation Using Particle Swarm Optimization , 2010, ESANN.

[24]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[25]  R. Adler,et al.  Newton's method on Riemannian manifolds and a geometric model for the human spine , 2002 .

[27]  R. Riddell,et al.  Minimax problems on Grassmann manifolds. Sums of eigenvalues , 1984 .

[28]  John Wright,et al.  Finding a Sparse Vector in a Subspace: Linear Sparsity Using Alternating Directions , 2014, IEEE Transactions on Information Theory.

[29]  C. Udriste,et al.  Convex Functions and Optimization Methods on Riemannian Manifolds , 1994 .

[30]  Daniel Azagra Rueda,et al.  Nonsmooth analysis and Hamilton-Jacobi equations on Riemannian manifolds , 2005 .

[31]  Philipp Grohs,et al.  ε-subgradient algorithms for locally lipschitz functions on Riemannian manifolds , 2015, Advances in Computational Mathematics.

[32]  Pierre-Antoine Absil,et al.  Riemannian BFGS Algorithm with Applications , 2010 .

[33]  Wen Huang,et al.  Intrinsic representation of tangent vectors and vector transports on matrix manifolds , 2017, Numerische Mathematik.

[34]  Pierre-Antoine Absil,et al.  Fast Oriented Bounding Box Computation Using Particle Swarm Optimization (abstract) , 2009, ESANN 2009.

[35]  Paulo Roberto Oliveira,et al.  Proximal point method for a special class of nonconvex functions on Hadamard manifolds , 2008, 0812.2201.

[36]  M. R. Pouryayevali,et al.  Generalized gradients and characterization of epi-Lipschitz sets in Riemannian manifolds , 2011 .

[37]  Nezam Mahdavi-Amiri,et al.  An Effective Nonsmooth Optimization Algorithm for Locally Lipschitz Functions , 2012, J. Optim. Theory Appl..

[38]  André Uschmajew,et al.  A Riemannian Gradient Sampling Algorithm for Nonsmooth Optimization on Manifolds , 2017, SIAM J. Optim..