A Collection of Nonsmooth Riemannian Optimization Problems

Nonsmooth Riemannian optimization is still a scarcely explored subfield of optimization theory that concerns the general problem of minimizing (or maximizing) over a domain endowed with a manifold structure, a real-valued function that is not everywhere differentiable. The purpose of this paper is to illustrate, by means of nine concrete examples, that nonsmooth Riemannian optimization finds numerous applications in engineering and the sciences.

[1]  M. Levas.  OBBTree : A Hierarchical Structure for Rapid Interference Detection , .

[2]  Pierre-Antoine Absil,et al.  A Riemannian subgradient algorithm for economic dispatch with valve-point effect , 2014, J. Comput. Appl. Math..

[3]  Wen Huang,et al.  A Broyden Class of Quasi-Newton Methods for Riemannian Optimization , 2015, SIAM J. Optim..

[4]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[5]  David W. Dreisigmeyer.  A Quasi-Isometric Embedding Algorithm , 2019, Pattern Recognition and Image Analysis.

[6]  Umberto Amato,et al.  A Derivative-Free Riemannian Powell’s Method, Minimizing Hartley-Entropy-Based ICA Contrast , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[7]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[8]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[9]  Philipp Grohs,et al.  Total variation regularization on Riemannian manifolds by iteratively reweighted minimization , 2016 .

[10]  Pierre-Antoine Absil,et al.  Robust Low-Rank Matrix Completion by Riemannian Optimization , 2016, SIAM J. Sci. Comput..

[11]  F. Bach,et al.  Low-rank optimization for semidefinite convex problems , 2008, 0807.4423.

[12]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[13]  Adrian S. Lewis,et al.  Nonsmooth optimization via quasi-Newton methods , 2012, Mathematical Programming.

[14]  Niklas Koep,et al.  Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiation , 2016, J. Mach. Learn. Res..

[15]  Pierre-Antoine Absil,et al.  Fast Oriented Bounding Box Computation Using Particle Swarm Optimization (abstract) , 2009, ESANN 2009.

[16]  Hiroyuki Sato,et al.  A Dai–Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions , 2014, Comput. Optim. Appl..

[17]  Bamdev Mishra,et al.  Manopt, a matlab toolbox for optimization on manifolds , 2013, J. Mach. Learn. Res..

[18]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[19]  Wen Huang,et al.  Intrinsic representation of tangent vectors and vector transports on matrix manifolds , 2017, Numerische Mathematik.

[20]  Pierre-Antoine Absil,et al.  Spherical Mesh Adaptive Direct Search for Separating Quasi-Uncorrelated Sources by Range-Based Independent Component Analysis , 2013, Neural Computation.

[22]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[23]  Michel Verleysen,et al.  A Minimum-Range Approach to Blind Extraction of Bounded Sources , 2007, IEEE Transactions on Neural Networks.

[24]  D. Broomhead,et al.  Dimensionality Reduction Using Secant-Based Projection Methods: The Induced Dynamics in Projected Systems , 2005 .

[25]  Benedikt Wirth,et al.  Optimization Methods on Riemannian Manifolds and Their Application to Shape Space , 2012, SIAM J. Optim..

[26]  Christer Ericson,et al.  Real-Time Collision Detection , 2004 .

[27]  Nickolay T. Trendafilov,et al.  From simple structure to sparse components: a review , 2014, Comput. Stat..

[28]  Philipp Grohs,et al.  ε-subgradient algorithms for locally lipschitz functions on Riemannian manifolds , 2015, Advances in Computational Mathematics.

[29]  David H. Eberly,et al.  Geometric Tools for Computer Graphics , 2002 .

[30]  S. Thomas George,et al.  Range-Based ICA Using a Nonsmooth Quasi-Newton Optimizer for Electroencephalographic Source Localization in Focal Epilepsy , 2015, Neural Computation.

[31]  V. A. Morozov,et al.  Methods for Solving Incorrectly Posed Problems , 1984 .

[32]  Francis R. Bach,et al.  Low-Rank Optimization on the Cone of Positive Semidefinite Matrices , 2008, SIAM J. Optim..

[33]  Chi-keung Chan,et al.  Minimum bounding boxes and volume decomposition of CAD models , 2003 .

[34]  Michael I. Jordan,et al.  A Direct Formulation for Sparse Pca Using Semidefinite Programming , 2004, NIPS 2004.

[35]  Allen J. Wood,et al.  Power Generation, Operation, and Control , 1984 .

[36]  Wen Huang,et al.  Weakly Correlated Sparse Components with Nearly Orthonormal Loadings , 2015, GSI.

[37]  Wang Jiaying,et al.  A modified particle swarm optimization algorithm , 2005 .

[38]  Gabriele Steidl,et al.  A Second Order Nonsmooth Variational Model for Restoring Manifold-Valued Images , 2015, SIAM J. Sci. Comput..

[39]  André Uschmajew,et al.  A Riemannian Gradient Sampling Algorithm for Nonsmooth Optimization on Manifolds , 2017, SIAM J. Optim..

[40]  Pierre-Antoine Absil,et al.  A Modified Particle Swarm Optimization Algorithm for the Best Low Multilinear Rank Approximation of Higher-Order Tensors , 2010, ANTS Conference.

[41]  John Wright,et al.  Finding a Sparse Vector in a Subspace: Linear Sparsity Using Alternating Directions , 2014, IEEE Transactions on Information Theory.

[42]  C. Udriste,et al.  Convex Functions and Optimization Methods on Riemannian Manifolds , 1994 .

[43]  V. Ivanov,et al.  The Theory of Approximate Methods and Their Application to the Numerical Solution of Singular Integr , 1978 .

[44]  Tony F. Chan,et al.  Guarantees of Riemannian Optimization for Low Rank Matrix Recovery , 2015, SIAM J. Matrix Anal. Appl..

[45]  U. Helmke,et al.  Nonsmooth Riemannian Optimization with Applications to Sphere Packing and Grasping , 2007 .

[46]  I. Jolliffe,et al.  A Modified Principal Component Technique Based on the LASSO , 2003 .

[47]  Michael M. Bronstein,et al.  MADMM: A Generic Algorithm for Non-smooth Optimization on Manifolds , 2015, ECCV.

[48]  M. Kleinsteuber,et al.  Intrinsic Newton’s Method on Oblique Manifolds for Overdetermined Blind Source Separation , 2010 .

[49]  P. Grohs,et al.  Nonsmooth trust region algorithms for locally Lipschitz functions on Riemannian manifolds , 2016 .

[50]  Yurii Nesterov,et al.  Generalized Power Method for Sparse Principal Component Analysis , 2008, J. Mach. Learn. Res..

[51]  Wen Huang,et al.  ROPTLIB , 2018, ACM Trans. Math. Softw..

[52]  D. Donoho,et al.  Sparse nonnegative solution of underdetermined linear equations by linear programming. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[53]  Laurent Demanet,et al.  Recovering the Sparsest Element in a Subspace , 2013, 1310.1654.

[54]  Wen Huang,et al.  Optimization algorithms on Riemannian manifolds with applications , 2013 .

[55]  G. Sheblé,et al.  Genetic algorithm solution of economic dispatch with valve point loading , 1993 .

[56]  D. Gabay.  Minimizing a differentiable function over a differential manifold , 1982 .

[57]  Pierre-Antoine Absil,et al.  Joint Diagonalization on the Oblique Manifold for Independent Component Analysis , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[58]  Timothy N. Davidson,et al.  Noncoherent MIMO communication: Grassmannian constellations and efficient detection , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[59]  W. Huang,et al.  Line Search Algorithms for Locally Lipschitz Functions on Riemannian Manifolds , 2018, SIAM J. Optim..

[60]  D. Botstein,et al.  Singular value decomposition for genome-wide expression data processing and modeling. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[61]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[62]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[63]  R. Adler,et al.  Newton's method on Riemannian manifolds and a geometric model for the human spine , 2002 .

[64]  D. Dreisigmeyer,et al.  DIRECT SEARCH ALGORITHMS OVER RIEMANNIAN MANIFOLDS , 2006 .

[65]  Lizhong Zheng,et al.  Communication on the Grassmann manifold: A geometric approach to the noncoherent multiple-antenna channel , 2002, IEEE Trans. Inf. Theory.