Smoothing algorithms for computing the projection onto a Minkowski sum of convex sets

In this paper, the problem of computing the projection, and therefore the minimum distance, from a point onto a Minkowski sum of general convex sets is studied. Our approach is based on the minimum norm duality theorem originally stated by Nirenberg and the Nesterov smoothing techniques. It is shown that projection points onto a Minkowski sum of sets can be represented as the sum of points on constituent sets so that, at these points, all of the sets share the same normal vector which is the negative of the dual solution. The proposed NESMINO algorithm improves the theoretical bound on number of iterations from $O(\frac{1}{\epsilon})$ by Gilbert [SIAM J. Contr., vol. 4, pp. 61--80, 1966] to $O\left(\frac{1}{\sqrt{\epsilon}}\ln(\frac{1}{\epsilon})\right)$, where $\epsilon$ is the desired accuracy for the objective function. Moreover, the algorithm also provides points on each component sets such that their sum is equal to the projection point.

[1]  Hua Zhou,et al.  Algorithms for Fitting the Constrained Lasso , 2016, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[2]  Francis Bach,et al.  SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives , 2014, NIPS.

[3]  Hyunjoong Kim,et al.  Functional Analysis I , 2017 .

[4]  Laurent Condat Fast projection onto the simplex and the l1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pmb {l}_\mathbf {1}$$\end{ , 2015, Mathematical Programming.

[5]  Stephen Cameron,et al.  Enhancing GJK: computing minimum and penetration distances between convex polyhedra , 1997, Proceedings of International Conference on Robotics and Automation.

[6]  Martin Jaggi,et al.  On the Global Linear Convergence of Frank-Wolfe Optimization Variants , 2015, NIPS.

[7]  Yoram Singer,et al.  Efficient projections onto the l1-ball for learning in high dimensions , 2008, ICML '08.

[8]  Y. Nesterov A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2) , 1983 .

[9]  Boris S. Mordukhovich,et al.  Limiting subgradients of minimal time functions in Banach spaces , 2010, J. Glob. Optim..

[10]  Jieping Ye,et al.  Efficient Methods for Overlapping Group Lasso , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Elad Hazan,et al.  Faster Rates for the Frank-Wolfe Method over Strongly-Convex Sets , 2014, ICML.

[12]  V. N. Malozemov,et al.  Finding the Point of a Polyhedron Closest to the Origin , 1974 .

[13]  S. Sathiya Keerthi,et al.  A fast iterative nearest point algorithm for support vector machine classifier design , 2000, IEEE Trans. Neural Networks Learn. Syst..

[14]  E. Gilbert An Iterative Procedure for Computing the Minimum of a Quadratic Form on a Convex Set , 1966 .

[15]  Joong-Ho Won,et al.  Projection onto Minkowski Sums with Application to Constrained Learning , 2019, ICML.

[16]  Mark W. Schmidt,et al.  Minimizing finite sums with the stochastic average gradient , 2013, Mathematical Programming.

[17]  Martin Jaggi,et al.  Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization , 2013, ICML.

[18]  P. Varaiya,et al.  Ellipsoidal Toolbox (ET) , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[19]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[20]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[21]  Laurent Condat,et al.  A Fast Projection onto the Simplex and the l 1 Ball , 2015 .

[22]  Z. Gabidullina The Problem of Projecting the Origin of Euclidean Space onto the Convex Polyhedron , 2016, 1605.05351.

[23]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[24]  Elmer G. Gilbert,et al.  Computing the distance between general convex objects in three-dimensional space , 1990, IEEE Trans. Robotics Autom..

[25]  J. Borwein,et al.  Convex Analysis And Nonlinear Optimization , 2000 .

[26]  Y. Nesterov A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[27]  R. Rockafellar Convex Analysis: (pms-28) , 1970 .

[28]  丸山 徹 Convex Analysisの二,三の進展について , 1977 .

[29]  Achiya Dax,et al.  A New Class of Minimum Norm Duality Theorems , 2008, SIAM J. Optim..

[30]  Philip Wolfe,et al.  An algorithm for quadratic programming , 1956 .

[31]  Shawn Martin Training support vector machines using Gilbert's algorithm , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[32]  Gino van den Bergen A Fast and Robust GJK Implementation for Collision Detection of Convex Objects , 1999, J. Graphics, GPU, & Game Tools.

[33]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[34]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[35]  Yu-Hong Dai,et al.  Fast Algorithms for Projection on an Ellipsoid , 2006, SIAM J. Optim..

[36]  Heinz H. Bauschke,et al.  On sums and convex combinations of projectors onto convex sets , 2018, J. Approx. Theory.

[37]  Jie Sun,et al.  Nonsmooth Algorithms and Nesterov's Smoothing Technique for Generalized Fermat-Torricelli Problems , 2014, SIAM J. Optim..

[38]  Sven Strauss,et al.  Convex Analysis And Global Optimization , 2016 .

[39]  Amir Beck,et al.  First-Order Methods in Optimization , 2017 .

[40]  Mato Baotic,et al.  Multi-Parametric Toolbox (MPT) , 2004, HSCC.

[41]  Philip Wolfe,et al.  Finding the nearest point in A polytope , 1976, Math. Program..

[42]  S. Sathiya Keerthi,et al.  A fast procedure for computing the distance between complex objects in three-dimensional space , 1988, IEEE J. Robotics Autom..

[43]  R. Tibshirani,et al.  The solution path of the generalized lasso , 2010, 1005.1971.

[44]  Liang Chang,et al.  An Improved Gilbert Algorithm with Rapid Convergence , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.