An accelerated newton's method for projections onto the ℓ1-ball

We present a simple and computationally efficient algorithm, based on the accelerated Newton's method, to solve the root finding problem associated with the projection onto the ℓ1-ball problem. Considering an interpretation of the Michelot's algorithm as Newton method, our algorithm can be understood as an accelerated version of the Michelot's algorithm, that needs significantly less major iterations to converge to the solution. Although the worst-case performance of the propose algorithm is O(n2), it exhibits in practice an O(n) performance and it is empirically demonstrated that it is competitive or faster than existing methods.

[1]  A. Fiacco.  A Finite Algorithm for Finding the Projection of a Point onto the Canonical Simplex of R " , 2009 .

[2]  M. A. Salanova,et al.  A family of newton type iterative processes , 1994 .

[3]  Jun Liu,et al.  Efficient Euclidean projections in linear time , 2009, ICML '09.

[4]  S. Sra.  Generalized Proximity and Projection with Norms and Mixed-norms , 2010 .

[5]  Jürgen Gerlach,et al.  Accelerated Convergence in Newton's Method , 1994, SIAM Rev..

[6]  Krzysztof C. Kiwiel,et al.  Breakpoint searching algorithms for the continuous quadratic knapsack problem , 2007, Math. Program..

[7]  Suvrit Sra,et al.  Fast projections onto mixed-norm balls with applications , 2012, Data Mining and Knowledge Discovery.

[8]  Brendt Wohlberg,et al.  An incremental principal component pursuit algorithm via projections onto the ℓ1 ball , 2017, 2017 IEEE XXIV International Conference on Electronics, Electrical Engineering and Computing (INTERCON).

[9]  K. Kiwiel.  Variable Fixing Algorithms for the Continuous Quadratic Knapsack Problem , 2008 .

[10]  Yoram Singer,et al.  Efficient projections onto the l1-ball for learning in high dimensions , 2008, ICML '08.

[11]  Abdulla Desmal,et al.  Sparse Nonlinear Electromagnetic Imaging Accelerated With Projected Steepest Descent Algorithm , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[12]  C. Michelot.  A finite algorithm for finding the projection of a point onto the canonical simplex of ∝n , 1986 .

[13]  Roberto Cominetti,et al.  A Newton’s method for the continuous quadratic knapsack problem , 2014, Math. Program. Comput..

[14]  I. Daubechies,et al.  Accelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints , 2007, 0706.4297.

[15]  L. Zanni,et al.  Accelerating gradient projection methods for ℓ1-constrained signal recovery by steplength selection rules , 2009 .

[16]  Laurent Condat,et al.  A Fast Projection onto the Simplex and the l 1 Ball , 2015 .

[17]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[18]  Philip Wolfe,et al.  Validation of subgradient optimization , 1974, Math. Program..

[19]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[20]  Changshui Zhang,et al.  Efficient Euclidean projections via Piecewise Root Finding and its application in gradient projection , 2011, Neurocomputing.