An $O(n\log(n))$ Algorithm for Projecting Onto the Ordered Weighted $\ell_1$ Norm Ball.

The ordered weighted '1 (OWL) norm is a newly developed generalization of the Octogonal Shrinkage and Clustering Algorithm for Regression (OSCAR) norm. This norm has desirable statistical properties and can be used to perform simultaneous clustering and regression. In this paper, we show how to compute the projection of an n-dimensional vector onto the OWL norm ball in O(n log(n)) operations. In addition, we illustrate the performance of our algorithm on a synthetic regression test.

[1]  Julien Mairal,et al.  Structured sparsity through convex optimization , 2011, ArXiv.

[2]  Philip Wolfe,et al.  Validation of subgradient optimization , 1974, Math. Program..

[3]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[4]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[5]  Noah Simon,et al.  A Sparse-Group Lasso , 2013 .

[6]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[7]  Philip Wolfe,et al.  An algorithm for quadratic programming , 1956 .

[8]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[9]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[10]  Weijie J. Su,et al.  Statistical estimation and testing via the sorted L1 norm , 2013, 1310.1969.

[11]  Mário A. T. Figueiredo,et al.  Decreasing Weighted Sorted ${\ell_1}$ Regularization , 2014, IEEE Signal Processing Letters.

[12]  H. Bondell,et al.  Simultaneous Regression Shrinkage, Variable Selection, and Supervised Clustering of Predictors with OSCAR , 2008, Biometrics.

[13]  Xiangrong Zeng,et al.  The Ordered Weighted $\ell_1$ Norm: Atomic Formulation, Projections, and Algorithms , 2014, 1409.4271.

[14]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[15]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[16]  Gregory B. Passty.  Ergodic convergence to a zero of the sum of monotone operators in Hilbert space , 1979 .

[17]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .