An Extension of Averaged-Operator-Based algorithms

Many of the algorithms used to solve minimization problems with sparsity-inducing regularizers are generic in the sense that they do not take into account the sparsity of the solution in any particular way. However, algorithms known as semismooth Newton are able to take advantage of this sparsity to accelerate their convergence. We show how to extend these algorithms in different directions, and study the convergence of the resulting algorithms by showing that they are a particular case of an extension of the well-known Krasnosel‘skiľ-Mann scheme.

[1]  P. L. Combettes,et al.  Variable metric forward–backward splitting with applications to monotone inclusions in duality , 2012, 1206.6791.

[2]  D. Lorenz,et al.  A semismooth Newton method for Tikhonov functionals with sparsity constraints , 2007, 0709.3186.

[3]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[4]  Michael Ulbrich,et al.  Semismooth Newton Methods for Variational Inequalities and Constrained Optimization Problems in Function Spaces , 2011, MOS-SIAM Series on Optimization.

[5]  Kazufumi Ito,et al.  The Primal-Dual Active Set Strategy as a Semismooth Newton Method , 2002, SIAM J. Optim..

[6]  Mohamed-Jalal Fadili,et al.  A quasi-Newton proximal splitting method , 2012, NIPS.

[7]  C. Byrne,et al.  A unified treatment of some iterative algorithms in signal processing and image reconstruction , 2003 .

[8]  Mark W. Schmidt,et al.  Projected Newton-type methods in machine learning , 2011 .

[9]  Liqun Qi,et al.  Convergence Analysis of Some Algorithms for Solving Nonsmooth Equations , 1993, Math. Oper. Res..

[10]  P. L. Combettes,et al.  Variable metric quasi-Fejér monotonicity , 2012, 1206.5705.

[11]  J. Moreau.  Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[12]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[13]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[14]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[15]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[16]  Piotr Indyk,et al.  Recent Developments in the Sparse Fourier Transform: A compressed Fourier transform for big data , 2014, IEEE Signal Processing Magazine.

[17]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[18]  Michael A. Saunders,et al.  Proximal Newton-Type Methods for Minimizing Composite Functions , 2012, SIAM J. Optim..

[19]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.