Dualization of Signal Recovery Problems

In convex optimization, duality theory can sometimes lead to simpler solution methods than those resulting from direct primal analysis. In this paper, this principle is applied to a class of composite variational problems arising in particular in signal recovery. These problems are not easily amenable to solution by current methods but they feature Fenchel–Moreau–Rockafellar dual problems that can be solved by forward-backward splitting. The proposed algorithm produces simultaneously a sequence converging weakly to a dual solution, and a sequence converging strongly to the primal solution. Our framework is shown to capture and extend several existing duality-based signal recovery methods and to be applicable to a variety of new problems beyond their scope.

[1]  W. Fenchel.  Convex cones, sets, and functions , 1953 .

[2]  L. Sandgren.  On convex cones , 1954 .

[3]  J. Moreau.  Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[4]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[5]  S. Twomey.  The application of numerical filtering to the solution of integral equations encountered in indirect sensing measurements , 1965 .

[6]  L. Levi,et al.  Fitting a bandlimited signal to given points , 1965, IEEE Trans. Inf. Theory.

[7]  F. Browder.  Convergence theorems for sequences of nonlinear operators in Banach spaces , 1967 .

[8]  R. Rockafellar,et al.  Duality and stability in extremum problems involving convex functions. , 1967 .

[9]  E. Polak,et al.  Computational methods in optimization : a unified approach , 1972 .

[10]  R. Tyrrell Rockafellar.  Conjugate Duality and Optimization , 1974 .

[11]  R. Gerchberg.  Super-resolution through Error Energy Reduction , 1974 .

[12]  R. Temam,et al.  Analyse convexe et problèmes variationnels , 1974 .

[13]  A. Papoulis.  A new algorithm in spectral analysis and band-limited extrapolation. , 1975 .

[14]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[15]  Dante C. Youla,et al.  Generalized Image Restoration by the Method of Alternating Orthogonal Projections , 1978 .

[16]  D. Youla,et al.  Image Restoration by the Method of Convex Projections: Part 1ߞTheory , 1982, IEEE Transactions on Medical Imaging.

[17]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[18]  H. Trussell,et al.  The feasible solution in signal restoration , 1984 .

[19]  Allan O. Steinhardt,et al.  Spectral estimation via minimum energy correlation extension , 1985, IEEE Trans. Acoust. Speech Signal Process..

[20]  Richard M. Leahy,et al.  An optimal technique for constraint-based image restoration and reconstruction , 1986, IEEE Trans. Acoust. Speech Signal Process..

[21]  D. Youla,et al.  Extensions of a result on the synthesis of signals in the presence of inconsistent constraints , 1986 .

[22]  Henry Stark,et al.  Image recovery: Theory and application , 1987 .

[23]  Marc Teboulle,et al.  A dual approach to multidimensional L p spectral estimation problems , 1988 .

[24]  L D Cromwell,et al.  Filtering noise from images with wavelet transforms , 1991, Magnetic resonance in medicine.

[25]  Patrick L. Combettes,et al.  The use of noise properties in set theoretic estimation , 1991, IEEE Trans. Signal Process..

[26]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[27]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[28]  Marc Teboulle,et al.  A regularized dual-based iterative method for a class of image reconstruction problems , 1993 .

[29]  Patrick L. Combettes,et al.  Signal recovery by best feasible approximation , 1993, IEEE Trans. Image Process..

[30]  L. Potter,et al.  A dual approach to linear inverse problems with convex constraints , 1993 .

[31]  Charles L. Byrne,et al.  Signal Processing: A Mathematical Approach , 1993 .

[32]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[33]  G. Bellettini,et al.  A notion of total variation depending on a metric with discontinuous coefficients , 1994 .

[34]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[35]  Patrick L. Combettes,et al.  Inconsistent signal feasibility problems: least-squares solutions in a product space , 1994, IEEE Trans. Signal Process..

[36]  H. Engl,et al.  Regularization of Inverse Problems , 1996 .

[37]  Jonathan M. Borwein,et al.  Maximum Entropy Reconstruction Using Derivative Information, Part 1: Fisher Information and Convex Duality , 1996, Math. Oper. Res..

[38]  P. L. Combettes,et al.  The Convex Feasibility Problem in Image Recovery , 1996 .

[39]  Y. Censor,et al.  Parallel Optimization: Theory, Algorithms, and Applications , 1997 .

[40]  Y. Censor,et al.  Parallel Optimization:theory , 1997 .

[41]  Dominikus Noll,et al.  Reconstruction with Noisy Data: An Approach via Eigenvalue Optimization , 1998, SIAM J. Optim..

[42]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[43]  Gene H. Golub,et al.  A Nonlinear Primal-Dual Method for Total Variation-Based Image Restoration , 1999, SIAM J. Sci. Comput..

[44]  Tommi Kärkkäinen,et al.  Comparison of formulations and solution methods for image restoration problems , 2001 .

[45]  Gjlles Aubert,et al.  Mathematical problems in image processing , 2001 .

[46]  S. Osher,et al.  IMAGE DECOMPOSITION AND RESTORATION USING TOTAL VARIATION MINIMIZATION AND THE H−1 NORM∗ , 2002 .

[47]  C. Zălinescu.  Convex analysis in general vector spaces , 2002 .

[48]  M Berteroi,et al.  Linear inverse problems with discrete data. I. General formulation and singular system analysis , 1985 .

[49]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[50]  Zdzisław Denkowski,et al.  Set-Valued Analysis , 2021 .

[51]  Antonin Chambolle,et al.  A l1-Unified Variational Framework for Image Restoration , 2004, ECCV.

[52]  Rachid Deriche,et al.  Regularizing Flows for Constrained Matrix-Valued Images , 2004 .

[53]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[54]  Yair Censor,et al.  A multiprojection algorithm using Bregman projections in a product space , 1994, Numerical Algorithms.

[55]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[56]  Antonin Chambolle,et al.  Total Variation Minimization and a Class of Binary MRF Models , 2005, EMMCVPR.

[57]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[58]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[59]  Michael Hintermüller,et al.  An Infeasible Primal-Dual Algorithm for Total Bounded Variation-Based Inf-Convolution-Type Image Restoration , 2006, SIAM J. Sci. Comput..

[60]  José M. Bioucas-Dias,et al.  A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration , 2007, IEEE Transactions on Image Processing.

[61]  Mila Nikolova,et al.  Denoising of Frame Coefficients Using ℓ1 Data-Fidelity Term and Edge-Preserving Regularization , 2007, Multiscale Model. Simul..

[62]  P. L. Combettes,et al.  A Dykstra-like algorithm for two monotone operators , 2007 .

[63]  Valérie R. Wajs,et al.  A variational formulation for frame-based inverse problems , 2007 .

[64]  M. Fornasier.  Domain decomposition methods for linear inverse problems with sparsity constraints , 2007 .

[65]  M. Jaoua,et al.  A dual algorithm for denoising and preserving edges in image processing , 2007 .

[66]  Patrick L. Combettes,et al.  Proximal Thresholding Algorithm for Minimization over Orthonormal Bases , 2007, SIAM J. Optim..

[67]  I. Daubechies,et al.  Iteratively solving linear inverse problems under general convex constraints , 2007 .

[68]  J.-C. Pesquet,et al.  A Douglas–Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery , 2007, IEEE Journal of Selected Topics in Signal Processing.

[69]  K. Bredies,et al.  Linear Convergence of Iterative Soft-Thresholding , 2007, 0709.1598.

[70]  P. L. Combettes,et al.  A proximal decomposition method for solving convex variational inverse problems , 2008, 0807.2617.

[71]  Jian-Feng Cai,et al.  A framelet-based image inpainting algorithm , 2008 .

[72]  Michael K. Ng,et al.  A Fast Total Variation Minimization Method for Image Restoration , 2008, Multiscale Model. Simul..

[73]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[74]  Gilles Aubert,et al.  Efficient Schemes for Total Variation Minimization Under Constraints in Image Processing , 2009, SIAM J. Sci. Comput..

[75]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[76]  Mohamed-Jalal Fadili,et al.  Total Variation Projection With First Order Schemes , 2011, IEEE Transactions on Image Processing.

[77]  Luis,et al.  Convex Variational Formulation with Smooth Coupling for Multicomponent Signal Decomposition and Recovery , 2009 .

[78]  Heinz H. Bauschke,et al.  The Baillon-Haddad Theorem Revisited , 2009, 0906.0807.

[79]  Raymond H. Chan,et al.  Convergence analysis of tight framelet approach for missing data recovery , 2009, Adv. Comput. Math..

[80]  Nelly Pustelnik,et al.  Nested Iterative Algorithms for Convex Constrained Image Recovery Problems , 2008, SIAM J. Imaging Sci..

[81]  Jonathan M. Borwein,et al.  Duality and convex programming , 2010 .