Signal Recovery With Certain Involved Convex Data-Fidelity Constraints

This paper proposes an optimization framework that can efficiently deal with convex data-fidelity constraints onto which the metric projections are difficult to compute. Although such an involved data-fidelity constraint is expected to play an important role in signal recovery under non-Gaussian noise contamination, the said difficulty precludes existing algorithms from solving convex optimization problems with the constraint. To resolve this dilemma, we introduce a fixed point set characterization of involved data-fidelity constraints based on a certain computable quasi-nonexpansive mapping. This characterization enables us to mobilize the hybrid steepest descent method to solve convex optimization problems with such a constraint. The proposed framework can handle a variety of involved data-fidelity constraints in a unified manner, without geometric approximation to them. In addition, it requires no computationally expensive procedure such as operator inversion and inner loop. As applications of the proposed framework, we provide image restoration under several types of non-Gaussian noise contamination with illustrative examples.

[1]  Nelly Pustelnik,et al.  Epigraphical Projection and Proximal Tools for Solving Constrained Convex Optimization Problems: Part I , 2012, ArXiv.

[2]  P. L. Combettes The foundations of set theoretic estimation , 1993 .

[3]  Shunsuke Ono,et al.  Hierarchical Convex Optimization With Primal-Dual Splitting , 2015, IEEE Transactions on Signal Processing.

[4]  Mohamed-Jalal Fadili,et al.  A Proximal Iteration for Deconvolving Poisson Noisy Images Using Sparse Representations , 2008, IEEE Transactions on Image Processing.

[5]  Sergios Theodoridis,et al.  Online Sparse System Identification and Signal Reconstruction Using Projections Onto Weighted $\ell_{1}$ Balls , 2010, IEEE Transactions on Signal Processing.

[6]  P. L. Combettes,et al.  Foundation of set theoretic estimation , 1993 .

[7]  Isao Yamada,et al.  Robust Wideband Beamforming by the Hybrid Steepest Descent Method , 2007, IEEE Transactions on Signal Processing.

[8]  Panos M. Pardalos,et al.  An algorithm for a singly constrained class of quadratic programs subject to upper and lower bounds , 1990, Math. Program..

[9]  J.-C. Pesquet,et al.  A Douglas–Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery , 2007, IEEE Journal of Selected Topics in Signal Processing.

[10]  Jianing Shi,et al.  A Nonlinear Inverse Scale Space Method for a Convex Multiplicative Noise Model , 2008, SIAM J. Imaging Sci..

[11]  José M. Bioucas-Dias,et al.  Multiplicative Noise Removal Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[12]  H. Wu,et al.  Adaptive impulse detection using center-weighted median filters , 2001, IEEE Signal Processing Letters.

[13]  J. Moreau Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[14]  Isao Yamada,et al.  An efficient robust adaptive filtering algorithm based on parallel subgradient projection techniques , 2002, IEEE Trans. Signal Process..

[15]  Patrick L. Combettes,et al.  A block-iterative surrogate constraint splitting method for quadratic signal recovery , 2003, IEEE Trans. Signal Process..

[16]  Ezequiel López-Rubio,et al.  Restoration of images corrupted by Gaussian and uniform impulsive noise , 2010, Pattern Recognit..

[17]  Emmanuel J. Candès,et al.  NESTA: A Fast and Accurate First-Order Method for Sparse Recovery , 2009, SIAM J. Imaging Sci..

[18]  Paul Petrus,et al.  Robust Huber adaptive filter , 1999, IEEE Trans. Signal Process..

[19]  Mila Nikolova,et al.  Minimizers of Cost-Functions Involving Nonsmooth Data-Fidelity Terms. Application to the Processing of Outliers , 2002, SIAM J. Numer. Anal..

[20]  Shunsuke Ono,et al.  Second-order total Generalized Variation constraint , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[21]  I. Yamada,et al.  Hybrid Steepest Descent Method for Variational Inequality Problem over the Fixed Point Set of Certain Quasi-nonexpansive Mappings , 2005 .

[22]  Michael B. Wakin Sparse Image and Signal Processing: Wavelets, Curvelets, Morphological Diversity (Starck, J.-L., et al; 2010) [Book Reviews] , 2011, IEEE Signal Processing Magazine.

[23]  R. Chan,et al.  Minimization and parameter estimation for seminorm regularization models with I-divergence constraints , 2013 .

[24]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[25]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[26]  Shunsuke Ono,et al.  Cartoon-Texture Image Decomposition Using Blockwise Low-Rank Texture Characterization , 2014, IEEE Transactions on Image Processing.

[27]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[28]  Shunsuke Ono,et al.  Image Recovery by Decomposition with Component-Wise Regularization , 2012, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[29]  Isao Yamada,et al.  Adaptive proximal forward-backward splitting for sparse system identification under impulsive noise , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[30]  Michael K. Ng,et al.  A New Total Variation Method for Multiplicative Noise Removal , 2009, SIAM J. Imaging Sci..

[31]  Shunsuke Ono,et al.  Poisson image restoration with likelihood constraint via hybrid steepest descent method , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[32]  Michael P. Friedlander,et al.  Variational Properties of Value Functions , 2012, SIAM J. Optim..

[33]  Isao Yamada,et al.  Minimizing the Moreau Envelope of Nonsmooth Convex Functions over the Fixed Point Set of Certain Quasi-Nonexpansive Mappings , 2011, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[34]  Isao Yamada,et al.  The Adaptive Projected Subgradient Method Constrained by Families of Quasi-nonexpansive Mappings and Its Application to Online Learning , 2010, SIAM J. Optim..

[35]  Gabriele Steidl,et al.  Homogeneous Penalizers and Constraints in Convex Image Restoration , 2013, Journal of Mathematical Imaging and Vision.

[36]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[37]  Kristian Bredies,et al.  Total Generalized Variation in Diffusion Tensor Imaging , 2013, SIAM J. Imaging Sci..

[38]  Y. Censor,et al.  Parallel Optimization:theory , 1997 .

[39]  Gilles Aubert,et al.  A Variational Approach to Removing Multiplicative Noise , 2008, SIAM J. Appl. Math..

[40]  Patrick L. Combettes,et al.  Strong Convergence of Block-Iterative Outer Approximation Methods for Convex Optimization , 2000, SIAM J. Control. Optim..

[41]  Nelly Pustelnik,et al.  Parallel Proximal Algorithm for Image Restoration Using Hybrid Regularization , 2009, IEEE Transactions on Image Processing.

[42]  Stanley Osher,et al.  Multiplicative Denoising and Deblurring: Theory and Algorithms , 2003 .

[43]  Laure Blanc-Féraud,et al.  Sparse Poisson Noisy Image Deblurring , 2012, IEEE Transactions on Image Processing.

[44]  Shunsuke Ono,et al.  A Convex Regularizer for Reducing Color Artifact in Color Image Recovery , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[46]  Irene Fonseca,et al.  A Higher Order Model for Image Restoration: The One-Dimensional Case , 2007, SIAM J. Math. Anal..

[47]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[48]  Y. Censor,et al.  Parallel Optimization: Theory, Algorithms, and Applications , 1997 .

[49]  Brendt Wohlberg,et al.  MIxed gaussian-impulse noise image restoration via total variation , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[50]  Karl Kunisch,et al.  Total Generalized Variation , 2010, SIAM J. Imaging Sci..

[51]  Jian Yu,et al.  Restoration of images corrupted by mixed Gaussian-impulse noise via l1-l0 minimization , 2011, Pattern Recognit..

[52]  Lixin Shen,et al.  Proximity algorithms for the L1/TV image denoising model , 2011, Advances in Computational Mathematics.

[53]  Yoram Singer,et al.  Efficient projections onto the l1-ball for learning in high dimensions , 2008, ICML '08.

[54]  I. Yamada The Hybrid Steepest Descent Method for the Variational Inequality Problem over the Intersection of Fixed Point Sets of Nonexpansive Mappings , 2001 .

[55]  I. Yamada,et al.  Total generalized variation for graph signals , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[56]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[57]  Gabriele Steidl,et al.  Deblurring Poissonian images by split Bregman techniques , 2010, J. Vis. Commun. Image Represent..

[58]  Heinz H. Bauschke,et al.  A Weak-to-Strong Convergence Principle for Fejé-Monotone Methods in Hilbert Spaces , 2001, Math. Oper. Res..

[59]  Sergios Theodoridis,et al.  Adaptive Learning in a World of Projections , 2011, IEEE Signal Processing Magazine.

[60]  Horst Bischof,et al.  Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation , 2013, 2013 IEEE International Conference on Computer Vision.

[61]  Patrick L. Combettes,et al.  Image restoration subject to a total variation constraint , 2004, IEEE Transactions on Image Processing.

[62]  Shunsuke Ono,et al.  Decorrelated Vectorial Total Variation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Tieyong Zeng,et al.  A Convex Variational Model for Restoring Blurred Images with Multiplicative Noise , 2013, SIAM J. Imaging Sci..

[64]  Gabriele Steidl,et al.  Removing Multiplicative Noise by Douglas-Rachford Splitting Methods , 2010, Journal of Mathematical Imaging and Vision.

[65]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[66]  Lixin Shen,et al.  Framelet Algorithms for De-Blurring Images Corrupted by Impulse Plus Gaussian Noise , 2011, IEEE Transactions on Image Processing.

[67]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[68]  José M. Bioucas-Dias,et al.  Restoration of Poissonian Images Using Alternating Direction Optimization , 2010, IEEE Transactions on Image Processing.

[69]  Michael K. Ng,et al.  Fast Image Restoration Methods for Impulse and Gaussian Noises Removal , 2009, IEEE Signal Processing Letters.

[70]  Volkan Cevher,et al.  Convex Optimization for Big Data: Scalable, randomized, and parallel algorithms for big data analytics , 2014, IEEE Signal Processing Magazine.

[71]  L. Vandenberghe,et al.  Convex Optimization in Signal Processing and Communications: Graphical models of autoregressive processes , 2009 .

[72]  Mohamed-Jalal Fadili,et al.  Multiplicative Noise Removal Using L1 Fidelity on Frame Coefficients , 2008, Journal of Mathematical Imaging and Vision.

[73]  Mohamed-Jalal Fadili,et al.  Wavelets, Ridgelets, and Curvelets for Poisson Noise Removal , 2008, IEEE Transactions on Image Processing.

[74]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[75]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[76]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[77]  I. Yamada,et al.  Adaptive Projected Subgradient Method for Asymptotic Minimization of Sequence of Nonnegative Convex Functions , 2005 .

[78]  Nelly Pustelnik,et al.  A Nonlocal Structure Tensor-Based Approach for Multicomponent Image Recovery Problems , 2014, IEEE Transactions on Image Processing.

[79]  Yang Gao,et al.  Speckle Reduction via Higher Order Total Variation Approach , 2014, IEEE Transactions on Image Processing.

[80]  Jian-Feng Cai,et al.  Two-phase approach for deblurring images corrupted by impulse plus gaussian noise , 2008 .

[81]  Andrzej Stachurski,et al.  Parallel Optimization: Theory, Algorithms and Applications , 2000, Parallel Distributed Comput. Pract..

[82]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.

[83]  Masahiro Yukawa,et al.  Online model selection and learning by multikernel adaptive filtering , 2013, 21st European Signal Processing Conference (EUSIPCO 2013).

[84]  Nelly Pustelnik,et al.  A proximal approach for constrained cosparse modelling , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).