Translation-invariant shrinkage/thresholding of group sparse signals

This paper addresses signal denoising when large-amplitude coefficients form clusters (groups). The L1-norm and other separable sparsity models do not capture the tendency of coefficients to cluster (group sparsity). This work develops an algorithm, called ‘overlapping group shrinkage’ (OGS), based on the minimization of a convex cost function involving a group-sparsity promoting penalty function. The groups are fully overlapping so the denoising method is translation-invariant and blocking artifacts are avoided. Based on the principle of majorization-minimization (MM), we derive a simple iterative minimization algorithm that reduces the cost function monotonically. A procedure for setting the regularization parameter, based on attenuating the noise to a specified level, is also described. The proposed approach is illustrated on speech enhancement, wherein the OGS approach is applied in the short-time Fourier transform (STFT) domain. The denoised speech produced by OGS does not suffer from musical noise.

[1]  Michael Elad,et al.  On the Role of Sparse and Redundant Representations in Image Processing , 2010, Proceedings of the IEEE.

[2]  Ilker Bayram,et al.  Mixed norms with overlapping groups as signal priors , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Robert D. Nowak,et al.  Wavelet-based image estimation: an empirical Bayes approach using Jeffrey's noninformative prior , 2001, IEEE Trans. Image Process..

[4]  S. Lelean Learning about research. , 1977, Nursing times.

[5]  Mohamed-Jalal Fadili,et al.  Analytical form for a Bayesian wavelet estimator of images using the Bessel K form densities , 2005, IEEE Transactions on Image Processing.

[6]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[7]  M. Kowalski Sparse regression using mixed norms , 2009 .

[8]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[9]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[10]  Francis R. Bach,et al.  Structured Variable Selection with Sparsity-Inducing Norms , 2009, J. Mach. Learn. Res..

[11]  Kai Siedenburg,et al.  STRUCTURED SPARSITY FOR AUDIO SIGNALS , 2011 .

[12]  S. Godsill,et al.  Bayesian variable selection and regularization for time–frequency surface estimation , 2004 .

[13]  Richard G. Baraniuk,et al.  Improved wavelet denoising via empirical Wiener filtering , 1997, Optics & Photonics.

[14]  Julien Mairal,et al.  Structured sparsity through convex optimization , 2011, ArXiv.

[15]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Levent Sendur,et al.  Bivariate shrinkage functions for wavelet-based denoising exploiting interscale dependency , 2002, IEEE Trans. Signal Process..

[17]  Nelly Pustelnik,et al.  Parallel Proximal Algorithm for Image Restoration Using Hybrid Regularization , 2009, IEEE Transactions on Image Processing.

[18]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[19]  Hong-Ye Gao,et al.  Wavelet Shrinkage Denoising Using the Non-Negative Garrote , 1998 .

[20]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[21]  Bruno Torrésani,et al.  Sparsity and persistence: mixed norms provide simple signal models with dependent coefficients , 2009, Signal Image Video Process..

[22]  Robert D. Nowak,et al.  Majorization–Minimization Algorithms for Wavelet-Based Image Restoration , 2007, IEEE Transactions on Image Processing.

[23]  Kannan Ramchandran,et al.  Low-complexity image denoising based on statistical modeling of wavelet coefficients , 1999, IEEE Signal Processing Letters.

[24]  Nick G. Kingsbury,et al.  Convex approaches to model wavelet sparsity patterns , 2011, 2011 18th IEEE International Conference on Image Processing.

[25]  Wotao Yin,et al.  Group sparse optimization by alternating direction method , 2013, Optics & Photonics - Optical Engineering + Applications.

[26]  Hyvarinen Sparse code shrinkage: denoising of nongaussian data by maximum likelihood estimation , 1999, Neural computation.

[27]  Reza Nezafat,et al.  Wavelet-Domain Medical Image Denoising Using Bivariate Laplacian Mixture Model , 2009, IEEE Transactions on Biomedical Engineering.

[28]  R. McAulay,et al.  Speech enhancement using a soft-decision noise suppression filter , 1980 .

[29]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[30]  Xi Chen,et al.  Smoothing proximal gradient method for general structured sparse regression , 2010, The Annals of Applied Statistics.

[31]  Amel Benazza-Benyahia,et al.  A hierarchical Bayesian model for frame representation , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[32]  Olivier Cappé,et al.  Elimination of the musical noise phenomenon with the Ephraim and Malah noise suppressor , 1994, IEEE Trans. Speech Audio Process..

[33]  Julien Mairal,et al.  Proximal Methods for Sparse Hierarchical Dictionary Learning , 2010, ICML.

[34]  Jean-Philippe Vert,et al.  Group lasso with overlap and graph lasso , 2009, ICML '09.

[35]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[36]  Mohamed-Jalal Fadili,et al.  Group sparsity with overlapping partition functions , 2011, 2011 19th European Signal Processing Conference.

[37]  Tien D. Bui,et al.  Multivariate statistical modeling for image denoising using wavelet transforms , 2005, Signal Process. Image Commun..

[38]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[39]  Richard M. Schwartz,et al.  Enhancement of speech corrupted by acoustic noise , 1979, ICASSP.

[40]  I. Cohen Optimal speech enhancement under signal presence uncertainty using log-spectral amplitude estimator , 2002, IEEE Signal Processing Letters.

[41]  Stéphane Mallat,et al.  Audio Denoising by Time-Frequency Block Thresholding , 2008, IEEE Transactions on Signal Processing.

[42]  S. Boll,et al.  Suppression of acoustic noise in speech using spectral subtraction , 1979 .

[43]  Lorenzo Rosasco,et al.  A Primal-Dual Algorithm for Group Sparse Regularization with Overlapping Groups , 2010, NIPS.

[44]  Yi Hu,et al.  A generalized subspace approach for enhancing speech corrupted by colored noise , 2003, IEEE Trans. Speech Audio Process..

[45]  Simon J. Godsill,et al.  Sparse Linear Regression With Structured Priors and Application to Denoising of Musical Audio , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[46]  Rainer Martin,et al.  MAP Estimators for Speech Enhancement Under Normal and Rayleigh Inverse Gaussian Distributions , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[47]  Ali Taylan Cemgil,et al.  Gamma Markov Random Fields for Audio Source Modeling , 2009, IEEE Transactions on Audio, Speech, and Language Processing.

[48]  Jerome M. Shapiro,et al.  Embedded image coding using zerotrees of wavelet coefficients , 1993, IEEE Trans. Signal Process..

[49]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[50]  Rainer Martin,et al.  Speech enhancement based on minimum mean-square error estimation and supergaussian priors , 2005, IEEE Transactions on Speech and Audio Processing.

[51]  Andrzej Cichocki,et al.  Improved M-FOCUSS Algorithm With Overlapping Blocks for Locally Smooth Sparse Signals , 2008, IEEE Transactions on Signal Processing.

[52]  José M. Bioucas-Dias,et al.  Adaptive total variation image deblurring: A majorization-minimization approach , 2009, Signal Process..

[53]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[54]  Pierre Moulin,et al.  Information-theoretic analysis of interscale and intrascale dependencies between image wavelet coefficients , 2001, IEEE Trans. Image Process..

[55]  Jean-Philippe Vert,et al.  Group Lasso with Overlaps: the Latent Group Lasso approach , 2011, ArXiv.

[56]  Gabriel Peyré,et al.  Proximal Splitting Derivatives for Risk Estimation , 2012 .

[57]  L. Sendur,et al.  Multivariate shrinkage functions for wavelet-based denoising , 2002, Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers, 2002..

[58]  Aleksandra Pizurica,et al.  Image Denoising Using Mixtures of Projected Gaussian Scale Mixtures , 2009, IEEE Transactions on Image Processing.

[59]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[60]  Monika Dörfler,et al.  Persistent Time-Frequency Shrinkage for Audio Denoising , 2013 .

[61]  Ivan W. Selesnick,et al.  The Estimation of Laplace Random Vectors in Additive White Gaussian Noise , 2008, IEEE Transactions on Signal Processing.

[62]  Bhaskar D. Rao,et al.  Subset selection in noise based on diversity measure minimization , 2003, IEEE Trans. Signal Process..

[63]  Alin Achim,et al.  Image denoising using bivariate α-stable distributions in the complex wavelet domain , 2005, IEEE Signal Processing Letters.

[64]  Ivan W. Selesnick,et al.  Sparse Signal Estimation by Maximally Sparse Convex Optimization , 2013, IEEE Transactions on Signal Processing.

[65]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[66]  Jieping Ye,et al.  Efficient Methods for Overlapping Group Lasso , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  B. Silverman,et al.  Incorporating Information on Neighboring Coefficients Into Wavelet Estimation , 2001 .

[68]  Aleksandra Pizurica,et al.  A joint inter- and intrascale statistical model for Bayesian wavelet based image denoising , 2002, IEEE Trans. Image Process..

[69]  Philipos C. Loizou,et al.  Speech Enhancement: Theory and Practice , 2007 .

[70]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[71]  Mihai Datcu,et al.  Gibbs Random Field Models for Model-Based Despeckling of SAR Images , 2010, IEEE Geoscience and Remote Sensing Letters.

[72]  Bhaskar D. Rao,et al.  An affine scaling methodology for best basis selection , 1999, IEEE Trans. Signal Process..

[73]  Penglang Shui,et al.  Image denoising algorithm using doubly local Wiener filtering with block-adaptive windows in wavelet domain , 2007, Signal Process..