Analysis of Multiresolution Image Denoising Schemes Using Generalized Gaussian and Complexity Priors

Research on universal and minimax wavelet shrinkage and thresholding methods has demonstrated near-ideal estimation performance in various asymptotic frameworks. However, image processing practice has shown that universal thresholding methods are outperformed by simple Bayesian estimators assuming independent wavelet coefficients and heavy-tailed priors such as generalized Gaussian distributions (GGDs). In this paper, we investigate various connections between shrinkage methods and maximum a posteriori (MAP) estimation using such priors. In particular, we state a simple condition under which MAP estimates are sparse. We also introduce a new family of complexity priors based upon Rissanen's universal prior on integers. One particular estimator in this class outperforms conventional estimators based on earlier applications of the minimum description length (MDL) principle. We develop analytical expressions for the shrinkage rules implied by GGD and complexity priors. This allows us to show the equivalence between universal hard thresholding, MAP estimation using a very heavy-tailed GGD, and MDL estimation using one of the new complexity priors. Theoretical analysis supported by numerous practical experiments shows the robustness of some of these estimates against mis-specifications of the prior-a basic concern in image processing applications.

[1]  Andrew R. Barron,et al.  Minimum complexity density estimation , 1991, IEEE Trans. Inf. Theory.

[2]  J. Berger Statistical Decision Theory and Bayesian Analysis , 1988 .

[3]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[4]  Sujit K. Ghosh,et al.  Essential Wavelets for Statistical Applications and Data Analysis , 2001, Technometrics.

[5]  A. Kolmogorov Three approaches to the quantitative definition of information , 1968 .

[6]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[8]  Ronald A. DeVore,et al.  Classifying the smoothness of images: theory and applications to wavelet image processing , 1994, Proceedings of 1st International Conference on Image Processing.

[9]  Jerome M. Shapiro,et al.  Embedded image coding using zerotrees of wavelet coefficients , 1993, IEEE Trans. Signal Process..

[10]  P. Massart,et al.  From Model Selection to Adaptive Estimation , 1997 .

[11]  Michael W. Marcellin,et al.  Comparison of different methods of classification in subband coding of images , 1997, IEEE Trans. Image Process..

[12]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[13]  Andrew G. Bruce,et al.  WaveShrink: shrinkage functions and thresholds , 1995, Optics + Photonics.

[14]  Pierre Moulin,et al.  Complexity-regularized image restoration , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[15]  D. Donoho Nonlinear Solution of Linear Inverse Problems by Wavelet–Vaguelette Decomposition , 1995 .

[16]  Pierre Moulin Signal estimation using adapted tree-structured bases and the MDL principle , 1996, Proceedings of Third International Symposium on Time-Frequency and Time-Scale Analysis (TFTS-96).

[17]  Pierre Moulin,et al.  Complexity-regularized image denoising , 1997, Proceedings of International Conference on Image Processing.

[18]  Ronald A. DeVore,et al.  Image compression through wavelet transform coding , 1992, IEEE Trans. Inf. Theory.

[19]  P. J. Huber The behavior of maximum likelihood estimates under nonstandard conditions , 1967 .

[20]  J. Rissanen Information in prediction and estimation , 1983, The 22nd IEEE Conference on Decision and Control.

[21]  David Leporini,et al.  Bayesian approach to best basis selection , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[22]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[23]  Pierre Moulin Model selection criteria and the orthogonal series method for function estimation , 1995, Proceedings of 1995 IEEE International Symposium on Information Theory.

[24]  P. J. Huber Robust Statistical Procedures , 1977 .

[25]  S. Orszag,et al.  Advanced Mathematical Methods For Scientists And Engineers , 1979 .

[26]  B. Vidakovic Nonlinear wavelet shrinkage with Bayes rules and Bayes factors , 1998 .

[27]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[28]  L. Breiman Better subset regression using the nonnegative garrote , 1995 .

[29]  Jan Biemond,et al.  Subband Coding of Color Images , 1991 .

[30]  Peter J. Huber,et al.  Robust Statistical Procedures: Second Edition , 1996 .

[31]  H. Chipman,et al.  Adaptive Bayesian Wavelet Shrinkage , 1997 .

[32]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[33]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.

[34]  G. Nason Wavelet Shrinkage using Cross-validation , 1996 .

[35]  Michael T. Orchard,et al.  Image coding based on mixture modeling of wavelet coefficients and a fast estimation-quantization framework , 1997, Proceedings DCC '97. Data Compression Conference.

[36]  Halbert White,et al.  Estimation, inference, and specification analysis , 1996 .

[37]  Hong-Ye Gao,et al.  Wavelet Shrinkage Denoising Using the Non-Negative Garrote , 1998 .

[38]  R. Ogden,et al.  Essential Wavelets for Statistical Applications and Data Analysis , 1996 .

[39]  M. Clyde,et al.  Multiple shrinkage and subset selection in wavelets , 1998 .

[40]  Martin Vetterli,et al.  Image denoising via lossy compression and wavelet thresholding , 1997, Proceedings of International Conference on Image Processing.

[41]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[42]  Edward H. Adelson,et al.  Noise removal via Bayesian wavelet coring , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.