Image Restoration Using Conditional Random Fields and Scale Mixtures of Gaussians

This paper proposes a general framework for internal patch-based image restoration based on Conditional Random Fields (CRF). Unlike related models based on Markov Random Fields (MRF), our approach explicitly formulates the posterior distribution for the entire image. The potential functions are taken as proportional to the product of a likelihood and prior for each patch. By assuming identical parameters for similar patches, our approach can be classified as a model-based non-local method. For the prior term in the potential function of the CRF model, multivariate Gaussians and multivariate scale-mixture of Gaussians are considered, with the latter being a novel prior for image patches. Our results show that the proposed approach outperforms methods based on Gaussian mixture models for image denoising and state-of-the-art methods for image interpolation/inpainting.

[1]  José M. Bioucas-Dias,et al.  Class-specific poisson denoising by patch-based importance sampling , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[2]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[3]  Guangming Shi,et al.  Nonlocal Image Restoration With Bilateral Variance Estimation: A Low-Rank Approach , 2013, IEEE Transactions on Image Processing.

[4]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[5]  Michael Elad,et al.  Expected Patch Log Likelihood with a Sparse Prior , 2014, EMMCVPR.

[6]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[7]  Yasubumi Sakakibara,et al.  RNA secondary structural alignment with conditional random fields , 2005, ECCB/JBI.

[8]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[9]  Kostadin Dabov,et al.  BM3D Image Denoising with Shape-Adaptive Principal Component Analysis , 2009 .

[10]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[11]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[12]  Mário A. T. Figueiredo.  Synthesis versus analysis in patch-based image priors , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Fernando Pereira,et al.  Shallow Parsing with Conditional Random Fields , 2003, NAACL.

[14]  R. Courant.  Variational methods for the solution of problems of equilibrium and vibrations , 1943 .

[15]  Mário A. T. Figueiredo,et al.  Single-frame Image Denoising and Inpainting Using Gaussian Mixtures , 2015, ICPRAM.

[16]  David B. Dunson,et al.  Nonparametric Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images , 2012, IEEE Transactions on Image Processing.

[17]  Eero P. Simoncelli,et al.  Statistical Modeling of Images with Fields of Gaussian Scale Mixtures , 2006, NIPS.

[18]  Truong Q. Nguyen,et al.  Image restoration with generalized Gaussian mixture model patch priors , 2018, ArXiv.

[19]  Michal Irani,et al.  Combining the power of Internal and External denoising , 2013, IEEE International Conference on Computational Photography (ICCP).

[20]  José M. Bioucas-Dias,et al.  Class-specific image denoising using importance sampling , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[21]  Edward H. Adelson,et al.  Learning Gaussian Conditional Random Fields for Low-Level Vision , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Matthias Bethge,et al.  Modeling Natural Image Statistics , 2015 .

[23]  Martial Hebert,et al.  Discriminative Fields for Modeling Spatial Dependencies in Natural Images , 2003, NIPS.

[24]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[25]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[26]  Truong Q. Nguyen,et al.  Adaptive Image Denoising by Targeted Databases , 2014, IEEE Transactions on Image Processing.

[27]  Truong Q. Nguyen,et al.  Adaptive Image Denoising by Mixture Adaptation , 2016, IEEE Transactions on Image Processing.

[28]  Stéphane Mallat,et al.  Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity , 2010, IEEE Transactions on Image Processing.

[29]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[30]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Guangming Shi,et al.  Image Restoration via Simultaneous Sparse Coding: Where Structured Sparsity Meets Gaussian Scale Mixture , 2015, International Journal of Computer Vision.

[32]  Massoud Babaie-Zadeh,et al.  Image Restoration Using Gaussian Mixture Models With Spatially Constrained Patch Clustering , 2015, IEEE Transactions on Image Processing.

[33]  Andrew McCallum,et al.  An Introduction to Conditional Random Fields , 2010, Found. Trends Mach. Learn..

[34]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[35]  Jean-Michel Morel,et al.  A Nonlocal Bayesian Image Denoising Algorithm , 2013, SIAM J. Imaging Sci..

[36]  Jian Zhang,et al.  Image Denoising via Bandwise Adaptive Modeling and Regularization Exploiting Nonlocal Similarity. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[37]  Martial Hebert,et al.  Discriminative random fields: a discriminative framework for contextual interaction in classification , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[38]  José M. Bioucas-Dias,et al.  Image restoration and reconstruction using variable splitting and class-adapted image priors , 2016, 2016 IEEE International Conference on Image Processing (ICIP).