UniHIST: A unified framework for image restoration with marginal histogram constraints

Marginal histograms provide valuable information for various computer vision problems. However, current image restoration methods do not fully exploit the potential of marginal histograms, in particular, their role as ensemble constraints on the marginal statistics of the restored image. In this paper, we introduce a new framework, UniHIST, to incorporate marginal histogram constraints into image restoration. The key idea of UniHIST is to minimize the discrepancy between the marginal histograms of the restored image and the reference histograms in pixel or gradient domains using the quadratic Wasserstein (W2) distance. The W2 distance can be computed directly from data without resorting to density estimation. It provides a differentiable metric between marginal histograms and allows easy integration with existing image restoration methods. We demonstrate the effectiveness of UniHIST through denoising of pattern images and non-blind deconvolution of natural images. We show that UniHIST enhances restoration performance and leads to visual and quantitative improvements over existing state-of-the-art methods.

[1]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[2]  David Zhang,et al.  A Generalized Iterated Shrinkage Algorithm for Non-convex Sparse Coding , 2013, 2013 IEEE International Conference on Computer Vision.

[3]  Shree K. Nayar,et al.  Multiresolution histograms and their use for recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[5]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[6]  A.C. Kokaram,et al.  N-dimensional probability density function transfer and its application to color transfer , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[7]  David Zhang,et al.  Gradient Histogram Estimation and Preservation for Texture Enhanced Image Denoising , 2014, IEEE Transactions on Image Processing.

[8]  Richard Szeliski,et al.  A content-aware image prior , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Vladimir Kolmogorov,et al.  A global perspective on MAP inference for low-level vision , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[10]  D. Geman,et al.  Nonlinear Image Recovery with , 1995 .

[11]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[12]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[13]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[14]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[15]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[16]  Jean-Michel Morel,et al.  Non-local dual image denoising , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[17]  David Mumford,et al.  Statistics of natural images and models , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[18]  Michael J. Black,et al.  Field of experts , 2011 .

[19]  Song-Chun Zhu.  Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling , 1998 .

[20]  Rafael C. González,et al.  Digital image processing, 3rd Edition , 2008 .

[21]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[22]  William T. Freeman,et al.  Removing camera shake from a single photograph , 2006, SIGGRAPH 2006.

[23]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[24]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[25]  Richard Szeliski,et al.  Image Restoration by Matching Gradient Distributions , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Julien Rabin,et al.  Wasserstein Regularization of Imaging Problems , 2017 .

[27]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[28]  Martin J. Wainwright,et al.  Scale Mixtures of Gaussians and the Statistics of Natural Images , 1999, NIPS.

[29]  Sung-Hyuk Cha,et al.  On measuring the distance between histograms , 2002, Pattern Recognit..

[30]  Aggelos K. Katsaggelos,et al.  Digital image restoration , 2012, IEEE Signal Process. Mag..

[31]  Julien Rabin,et al.  Wasserstein regularization of imaging problem , 2011, 2011 18th IEEE International Conference on Image Processing.

[32]  Julien Rabin,et al.  Sliced and Radon Wasserstein Barycenters of Measures , 2014, Journal of Mathematical Imaging and Vision.

[33]  Qi Gao,et al.  A generative perspective on MRFs in low-level vision , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  C. Villani.  Optimal Transport: Old and New , 2008 .

[35]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[36]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, CVPR.

[37]  Edward H. Adelson,et al.  Noise removal via Bayesian wavelet coring , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[38]  Kari Pulli,et al.  FlexISP , 2014, ACM Trans. Graph..

[39]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.