A Bayesian Hyperprior Approach for Joint Image Denoising and Interpolation, With an Application to HDR Imaging

Recently, impressive denoising results have been achieved by Bayesian approaches which assume Gaussian models for the image patches. This improvement in performance can be attributed to the use of per-patch models. Unfortunately such an approach is particularly unstable for most inverse problems beyond denoising. In this paper, we propose the use of a hyperprior to model image patches, in order to stabilize the estimation procedure. There are two main advantages to the proposed restoration scheme: First, it is adapted to diagonal degradation matrices, and in particular to missing data problems (e.g., inpainting of missing pixels or zooming). Second, it can deal with signal dependent noise models, particularly suited to digital cameras. As such, the scheme is especially adapted to computational photography. In order to illustrate this point, we provide an application to high dynamic range imaging from a single image taken with a modified sensor, which shows the effectiveness of the proposed scheme.

[1]  J. Delon,et al.  Study of the digital camera acquisition process and statistical modeling of the sensor raw data , 2013 .

[2]  Peyman Milanfar,et al.  Patch-Based Near-Optimal Image Denoising , 2012, IEEE Transactions on Image Processing.

[3]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[4]  A. Ostrowski Solution of equations and systems of equations , 1967 .

[5]  Julie Delon,et al.  Simultaneous HDR image reconstruction and denoising for dynamic scenes , 2013, IEEE International Conference on Computational Photography (ICCP).

[6]  Julie Delon,et al.  Best Algorithms for HDR Image Generation. A Study of Performance Bounds , 2014, SIAM J. Imaging Sci..

[7]  Stéphane Mallat,et al.  Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity , 2010, IEEE Transactions on Image Processing.

[8]  Vicent Caselles,et al.  Analysis of a Variational Framework for Exemplar-Based Image Inpainting , 2012, Multiscale Model. Simul..

[9]  Rafael Molina,et al.  On the Hierarchical Bayesian Approach to Image Restoration: Applications to Astronomical Images , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[11]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[12]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[13]  William Puech,et al.  Ghost detection and removal in High Dynamic Range Images , 2009, 2009 17th European Signal Processing Conference.

[14]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[15]  Jürgen Seiler,et al.  Building a high dynamic range video sensor with spatially nonregular optical filtering , 2012, Other Conferences.

[16]  Aggelos K. Katsaggelos,et al.  Bayesian and regularization methods for hyperparameter estimation in image restoration , 1999, IEEE Trans. Image Process..

[17]  André Kaup,et al.  High dynamic range video by spatially non-regular optical filtering , 2012, 2012 19th IEEE International Conference on Image Processing.

[18]  Yi-Qing Wang,et al.  E-PLE: an Algorithm for Image Inpainting , 2013, Image Process. Line.

[19]  Kathrin Klamroth,et al.  Biconvex sets and optimization with biconvex functions: a survey and extensions , 2007, Math. Methods Oper. Res..

[20]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[21]  Chein-I Chang,et al.  Semi-Supervised Linear Spectral Unmixing Using a Hierarchical Bayesian Model for Hyperspectral Imagery , 2008, IEEE Transactions on Signal Processing.

[22]  Jean-Michel Morel,et al.  Implementation of the "Non-Local Bayes" (NL-Bayes) Image Denoising Algorithm , 2013, Image Process. Line.

[23]  Yi-Qing Wang,et al.  The Implementation of SURE Guided Piecewise Linear Image Denoising , 2013, Image Process. Line.

[24]  Shree K. Nayar,et al.  Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum , 2010, IEEE Transactions on Image Processing.

[25]  Hans-Peter Seidel,et al.  Optimal HDR reconstruction with linear digital cameras , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Jean-Michel Morel,et al.  A Nonlocal Bayesian Image Denoising Algorithm , 2013, SIAM J. Imaging Sci..

[27]  Shree K. Nayar,et al.  High dynamic range imaging: spatially varying pixel exposures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[28]  Eero P. Simoncelli,et al.  Modeling Multiscale Subbands of Photographic Images with Fields of Gaussian Scale Mixtures , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Jean-Michel Morel,et al.  SURE Guided Gaussian Mixture Image Denoising , 2013, SIAM J. Imaging Sci..

[30]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[31]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[32]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[33]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[34]  Eli Shechtman,et al.  Robust patch-based hdr reconstruction of dynamic scenes , 2012, ACM Trans. Graph..

[35]  Jean-Michel Morel,et al.  Secrets of image denoising cuisine* , 2012, Acta Numerica.

[36]  Rafal Mantiuk,et al.  Display adaptive tone mapping , 2008, ACM Trans. Graph..

[37]  Julie Delon,et al.  Single shot high dynamic range imaging using piecewise linear estimators , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).