Generative Bayesian Image Super Resolution With Natural Image Prior

We propose a new single image super resolution (SR) algorithm via Bayesian modeling with a natural image prior modeled by a high-order Markov random field (MRF). SR is one of the long-standing and active topics in image processing community. It is of great use in many practical applications, such as astronomical observation, medical imaging, and the adaptation of low-resolution contents onto high-resolution displays. One category of the conventional approaches for image SR is formulating the problem with Bayesian modeling techniques and then obtaining its maximum-a-posteriori solution, which actually boils down to a regularized regression task. Although straightforward, this approach cannot exploit the full potential offered by the probabilistic modeling, as only the posterior mode is sought. On the other hand, current Bayesian SR approaches using the posterior mean estimation typically use very simple prior models for natural images to ensure the computational tractability. In this paper, we present a Bayesian image SR approach with a flexible high-order MRF model as the prior for natural images. The minimum mean square error (MMSE) criteria are used for estimating the HR image. A Markov chain Monte Carlo-based sampling algorithm is presented for obtaining the MMSE solution. The proposed method cannot only enjoy the benefits offered by the flexible prior, but also has the advantage of making use of the probabilistic modeling to perform a posterior mean estimation, thus is less sensitive to the local minima problem as the MAP solution. Experimental results indicate that the proposed method can generate competitive or better results than state-of-the-art SR algorithms.

[1]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[2]  George Papandreou,et al.  Gaussian sampling by local perturbations , 2010, NIPS.

[3]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[4]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[5]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[6]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[7]  Paul W. Fieguth,et al.  Posterior Sampling of Scientific Images , 2006, ICIAR.

[8]  Chi-Keung Tang,et al.  Fast image/video upsampling , 2008, SIGGRAPH Asia '08.

[9]  Thomas S. Huang,et al.  Non-Local Kernel Regression for Image and Video Restoration , 2010, ECCV.

[10]  Raanan Fattal,et al.  Image upsampling via texture hallucination , 2010, 2010 IEEE International Conference on Computational Photography (ICCP).

[11]  William T. Freeman,et al.  What makes a good model of natural images? , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Bryan C. Russell,et al.  Exploiting the sparse derivative prior for super-resolution , 2003 .

[13]  Qi Gao,et al.  A generative perspective on MRFs in low-level vision , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[15]  Jean-François Giovannelli,et al.  Efficient sampling of high-dimensional Gaussian fields: the non-stationary / non-sparse case , 2011, ArXiv.

[16]  George Papandreou,et al.  Perturb-and-MAP random fields: Using discrete optimization to learn and sample from energy models , 2011, 2011 International Conference on Computer Vision.

[17]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[18]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[19]  Masato Inoue,et al.  Posterior-Mean Super-Resolution With a Causal Gaussian Markov Random Field Prior , 2011, IEEE Transactions on Image Processing.

[20]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[21]  Aggelos K. Katsaggelos,et al.  Variational Bayesian Super Resolution , 2011, IEEE Transactions on Image Processing.

[22]  Lawrence Carin,et al.  Tree-Structured Compressive Sensing With Variational Bayesian Analysis , 2010, IEEE Signal Processing Letters.

[23]  Christopher M. Bishop,et al.  Bayesian Image Super-Resolution , 2002, NIPS.

[24]  Richard Szeliski,et al.  Image Restoration by Matching Gradient Distributions , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Raanan Fattal,et al.  Image upsampling via imposed edge statistics , 2007, ACM Trans. Graph..

[27]  Stephen J. Roberts,et al.  A Sampled Texture Prior for Image Super-Resolution , 2003, NIPS.

[28]  Stephen Lin,et al.  Super resolution using edge prior and single image detail synthesis , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Michal Irani,et al.  Internal statistics of a single natural image , 2011, CVPR 2011.

[30]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[31]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[32]  Stefan Roth,et al.  Bayesian deblurring with integrated noise estimation , 2011, CVPR 2011.

[33]  Siwei Lyu,et al.  Interpretation and Generalization of Score Matching , 2009, UAI.

[34]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[35]  Sundaresh Ram,et al.  Removing Camera Shake from a Single Photograph , 2009 .

[36]  Michael J. Black,et al.  Fields of Experts , 2009, International Journal of Computer Vision.

[37]  Mei Han,et al.  SoftCuts: A Soft Edge Smoothness Prior for Color Image Super-Resolution , 2009, IEEE Transactions on Image Processing.

[38]  David J. C. MacKay,et al.  Ensemble Learning for Blind Image Separation and Deconvolution , 2000 .

[39]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.