Gradient Profile Prior and Its Applications in Image Super-Resolution and Enhancement

In this paper, we propose a novel generic image prior-gradient profile prior, which implies the prior knowledge of natural image gradients. In this prior, the image gradients are represented by gradient profiles, which are 1-D profiles of gradient magnitudes perpendicular to image structures. We model the gradient profiles by a parametric gradient profile model. Using this model, the prior knowledge of the gradient profiles are learned from a large collection of natural images, which are called gradient profile prior. Based on this prior, we propose a gradient field transformation to constrain the gradient fields of the high resolution image and the enhanced image when performing single image super-resolution and sharpness enhancement. With this simple but very effective approach, we are able to produce state-of-the-art results. The reconstructed high resolution images or the enhanced images are sharp while have rare ringing or jaggy artifacts.

[1]  Hsieh Hou,et al.  Cubic splines for image interpolation and digital filtering , 1978 .

[2]  T. Poggio,et al.  Fingerprints theorems for zero crossings , 1985 .

[3]  L. Rudin.  Images, Numerical Analysis of Singularities and Shock Filters , 1987 .

[4]  M. Varanasi,et al.  Parametric generalized Gaussian density estimation , 1989 .

[5]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  L. Rudin,et al.  Feature-oriented image enhancement using shock filters , 1990 .

[7]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[8]  Michal Irani,et al.  Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency , 1993, J. Vis. Commun. Image Represent..

[9]  Song-Chun Zhu,et al.  Prior Learning and Gibbs Reaction-Diffusion , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Song-Chun Zhu,et al.  Learning generic prior models for visual computation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  P. Moulin,et al.  Analysis of multiresolution image denoising schemes using generalized-Gaussian priors , 1998, Proceedings of the IEEE-SP International Symposium on Time-Frequency and Time-Scale Analysis (Cat. No.98TH8380).

[12]  Pierre Moulin,et al.  Analysis of Multiresolution Image Denoising Schemes Using Generalized Gaussian and Complexity Priors , 1999, IEEE Trans. Inf. Theory.

[13]  David Mumford,et al.  Statistics of natural images and models , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[14]  Marcel J. T. Reinders,et al.  Image sharpening by morphological filtering , 2000, Pattern Recognit..

[15]  Tony F. Chan,et al.  High-Order Total Variation-Based Image Restoration , 2000, SIAM J. Sci. Comput..

[16]  Michael Unser,et al.  Image interpolation and resampling , 2000 .

[17]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[18]  Giovanni Ramponi,et al.  Image enhancement via adaptive unsharp masking , 2000, IEEE Trans. Image Process..

[19]  M. Orchard,et al.  New edge-directed interpolation , 2001, IEEE Trans. Image Process..

[20]  Robert Allen,et al.  Handbook of Medical Imaging—Processing and Analysis , 2001 .

[21]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[22]  Joachim Weickert,et al.  Coherence-Enhancing Shock Filters , 2003, DAGM-Symposium.

[23]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[24]  Assaf Zomet,et al.  Learning how to inpaint from global image statistics , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[25]  Bryan C. Russell,et al.  Exploiting the sparse derivative prior for super-resolution , 2003 .

[26]  Nanning Zheng,et al.  Image hallucination with primal sketch priors , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[27]  Zhou Wang,et al.  Image Quality Assessment: From Error Measurement to Structural Similarity , 2004 .

[28]  Zhouchen Lin,et al.  Response to the Comments on "Fundamental Limits of Reconstruction-Based Superresolution Algorithms under Local Translation' , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[30]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[31]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[32]  Jean-Michel Morel,et al.  A Note on Two Classical Enhancement Filters and Their Associated PDE's , 2003, International Journal of Computer Vision.

[33]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[34]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[35]  Harry Shum,et al.  Patch based blind image super resolution , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[36]  Eero P. Simoncelli,et al.  Statistical Modeling of Images with Fields of Gaussian Scale Mixtures , 2006, NIPS.

[37]  Rama Chellappa,et al.  What Is the Range of Surface Reconstructions from a Gradient Field? , 2006, ECCV.

[38]  Harry Shum,et al.  Face Hallucination: Theory and Practice , 2007, International Journal of Computer Vision.

[39]  Peyman Milanfar,et al.  Modeling multiscale differential pixel statistics , 2006, Electronic Imaging.

[40]  Mei Han,et al.  Soft Edge Smoothness Prior for Alpha Channel Super Resolution , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Raanan Fattal.  Image upsampling via imposed edge statistics , 2007, SIGGRAPH 2007.

[42]  Raanan Fattal,et al.  Image upsampling via imposed edge statistics , 2007, ACM Trans. Graph..

[43]  Moshe Ben-Ezra,et al.  Penrose Pixels Super-Resolution in the Detector Layout Domain , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[44]  Michael J. Black,et al.  Steerable Random Fields , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[45]  H. Shum,et al.  Image super-resolution using gradient profile prior , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Xiangjun Zhang,et al.  Image Interpolation by Adaptive 2-D Autoregressive Modeling and Soft-Decision Estimation , 2008, IEEE Transactions on Image Processing.

[47]  Richard Szeliski,et al.  PSF estimation using sharp edge prediction , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Mohamed-Jalal Fadili,et al.  Inpainting and Zooming Using Sparse Representations , 2009, Comput. J..

[50]  Sundaresh Ram,et al.  Removing Camera Shake from a Single Photograph , 2009 .

[51]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[52]  Stéphane Mallat,et al.  Super-Resolution With Sparse Mixing Estimators , 2010, IEEE Transactions on Image Processing.