Context-constrained hallucination for image super-resolution

This paper proposes a context-constrained hallucination approach for image super-resolution. Through building a training set of high-resolution/low-resolution image segment pairs, the high-resolution pixel is hallucinated from its texturally similar segments which are retrieved from the training set by texture similarity. Given the discrete hallucinated examples, a continuous energy function is designed to enforce the fidelity of high-resolution image to low-resolution input and the constraints imposed by the hallucinated examples and the edge smoothness prior. The reconstructed high-resolution image is sharp with minimal artifacts both along the edges and in the textural regions.

[1]  Michal Irani,et al.  Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency , 1993, J. Vis. Commun. Image Represent..

[2]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[3]  Chi-Keung Tang,et al.  Fast image/video upsampling , 2008, SIGGRAPH Asia '08.

[4]  Harry Shum,et al.  Patch based blind image super resolution , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[5]  Wai-kuen Cham,et al.  Learning-based face hallucination in DCT domain , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[7]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[8]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[9]  E. Adelson,et al.  Early vision and texture perception , 1988, Nature.

[10]  Yihong Gong,et al.  Resolution-Invariant Image Representation and its applications , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Harry Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[14]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[15]  Feng Wu,et al.  Image hallucination with feature enhancement , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Harry Shum,et al.  Face Hallucination: Theory and Practice , 2007, International Journal of Computer Vision.

[17]  Mei Han,et al.  Soft Edge Smoothness Prior for Alpha Channel Super Resolution , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Bryan C. Russell,et al.  Exploiting the sparse derivative prior for super-resolution , 2003 .

[19]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[20]  Zhou Wang,et al.  Image Quality Assessment: From Error Measurement to Structural Similarity , 2004 .

[21]  Raanan Fattal,et al.  Image upsampling via imposed edge statistics , 2007, ACM Trans. Graph..

[22]  Ying Wu,et al.  Contextual flow , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Bryan S. Morse,et al.  Image magnification using level-set reconstruction , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[25]  Sylvain Lefebvre,et al.  Appearance-space texture synthesis , 2006, ACM Trans. Graph..

[26]  Chi-Keung Tang,et al.  Perceptually-Inspired and Edge-Directed Color Image Super-Resolution , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[27]  Jitendra Malik,et al.  Learning affinity functions for image segmentation: combining patch-based and gradient-based approaches , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[28]  Stephen J. Roberts,et al.  A Sampled Texture Prior for Image Super-Resolution , 2003, NIPS.

[29]  Sylvain Lefebvre,et al.  Parallel controllable texture synthesis , 2005, ACM Trans. Graph..

[30]  DeLiang Wang,et al.  Texture classification using spectral histograms , 2003, IEEE Trans. Image Process..

[31]  Nanning Zheng,et al.  Image hallucination with primal sketch priors , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[32]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.