Semantic super-resolution: When and where is it useful?

Abstract Recent algorithms for exemplar-based single image super-resolution have shown impressive results, mainly due to well-chosen priors and recently also due to more accurate blur kernels. Some methods exploit clustering of patches, local gradients or some context information. However, to the best of our knowledge, there is no literature studying the benefits of using semantic information at the image level. By semantic information we mean image segments with corresponding categorical labels. In this paper we investigate the use of semantic information in conjunction with A+, a state-of-the-art super-resolution method. We conduct experiments on large standard datasets of natural images with semantic annotations, and discuss the benefits vs. the drawbacks of using semantic information. Experimental results show that our semantic driven super-resolution can significantly improve over the original settings.

[1]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[2]  Ce Liu,et al.  A Bayesian Approach to Alignment-Based Image Hallucination , 2012, ECCV.

[3]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Luc Van Gool,et al.  Pedestrian detection at 100 frames per second , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  William T. Freeman,et al.  Learning low-level vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[7]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[8]  Leon Hirsch,et al.  Super Resolution From A Single Image , 2016 .

[9]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[11]  Michal Irani,et al.  Nonparametric Blind Super-resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[12]  Zhe L. Lin,et al.  Fast Image Super-Resolution Based on In-Place Example Regression , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Stephen J. Roberts,et al.  A Sampled Texture Prior for Image Super-Resolution , 2003, NIPS.

[14]  Jiejie Zhu,et al.  Context-constrained hallucination for image super-resolution , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[16]  Tetsuya Takiguchi,et al.  Super-resolution by GMM based conversion using self-reduction image , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  Raanan Fattal,et al.  Image upsampling via texture hallucination , 2010, 2010 IEEE International Conference on Computational Photography (ICCP).

[18]  Bhabatosh Chanda,et al.  Image Upscaling Using Multiple Dictionaries of Natural Image Patches , 2012, ACCV.

[19]  Antonio Torralba,et al.  Contextual Priming for Object Detection , 2003, International Journal of Computer Vision.

[20]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Luc Van Gool,et al.  Nonuniform image patch exemplars for low level vision , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[22]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[23]  Thomas Hofmann,et al.  Probabilistic latent semantic indexing , 1999, SIGIR '99.

[24]  Marc Pollefeys,et al.  Pulling Things out of Perspective , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  David A. Forsyth,et al.  30Hz Object Detection with DPM V5 , 2014, ECCV.

[26]  James Hays,et al.  Super-resolution from internet-scale scene matching , 2012, 2012 IEEE International Conference on Computational Photography (ICCP).

[27]  Stephen Lin,et al.  Super resolution using edge prior and single image detail synthesis , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.