Fidelity-Naturalness Evaluation of Single Image Super Resolution

We study the problem of evaluating super resolution methods. Traditional evaluation methods usually judge the quality of super resolved images based on a single measure of their difference with the original high resolution images. In this paper, we proposed to use both fidelity (the difference with original images) and naturalness (human visual perception of super resolved images) for evaluation. For fidelity evaluation, a new metric is proposed to solve the bias problem of traditional evaluation. For naturalness evaluation, we let humans label preference of super resolution results using pair-wise comparison, and test the correlation between human labeling results and image quality assessment metrics' outputs. Experimental results show that our fidelity-naturalness method is better than the traditional evaluation method for super resolution methods, which could help future research on single-image super resolution.

[1]  Zhe L. Lin,et al.  Fast Image Super-Resolution Based on In-Place Example Regression , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Ashish Kapoor,et al.  Learning a blind measure of perceptual image quality , 2011, CVPR 2011.

[3]  Raanan Fattal,et al.  Image upsampling via imposed edge statistics , 2007, ACM Trans. Graph..

[4]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[5]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[6]  Harry Shum,et al.  Gradient Profile Prior and Its Applications in Image Super-Resolution and Enhancement , 2011, IEEE Transactions on Image Processing.

[7]  Gordon Erlebacher,et al.  Hybrid No-Reference Natural Image Quality Assessment of Noisy, Blurry, JPEG2000, and JPEG Images , 2011, IEEE Transactions on Image Processing.

[8]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[9]  David S. Doermann,et al.  No-Reference Image Quality Assessment Using Visual Codebooks , 2012, IEEE Transactions on Image Processing.

[10]  Lei Zhang,et al.  Blind Image Quality Assessment Using Joint Statistics of Gradient Magnitude and Laplacian Features , 2014, IEEE Transactions on Image Processing.

[11]  Gustavo de Veciana,et al.  An information fidelity criterion for image quality assessment using natural scene statistics , 2005, IEEE Transactions on Image Processing.

[12]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[13]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[14]  David S. Doermann,et al.  Unsupervised feature learning framework for no-reference image quality assessment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Shiguang Shan,et al.  Deep Network Cascade for Image Super-resolution , 2014, ECCV.

[16]  Raanan Fattal,et al.  Image and video upscaling from local self-examples , 2011, TOGS.

[17]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Nikolay N. Ponomarenko,et al.  TID2008 – A database for evaluation of full-reference visual quality assessment metrics , 2004 .

[19]  David S. Doermann,et al.  Real-Time No-Reference Image Quality Assessment Based on Filter Learning , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Alan C. Bovik,et al.  A Two-Step Framework for Constructing Blind Image Quality Indices , 2010, IEEE Signal Processing Letters.

[21]  Hua Huang,et al.  Neighbor embedding based super-resolution algorithm through edge detection and feature selection , 2009, Pattern Recognit. Lett..

[22]  Alan C. Bovik,et al.  Image information and visual quality , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[23]  Yanning Zhang,et al.  Single Image Super-resolution Using Deformable Patches , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[25]  Jing Hu,et al.  Single-image superresolution based on local regression and nonlocal self-similarity , 2014, J. Electronic Imaging.

[26]  Nikolay N. Ponomarenko,et al.  Color image database TID2013: Peculiarities and preliminary results , 2013, European Workshop on Visual Information Processing (EUVIP).

[27]  Russell Zaretzki,et al.  Beta Process Joint Dictionary Learning for Coupled Feature Spaces with Application to Single Image Super-Resolution , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[29]  Xuelong Li,et al.  Partially Supervised Neighbor Embedding for Example-Based Image Super-Resolution , 2011, IEEE Journal of Selected Topics in Signal Processing.

[30]  Joydeep Ghosh,et al.  Blind image quality assessment without training on human opinion scores , 2013, Electronic Imaging.

[31]  Alan C. Bovik,et al.  No-reference quality assessment using natural scene statistics: JPEG2000 , 2005, IEEE Transactions on Image Processing.

[32]  Xuelong Li,et al.  Image Super-Resolution With Sparse Neighbor Embedding , 2012, IEEE Transactions on Image Processing.

[33]  Stephen Lin,et al.  Super resolution using edge prior and single image detail synthesis , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Yi Li,et al.  Convolutional Neural Networks for No-Reference Image Quality Assessment , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Xiaojun Wu,et al.  Blind Image Quality Assessment Using a General Regression Neural Network , 2011, IEEE Transactions on Neural Networks.

[37]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[38]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[39]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[40]  Ashish Kapoor,et al.  Blind Image Quality Assessment Using Semi-supervised Rectifier Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Thomas S. Huang,et al.  Non-Local Kernel Regression for Image and Video Restoration , 2010, ECCV.