Face quality analysis of single-image super-resolution based on SIFT

Single-image super-resolution (SISR) aims at improving image quality, and there so far exist many SISR algorithms to hallucinate super-resolution ( super - res ) image from simulated low - res image. In order to evaluate SISR algorithms, objective image quality assessment (IQA), e.g., full reference IQA and no-reference IQA, and subjective quality are usually estimated. However, the objective IQA usually does not well match with the subjective quality. This paper therefore introduces a new measurement based on SIFT key-points. Both descriptors and locations of SIFT key-points are used to detect the matched SIFT key-points between super - res image and its high - res label image. The more the matched SIFT key-points are, the closer super - res image should be to its high - res label image, that is the SISR algorithm is able to recover more SIFT key-points. Both simulated low - res faces and real low - res face are employed to validate the evaluation strategy. The normalization of the number of SIFT key-points is proposed and mean opinion score from 30 raters are collected to evaluate SISR algorithms. The experimental results show that the objective IQA based on SIFT key-points are able to effectively evaluate SISR algorithms, and can well match with the subjective IQA.

[1]  Harry Shum,et al.  Face Hallucination: Theory and Practice , 2007, International Journal of Computer Vision.

[2]  King Ngi Ngan,et al.  Blind Image Quality Assessment Based on Multichannel Feature Fusion and Label Transfer , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Patrick J. Flynn,et al.  Pose-robust recognition of low-resolution face images , 2013, CVPR 2011.

[5]  Xiaogang Wang,et al.  Hallucinating face by eigentransformation , 2005, IEEE Trans. Syst. Man Cybern. Part C.

[6]  Hongliang Li,et al.  A Perceptually Weighted Rank Correlation Indicator for Objective Image Quality Assessment , 2017, IEEE Transactions on Image Processing.

[7]  Joumana Farah,et al.  Full-reference and reduced-reference quality metrics based on SIFT , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Mislav Grgic,et al.  SCface – surveillance cameras face database , 2011, Multimedia Tools and Applications.

[9]  Tao Jiang,et al.  Person re-identification by discriminant analytical least squares metric learning , 2018, Machine Vision and Applications.

[10]  Shengcai Liao,et al.  Nighttime Face Recognition at Long Distance: Cross-Distance and Cross-Spectral Matching , 2012, ACCV.

[11]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[12]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Aly A. Farag,et al.  CSIFT: A SIFT Descriptor with Color Invariant Characteristics , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[14]  Yu Qiao,et al.  ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks , 2018, ECCV Workshops.

[15]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[16]  Jie Li,et al.  No-reference image quality assessment based on hybrid model , 2017, Signal Image Video Process..

[17]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[18]  Sebastian Bosse,et al.  Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment , 2016, IEEE Transactions on Image Processing.

[19]  Anil K. Jain,et al.  Nighttime face recognition at large standoff: Cross-distance and cross-spectral matching , 2014, Pattern Recognit..

[20]  Georgios Tzimiropoulos,et al.  Super-FAN: Integrated Facial Landmark Localization and Super-Resolution of Real-World Low Resolution Faces in Arbitrary Poses with GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Meisam Rakhshanfar,et al.  Sparsity-based no-reference image quality assessment for automatic denoising , 2018, Signal Image Video Process..

[23]  Li Wang,et al.  Surveillance video face recognition with single sample per person based on 3D modeling and blurring , 2017, Neurocomputing.

[24]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[26]  Mehran Yazdi,et al.  Face image super-resolution via sparse representation and wavelet transform , 2018, Signal Image Video Process..

[27]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[28]  Li Wang,et al.  Face hallucination from low quality images using definition-scalable inference , 2019, Pattern Recognit..

[29]  Yuehua Li,et al.  Single image super-resolution under multi-frame method , 2018, Signal Image Video Process..