A shallow convolutional neural network for blind image sharpness assessment

Blind image quality assessment can be modeled as feature extraction followed by score prediction. It necessitates considerable expertise and efforts to handcraft features for optimal representation of perceptual image quality. This paper addresses blind image sharpness assessment by using a shallow convolutional neural network (CNN). The network takes single feature layer to unearth intrinsic features for image sharpness representation and utilizes multilayer perceptron (MLP) to rate image quality. Different from traditional methods, CNN integrates feature extraction and score prediction into an optimization procedure and retrieves features automatically from raw images. Moreover, its prediction performance can be enhanced by replacing MLP with general regression neural network (GRNN) and support vector regression (SVR). Experiments on Gaussian blur images from LIVE-II, CSIQ, TID2008 and TID2013 demonstrate that CNN features with SVR achieves the best overall performance, indicating high correlation with human subjective judgment.

[1]  Alex ChiChung Kot,et al.  A Fast Approach for No-Reference Image Sharpness Assessment Based on Maximum Local Variation , 2014, IEEE Signal Processing Letters.

[2]  Mei Yu,et al.  Difference of Gaussian statistical features based blind image quality assessment: A deep learning approach , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[3]  King Ngi Ngan,et al.  Blind Image Quality Assessment Based on Multichannel Feature Fusion and Label Transfer , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Lina J. Karam,et al.  A No-Reference Image Blur Metric Based on the Cumulative Probability of Blur Detection (CPBD) , 2011, IEEE Transactions on Image Processing.

[5]  Lei Zhang,et al.  Blind Image Quality Assessment Using Joint Statistics of Gradient Magnitude and Laplacian Features , 2014, IEEE Transactions on Image Processing.

[6]  Mariusz Oszust,et al.  Full-Reference Image Quality Assessment with Linear Combination of Genetically Selected Quality Measures , 2016, PloS one.

[7]  Xuelong Li,et al.  Universal Blind Image Quality Assessment Metrics Via Natural Scene Statistics and Multiple Kernel Learning , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Wenjun Zhang,et al.  Using Free Energy Principle For Blind Image Quality Assessment , 2015, IEEE Transactions on Multimedia.

[9]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[10]  King Ngi Ngan,et al.  Reorganized DCT-based image representation for reduced reference stereoscopic image quality assessment , 2016, Neurocomputing.

[11]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[12]  Weisi Lin,et al.  No-reference quality assessment of deblocked images , 2016, Neurocomputing.

[13]  Xinbo Gao,et al.  Saliency-Guided Deep Framework for Image Quality Assessment , 2015, IEEE MultiMedia.

[14]  Xiao-Jun Wu,et al.  Blind Image Blur Assessment Using Singular Value Similarity and Blur Comparisons , 2014, PloS one.

[15]  D. Ruderman The statistics of natural images , 1994 .

[16]  Alan C. Bovik,et al.  RRED Indices: Reduced Reference Entropic Differencing for Image Quality Assessment , 2012, IEEE Transactions on Image Processing.

[17]  Lai-Man Po,et al.  No-reference image quality assessment with shearlet transform and deep neural networks , 2015, Neurocomputing.

[18]  Yaoqin Xie,et al.  Edge preservation ratio for image sharpness assessment , 2016, 2016 12th World Congress on Intelligent Control and Automation (WCICA).

[19]  Weisi Lin,et al.  No-Reference and Robust Image Sharpness Evaluation Based on Multiscale Spatial and Spectral Features , 2017, IEEE Transactions on Multimedia.

[20]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[22]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[23]  Mikko Nuutinen,et al.  CVD2014—A Database for Evaluating No-Reference Video Quality Assessment Algorithms , 2016, IEEE Transactions on Image Processing.

[24]  D. Basak,et al.  Support Vector Regression , 2008 .

[25]  Zhou Wang,et al.  A highly efficient method for blind image quality assessment , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[26]  Weisi Lin,et al.  No-Reference Image Sharpness Assessment in Autoregressive Parameter Space , 2015, IEEE Transactions on Image Processing.

[27]  Xiaojun Wu,et al.  No-reference image blur index based on singular value curve , 2014, J. Vis. Commun. Image Represent..

[28]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[29]  Weisi Lin,et al.  Objective Image Quality Assessment Based on Support Vector Regression , 2010, IEEE Transactions on Neural Networks.

[30]  Ling Shao,et al.  Non-distortion-specific no-reference image quality assessment: A survey , 2015, Inf. Sci..

[31]  Xuelong Li,et al.  Blind Image Quality Assessment via Deep Learning , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Weisi Lin,et al.  The Analysis of Image Contrast: From Quality Assessment to Automatic Enhancement , 2016, IEEE Transactions on Cybernetics.

[33]  Zhou Wang,et al.  Reduced- and No-Reference Image Quality Assessment , 2011, IEEE Signal Processing Magazine.

[34]  Lei Zhang,et al.  A Feature-Enriched Completely Blind Image Quality Evaluator , 2015, IEEE Transactions on Image Processing.

[35]  King Ngi Ngan,et al.  No reference image quality assessment metric via multi-domain structural information and piecewise regression , 2015, J. Vis. Commun. Image Represent..

[36]  Fei Zhou,et al.  Image Quality Assessment Based on Inter-Patch and Intra-Patch Similarity , 2015, PloS one.

[37]  P. Lennie,et al.  The machinery of colour vision , 2007, Nature Reviews Neuroscience.

[38]  J. Astola,et al.  Image database TID 2013 : Peculiarities , results and perspectives , 2016 .

[39]  Alexandre G. Ciancio,et al.  No-Reference Blur Assessment of Digital Pictures Based on Multifeature Classifiers , 2011, IEEE Transactions on Image Processing.

[40]  Tingting Wang,et al.  No reference image quality assessment using sparse feature representation in two dimensions spatial correlation , 2016, Neurocomputing.

[41]  Lina J. Karam,et al.  A No-Reference Objective Image Sharpness Metric Based on the Notion of Just Noticeable Blur (JNB) , 2009, IEEE Transactions on Image Processing.

[42]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[43]  Yaoqin Xie,et al.  CNN-GRNN for Image Sharpness Assessment , 2016, ACCV Workshops.

[44]  Weisi Lin,et al.  No-Reference Image Blur Assessment Based on Discrete Orthogonal Moments , 2016, IEEE Transactions on Cybernetics.

[45]  Wei Wu,et al.  User models of subjective image quality assessment on virtual viewpoint in free-viewpoint video system , 2016, Multimedia Tools and Applications.

[46]  Zhuo Chen,et al.  Natural image statistics based 3D reduced reference image quality assessment in contourlet domain , 2015, Neurocomputing.

[47]  Xuesong Wang,et al.  Image quality assessment based on multi-scale representation of structure , 2014, Digit. Signal Process..

[48]  Nikolay N. Ponomarenko,et al.  TID2008 – A database for evaluation of full-reference visual quality assessment metrics , 2004 .

[49]  Nikolay N. Ponomarenko,et al.  Image database TID2013: Peculiarities, results and perspectives , 2015, Signal Process. Image Commun..

[50]  Damon M. Chandler,et al.  ${\bf S}_{3}$: A Spectral and Spatial Measure of Local Perceived Sharpness in Natural Images , 2012, IEEE Transactions on Image Processing.

[51]  Guangming Shi,et al.  Reduced-Reference Image Quality Assessment With Visual Information Fidelity , 2013, IEEE Transactions on Multimedia.

[52]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[53]  Yun Zhang,et al.  A Virtual View PSNR Estimation Method for 3-D Videos , 2016, IEEE Transactions on Broadcasting.

[54]  Elsevier Sdol,et al.  Journal of Visual Communication and Image Representation , 2009 .

[55]  Weisi Lin,et al.  Image Sharpness Assessment by Sparse Representation , 2016, IEEE Transactions on Multimedia.

[56]  R. Paramesran,et al.  Correlation between subjective and objective assessment of magnetic resonance (MR) images. , 2016, Magnetic resonance imaging.

[57]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[58]  Weisi Lin,et al.  Perceptual visual quality metrics: A survey , 2011, J. Vis. Commun. Image Represent..

[59]  Xuelong Li,et al.  Learning to Rank for Blind Image Quality Assessment , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[60]  Weisi Lin,et al.  Learning Structural Regularity for Evaluating Blocking Artifacts in JPEG Images , 2014, IEEE Signal Processing Letters.

[61]  Jie Li,et al.  No-reference image quality assessment using Prewitt magnitude based on convolutional neural networks , 2016, Signal Image Video Process..

[62]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[63]  Gaobo Yang,et al.  Referenceless Measure of Blocking Artifacts by Tchebichef Kernel Analysis , 2014, IEEE Signal Processing Letters.

[64]  Yi Li,et al.  Convolutional Neural Networks for No-Reference Image Quality Assessment , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[67]  Baojun Zhao,et al.  NMF-Based Image Quality Assessment Using Extreme Learning Machine , 2017, IEEE Transactions on Cybernetics.

[68]  Phong V. Vu,et al.  A Fast Wavelet-Based Algorithm for Global and Local Image Sharpness Estimation , 2012, IEEE Signal Processing Letters.

[69]  Zhou Wang,et al.  Image Sharpness Assessment Based on Local Phase Coherence , 2013, IEEE Transactions on Image Processing.

[70]  Mikko Nuutinen,et al.  CID2013: A Database for Evaluating No-Reference Image Quality Assessment Algorithms , 2015, IEEE Transactions on Image Processing.