Local and global sparse representation for no-reference quality assessment of stereoscopic images

Abstract No-reference/blind quality assessment of stereoscopic 3D images is much more challenging than 2D images due to the poor understanding of binocular vision. In this paper, we propose a BLind Quality Evaluator for stereoscopic 3D images by learning Local and Global Sparse Representations (BLQELGSR). Specifically, at the training stage, we first construct a large-scale training set by simulating some common distortions that are likely encountered by stereoscopic images, and propose a multi-modal sparse representation framework to characterize the relationship between the feature and quality spaces for all sources of information from left, right and cyclopean views in local and global manners. At the testing stage, based on the derived 3D quality prediction framework, the local and global quality scores from different sources are predicted and combined to drive a final 3D quality score. Experimental results on three 3D image quality databases show that in comparison with the existing methods, the devised BLQELGSR can achieve better prediction performance to be in line with subjective assessment.

[1]  Zhou Wang,et al.  No-reference perceptual quality assessment of JPEG compressed images , 2002, Proceedings. International Conference on Image Processing.

[2]  Weisi Lin,et al.  Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics , 2013, IEEE Transactions on Image Processing.

[3]  Lei Zhang,et al.  Blind Image Quality Assessment Using Joint Statistics of Gradient Magnitude and Laplacian Features , 2014, IEEE Transactions on Image Processing.

[4]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[5]  Yuukou Horita,et al.  Objective No-Reference Stereoscopic Image Quality Prediction Based on 2D Image Features and Relative Disparity , 2012, Adv. Multim..

[6]  Havani,et al.  Quality Assessment of Stereoscopic 3 D Image Compression by Binocular Integration Behaviors , 2016 .

[7]  G. Nur Yilmaz A no reference depth perception assessment metric for 3D video , 2015 .

[8]  Alan C. Bovik,et al.  No-Reference Quality Assessment of Natural Stereopairs , 2013, IEEE Transactions on Image Processing.

[9]  Kwanghoon Sohn,et al.  No-Reference Quality Assessment for Stereoscopic Images Based on Binocular Quality Perception , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Damon M. Chandler,et al.  3D-MAD: A Full Reference Stereoscopic Image Quality Estimator Based on Binocular Lightness and Contrast Perception , 2015, IEEE Transactions on Image Processing.

[11]  Mohamed-Chaker Larabi,et al.  A perceptual metric for stereoscopic image quality assessment based on the binocular energy , 2013, Multidimens. Syst. Signal Process..

[12]  S. Hochstein,et al.  View from the Top Hierarchies and Reverse Hierarchies in the Visual System , 2002, Neuron.

[13]  Lin Ma,et al.  Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network , 2016, Pattern Recognit..

[14]  Larry S. Davis,et al.  Label Consistent K-SVD: Learning a Discriminative Dictionary for Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Asok Ray,et al.  Multimodal Task-Driven Dictionary Learning for Image Classification , 2015, IEEE Transactions on Image Processing.

[17]  Kwanghyun Lee,et al.  3D Perception Based Quality Pooling: Stereopsis, Binocular Rivalry, and Binocular Suppression , 2015, IEEE Journal of Selected Topics in Signal Processing.

[18]  Wenjun Zhang,et al.  Using Free Energy Principle For Blind Image Quality Assessment , 2015, IEEE Transactions on Multimedia.

[19]  Alan C. Bovik,et al.  Subjective evaluation of stereoscopic image quality , 2013, Signal Process. Image Commun..

[20]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[21]  Xuelong Li,et al.  Sparse representation for blind image quality assessment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Alan C. Bovik,et al.  3D Visual Discomfort Predictor: Analysis of Disparity and Neural Activity Statistics , 2015, IEEE Transactions on Image Processing.

[23]  Ashish Kapoor,et al.  Blind Image Quality Assessment Using Semi-supervised Rectifier Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[25]  Rama Chellappa,et al.  Joint Sparse Representation for Robust Multimodal Biometrics Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Phong V. Vu,et al.  A Fast Wavelet-Based Algorithm for Global and Local Image Sharpness Estimation , 2012, IEEE Signal Processing Letters.

[27]  Ja-Ling Wu,et al.  Quality Assessment of Stereoscopic 3D Image Compression by Binocular Integration Behaviors , 2014, IEEE Transactions on Image Processing.

[28]  Stefan Winkler,et al.  A no-reference perceptual blur metric , 2002, Proceedings. International Conference on Image Processing.

[29]  Yi Li,et al.  Convolutional Neural Networks for No-Reference Image Quality Assessment , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Kai Zeng,et al.  Quality Prediction of Asymmetrically Distorted Stereoscopic 3D Images , 2015, IEEE Transactions on Image Processing.

[31]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[32]  Mei Yu,et al.  Supervised dictionary learning for blind image quality assessment , 2015, 2015 Visual Communications and Image Processing (VCIP).

[33]  Baihua Li,et al.  Quality assessment metric of stereo images considering cyclopean integration and visual saliency , 2016, Inf. Sci..

[34]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[35]  Patrick Le Callet,et al.  Quality Assessment of Stereoscopic Images , 2008, EURASIP J. Image Video Process..

[36]  Lei Zhang,et al.  A Feature-Enriched Completely Blind Image Quality Evaluator , 2015, IEEE Transactions on Image Processing.

[37]  Alan C. Bovik,et al.  Oriented Correlation Models of Distorted Natural Images With Application to Natural Stereopair Quality Evaluation , 2015, IEEE Transactions on Image Processing.

[38]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[39]  Alexander Raake,et al.  Evaluating Depth Perception of 3D Stereoscopic Videos , 2012, IEEE Journal of Selected Topics in Signal Processing.

[40]  Qionghai Dai,et al.  Learning Receptive Fields and Quality Lookups for Blind Quality Assessment of Stereoscopic Images , 2016, IEEE Transactions on Cybernetics.

[41]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[42]  Do-Kyoung Kwon,et al.  Full-reference quality assessment of stereopairs accounting for rivalry , 2013, Signal Process. Image Commun..

[43]  Peter Schelkens,et al.  Qualinet White Paper on Definitions of Quality of Experience , 2013 .

[44]  Wenjun Zhang,et al.  No-Reference Stereoscopic IQA Approach: From Nonlinear Effect to Parallax Compensation , 2012, J. Electr. Comput. Eng..

[45]  Alan C. Bovik,et al.  Image information and visual quality , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[46]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[47]  Decebal Constantin Mocanu,et al.  Deep learning for objective quality assessment of 3D images , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[48]  Qionghai Dai,et al.  Full-Reference Quality Assessment of Stereoscopic Images by Learning Binocular Receptive Field Properties , 2015, IEEE Transactions on Image Processing.

[49]  Yanqing Li,et al.  No-Reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics , 2017, 2017 2nd International Conference on Multimedia and Image Processing (ICMIP).

[50]  Lei Zhang,et al.  Learning without Human Scores for Blind Image Quality Assessment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Weisi Lin,et al.  On Predicting Visual Comfort of Stereoscopic Images: A Learning to Rank Based Approach , 2016, IEEE Signal Processing Letters.