No-Reference Stereoscopic Image Quality Assessment Based on Visual Attention and Perception

In recent years, the methods of no-reference stereoscopic image quality assessment (NR-SIQA) have been well investigated, but there still remain challenges due to the inaccurate extraction of binocular perception information. In this paper, we propose an NR-SIQA method based on visual attention and perception. We combine saliency and just noticeable difference (JND) to model visual attention and perception, respectively, and weight the global and local features extracted from the left and right views. Meanwhile, in order to obtain the accurate binocular perception information, the global structural features reflecting spatial correlation are extracted from the cyclopean map that is synthesized by the left and right views. Then, a regression model is learned based on a support vector machine regression (SVR) to evaluate the quality of stereoscopic images. The experiments on popular SIQA datasets demonstrate that the proposed NR-SIQA method has better and more reliable performance than the state-of-the-art methods.

[1]  Weisi Lin,et al.  Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics , 2013, IEEE Transactions on Image Processing.

[2]  Baihua Li,et al.  Quality assessment metric of stereo images considering cyclopean integration and visual saliency , 2016, Inf. Sci..

[3]  Chunping Hou,et al.  Blind stereoscopic 3D image quality assessment via analysis of naturalness, structure, and binocular asymmetry , 2018, Signal Process..

[4]  Faouzi Alaya Cheikh,et al.  Stereoscopic image quality assessment based on the binocular properties of the human visual system , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  Ju Liu,et al.  Pattern complexity-based JND estimation for quantization watermarking , 2020, Pattern Recognit. Lett..

[6]  Weisi Lin,et al.  No-Reference Quality Assessment for Multiply-Distorted Images in Gradient Domain , 2016, IEEE Signal Processing Letters.

[7]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[8]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[9]  Alan Conrad Bovik,et al.  Binocular spatial activity and reverse saliency driven no-reference stereopair quality assessment , 2017, Signal Process. Image Commun..

[10]  Zhihan Lv,et al.  Stereoscopic image quality assessment method based on binocular combination saliency model , 2016, Signal Process..

[11]  Eero P. Simoncelli,et al.  On Advances in Statistical Modeling of Natural Images , 2004, Journal of Mathematical Imaging and Vision.

[12]  Kwanghoon Sohn,et al.  No-Reference Quality Assessment for Stereoscopic Images Based on Binocular Quality Perception , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Alan C. Bovik,et al.  Oriented Correlation Models of Distorted Natural Images With Application to Natural Stereopair Quality Evaluation , 2015, IEEE Transactions on Image Processing.

[14]  Chaminda T. E. R. Hewage,et al.  Reduced-reference quality metric for 3D depth map transmission , 2010, 2010 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[15]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[16]  Kai Li,et al.  No-reference stereo image quality assessment by learning gradient dictionary-based color visual characteristics , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[17]  Susu Yao,et al.  Just noticeable distortion model and its applications in video coding , 2005, Signal Process. Image Commun..

[18]  Xiaojun Chang,et al.  Adaptive Semi-Supervised Feature Selection for Cross-Modal Retrieval , 2019, IEEE Transactions on Multimedia.

[19]  Guangming Shi,et al.  Visual Orientation Selectivity Based Structure Description , 2015, IEEE Transactions on Image Processing.

[20]  Weisi Lin,et al.  Objective Image Quality Assessment Based on Support Vector Regression , 2010, IEEE Transactions on Neural Networks.

[21]  Hua Huang,et al.  No-reference image quality assessment based on spatial and spectral entropies , 2014, Signal Process. Image Commun..

[22]  Mei Yu,et al.  Reduced-reference stereoscopic image quality assessment based on view and disparity zero-watermarks , 2014, Signal Process. Image Commun..

[23]  Andrey S. Krylov,et al.  No-Reference Stereoscopic Image Quality Assessment Using Convolutional Neural Network for Adaptive Feature Extraction , 2018, IEEE Access.

[24]  Baihua Li,et al.  A Blind Stereoscopic Image Quality Evaluator With Segmented Stacked Autoencoders Considering the Whole Visual Perception Route , 2019, IEEE Transactions on Image Processing.

[25]  Alan C. Bovik,et al.  No-Reference Quality Assessment of Natural Stereopairs , 2013, IEEE Transactions on Image Processing.

[26]  Damon M. Chandler,et al.  3D-MAD: A Full Reference Stereoscopic Image Quality Estimator Based on Binocular Lightness and Contrast Perception , 2015, IEEE Transactions on Image Processing.

[27]  Ting Luo,et al.  Blind quality estimator for 3D images based on binocular combination and extreme learning machine , 2017, Pattern Recognit..

[28]  Kai Zeng,et al.  Quality Prediction of Asymmetrically Distorted Stereoscopic 3D Images , 2015, IEEE Transactions on Image Processing.

[29]  Zhou Wang,et al.  Reduced-reference image quality assessment using a wavelet-domain natural image statistic model , 2005, IS&T/SPIE Electronic Imaging.

[30]  D. Heeger,et al.  Feature-based attention enhances performance by increasing response gain , 2011, Vision Research.

[31]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[33]  Min Liu,et al.  Using Structural degradation and Parallax for reduced-reference quality assessment of 3D images , 2014, 2014 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting.

[34]  Guangming Shi,et al.  Enhanced Just Noticeable Difference Model for Images With Pattern Complexity , 2017, IEEE Transactions on Image Processing.

[35]  Sumohana S. Channappayya,et al.  Full-Reference Stereo Image Quality Assessment Using Natural Stereo Scene Statistics , 2015, IEEE Signal Processing Letters.

[36]  Yong Ding,et al.  No-reference Stereoscopic Image Quality Assessment Based on Saliency-guided Binocular Feature Consolidation , 2017 .

[37]  Kamel Mohamed Faraoun,et al.  Stereoscopic image quality metric based on local entropy and binocular just noticeable difference , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[38]  Kwanghoon Sohn,et al.  Stereoscopic image quality metric based on binocular perception model , 2012, 2012 19th IEEE International Conference on Image Processing.

[39]  Alan C. Bovik,et al.  RRED Indices: Reduced Reference Entropic Differencing for Image Quality Assessment , 2012, IEEE Transactions on Image Processing.

[40]  Soo-Chang Pei,et al.  Blind Stereoscopic Image Quality Assessment Based on Hierarchical Learning , 2019, IEEE Access.

[41]  Lin Ma,et al.  Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network , 2016, Pattern Recognit..

[42]  H. Sompolinsky,et al.  Theory of orientation tuning in visual cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[43]  Xuelong Li,et al.  Blind Image Quality Assessment via Deep Learning , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[44]  Tat Chee Avenue,et al.  Considering Binocular Spatial Sensitivity in Stereoscopic Image Quality Assessment , 2011 .

[45]  Yu Zhou,et al.  Quaternion representation based visual saliency for stereoscopic image quality assessment , 2018, Signal Process..

[46]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[47]  D. Ruderman The statistics of natural images , 1994 .

[48]  Manoranjan Paul,et al.  Just Noticeable Difference for Images With Decomposition Model for Separating Edge and Textured Regions , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[49]  Ingrid Heynderickx,et al.  Visual Attention in Objective Image Quality Assessment: Based on Eye-Tracking Data , 2011, IEEE Transactions on Circuits and Systems for Video Technology.