Jointly learning perceptually heterogeneous features for blind 3D video quality assessment

Abstract 3D videos quality assessment (3D-VQA) is essential to various 3D video processing applications. However, it has not been well investigated on how to make use of perceptual multi-channel video information to improve 3D-VQA under different distortion categories and degrees, especially under asymmetrical distortions. In the paper, we propose a new blind 3D-VQA metric by jointly learning perceptually heterogeneous features. Firstly, a binocular spatio-temporal internal generative mechanism (BST-IGM) is proposed to decompose the views of 3D video into multi-channel videos. Then, we extract perceptually heterogeneous features by proposed multi-channel natural video statistics (MNVS) model, which are characterized 3D video information. Furthermore, a robust AdaBoosting Radial Basis Function (RBF) neural network is utilized to map the features to the overall quality of 3D video. On two benchmark databases, the extensive evaluations demonstrate that the proposed algorithm significantly outperforms several state-of-the-art quality metrics in term of prediction accuracy and robustness.

[1]  Wujie Zhou,et al.  Binocular Responses for No-Reference 3D Image Quality Assessment , 2016, IEEE Transactions on Multimedia.

[2]  Wei Zhou,et al.  3D-HEVC visual quality assessment: Database and bitstream model , 2016, 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX).

[3]  Karl J. Friston.  The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[4]  Guangming Shi,et al.  Perceptual Quality Metric With Internal Generative Mechanism , 2013, IEEE Transactions on Image Processing.

[5]  Ting Luo,et al.  Blind quality estimator for 3D images based on binocular combination and extreme learning machine , 2017, Pattern Recognit..

[6]  Weisi Lin,et al.  No-Reference Quality Assessment for Multiply-Distorted Images in Gradient Domain , 2016, IEEE Signal Processing Letters.

[7]  Do-Kyoung Kwon,et al.  Full-reference quality assessment of stereopairs accounting for rivalry , 2013, Signal Process. Image Commun..

[8]  Alan C. Bovik,et al.  Assessment of video naturalness using time-frequency statistics , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[9]  Jean-Yves Guillemaut,et al.  Stereoscopic Video Quality Assessment Using Binocular Energy , 2017, IEEE Journal of Selected Topics in Signal Processing.

[10]  Narciso García,et al.  NAMA3DS1-COSPAD1: Subjective video quality assessment database on coding conditions introducing freely available high quality 3D stereoscopic sequences , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[11]  Yingyun Yang,et al.  A novel non-reference image quality assessment algorithm , 2017, 2017 IEEE 2nd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC).

[12]  Qingjie Zhao,et al.  Blind image quality assessment by relative gradient statistics and adaboosting neural network , 2016, Signal Process. Image Commun..

[13]  Damon M. Chandler,et al.  A spatiotemporal most-apparent-distortion model for video quality assessment , 2011, 2011 18th IEEE International Conference on Image Processing.

[14]  Sumohana S. Channappayya,et al.  Full-Reference Stereo Image Quality Assessment Using Natural Stereo Scene Statistics , 2015, IEEE Signal Processing Letters.

[15]  Sumohana S. Channappayya,et al.  An optical flow-based no-reference video quality assessment algorithm , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[16]  Hua Huang,et al.  No-reference stereopair quality assessment based on singular value decomposition , 2018, Neurocomputing.

[17]  Sumohana S. Channappayya,et al.  A full reference stereoscopic video quality assessment metric , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[18]  Ting Luo,et al.  Blind 3D image quality assessment based on self-similarity of binocular features , 2017, Neurocomputing.

[19]  Kwanghoon Sohn,et al.  An Objective Video Quality Metric for Compressed Stereoscopic Video , 2012, Circuits Syst. Signal Process..

[20]  Touradj Ebrahimi,et al.  A perceptual quality metric for stereoscopic crosstalk perception , 2010, 2010 IEEE International Conference on Image Processing.

[21]  Yun Zhu,et al.  Blind video quality assessment based on spatio-temporal internal generative mechanism , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[22]  Qian Zhao,et al.  Application of RBF Neural Network in Fault Diagnosis for Transmission Gear , 2012 .

[23]  Weisi Lin,et al.  Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics , 2013, IEEE Transactions on Image Processing.

[24]  Narasimhan Sundararajan,et al.  An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  José Manuel Menéndez,et al.  Stereoscopic 3D video quality assessment based on depth maps and video motion , 2013, EURASIP J. Image Video Process..

[26]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[27]  Alberto Leon-Garcia,et al.  Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video , 1995, IEEE Trans. Circuits Syst. Video Technol..

[28]  Mei Yu,et al.  PMFS: A Perceptual Modulated Feature Similarity Metric for Stereoscopic Image Quality Assessment , 2014, IEEE Signal Processing Letters.