Depth perceptual quality assessment for symmetrically and asymmetrically distorted stereoscopic 3D videos

Abstract Depth perception is one of the most important features provided by stereoscopic 3D videos, which is also the major difference to 2D videos. Its quality, called depth perceptual quality or depth quality, is one of the key factors that directly affects the quality of experience of 3D videos. Since 3D video shall be compressed, transmitted and reconstructed in 3D video system, distortion introduced from compression and processing affects the video quality as well as the depth perceptual quality. However, understandings of how the distortions affect the depth perception and how to evaluate the depth perceptual quality of 3D video remain limited. In this paper, subjective experiments are firstly conducted for investigating the compression and processing distortions’ impacts on the perceived depth quality of 3D videos. The subjective experiments over the datasets show that the loss of video details can cause the degradation of the depth perceptual quality in monocular and binocular perspectives. In addition, the relationships between video quality and depth quality for both symmetric and asymmetric distorted 3D stereoscopic videos are analyzed. Both the subjective video quality and depth quality scores are released and available for public. Secondly, an objective depth quality assessment algorithm is proposed for measuring the depth perceptual quality degradation of symmetrically and asymmetrically distorted stereoscopic 3D videos. The experimental results demonstrate that the proposed metric has better consistency with the subjective scores of human vision system compared with the state-of-the-art schemes.

[1]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[2]  Mohamed-Chaker Larabi,et al.  Perceptually Driven Nonuniform Asymmetric Coding of Stereoscopic 3D Video , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Jun Okamoto,et al.  Subjective characteristics for stereoscopic high definition video , 2011, 2011 Third International Workshop on Quality of Multimedia Experience.

[4]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[5]  Alan C. Bovik,et al.  3D Visual Discomfort Prediction: Vergence, Foveation, and the Physiological Optics of Accommodation , 2014, IEEE Journal of Selected Topics in Signal Processing.

[6]  Zhibo Chen,et al.  Blind Stereoscopic Video Quality Assessment: From Depth Perception to Overall Experience , 2018, IEEE Transactions on Image Processing.

[7]  Zhou Wang,et al.  Perceptual Depth Quality in Distorted Stereoscopic Images , 2017, IEEE Transactions on Image Processing.

[8]  Ahmet M. Kondoz,et al.  Sensitivity Analysis of the Human Visual System for Depth Cues in Stereoscopic 3-D Displays , 2011, IEEE Transactions on Multimedia.

[9]  Hans-Peter Seidel,et al.  Motion parallax in stereo 3D , 2016, ACM Trans. Graph..

[10]  Li Yu,et al.  No-Reference Depth Assessment Based on Edge Misalignment Errors for T + D Images , 2016, IEEE Transactions on Image Processing.

[11]  Alexander Raake,et al.  Measuring perceived depth in natural images and study of its relation with monocular and binocular depth cues , 2014, Electronic Imaging.

[12]  Gangyi Jiang,et al.  High-Efficiency 3D Depth Coding Based on Perceptual Quality of Synthesized Video , 2016, IEEE Transactions on Image Processing.

[13]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[14]  Ke Gu,et al.  Quality Assessment of DIBR-Synthesized Images by Measuring Local Geometric Distortions and Global Sharpness , 2018, IEEE Transactions on Multimedia.

[15]  Zhou Wang,et al.  Depth perception of distorted stereoscopic images , 2015, 2015 IEEE 17th International Workshop on Multimedia Signal Processing (MMSP).

[16]  Alexander Raake,et al.  Evaluating Depth Perception of 3D Stereoscopic Videos , 2012, IEEE Journal of Selected Topics in Signal Processing.

[17]  Atanas P. Gotchev,et al.  Stereoscopic Depth Cues Outperform Monocular Ones on Autostereoscopic Display , 2012, IEEE Journal of Selected Topics in Signal Processing.

[18]  Hongyu Li,et al.  VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment , 2014, IEEE Transactions on Image Processing.

[19]  Mei Yu,et al.  Leveraging visual attention and neural activity for stereoscopic 3D visual comfort assessment , 2016, Multimedia Tools and Applications.

[20]  Yo-Sung Ho,et al.  Sparse Representation-Based Video Quality Assessment for Synthesized 3D Videos , 2020, IEEE Transactions on Image Processing.

[21]  Yu Zhou,et al.  Quaternion representation based visual saliency for stereoscopic image quality assessment , 2018, Signal Process..

[22]  Do-Kyoung Kwon,et al.  Full-reference quality assessment of stereopairs accounting for rivalry , 2013, Signal Process. Image Commun..

[23]  Qionghai Dai,et al.  Full-Reference Quality Assessment of Stereoscopic Images by Learning Binocular Receptive Field Properties , 2015, IEEE Transactions on Image Processing.

[24]  C.-C. Jay Kuo,et al.  Subjective and Objective Video Quality Assessment of 3D Synthesized Views With Texture/Depth Compression Distortion , 2015, IEEE Transactions on Image Processing.

[25]  Yun Zhang,et al.  Visibility threshold of compressed stereoscopic image: effects of asymmetrical coding , 2013 .

[26]  Zhihan Lv,et al.  Stereoscopic image quality assessment method based on binocular combination saliency model , 2016, Signal Process..

[27]  Taewan Kim,et al.  Perceptual Crosstalk Prediction on Autostereoscopic 3D Display , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Narciso García,et al.  NAMA3DS1-COSPAD1: Subjective video quality assessment database on coding conditions introducing freely available high quality 3D stereoscopic sequences , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[29]  Zhuo Chen,et al.  Unified No-Reference Quality Assessment of Singly and Multiply Distorted Stereoscopic Images , 2019, IEEE Transactions on Image Processing.

[30]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[31]  Yun Zhang,et al.  View synthesis distortion model based frame level rate control optimization for multiview depth video coding , 2015, Signal Process..

[32]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[33]  Sang Uk Lee,et al.  Joint Depth Map and Color Consistency Estimation for Stereo Images with Different Illuminations and Cameras , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Yu Zhang,et al.  A SVR based quality metric for depth quality assessment , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[35]  Baihua Li,et al.  A no-reference optical flow-based quality evaluator for stereoscopic videos in curvelet domain , 2017, Inf. Sci..

[36]  Tom Drummond,et al.  Faster and Better: A Machine Learning Approach to Corner Detection , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Zhou Wang,et al.  Asymmetrically Compressed Stereoscopic 3D Videos: Quality Assessment and Rate-Distortion Performance Evaluation , 2017, IEEE Transactions on Image Processing.

[38]  Ja-Ling Wu,et al.  Quality Assessment of Stereoscopic 3D Image Compression by Binocular Integration Behaviors , 2014, IEEE Transactions on Image Processing.

[39]  Chunping Hou,et al.  Blind stereoscopic 3D image quality assessment via analysis of naturalness, structure, and binocular asymmetry , 2018, Signal Process..

[40]  Xin Jin,et al.  VideoSet: A large-scale compressed video quality dataset based on JND measurement , 2017, J. Vis. Commun. Image Represent..

[41]  Fernando Jaureguizar,et al.  MultiView Perceptual Disparity Model for Super MultiView Video , 2017, IEEE Journal of Selected Topics in Signal Processing.

[42]  Lu Yu,et al.  A Spatio-Temporal Perceptual Quality Index Measuring Compression Distortions of Three-Dimensional Video , 2018, IEEE Signal Processing Letters.

[43]  Ahmet M. Kondoz,et al.  Toward an Impairment Metric for Stereoscopic Video: A Full-Reference Video Quality Metric to Assess Compressed Stereoscopic Video , 2013, IEEE Transactions on Image Processing.

[44]  G. Nur Yilmaz A no reference depth perception assessment metric for 3D video , 2015 .