Asymmetrically Compressed Stereoscopic 3D Videos: Quality Assessment and Rate-Distortion Performance Evaluation

Objective quality assessment of stereoscopic 3D video is challenging but highly desirable, especially in the application of stereoscopic video compression and transmission, where useful quality models are missing, that can guide the critical decision making steps in the selection of mixed-resolution coding, asymmetric quantization, and pre- and post-processing schemes. Here we first carry out subjective quality assessment experiments on two databases that contain various asymmetrically compressed stereoscopic 3D videos obtained from mixed-resolution coding, asymmetric transform-domain quantization coding, their combinations, and the multiple choices of postprocessing techniques. We compare these asymmetric stereoscopic video coding schemes with symmetric coding methods and verify their potential coding gains. We observe a strong systematic bias when using direct averaging of 2D video quality of both views to predict 3D video quality. We then apply a binocular rivalry inspired model to account for the prediction bias, leading to a significantly improved full reference quality prediction model of stereoscopic videos. The model allows us to quantitatively predict the coding gain of different variations of asymmetric video compression, and provides new insight on the development of high efficiency 3D video coding schemes.

[1]  Yuan Zhou,et al.  Objective quality assessment method of stereo images , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[2]  Do-Kyoung Kwon,et al.  Study of subject agreement on stereoscopic video quality , 2012, 2012 IEEE Southwest Symposium on Image Analysis and Interpretation.

[3]  Kai Zeng,et al.  Polyview Fusion: A Strategy to Enhance Video-Denoising Algorithms , 2012, IEEE Transactions on Image Processing.

[4]  Kai Zeng,et al.  Quality Prediction of Asymmetrically Distorted Stereoscopic 3D Images , 2015, IEEE Transactions on Image Processing.

[5]  Jiachen Yang,et al.  Stereo Picture Quality Estimation Based on a Multiple Channel HVS Model , 2009, 2009 2nd International Congress on Image and Signal Processing.

[6]  A. Gotchev,et al.  Quality assessment of 3D video in rate allocation experiments , 2008, 2008 IEEE International Symposium on Consumer Electronics.

[7]  Luís Alberto da Silva Cruz,et al.  3D video subjective quality: a new database and grade comparison study , 2015, Multimedia Tools and Applications.

[8]  Nick Holliman,et al.  Stereoscopic image quality metrics and compression , 2008, Electronic Imaging.

[9]  Alan C. Bovik,et al.  3D Visual Discomfort Predictor: Analysis of Disparity and Neural Activity Statistics , 2015, IEEE Transactions on Image Processing.

[10]  Karen O. Egiazarian,et al.  Validation of a new full reference metric for quality assessment of mobile 3DTV content , 2011, 2011 19th European Signal Processing Conference.

[11]  Ahmet M. Kondoz,et al.  Prediction of stereoscopic video quality using objective quality models of 2-D video , 2008 .

[12]  Zhou Wang,et al.  Information Content Weighting for Perceptual Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[13]  Mei Yu,et al.  Three-dimensional visual comfort assessment via preference learning , 2015, J. Electronic Imaging.

[14]  Zhou Wang,et al.  Depth perception of distorted stereoscopic images , 2015, 2015 IEEE 17th International Workshop on Multimedia Signal Processing (MMSP).

[15]  Zhou Wang,et al.  Spatial Pooling Strategies for Perceptual Image Quality Assessment , 2006, 2006 International Conference on Image Processing.

[16]  A. Murat Tekalp,et al.  Quality assessment of asymmetric stereo video coding , 2010, 2010 IEEE International Conference on Image Processing.

[17]  Ja-Ling Wu,et al.  Quality Assessment of Stereoscopic 3D Image Compression by Binocular Integration Behaviors , 2014, IEEE Transactions on Image Processing.

[18]  Ahmet M. Kondoz,et al.  Quality analysis for 3D video using 2D video quality models , 2008, IEEE Transactions on Consumer Electronics.

[19]  Alan C. Bovik,et al.  Visual Quality Assessment of Stereoscopic Image and Video: Challenges, Advances, and Future Trends , 2015 .

[20]  Zhongjie Zhu,et al.  Perceptual distortion metric for stereo video quality evaluation , 2009 .

[21]  Panos Nasiopoulos,et al.  Effect of high frame rates on 3D video quality of experience , 2014, 2014 IEEE International Conference on Consumer Electronics (ICCE).

[22]  Junyong You,et al.  PERCEPTUAL QUALITY ASSESSMENT FOR STEREOSCOPIC IMAGES BASED ON 2 D IMAGE QUALITY METRICS AND DISPARITY ANALYSIS , 2010 .

[23]  Zhou Wang,et al.  Quality prediction of asymmetrically compressed stereoscopic videos , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[24]  Roushain Akhter,et al.  No-reference stereoscopic image quality assessment , 2010, Electronic Imaging.

[25]  Margaret H. Pinson,et al.  Comparing subjective video quality testing methodologies , 2003, Visual Communications and Image Processing.

[26]  Zhou Wang,et al.  PERCEPTUAL QUALITY OF ASYMMETRICALLY DISTORTED STEREOSCOPIC IMAGES : THE ROLE OF IMAGE DISTORTION TYPES , 2013 .

[27]  Jean-Bernard Martens,et al.  Quality asessment of coded images using numerical category scaling , 1995, Other Conferences.

[28]  Narciso García,et al.  NAMA3DS1-COSPAD1: Subjective video quality assessment database on coding conditions introducing freely available high quality 3D stereoscopic sequences , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[29]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[30]  Thomas Wiegand,et al.  Mixed resolution coding of stereoscopic video for Mobile devices , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[31]  Hiroshi Ono,et al.  What does the dominant eye dominate? A brief and somewhat contentious review , 2003, Perception & psychophysics.

[32]  Kwanghoon Sohn,et al.  An Objective Video Quality Metric for Compressed Stereoscopic Video , 2012, Circuits Syst. Signal Process..

[33]  Alan C. Bovik,et al.  Subjective evaluation of stereoscopic image quality , 2013, Signal Process. Image Commun..

[34]  Panos Nasiopoulos,et al.  The Effect of Frame Rate on 3D Video Quality and Bitrate , 2015, 1803.04826.

[35]  Margaret H. Pinson,et al.  The Consumer Digital Video Library [Best of the Web] , 2013, IEEE Signal Processing Magazine.

[36]  Alan C. Bovik,et al.  3D Visual Discomfort Prediction: Vergence, Foveation, and the Physiological Optics of Accommodation , 2014, IEEE Journal of Selected Topics in Signal Processing.

[37]  Atanas Gotchev,et al.  Subjective evaluation of mobile 3D video content: depth range versus compression artifacts , 2011, Electronic Imaging.

[38]  Weisi Lin,et al.  Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics , 2013, IEEE Transactions on Image Processing.

[39]  Zhou Wang,et al.  Video quality assessment based on structural distortion measurement , 2004, Signal Process. Image Commun..

[40]  Ahmet M. Kondoz,et al.  Toward an Impairment Metric for Stereoscopic Video: A Full-Reference Video Quality Metric to Assess Compressed Stereoscopic Video , 2013, IEEE Transactions on Image Processing.

[41]  Alan C. Bovik,et al.  Stereoscopic 3D Visual Discomfort Prediction: A Dynamic Accommodation and Vergence Interaction Model , 2016, IEEE Transactions on Image Processing.

[42]  R. Blake © 2001 Kluwer Academic Publishers. Printed in the Netherlands. 5 A Primer on Binocular Rivalry, Including Current Controversies , 2000 .

[43]  Michael G. Perkins,et al.  Data compression of stereopairs , 1992, IEEE Trans. Commun..

[44]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[45]  Kai Zeng,et al.  3D-SSIM for video quality assessment , 2012, 2012 19th IEEE International Conference on Image Processing.

[46]  Miska M. Hannuksela,et al.  Subjective quality assessment of asymmetric stereoscopic 3D video , 2015, Signal Image Video Process..

[47]  Margaret H. Pinson,et al.  A new standardized method for objectively measuring video quality , 2004, IEEE Transactions on Broadcasting.

[48]  Khan,et al.  PII: S0042-6989(01)00079-7 , 2001 .

[49]  Do-Kyoung Kwon,et al.  Full-reference quality assessment of stereopairs accounting for rivalry , 2013, Signal Process. Image Commun..

[50]  Jiheng Wang Perceptual Quality-of-Experience of Stereoscopic 3D Images and Videos , 2016 .

[51]  Ahmet M. Kondoz,et al.  Psycho-physical limits of interocular blur suppression and its application to asymmetric stereoscopic video delivery , 2012, 2012 19th International Packet Video Workshop (PV).

[52]  Panos Nasiopoulos,et al.  Subjective study on asymmetric stereoscopic video with low-pass filtered slices , 2012, 2012 International Conference on Computing, Networking and Communications (ICNC).

[53]  Gozde Bozdagi Akar,et al.  Quality evaluation of stereoscopic videos using depth map segmentation , 2011, 2011 Third International Workshop on Quality of Multimedia Experience.

[54]  Touradj Ebrahimi,et al.  A comprehensive database and subjective evaluation methodology for quality of experience in stereoscopic video , 2010, Electronic Imaging.

[55]  Chaminda T. E. R. Hewage,et al.  Reduced-reference quality metric for 3D depth map transmission , 2010, 2010 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[56]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[57]  Patrick Le Callet,et al.  Quality Assessment of Stereoscopic Images , 2008, EURASIP J. Image Video Process..

[58]  Alan C. Bovik,et al.  No-Reference Quality Assessment of Natural Stereopairs , 2013, IEEE Transactions on Image Processing.

[59]  Karen O. Egiazarian,et al.  3D-DCT based perceptual quality assessment of stereo video , 2011, 2011 18th IEEE International Conference on Image Processing.

[60]  André Vincent,et al.  Stereo image quality: effects of mixed spatio-temporal resolution , 2000, IEEE Trans. Circuits Syst. Video Technol..

[61]  Jianjun Lei,et al.  New metric for stereo image quality assessment based on HVS , 2010, Int. J. Imaging Syst. Technol..

[62]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[63]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[64]  Kai Zeng,et al.  Quality prediction of asymmetrically distorted stereoscopic images from single views , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).