Toward the Assessment of Quality of Experience for Asymmetric Encoding in Immersive Media

The assessment of Quality of Experience (QoE) for stereoscopic 3-D video is a challenging task, especially in 3-D video compression and transmission applications. The focus of this contribution is the development of a QoE assessment framework, in line with the latest standardization progress in the field of QoE assessment, for understanding the visual effect of asymmetric and symmetric encoding for immersive media. Asymmetric stereoscopic video coding exploits the binocular suppression of the human vision system by representing one of the two views with a lower quality. This processing, while of limited effects on image quality, may influence the overall QoE. Many studies show that the QoE of immersive media such as 3-DTV can be thought as the combination of the perceived visual quality, the perceived depth quality, the visual fatigue, and visual discomfort. In this paper, we aim at: 1) exploiting the concept of preference of experience and protocols recently standardized for characterizing QoE; 2) conducting a case study using these standardized protocols to investigate the factors involving visual discomfort in stereoscopic video sequences with a focus on binocular rivalry; and 3) presenting the results of subjective experiments performed, by using the perceptual quality and preference of experience assessment protocols, for evaluating the impact of symmetrical, asymmetrical, and alternate coding schemes.

[1]  Namho Hur,et al.  Asymmetric Coding of Stereoscopic Video for Transmission Over T-DMB , 2007, 2007 3DTV Conference.

[2]  R. A. Bradley,et al.  Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons , 1952 .

[3]  Patrick Le Callet,et al.  Quality of experience model for 3DTV , 2012, Electronic Imaging.

[4]  Patrick Le Callet,et al.  Stereoscopic images quality assessment , 2007, 2007 15th European Signal Processing Conference.

[5]  Kamel Mohamed Faraoun,et al.  Asymmetric coding using Binocular Just Noticeable Difference and depth information for stereoscopic 3D , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Ken Chen,et al.  Stereoscopic video coding with asymmetric luminance and chrominance qualities , 2010, IEEE Transactions on Consumer Electronics.

[7]  Patrick Le Callet,et al.  Using disparity for quality assessment of stereoscopic images , 2008, 2008 15th IEEE International Conference on Image Processing.

[8]  Chang-Su Kim,et al.  Efficient stereo video coding based on frame skipping for real-time mobile applications , 2008, IEEE Transactions on Consumer Electronics.

[9]  Marcus Barkowsky,et al.  Analysis and improvement of a paired comparison method in the application of 3DTV subjective experiment , 2012, 2012 19th IEEE International Conference on Image Processing.

[10]  B. Julesz Foundations of Cyclopean Perception , 1971 .

[11]  Luís Alberto da Silva Cruz,et al.  3D video subjective quality: a new database and grade comparison study , 2015, Multimedia Tools and Applications.

[12]  Houqiang Li,et al.  Multiview-Video-Plus-Depth Coding Based on the Advanced Video Coding Standard , 2013, IEEE Transactions on Image Processing.

[13]  Jonathan Burton,et al.  RMIT3DV: Pre-announcement of a creative commons uncompressed HD 3D video database , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[14]  Ying Chen,et al.  Picture-level adaptive filter for asymmetric stereoscopic video , 2008, 2008 15th IEEE International Conference on Image Processing.

[15]  A. Murat Tekalp,et al.  Quality assessment of asymmetric stereo video coding , 2010, 2010 IEEE International Conference on Image Processing.

[16]  Atanas Gotchev,et al.  Subjective evaluation of mobile 3D video content: depth range versus compression artifacts , 2011, Electronic Imaging.

[17]  Jean-Yves Guillemaut,et al.  Stereoscopic Video Quality Assessment Using Binocular Energy , 2017, IEEE Journal of Selected Topics in Signal Processing.

[18]  Federica Battisti,et al.  Evaluation of the effects of transmission impairments on perceived video quality by exploiting ReTRiEVED dataset , 2017, J. Electronic Imaging.

[19]  Junyong You,et al.  PERCEPTUAL QUALITY ASSESSMENT FOR STEREOSCOPIC IMAGES BASED ON 2 D IMAGE QUALITY METRICS AND DISPARITY ANALYSIS , 2010 .

[20]  Alan C. Bovik,et al.  No-Reference Quality Assessment of Natural Stereopairs , 2013, IEEE Transactions on Image Processing.

[21]  Gal A. Kaminka,et al.  Color stereoscopic images requiring only one color image , 2007 .

[22]  Tao Chen,et al.  3D-TV Content Storage and Transmission , 2011, IEEE Transactions on Broadcasting.

[23]  Romain Cousseau,et al.  Subjective quality assessment of error concealment strategies for 3DTV in the presence of asymmetric transmission errors , 2010, 2010 18th International Packet Video Workshop.

[24]  Patrick Le Callet,et al.  Cross-lab study on preference of experience in 3DTV: Influence from display technology and test environment , 2013, 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX).

[25]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[26]  Ying Chen,et al.  Overview of the Multiview and 3D Extensions of High Efficiency Video Coding , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Zhou Wang,et al.  Asymmetrically Compressed Stereoscopic 3D Videos: Quality Assessment and Rate-Distortion Performance Evaluation , 2017, IEEE Transactions on Image Processing.

[28]  Federica Battisti,et al.  Study of the effects of video content on quality of experience , 2015, Electronic Imaging.

[29]  Patrick Le Callet,et al.  Objective image quality assessment of 3D synthesized views , 2015, Signal Process. Image Commun..

[30]  Nikolay N. Ponomarenko,et al.  Image database TID2013: Peculiarities, results and perspectives , 2015, Signal Process. Image Commun..

[31]  Jong-Seok Lee,et al.  On Designing Paired Comparison Experiments for Subjective Multimedia Quality Assessment , 2014, IEEE Transactions on Multimedia.

[32]  T. Tominaga,et al.  Performance comparisons of subjective quality assessment methods for mobile video , 2010, 2010 Second International Workshop on Quality of Multimedia Experience (QoMEX).

[33]  Ken Chen,et al.  Asymmetric Coding of Multi-View Video Plus Depth Based 3-D Video for View Rendering , 2012, IEEE Transactions on Multimedia.

[34]  Panos Nasiopoulos,et al.  Subjective study on asymmetric stereoscopic video with low-pass filtered slices , 2012, 2012 International Conference on Computing, Networking and Communications (ICNC).

[35]  Truong Q. Nguyen,et al.  Comparing Perceived Quality and Fatigue for Two Methods of Mixed Resolution Stereoscopic Coding , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Rafal Mantiuk,et al.  Comparison of Four Subjective Methods for Image Quality Assessment , 2012, Comput. Graph. Forum.

[37]  Chen,et al.  Methodologies for Assessing 3D QoE: Standards and Explorative Studies , 2013 .

[38]  Wa James Tam,et al.  Stereoscopic video: asymmetrical coding with temporal interleaving , 2001, IS&T/SPIE Electronic Imaging.

[39]  Aljoscha Smolic,et al.  Multi-View Video Plus Depth Representation and Coding , 2007, 2007 IEEE International Conference on Image Processing.

[40]  Panos Nasiopoulos,et al.  The Effect of Frame Rate on 3D Video Quality and Bitrate , 2015, 1803.04826.

[41]  Takanori Hayashi,et al.  Performance comparison of subjective assessment methods for 3D video quality , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[42]  Miska M. Hannuksela,et al.  Subjective quality assessment of asymmetric stereoscopic 3D video , 2015, Signal Image Video Process..

[43]  R. Blake © 2001 Kluwer Academic Publishers. Printed in the Netherlands. 5 A Primer on Binocular Rivalry, Including Current Controversies , 2000 .

[44]  Michael G. Perkins,et al.  Data compression of stereopairs , 1992, IEEE Trans. Commun..

[45]  A. Murat Tekalp,et al.  Temporal and spatial scaling for stereoscopic video compression , 2006, 2006 14th European Signal Processing Conference.

[46]  Margaret H. Pinson,et al.  Comparing subjective video quality testing methodologies , 2003, Visual Communications and Image Processing.

[47]  Marcus Barkowsky,et al.  Subjective assessment methodology for preference of experience in 3DTV , 2013, IVMSP 2013.

[48]  A. Murat Tekalp,et al.  Evaluation of Asymmetric Stereo Video Coding and Rate Scaling for Adaptive 3D Video Streaming , 2011, IEEE Transactions on Broadcasting.

[49]  S. Thorpe,et al.  Speed of processing in the human visual system , 1996, Nature.

[50]  Junle Wang,et al.  Study on visual discomfort induced by stimulus movement at fixed depth on stereoscopic displays using shutter glasses , 2011, 2011 17th International Conference on Digital Signal Processing (DSP).

[51]  Federica Battisti,et al.  Impact of video content and transmission impairments on quality of experience , 2016, Multimedia Tools and Applications.

[52]  Karel Fliegel,et al.  Preference of Experience in Image Tone-Mapping: Dataset and Framework for Objective Measures Comparison , 2017, IEEE Journal of Selected Topics in Signal Processing.

[53]  M. Webster,et al.  Visual adaptation: Neural, psychological and computational aspects , 2007, Vision Research.

[54]  Munchurl Kim,et al.  A Hybrid Stereoscopic Video Coding Scheme Based on MPEG-2 and HEVC for 3DTV Services , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[55]  Touradj Ebrahimi,et al.  Paired comparison-based subjective quality assessment of stereoscopic images , 2013, Multimedia Tools and Applications.

[56]  Shu Liu,et al.  Asymmetric stereoscopic video encoding algorithm based on subjective visual characteristic , 2009, 2009 International Conference on Wireless Communications & Signal Processing.

[57]  Jing Li,et al.  Boosting paired comparison methodology in measuring visual discomfort of 3DTV: performances of three different designs , 2013, Electronic Imaging.

[58]  Wijnand A. IJsselsteijn,et al.  Perceived quality of compressed stereoscopic images: Effects of symmetric and asymmetric JPEG coding and camera separation , 2006, TAP.

[59]  Munchurl Kim,et al.  Binocular Suppression-Based Stereoscopic Video Coding by Joint Rate Control With KKT Conditions for a Hybrid Video Codec System , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[60]  Y. Ejima,et al.  Estimation of the Timing of Human Visual Perception from Magnetoencephalography , 2006, The Journal of Neuroscience.

[61]  Do-Kyoung Kwon,et al.  Full-reference quality assessment of stereopairs accounting for rivalry , 2013, Signal Process. Image Commun..

[62]  Borko Furht,et al.  3D TV using MPEG-2 and H.264 view coding and autostereoscopic displays , 2006, MM '06.

[63]  Ahmet M. Kondoz,et al.  Psycho-physical limits of interocular blur suppression and its application to asymmetric stereoscopic video delivery , 2012, 2012 19th International Packet Video Workshop (PV).

[64]  Yuan Zhou,et al.  Objective quality assessment method of stereo images , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[65]  Hitoshi Urabe,et al.  ISO 20462: a psychophysical image quality measurement standard , 2003, IS&T/SPIE Electronic Imaging.

[66]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[67]  Weisi Lin,et al.  No-Reference View Synthesis Quality Prediction for 3-D Videos Based on Color–Depth Interactions , 2018, IEEE Transactions on Multimedia.

[68]  Marcus Barkowsky,et al.  Perceived 3D TV Transmission Quality Assessment: Multi-Laboratory Results Using Absolute Category Rating on Quality of Experience Scale , 2012, IEEE Transactions on Broadcasting.

[69]  Zhou Wang,et al.  PERCEPTUAL QUALITY OF ASYMMETRICALLY DISTORTED STEREOSCOPIC IMAGES : THE ROLE OF IMAGE DISTORTION TYPES , 2013 .

[70]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[71]  Touradj Ebrahimi,et al.  A comprehensive database and subjective evaluation methodology for quality of experience in stereoscopic video , 2010, Electronic Imaging.

[72]  Junle Wang,et al.  Exploring the effects of subjective methodology on assessing visual discomfort in immersive multimedia , 2018, HVEI.

[73]  Krzysztof Wegner,et al.  Immersive visual media — MPEG-I: 360 video, virtual navigation and beyond , 2017, 2017 International Conference on Systems, Signals and Image Processing (IWSSIP).

[74]  Christian Schmid,et al.  A Matlab function to estimate choice model parameters from paired-comparison data , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[75]  Kwanghoon Sohn,et al.  No-Reference Quality Assessment for Stereoscopic Images Based on Binocular Quality Perception , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[76]  Mei Yu,et al.  Asymmetric multi-view video coding based on chrominance reconstruction , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[77]  Peter Gregor,et al.  Binocular single vision achieved by fusion and suppression , 1977 .

[78]  Weisi Lin,et al.  Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics , 2013, IEEE Transactions on Image Processing.

[79]  Gary R. Bradski,et al.  Peripheral-Foveal Vision for Real-time Object Recognition and Tracking in Video , 2007, IJCAI.

[80]  Marcus Barkowsky,et al.  How visual fatigue and discomfort impact 3D-TV quality of experience: a comprehensive review of technological, psychophysical, and psychological factors , 2013, annals of telecommunications - annales des télécommunications.

[81]  Clifton M. Schor,et al.  The spatial properties of binocular suppression zone , 1994, Vision Research.

[82]  Qionghai Dai,et al.  Learning Sparse Representation for No-Reference Quality Assessment of Multiply Distorted Stereoscopic Images , 2017, IEEE Transactions on Multimedia.

[83]  Kai Zeng,et al.  Quality Prediction of Asymmetrically Distorted Stereoscopic 3D Images , 2015, IEEE Transactions on Image Processing.

[84]  Narciso García,et al.  NAMA3DS1-COSPAD1: Subjective video quality assessment database on coding conditions introducing freely available high quality 3D stereoscopic sequences , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[85]  Zhibo Chen,et al.  Blind Stereoscopic Video Quality Assessment: From Depth Perception to Overall Experience. , 2018, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[86]  Alan C. Bovik,et al.  Subjective evaluation of stereoscopic image quality , 2013, Signal Process. Image Commun..

[87]  Miska M. Hannuksela,et al.  Subjective study on compressed asymmetric stereoscopic video , 2010, 2010 IEEE International Conference on Image Processing.