Quality Prediction of Asymmetrically Distorted Stereoscopic 3D Images

Objective quality assessment of distorted stereoscopic images is a challenging problem, especially when the distortions in the left and right views are asymmetric. Existing studies suggest that simply averaging the quality of the left and right views well predicts the quality of symmetrically distorted stereoscopic images, but generates substantial prediction bias when applied to asymmetrically distorted stereoscopic images. In this paper, we first build a database that contains both single-view and symmetrically and asymmetrically distorted stereoscopic images. We then carry out a subjective test, where we find that the quality prediction bias of the asymmetrically distorted images could lean toward opposite directions (overestimate or underestimate), depending on the distortion types and levels. Our subjective test also suggests that eye dominance effect does not have strong impact on the visual quality decisions of stereoscopic images. Furthermore, we develop an information content and divisive normalization-based pooling scheme that improves upon structural similarity in estimating the quality of single-view images. Finally, we propose a binocular rivalry-inspired multi-scale model to predict the quality of stereoscopic images from that of the single-view images. Our results show that the proposed model, without explicitly identifying image distortion types, successfully eliminates the prediction bias, leading to significantly improved quality prediction of the stereoscopic images.

[1]  Jiachen Yang,et al.  Stereo Picture Quality Estimation Based on a Multiple Channel HVS Model , 2009, 2009 2nd International Congress on Image and Signal Processing.

[2]  Mohamed-Chaker Larabi,et al.  Towards a perceptual quality metric for color stereo images , 2010, 2010 IEEE International Conference on Image Processing.

[3]  Alan C. Bovik,et al.  No-Reference Quality Assessment of Natural Stereopairs , 2013, IEEE Transactions on Image Processing.

[4]  Karen O. Egiazarian,et al.  3D-DCT based perceptual quality assessment of stereo video , 2011, 2011 18th IEEE International Conference on Image Processing.

[5]  Ahmet M. Kondoz,et al.  Prediction of stereoscopic video quality using objective quality models of 2-D video , 2008 .

[6]  Ja-Ling Wu,et al.  Quality Assessment of Stereoscopic 3D Image Compression by Binocular Integration Behaviors , 2014, IEEE Transactions on Image Processing.

[7]  George Sperling,et al.  A gain-control theory of binocular combination. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Patrick Le Callet,et al.  Using disparity for quality assessment of stereoscopic images , 2008, 2008 15th IEEE International Conference on Image Processing.

[9]  Ahmet M. Kondoz,et al.  Quality analysis for 3D video using 2D video quality models , 2008, IEEE Transactions on Consumer Electronics.

[10]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[11]  M. Fahle Binocular rivalry: Suppression depends on orientation and spatial frequency , 1982, Vision Research.

[12]  Patrick Le Callet,et al.  Quality Assessment of Stereoscopic Images , 2008, EURASIP J. Image Video Process..

[13]  Peter G. J. Barten,et al.  Formula for the contrast sensitivity of the human eye , 2003, IS&T/SPIE Electronic Imaging.

[14]  Narciso García,et al.  NAMA3DS1-COSPAD1: Subjective video quality assessment database on coding conditions introducing freely available high quality 3D stereoscopic sequences , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[15]  Touradj Ebrahimi,et al.  Impact of Acquisition Distortion on the Quality of Stereoscopic Images , 2010 .

[16]  Zhongjie Zhu,et al.  Perceptual distortion metric for stereo video quality evaluation , 2009 .

[17]  Kwanghoon Sohn,et al.  Stereoscopic image quality metric based on binocular perception model , 2012, 2012 19th IEEE International Conference on Image Processing.

[18]  Marcus Barkowsky,et al.  EXPLORATION OF QUALITY OF EXPERIENCE OF STEREOSCOPIC IMAGES: BINOCULAR DEPTH , 2012 .

[19]  R. Blake,et al.  Spatial frequency tuning of human stereopsis , 1991, Vision Research.

[20]  Mei Yu,et al.  Subjective quality analyses of stereoscopic images in 3DTV system , 2011, 2011 Visual Communications and Image Processing (VCIP).

[21]  Zhou Wang,et al.  Reduced-reference image quality assessment using a wavelet-domain natural image statistic model , 2005, IS&T/SPIE Electronic Imaging.

[22]  Gangyi Jiang,et al.  Research on subjective stereoscopic image quality assessment , 2009, Electronic Imaging.

[23]  Khan,et al.  PII: S0042-6989(01)00079-7 , 2001 .

[24]  Thomas Wiegand,et al.  Mixed resolution coding of stereoscopic video for Mobile devices , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[25]  Christopher Joseph Pal,et al.  Learning Conditional Random Fields for Stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Do-Kyoung Kwon,et al.  Full-reference quality assessment of stereopairs accounting for rivalry , 2013, Signal Process. Image Commun..

[27]  Hiroshi Ono,et al.  What does the dominant eye dominate? A brief and somewhat contentious review , 2003, Perception & psychophysics.

[28]  Rabab Kreidieh Ward,et al.  HDR image construction from multi-exposed stereo LDR images , 2010, 2010 IEEE International Conference on Image Processing.

[29]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[30]  Patrick Le Callet,et al.  An image quality assessment method based on perception of structural information , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[31]  Weisi Lin,et al.  Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics , 2013, IEEE Transactions on Image Processing.

[32]  Zhou Wang,et al.  Video quality assessment based on structural distortion measurement , 2004, Signal Process. Image Commun..

[33]  Chaminda T. E. R. Hewage,et al.  Reduced-reference quality metric for 3D depth map transmission , 2010, 2010 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[34]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[35]  D V Meegan,et al.  Unequal weighting of monocular inputs in binocular combination: implications for the compression of stereoscopic imagery. , 2001, Journal of experimental psychology. Applied.

[36]  Joel Pokorny,et al.  Sight and Mind. An Introduction to Visual Perception , 1974 .

[37]  Zhou Wang,et al.  Information Content Weighting for Perceptual Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[38]  Pjh Pieter Seuntiëns,et al.  Visual experience of 3D TV , 2006 .

[39]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[40]  Junyong You,et al.  PERCEPTUAL QUALITY ASSESSMENT FOR STEREOSCOPIC IMAGES BASED ON 2 D IMAGE QUALITY METRICS AND DISPARITY ANALYSIS , 2010 .

[41]  Zhou Wang,et al.  Quality prediction of asymmetrically compressed stereoscopic videos , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[42]  Roushain Akhter,et al.  No-reference stereoscopic image quality assessment , 2010, Electronic Imaging.

[43]  Miska M. Hannuksela,et al.  Subjective quality assessment of asymmetric stereoscopic 3D video , 2015, Signal Image Video Process..

[44]  Nick Holliman,et al.  Stereoscopic image quality metrics and compression , 2008, Electronic Imaging.

[45]  Zhou Wang,et al.  PERCEPTUAL QUALITY OF ASYMMETRICALLY DISTORTED STEREOSCOPIC IMAGES : THE ROLE OF IMAGE DISTORTION TYPES , 2013 .

[46]  W. Levelt The alternation process in binocular rivalry , 1966 .

[47]  R. Blake © 2001 Kluwer Academic Publishers. Printed in the Netherlands. 5 A Primer on Binocular Rivalry, Including Current Controversies , 2000 .

[48]  Yuukou Horita,et al.  Stereoscopic image quality prediction , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[49]  Atanas Gotchev,et al.  Subjective evaluation of mobile 3D video content: depth range versus compression artifacts , 2011, Electronic Imaging.

[50]  André Vincent,et al.  Stereo image quality: effects of mixed spatio-temporal resolution , 2000, IEEE Trans. Circuits Syst. Video Technol..

[51]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[52]  Jianjun Lei,et al.  New metric for stereo image quality assessment based on HVS , 2010, Int. J. Imaging Syst. Technol..

[53]  Martin J. Wainwright,et al.  Scale Mixtures of Gaussians and the Statistics of Natural Images , 1999, NIPS.

[54]  Wijnand A. IJsselsteijn,et al.  Perceived quality of compressed stereoscopic images: Effects of symmetric and asymmetric JPEG coding and camera separation , 2006, TAP.

[55]  Margaret H. Pinson,et al.  The Consumer Digital Video Library [Best of the Web] , 2013, IEEE Signal Processing Magazine.

[56]  Alan C. Bovik,et al.  Image information and visual quality , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[57]  Kai Zeng,et al.  Quality prediction of asymmetrically distorted stereoscopic images from single views , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[58]  Alan C. Bovik,et al.  Subjective evaluation of stereoscopic image quality , 2013, Signal Process. Image Commun..

[59]  R. Blake Threshold conditions for binocular rivalry. , 1977, Journal of experimental psychology. Human perception and performance.

[60]  Margaret H. Pinson,et al.  A new standardized method for objectively measuring video quality , 2004, IEEE Transactions on Broadcasting.

[61]  Yuan Zhou,et al.  Objective quality assessment method of stereo images , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[62]  Do-Kyoung Kwon,et al.  Study of subject agreement on stereoscopic video quality , 2012, 2012 IEEE Southwest Symposium on Image Analysis and Interpretation.

[63]  A. Gotchev,et al.  Quality assessment of 3D video in rate allocation experiments , 2008, 2008 IEEE International Symposium on Consumer Electronics.

[64]  Patrick Le Callet,et al.  Stereoscopic images quality assessment , 2007, 2007 15th European Signal Processing Conference.

[65]  J. Pokorny Foundations of Cyclopean Perception , 1972 .

[66]  Zhou Wang,et al.  Spatial Pooling Strategies for Perceptual Image Quality Assessment , 2006, 2006 International Conference on Image Processing.