Quality Assessment of Stereoscopic 3D Image Compression by Binocular Integration Behaviors

The objective approaches of 3D image quality assessment play a key role for the development of compression standards and various 3D multimedia applications. The quality assessment of 3D images faces more new challenges, such as asymmetric stereo compression, depth perception, and virtual view synthesis, than its 2D counterparts. In addition, the widely used 2D image quality metrics (e.g., PSNR and SSIM) cannot be directly applied to deal with these newly introduced challenges. This statement can be verified by the low correlation between the computed objective measures and the subjectively measured mean opinion scores (MOSs), when 3D images are the tested targets. In order to meet these newly introduced challenges, in this paper, besides traditional 2D image metrics, the binocular integration behaviors-the binocular combination and the binocular frequency integration, are utilized as the bases for measuring the quality of stereoscopic 3D images. The effectiveness of the proposed metrics is verified by conducting subjective evaluations on publicly available stereoscopic image databases. Experimental results show that significant consistency could be reached between the measured MOS and the proposed metrics, in which the correlation coefficient between them can go up to 0.88. Furthermore, we found that the proposed metrics can also address the quality assessment of the synthesized color-plus-depth 3D images well. Therefore, it is our belief that the binocular integration behaviors are important factors in the development of objective quality assessment for 3D images.

[1]  Sumohana S. Channappayya,et al.  No- Reference Stereoscopic Image Quality Assessment , 2015 .

[2]  Ja-Ling Wu,et al.  A Digital Blind Watermarking for Depth-Image-Based Rendering 3D Images , 2011, IEEE Transactions on Broadcasting.

[3]  Chaminda T. E. R. Hewage,et al.  Reduced-reference quality metric for 3D depth map transmission , 2010, 2010 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[4]  Ghassan Al-Regib,et al.  MIQM: A Multicamera Image Quality Measure , 2012, IEEE Transactions on Image Processing.

[5]  J. Bergen,et al.  A four mechanism model for threshold spatial vision , 1979, Vision Research.

[6]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[7]  Alan C. Bovik,et al.  Statistical Modeling of 3-D Natural Scenes With Application to Bayesian Stereopsis , 2011, IEEE Transactions on Image Processing.

[8]  Scott B. Steinman,et al.  Foundations of Binocular Vision: A Clinical Perspective , 2000 .

[9]  Yuan Zhou,et al.  Objective quality assessment method of stereo images , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[10]  George Mather,et al.  Foundations of Sensation and Perception , 2016 .

[11]  Zhenzhong Chen,et al.  Binocular Just-Noticeable-Difference Model for Stereoscopic Images , 2011, IEEE Signal Processing Letters.

[12]  Ahmet M. Kondoz,et al.  Quality Evaluation of Color Plus Depth Map-Based Stereoscopic Video , 2009, IEEE Journal of Selected Topics in Signal Processing.

[13]  Zhou Wang,et al.  Complex Wavelet Structural Similarity: A New Image Similarity Index , 2009, IEEE Transactions on Image Processing.

[14]  T. Meese,et al.  Binocular contrast interactions: Dichoptic masking is not a single process , 2007, Vision Research.

[15]  Alexander Raake,et al.  Evaluating Depth Perception of 3D Stereoscopic Videos , 2012, IEEE Journal of Selected Topics in Signal Processing.

[16]  I. Ohzawa,et al.  Stereoscopic depth discrimination in the visual cortex: neurons ideally suited as disparity detectors. , 1990, Science.

[17]  Alan C. Bovik,et al.  No-Reference Quality Assessment of Natural Stereopairs , 2013, IEEE Transactions on Image Processing.

[18]  Karen O. Egiazarian,et al.  3D-DCT based perceptual quality assessment of stereo video , 2011, 2011 18th IEEE International Conference on Image Processing.

[19]  Aldo Maalouf,et al.  CYCLOP: A stereo color image quality assessment metric , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  David J. Sakrison,et al.  The effects of a visual fidelity criterion of the encoding of images , 1974, IEEE Trans. Inf. Theory.

[21]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[22]  Junyong You,et al.  PERCEPTUAL QUALITY ASSESSMENT FOR STEREOSCOPIC IMAGES BASED ON 2 D IMAGE QUALITY METRICS AND DISPARITY ANALYSIS , 2010 .

[23]  R. Weale.  Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[24]  Hubert Konik,et al.  Full Reference Image Quality Assessment Based on Saliency Map Analysis , 2010 .

[25]  George Sperling,et al.  A gain-control theory of binocular combination. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[26]  Patrick Le Callet,et al.  Quality Assessment of Stereoscopic Images , 2008, EURASIP J. Image Video Process..

[27]  Jiebo Luo,et al.  Toward Assessing and Improving the Quality of Stereo Images , 2012, IEEE Journal of Selected Topics in Signal Processing.

[28]  S. Grossberg,et al.  Neural dynamics of binocular brightness perception , 1999, Vision Research.

[29]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[30]  A. Murat Tekalp,et al.  Evaluation of Asymmetric Stereo Video Coding and Rate Scaling for Adaptive 3D Video Streaming , 2011, IEEE Transactions on Broadcasting.

[31]  Gordon E. Legge,et al.  Binocular contrast summation—II. Quadratic summation , 1984, Vision Research.

[32]  Zhongjie Zhu,et al.  Perceptual distortion metric for stereo video quality evaluation , 2009 .

[33]  Rabab Kreidieh Ward,et al.  Rendering 3-D High Dynamic Range Images: Subjective Evaluation of Tone-Mapping Methods and Preferred 3-D Image Attributes , 2012, IEEE Journal of Selected Topics in Signal Processing.

[34]  G. Legge.  Spatial frequency masking in human vision: binocular interactions. , 1979, Journal of the Optical Society of America.

[35]  Robert Fletcher,et al.  Normal Binocular Vision: Theory, Investigation and Practical Aspects , 2010 .

[36]  Do-Kyoung Kwon,et al.  Full-reference quality assessment of stereopairs accounting for rivalry , 2013, Signal Process. Image Commun..

[37]  M. Georgeson,et al.  Binocular contrast vision at and above threshold. , 2006, Journal of vision.

[38]  Mark A. Georgeson,et al.  Nonlinearities in the binocular combination of luminance and contrast , 2012, Vision Research.

[39]  Subjective methods for the assessment of stereoscopic 3DTV systems , 2015 .

[40]  Ahmet M. Kondoz,et al.  Prediction of stereoscopic video quality using objective quality models of 2-D video , 2008 .

[41]  A. B. Nutt.  Binocular vision. , 1945, The British orthoptic journal.

[42]  Ian P. Howard,et al.  Seeing in Depth , 2008 .

[43]  Gordon E. Legge,et al.  Binocular contrast summation—I. Detection and discrimination , 1984, Vision Research.

[44]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[45]  Atanas P. Gotchev,et al.  Stereoscopic Depth Cues Outperform Monocular Ones on Autostereoscopic Display , 2012, IEEE Journal of Selected Topics in Signal Processing.

[46]  Jiachen Yang,et al.  Stereo Picture Quality Estimation Based on a Multiple Channel HVS Model , 2009, 2009 2nd International Congress on Image and Signal Processing.

[47]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[48]  Jing Liu,et al.  3D+2DTV: 3D displays with no ghosting for viewers without glasses , 2013, TOGS.

[49]  Wijnand A. IJsselsteijn,et al.  A survey of perceptual evaluations and requirements of three-dimensional TV , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[50]  A. Watson,et al.  A standard model for foveal detection of spatial contrast. , 2005, Journal of vision.

[51]  Mohamed-Chaker Larabi,et al.  A perceptual metric for stereoscopic image quality assessment based on the binocular energy , 2013, Multidimens. Syst. Signal Process..

[52]  Jianfei Cai,et al.  Three Dimensional Scalable Video Adaptation via User-End Perceptual Quality Assessment , 2008, IEEE Transactions on Broadcasting.

[53]  H. Wilson.  Quantitative characterization of two types of line-spread function near the fovea , 1978, Vision Research.

[54]  Miska M. Hannuksela,et al.  Subjective study on compressed asymmetric stereoscopic video , 2010, 2010 IEEE International Conference on Image Processing.

[55]  D Marr,et al.  A computational theory of human stereo vision. , 1979, Proceedings of the Royal Society of London. Series B, Biological sciences.

[56]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[57]  Nick Holliman,et al.  Stereoscopic image quality metrics and compression , 2008, Electronic Imaging.

[58]  Ja-Ling Wu,et al.  Rendering Lossless Compression of Depth Image , 2011, 2011 Data Compression Conference.

[59]  Weisi Lin,et al.  Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics , 2013, IEEE Transactions on Image Processing.

[60]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[61]  Wijnand A. IJsselsteijn,et al.  Evaluation of Stereoscopic Images: Beyond 2D Quality , 2011, IEEE Transactions on Broadcasting.

[62]  Ahmet M. Kondoz,et al.  Toward an Impairment Metric for Stereoscopic Video: A Full-Reference Video Quality Metric to Assess Compressed Stereoscopic Video , 2013, IEEE Transactions on Image Processing.

[63]  Weisi Lin,et al.  Perceptual visual quality metrics: A survey , 2011, J. Vis. Commun. Image Represent..

[64]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[65]  Wijnand A. IJsselsteijn,et al.  Perceived quality of compressed stereoscopic images: Effects of symmetric and asymmetric JPEG coding and camera separation , 2006, TAP.

[66]  Ichiro Fujita,et al.  Spatial frequency integration for binocular correspondence in macaque area V4. , 2008, Journal of neurophysiology.

[67]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[68]  W. Marsden.  I and J , 2012 .

[69]  Weisi Lin,et al.  Image Quality Assessment Using Multi-Method Fusion , 2013, IEEE Transactions on Image Processing.

[70]  D. Chandler,et al.  Supplement to “ VSNR : A Visual Signal-to-Noise Ratio for Natural Images Based on Near-Threshold and Suprathreshold Vision ” , 2007 .