No-Reference Quality Assessment for Stereoscopic Images Based on Binocular Quality Perception

Quality perception of 3-D images is one of the most important parameters for accelerating advances in 3-D imaging fields. Despite active research in recent years for understanding the quality perception of 3-D images, binocular quality perception of asymmetric distortions in stereoscopic images is not thoroughly comprehended. In this paper, we explore the relationship between the perceptual quality of stereoscopic images and visual information, and introduce a model for binocular quality perception. Based on this binocular quality perception model, a no-reference quality metric for stereoscopic images is proposed. The proposed metric is a top-down method modeling the binocular quality perception of the human visual system in the context of blurriness and blockiness. Perceptual blurriness and blockiness scores of left and right images were computed using local blurriness, blockiness, and visual saliency information and then combined into an overall quality index using the binocular quality perception model. Experiments for image and video databases show that the proposed metric provides consistent correlations with subjective quality scores. The results also show that the proposed metric provides higher performance than existing full-reference methods even though the proposed method is a no-reference approach.

[1]  Wijnand A. IJsselsteijn,et al.  A survey of perceptual evaluations and requirements of three-dimensional TV , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Woo-Shik Kim,et al.  3-D video coding system with enhanced rendered view quality , 2011 .

[3]  L. Kaufman.  Sight and mind : an introduction to visual perception , 1974 .

[4]  W.A.C. Fernando,et al.  A compound depth and image quality metric for measuring the effects of packet loss on 3D video , 2011, 2011 17th International Conference on Digital Signal Processing (DSP).

[5]  David M. Hoffman,et al.  Perceptual Issues in Stereoscopic Signal Processing , 2011, IEEE Transactions on Broadcasting.

[6]  Cable Ieee Broadcast.  IEEE transactions on broadcasting , 1963 .

[7]  Alan C. Bovik,et al.  Range image quality assessment by Structural Similarity , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[8]  B. Julesz.  Binocular depth perception of computer-generated patterns , 1960 .

[9]  Filippo Speranza,et al.  Stereoscopic 3D-TV: Visual Comfort , 2011, IEEE Transactions on Broadcasting.

[10]  Chaminda T. E. R. Hewage,et al.  Reduced-reference quality assessment for 3D video compression and transmission , 2011, IEEE Transactions on Consumer Electronics.

[11]  Yuukou Horita,et al.  Stereoscopic image quality prediction , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[12]  Ashraf A. Kassim,et al.  Digital Video Image Quality and Perceptual Coding , 2005, J. Electronic Imaging.

[13]  Stephan Reichelt,et al.  Depth cues in human visual perception and their realization in 3D displays , 2010, Defense + Commercial Sensing.

[14]  Patrick Le Callet,et al.  Towards a New Quality Metric for 3-D Synthesized View Assessment , 2011, IEEE Journal of Selected Topics in Signal Processing.

[15]  Kwanghoon Sohn,et al.  Visual fatigue evaluation and enhancement for 2D-plus-depth video , 2010, 2010 IEEE International Conference on Image Processing.

[16]  Sumio Yano,et al.  A study of visual fatigue and visual comfort for 3D HDTV/HDTV images , 2002 .

[17]  J. Moran,et al.  Sensation and perception , 1980 .

[18]  Filippo Speranza,et al.  Effect of disparity and motion on visual comfort of stereoscopic images , 2006, Electronic Imaging.

[19]  Margaret H. Pinson,et al.  A new standardized method for objectively measuring video quality , 2004, IEEE Transactions on Broadcasting.

[20]  S. Anstis,et al.  Nonlinear combination of luminance excursions during flicker, simultaneous contrast, afterimages and binocular fusion , 1998, Vision Research.

[21]  A. C. Bovik.  SSIM BASED RANGE IMAGE QUALITY ASSESSMENT , 2008 .

[22]  Pjh Pieter Seuntiëns,et al.  Visual experience of 3D TV , 2006 .

[23]  A. Aksay,et al.  Towards compound stereo-video quality metric: a specific encoder-based framework , 2006, 2006 IEEE Southwest Symposium on Image Analysis and Interpretation.

[24]  Heiko Hirschmüller,et al.  Evaluation of Cost Functions for Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Wijnand A. IJsselsteijn,et al.  Subjective evaluation of stereoscopic images: effects of camera parameters and display duration , 2000, IEEE Trans. Circuits Syst. Video Technol..

[26]  Gary J. Sullivan,et al.  Overview of the Stereo and Multiview Video Coding Extensions of the H.264/MPEG-4 AVC Standard , 2011, Proceedings of the IEEE.

[27]  Mohamed-Chaker Larabi,et al.  Towards a perceptual quality metric for color stereo images , 2010, 2010 IEEE International Conference on Image Processing.

[28]  Gozde Bozdagi Akar,et al.  Quality evaluation of stereoscopic videos using depth map segmentation , 2011, 2011 Third International Workshop on Quality of Multimedia Experience.

[29]  Mtm Marc Lambooij,et al.  Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review , 2009 .

[30]  Wijnand A. IJsselsteijn,et al.  Perceptual evaluation of JPEG-coded stereoscopic images , 2003, IS&T/SPIE Electronic Imaging.

[31]  Ahmet M. Kondoz,et al.  Perceptual Video Quality Metric for 3D video quality assessment , 2010, 2010 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[32]  Junyong You,et al.  PERCEPTUAL QUALITY ASSESSMENT FOR STEREOSCOPIC IMAGES BASED ON 2 D IMAGE QUALITY METRICS AND DISPARITY ANALYSIS , 2010 .

[33]  Kwanghoon Sohn,et al.  Visual fatigue modeling and analysis for stereoscopic video , 2012 .

[34]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[35]  Patrick Le Callet,et al.  An image quality assessment method based on perception of structural information , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[36]  Ahmet M. Kondoz,et al.  Quality Evaluation of Color Plus Depth Map-Based Stereoscopic Video , 2009, IEEE Journal of Selected Topics in Signal Processing.

[37]  Karen O. Egiazarian,et al.  3D-DCT based perceptual quality assessment of stereo video , 2011, 2011 18th IEEE International Conference on Image Processing.

[38]  Subjective methods for the assessment of stereoscopic 3DTV systems , 2015 .

[39]  Zhou Wang,et al.  Multi-scale structural similarity for image quality assessment , 2003 .

[40]  Kwanghoon Sohn,et al.  Visual Fatigue Prediction for Stereoscopic Image , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Nikolay N. Ponomarenko,et al.  TID2008 – A database for evaluation of full-reference visual quality assessment metrics , 2004 .

[42]  Wijnand A. IJsselsteijn,et al.  Evaluation of Stereoscopic Images: Beyond 2D Quality , 2011, IEEE Transactions on Broadcasting.

[43]  Wa Wijnand IJsselsteijn,et al.  Visual discomfort in stereoscopic displays: a review , 2007, Electronic Imaging.

[44]  Ghassan Al-Regib,et al.  3VQM: A vision-based quality measure for DIBR-based 3D videos , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[45]  Aldo Maalouf,et al.  CYCLOP: A stereo color image quality assessment metric , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[46]  Philip J. Corriveau,et al.  Psychovisual aspects of viewing stereoscopic video sequences , 1998, Electronic Imaging.

[47]  Ahmet M. Kondoz,et al.  Predicting sensation of depth in 3D video , 2010 .

[48]  B. Julesz.  Foundations of Cyclopean Perception , 1971 .

[49]  Patrick Le Callet,et al.  Stereoscopic images quality assessment , 2007, 2007 15th European Signal Processing Conference.

[50]  Y. Horita,et al.  Spatio-temporal segmentation based continuous no-reference stereoscopic video quality prediction , 2010, 2010 Second International Workshop on Quality of Multimedia Experience (QoMEX).

[51]  Alexander Toet,et al.  Visual comfort of binocular and 3D displays , 2004 .

[52]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[53]  Zhou Wang,et al.  Modern Image Quality Assessment , 2006, Modern Image Quality Assessment.

[54]  Patrick Le Callet,et al.  Quality Assessment of Stereoscopic Images , 2008, EURASIP J. Image Video Process..

[55]  Yong Man Ro,et al.  Visual comfort assessment metric based on salient object motion information in stereoscopic video , 2012, J. Electronic Imaging.

[56]  Jianjun Lei,et al.  New metric for stereo image quality assessment based on HVS , 2010, Int. J. Imaging Syst. Technol..

[57]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[58]  Robert Patterson,et al.  Human factors of 3‐D displays , 2007 .

[59]  Kwanghoon Sohn,et al.  Blind blockiness measure based on marginal distribution of wavelet coefficient and saliency , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[60]  Patrick Le Callet,et al.  Using disparity for quality assessment of stereoscopic images , 2008, 2008 15th IEEE International Conference on Image Processing.

[61]  Kwanghoon Sohn,et al.  Stereoscopic image quality metric based on binocular perception model , 2012, 2012 19th IEEE International Conference on Image Processing.

[62]  Guihua Er,et al.  Objective quality assessment of depth image based rendering in 3DTV system , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[63]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[64]  Zhou Wang,et al.  Reduced-reference image quality assessment using a wavelet-domain natural image statistic model , 2005, IS&T/SPIE Electronic Imaging.

[65]  Zhou Wang,et al.  Translation insensitive image similarity in complex wavelet domain , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[66]  Ghassan Al-Regib,et al.  A no-reference quality measure for DIBR-based 3D videos , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[67]  Karen O. Egiazarian,et al.  Validation of a new full reference metric for quality assessment of mobile 3DTV content , 2011, 2011 19th European Signal Processing Conference.

[68]  Ahmet M. Kondoz,et al.  Prediction of stereoscopic video quality using objective quality models of 2-D video , 2008 .

[69]  Yuan Zhou,et al.  Objective quality assessment method of stereo images , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[70]  R. Haber,et al.  Visual Perception , 2018, Encyclopedia of Database Systems.

[71]  Alan C. Bovik,et al.  Subjective evaluation of stereoscopic image quality , 2013, Signal Process. Image Commun..

[72]  D V Meegan,et al.  Unequal weighting of monocular inputs in binocular combination: implications for the compression of stereoscopic imagery. , 2001, Journal of experimental psychology. Applied.

[73]  Wa Wijnand IJsselsteijn,et al.  Capturing the added value of three-dimensional television : viewing experience and naturalness of stereoscopic images , 2008 .

[74]  Wijnand A. IJsselsteijn,et al.  Perceived quality of compressed stereoscopic images: Effects of symmetric and asymmetric JPEG coding and camera separation , 2006, TAP.

[75]  F. Okano,et al.  Repeated vergence adaptation causes the decline of visual functions in watching stereoscopic television , 2005, Journal of Display Technology.

[76]  Kwanghoon Sohn,et al.  No-reference sharpness metric based on inherent sharpness , 2011 .

[77]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .