Sharpness Mismatch Detection in Stereoscopic Content with 360-Degree Capability

This paper presents a novel sharpness mismatch detection method for stereoscopic images based on the comparison of edge width histograms of the left and right view. The new method is evaluated on the LIVE 3D Phase II and Ningbo 3D Phase I datasets and compared with two state-of-the-art methods. Experimental results show that the new method highly correlates with user scores of subjective tests and that it outperforms the current state-of-the-art. We then extend the method to stereoscopic omnidirectional images by partitioning the images into patches using a spherical Voronoi diagram. Furthermore, we integrate visual attention data into the detection process in order to weight sharpness mismatch according to the likelihood of its appearance in the viewport of the end-user's virtual reality device. For obtaining visual attention data, we performed a subjective experiment with 17 test subjects and 96 stereoscopic omnidirectional images. The entire dataset including the viewport trajectory data and resulting visual attention maps are publicly available with this paper.

[1]  Dmitriy Vatolin,et al.  Sharpness Mismatch and 6 Other Stereoscopic Artifacts Measured on 10 Chinese S3D Movies , 2017 .

[2]  Stefan Winkler,et al.  Perceptual blur and ringing metrics: application to JPEG2000 , 2004, Signal Process. Image Commun..

[3]  Gwendal Simon,et al.  360-Degree Video Head Movement Dataset , 2017, MMSys.

[4]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  H. Hirschmüller Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information , 2005, CVPR.

[6]  Simone Croci,et al.  A Modular Scheme for Artifact Detection in Stereoscopic Omni-Directional Images , 2017 .

[7]  Thomas Sikora,et al.  The Avoidance of Visual Discomfort and Basic Rules for Producing “Good 3D” Pictures , 2012 .

[8]  Gangyi Jiang,et al.  Research on subjective stereoscopic image quality assessment , 2009, Electronic Imaging.

[9]  Mtm Marc Lambooij,et al.  Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review , 2009 .

[10]  Aljoscha Smolic,et al.  Saliency-Based Sharpness Mismatch Detection For Stereoscopic Omnidirectional Images , 2017, CVMP.

[11]  Alexander Raake,et al.  Efficient no-reference metric for sharpness mismatch artifact between stereoscopic views , 2016, J. Vis. Commun. Image Represent..

[12]  Dmitriy Vatolin,et al.  Automatic detection of artifacts in converted S3D video , 2014, Electronic Imaging.

[13]  Touradj Ebrahimi,et al.  Testbed for subjective evaluation of omnidirectional visual content , 2016, 2016 Picture Coding Symposium (PCS).

[14]  Michael Werman,et al.  Fast and robust Earth Mover's Distances , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[15]  Heinz Hügli,et al.  Dynamic visual attention on the sphere , 2010, Comput. Vis. Image Underst..

[16]  Kwanghoon Sohn,et al.  No-Reference Quality Assessment for Stereoscopic Images Based on Binocular Quality Perception , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Alan C. Bovik,et al.  No-Reference Quality Assessment of Natural Stereopairs , 2013, IEEE Transactions on Image Processing.

[19]  Cagri Ozcinar,et al.  Look around you: Saliency maps for omnidirectional images in VR applications , 2017, 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX).

[20]  David M. Hoffman,et al.  The zone of comfort: Predicting visual discomfort with stereo displays. , 2011, Journal of vision.

[21]  Dmitriy Vatolin,et al.  Trends in S3D-Movie Quality Evaluated on 105 Films Using 10 Metrics , 2016 .

[22]  Lina J. Karam,et al.  A no-reference perceptual image sharpness metric based on a cumulative probability of blur detection , 2009, 2009 International Workshop on Quality of Multimedia Experience.