Stereoscopic saliency model using contrast and depth-guided-background prior

Abstract Many successful models of saliency have been proposed to detect salient regions for 2D images. Because stereopsis, with its distinctive depth information, influences human viewing, it is necessary for stereoscopic saliency detection to consider depth information as an additional cue. In this paper, we propose a 3D stereoscopic saliency model based on both contrast and depth-guided-background prior. First, a depth-guided-background prior is specifically detected from a disparity map apart from the conventional prior, assuming boundary super-pixels as background. Then, saliency based on disparity with the help of the proposed prior is proposed to prioritize the contrasts among super-pixels. In addition, a scheme to combine the contrast of disparity and the contrast of color is presented. Finally, 2D spatial dissimilarity features are further employed to refine the saliency map. Experimental results on the PSU stereo saliency benchmark dataset (SSB) show that the proposed method performs better than existing saliency models.

[1]  Christel Chamaret,et al.  Adaptive 3D rendering based on region-of-interest , 2010, Electronic Imaging.

[2]  Ali Borji,et al.  Boosting bottom-up and top-down visual features for saliency estimation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Jian Sun,et al.  Geodesic Saliency Using Background Priors , 2012, ECCV.

[4]  Weisi Lin,et al.  Saliency detection for stereoscopic images , 2013, 2013 Visual Communications and Image Processing (VCIP).

[5]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[6]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[7]  Baihua Li,et al.  Quality assessment metric of stereo images considering cyclopean integration and visual saliency , 2016, Inf. Sci..

[8]  Wen Gao,et al.  Probabilistic Multi-Task Learning for Visual Saliency Estimation in Video , 2010, International Journal of Computer Vision.

[9]  Nanning Zheng,et al.  Salient Object Detection: A Discriminative Regional Feature Integration Approach , 2013, International Journal of Computer Vision.

[10]  Laurent Itti,et al.  Models of Bottom-up Attention and Saliency , 2005 .

[11]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Huchuan Lu,et al.  Inner and Inter Label Propagation: Salient Object Detection in the Wild , 2015, IEEE Transactions on Image Processing.

[13]  Li Xu,et al.  Hierarchical Saliency Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Ling-Yu Duan,et al.  Saliency detection based on 2D log-gabor wavelets and center bias , 2010, ACM Multimedia.

[15]  Lijuan Duan,et al.  A spatiotemporal weighted dissimilarity-based method for video saliency detection , 2015, Signal Process. Image Commun..

[16]  S. Yantis,et al.  Visual Attention: Bottom-Up Versus Top-Down , 2004, Current Biology.

[17]  Xuelong Li,et al.  Detection of Co-salient Objects by Looking Deep and Wide , 2016, International Journal of Computer Vision.

[18]  Hanling Zhang,et al.  Combining multi-layer integration algorithm with background prior and label propagation for saliency detection , 2017, J. Vis. Commun. Image Represent..

[19]  Tae-Sun Choi,et al.  Rician noise removal from MR images using novel adapted selective non-local means filter , 2012, Multimedia Tools and Applications.

[20]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Scenes and Its Applications , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Nianyi Li,et al.  A weighted sparse coding framework for saliency detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Xiaojun Chang,et al.  Revealing Event Saliency in Unconstrained Video Collection , 2017, IEEE Transactions on Image Processing.

[24]  Fatih Murat Porikli,et al.  Saliency-aware geodesic video object segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Peter König,et al.  Influence of disparity on fixation and saccades in free viewing of natural scenes. , 2009, Journal of vision.

[26]  Feng Wu,et al.  Background Prior-Based Salient Object Detection via Deep Reconstruction Residual , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[28]  J. Geng.  Three-dimensional display technologies. , 2013, Advances in optics and photonics.

[29]  Xueqing Li,et al.  Leveraging stereopsis for saliency analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Ling Shao,et al.  Consistent Video Saliency Using Local Gradient Flow Optimization and Global Refinement , 2015, IEEE Transactions on Image Processing.

[31]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[32]  Bruno Arnaldi,et al.  A new application for saliency maps: synthetic vision of autonomous actors , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[33]  Ling Shao,et al.  Cosaliency Detection Based on Intrasaliency Prior Transfer and Deep Intersaliency Mining , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Jianjun Lei,et al.  Stereoscopic Visual Attention Guided Disparity Control for Multiview Images , 2014, Journal of Display Technology.

[35]  Ling Shao,et al.  Correspondence Driven Saliency Transfer , 2016, IEEE Transactions on Image Processing.

[36]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Yong Man Ro,et al.  Attention model-based visual comfort assessment for stereoscopic depth perception , 2011, 2011 17th International Conference on Digital Signal Processing (DSP).

[38]  Marcus Barkowsky,et al.  The Importance of Visual Attention in Improving the 3D-TV Viewing Experience: Overview and New Perspectives , 2011, IEEE Transactions on Broadcasting.

[39]  Richard T Born,et al.  Feature attention for binocular disparity in primate area MT depends on tuning strength. , 2015, Journal of neurophysiology.

[40]  Feng Wu,et al.  Background-Driven Salient Object Detection , 2017, IEEE Transactions on Multimedia.

[41]  Lei Guo,et al.  Saliency detection by selective color features , 2016, Neurocomputing.

[42]  Jari Takatalo,et al.  What do people look at when they watch stereoscopic movies? , 2010, Electronic Imaging.

[43]  J. Theeuwes.  Top-down and bottom-up control of visual selection. , 2010, Acta psychologica.

[44]  Yang Liu,et al.  Dichotomy between luminance and disparity features at binocular fixations. , 2010, Journal of vision.

[45]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Feng Qi,et al.  3D visual saliency detection model with generated disparity map , 2016, Multimedia Tools and Applications.

[47]  Weisi Lin,et al.  Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics , 2013, IEEE Transactions on Image Processing.

[48]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[50]  P. König,et al.  Does luminance‐contrast contribute to a saliency map for overt visual attention? , 2003, The European journal of neuroscience.

[51]  Yu Fu,et al.  Visual saliency detection by spatially weighted dissimilarity , 2011, CVPR 2011.

[52]  Kwan-Liu Ma,et al.  Stereoscopic Thumbnail Creation via Efficient Stereo Saliency Detection , 2017, IEEE Transactions on Visualization and Computer Graphics.

[53]  King Ngi Ngan,et al.  A Co-Saliency Model of Image Pairs , 2011, IEEE Transactions on Image Processing.

[54]  Junle Wang,et al.  Study of center-bias in the viewing of stereoscopic image and a framework for extending 2D visual attention models to 3D , 2013, Electronic Imaging.

[55]  Atsuto Maki,et al.  Attentional Scene Segmentation: Integrating Depth and Motion , 2000, Comput. Vis. Image Underst..

[56]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Sehoon Yea,et al.  Content-Driven Retargeting of Stereoscopic Images , 2013, IEEE Signal Processing Letters.

[58]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[59]  Xuelong Li,et al.  Lazy Random Walks for Superpixel Segmentation , 2014, IEEE Transactions on Image Processing.

[60]  Deyu Meng,et al.  Co-Saliency Detection via a Self-Paced Multiple-Instance Learning Framework , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Xiaochun Cao,et al.  Cluster-Based Co-Saliency Detection , 2013, IEEE Transactions on Image Processing.

[62]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[63]  Lin Li,et al.  Saliency Detection With Spaces of Background-Based Distribution , 2016, IEEE Signal Processing Letters.

[64]  N. Qian.  Binocular Disparity and the Perception of Depth , 1997, Neuron.