Spatiotemporal saliency detection using border connectivity

This paper proposes a border connectivity-based spatiotemporal saliency model for videos with complicated motion and complex scenes. Based on the superpixel segmentation results of video frames, feature extraction is performed to obtain the three features, including motion orientation histogram, motion amplitude histogram and color histogram. Then the border connectivity is exploited to evaluate the importance of three features for distance fusion. Finally the background weighted contrast and saliency optimization are utilized to generate superpixel-level spatiotemporal saliency maps. Experimental results on a public benchmark dataset demonstrate that the proposed model outperforms the state-of-the- art saliency models on saliency detection performance.

[1]  James M. Rehg,et al.  Video Segmentation by Tracking Many Figure-Ground Segments , 2013, 2013 IEEE International Conference on Computer Vision.

[2]  Liming Zhang,et al.  A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression , 2010, IEEE Transactions on Image Processing.

[3]  Liquan Shen,et al.  Spatiotemporal saliency detection based on superpixel-level trajectory , 2015, Signal Process. Image Commun..

[4]  Wenbin Zou,et al.  Co-saliency detection based on region-level fusion and pixel-level refinement , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[5]  Peyman Milanfar,et al.  Static and space-time visual saliency detection by self-resemblance. , 2009, Journal of vision.

[6]  Ingrid Heynderickx,et al.  Visual Attention in Objective Image Quality Assessment: Based on Eye-Tracking Data , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Zhi Liu,et al.  Stretchability-aware block scaling for image retargeting , 2013, J. Vis. Commun. Image Represent..

[8]  Laurent Itti,et al.  Visual attention guided bit allocation in video compression , 2011, Image Vis. Comput..

[9]  Linwei Ye,et al.  Co-Saliency Detection via Co-Salient Object Discovery and Recovery , 2015, IEEE Signal Processing Letters.

[10]  Xiang Zhang,et al.  Superpixel-Based Spatiotemporal Saliency Detection , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Wenbin Zou,et al.  Co-Saliency Detection Based on Hierarchical Segmentation , 2014, IEEE Signal Processing Letters.

[12]  Chun-Rong Huang,et al.  Video Saliency Map Detection by Dominant Camera Motion Removal , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Xiang Zhang,et al.  Region Diversity Maximization for Salient Object Detection , 2012, IEEE Signal Processing Letters.

[14]  Vladimir Zlokolica,et al.  Salient Motion Features for Video Quality Assessment , 2011, IEEE Transactions on Image Processing.

[15]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Zhi Liu,et al.  A novel H.264 rate control algorithm with consideration of visual attention , 2011, Multimedia Tools and Applications.

[17]  Yu Huang,et al.  Addressing Visual Consistency in Video Retargeting: A Refined Homogeneous Approach , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Yin Li,et al.  Visual Saliency Based on Conditional Entropy , 2009, ACCV.

[19]  King Ngi Ngan,et al.  Unsupervised Salient Object Segmentation Based on Kernel Density Estimation and Two-Phase Graph Cut , 2012, IEEE Transactions on Multimedia.

[20]  Zhi Liu,et al.  Efficient Saliency-Model-Guided Visual Co-Saliency Detection , 2015, IEEE Signal Processing Letters.

[21]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Zhi Liu,et al.  Saccadic model of eye movements for free-viewing condition , 2015, Vision Research.

[23]  Wenbin Zou,et al.  Saliency Tree: A Novel Saliency Detection Framework , 2014, IEEE Transactions on Image Processing.

[24]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.