Automatic Foreground Seeds Discovery for Robust Video Saliency Detection

In this paper, we propose a novel algorithm for saliency object detection in unconstrained videos. Even though various methods have been proposed to solve this task, video saliency detection is still challenging due to the complication in object discovery as well as the utilization of motion cues. Most of existing methods adopt background prior to detect salient objects. However, they are prone to fail in the case that foreground objects are similar with the background. In this work, we aim to discover robust foreground priors as a complement to background priors so that we can improve the performance. Given an input video, we consider motion and appearance cues separately to generate initial foreground/background seeds. Then, we learn a global object appearance model using the initial seeds and remove unreliable seeds according to foreground likelihood. Finally, the seeds work as queries to rank all the superpixels in images to generate saliency maps. Experimental results on challenging public dataset demonstrate the advantage of our algorithm over state-of-the-art algorithms.

[1]  Svetlana Lazebnik,et al.  Iterative quantization: A procrustean approach to learning binary codes , 2011, CVPR 2011.

[2]  Byoung Chul Ko,et al.  Object-of-interest image segmentation based on human attention and semantic region clustering. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.

[3]  Weisi Lin,et al.  Saliency Detection in the Compressed Domain for Adaptive Image Retargeting , 2012, IEEE Transactions on Image Processing.

[4]  Fatih Murat Porikli,et al.  Saliency-aware geodesic video object segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Xiaochun Cao,et al.  Motion saliency detection using low-rank and sparse decomposition , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Tiejun Huang,et al.  Automatic interesting object extraction from images using complementary saliency maps , 2010, ACM Multimedia.

[7]  Zhuwen Li,et al.  Video Co-segmentation for Meaningful Action Extraction , 2013, 2013 IEEE International Conference on Computer Vision.

[8]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Xiaochun Cao,et al.  Cluster-Based Co-Saliency Detection , 2013, IEEE Transactions on Image Processing.

[10]  Yizhou Yu,et al.  Deep Contrast Learning for Salient Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Luc Van Gool,et al.  A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Ling Shao,et al.  Consistent Video Saliency Using Local Gradient Flow Optimization and Global Refinement , 2015, IEEE Transactions on Image Processing.

[13]  Jian Zhang,et al.  Video object segmentation aggregation , 2016, 2016 IEEE International Conference on Multimedia and Expo (ICME).

[14]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[15]  Zhou Wang,et al.  Video saliency incorporating spatiotemporal cues and uncertainty weighting , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[16]  Kurt Keutzer,et al.  Dense Point Trajectories by GPU-Accelerated Large Displacement Optical Flow , 2010, ECCV.

[17]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Bernhard Schölkopf,et al.  Ranking on Data Manifolds , 2003, NIPS.

[19]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[20]  Han Wang,et al.  Salient Object Detection With Spatiotemporal Background Priors for Video , 2017, IEEE Transactions on Image Processing.

[21]  R. Born,et al.  Segregation of Object and Background Motion in Visual Area MT Effects of Microstimulation on Eye Movements , 2000, Neuron.

[22]  Jonathan T. Barron,et al.  Multiscale Combinatorial Grouping , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.