Spatiotemporal salient object detection by integrating with objectness

This paper proposes a novel spatiotemporal salient object detection method by integrating saliency and objectness, for videos with complicated motion and complex scenes. The initial salient object detection result is first built upon both saliency map and objectness map. Afterwards, the region size of salient object is adjusted to obtain the frame-wise salient object detection result by iteratively updating the object probability map, which is the combination of saliency map and objectness map. Finally, in order to enhance the temporal coherence, the sequence-level refinement is performed to generate the final salient object detection result. Experimental results on public benchmark datasets demonstrate that the proposed method consistently outperforms the state-of-the-art salient object detection methods.

[1]  Yu Huang,et al.  Addressing Visual Consistency in Video Retargeting: A Refined Homogeneous Approach , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Linwei Ye,et al.  Saliency Detection for Unconstrained Videos Using Superpixel-Level Graph and Spatiotemporal Propagation , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Deepu Rajan,et al.  Random Walks on Graphs for Salient Object Detection in Images , 2010, IEEE Transactions on Image Processing.

[4]  Michael Gleicher,et al.  Automatic image retargeting with fisheye-view warping , 2005, UIST.

[5]  HongJiang Zhang,et al.  Contrast-based image attention analysis by using fuzzy growing , 2003, MULTIMEDIA '03.

[6]  Zhi Liu,et al.  Spatiotemporal saliency detection using border connectivity , 2016, International Conference on Digital Image Processing.

[7]  Weisi Lin,et al.  A Video Saliency Detection Model in Compressed Domain , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Junsong Yuan,et al.  Finding spatio-temporal salient paths for video objects discovery , 2016, J. Vis. Commun. Image Represent..

[9]  Yang Li,et al.  Video saliency detection using 3D shearlet transform , 2015, Multimedia Tools and Applications.

[10]  Christoph H. Lampert,et al.  Efficient Subwindow Search: A Branch and Bound Framework for Object Localization , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Zhi Liu,et al.  Stretchability-aware block scaling for image retargeting , 2013, J. Vis. Commun. Image Represent..

[12]  Yongdong Zhang,et al.  A Highly Parallel Framework for HEVC Coding Unit Partitioning Tree Decision on Many-core Processors , 2014, IEEE Signal Processing Letters.

[13]  Qiang Liu,et al.  A two step salient objects extraction framework based on image segmentation and saliency detection , 2012, Multimedia Tools and Applications.

[14]  Xiang Zhang,et al.  Superpixel-Based Spatiotemporal Saliency Detection , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[16]  Laurent Itti,et al.  Visual attention guided bit allocation in video compression , 2011, Image Vis. Comput..

[17]  Ze-Nian Li,et al.  Image trimming via saliency region detection and iterative feature matching , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[18]  Xinge You,et al.  Kernel Learning for Dynamic Texture Synthesis , 2016, IEEE Transactions on Image Processing.

[19]  Qi Tian,et al.  Saliency Density Maximization for Object Detection and Localization , 2010, ACCV.

[20]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[21]  Ce Liu,et al.  Exploring new representations and applications for motion analysis , 2009 .

[22]  Zhi Liu,et al.  A novel H.264 rate control algorithm with consideration of visual attention , 2011, Multimedia Tools and Applications.

[23]  Linwei Ye,et al.  Evaluation on fusion of saliency and objectness for salient object segmentation , 2015, ICIMCS '15.

[24]  Ali Borji,et al.  Salient object detection: A survey , 2014, Computational Visual Media.

[25]  Nuno Vasconcelos,et al.  Classifying Video with Kernel Dynamic Textures , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Shui Yu,et al.  Learning Complementary Saliency Priors for Foreground Object Segmentation in Complex Scenes , 2014, International Journal of Computer Vision.

[28]  Yang Li,et al.  A saliency detection model using shearlet transform , 2014, Multimedia Tools and Applications.

[29]  Fatih Murat Porikli,et al.  Saliency-aware geodesic video object segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  C. Lawrence Zitnick,et al.  Edge Boxes: Locating Object Proposals from Edges , 2014, ECCV.

[31]  Luc Van Gool,et al.  A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Liming Zhang,et al.  A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression , 2010, IEEE Transactions on Image Processing.

[33]  Liquan Shen,et al.  Spatiotemporal saliency detection based on superpixel-level trajectory , 2015, Signal Process. Image Commun..

[34]  Xu Dan Object detection based on saliency map , 2010 .

[35]  Byoung Chul Ko,et al.  Object-of-interest image segmentation based on human attention and semantic region clustering. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.

[36]  Yongdong Zhang,et al.  Efficient Parallel Framework for HEVC Motion Estimation on Many-Core Processors , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  James M. Rehg,et al.  Video Segmentation by Tracking Many Figure-Ground Segments , 2013, 2013 IEEE International Conference on Computer Vision.

[38]  Ariel Shamir,et al.  Seam carving for media retargeting , 2009, CACM.

[39]  Ling Shao,et al.  Consistent Video Saliency Using Local Gradient Flow Optimization and Global Refinement , 2015, IEEE Transactions on Image Processing.

[40]  Bo Wu,et al.  Integrating bottom-up and top-down visual stimulus for saliency detection in news video , 2013, Multimedia Tools and Applications.

[41]  Bing Li,et al.  Salient Object Detection via Structured Matrix Decomposition. , 2017, IEEE transactions on pattern analysis and machine intelligence.

[42]  Nicu Sebe,et al.  Image saliency by isocentric curvedness and color , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[43]  Narendra Ahuja,et al.  Learning Nonlinear Manifolds from Time Series , 2006, ECCV.

[44]  J.-L. Wu,et al.  Video Adaptation for Small Display Based on Content Recomposition , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[45]  Xiang Zhang,et al.  Region Diversity Maximization for Salient Object Detection , 2012, IEEE Signal Processing Letters.

[46]  Yi Yang,et al.  Image Classification by Cross-Media Active Learning With Privileged Information , 2016, IEEE Transactions on Multimedia.

[47]  Wenbin Zou,et al.  Saliency Tree: A Novel Saliency Detection Framework , 2014, IEEE Transactions on Image Processing.

[48]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[49]  Hai Jin,et al.  MsLRR: A Unified Multiscale Low-Rank Representation for Image Segmentation , 2014, IEEE Transactions on Image Processing.