Detection of Co-salient Objects by Looking Deep and Wide

In this paper, we propose a unified co-salient object detection framework by introducing two novel insights: (1) looking deep to transfer higher-level representations by using the convolutional neural network with additional adaptive layers could better reflect the sematic properties of the co-salient objects; (2) looking wide to take advantage of the visually similar neighbors from other image groups could effectively suppress the influence of the common background regions. The wide and deep information are explored for the object proposal windows extracted in each image. The window-level co-saliency scores are calculated by integrating the intra-image contrast, the intra-group consistency, and the inter-group separability via a principled Bayesian formulation and are then converted to the superpixel-level co-saliency maps through a foreground region agreement strategy. Comprehensive experiments on two existing and one newly established datasets have demonstrated the consistent performance gain of the proposed approach.

[1]  King Ngi Ngan,et al.  Co-Salient Object Detection From Multiple Images , 2013, IEEE Transactions on Multimedia.

[2]  Akisato Kimura,et al.  Fully Automatic Extraction of Salient Objects from Videos in Near Real Time , 2010, Comput. J..

[3]  Xiaochun Cao,et al.  Cluster-Based Co-Saliency Detection , 2013, IEEE Transactions on Image Processing.

[4]  Vittorio Ferrari,et al.  Human Pose Co-Estimation and Applications , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Yu-Chiang Frank Wang,et al.  Exploring Visual and Motion Saliency for Automatic Video Object Extraction , 2013, IEEE Transactions on Image Processing.

[6]  Wenbin Zou,et al.  Co-Saliency Detection Based on Hierarchical Segmentation , 2014, IEEE Signal Processing Letters.

[7]  Zhuowen Tu,et al.  Unsupervised object class discovery via saliency-guided multiple class learning , 2012, CVPR.

[8]  Ying Wu,et al.  A unified approach to salient object detection via low rank matrix recovery , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[10]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Nanning Zheng,et al.  Automatic salient object segmentation based on context and shape prior , 2011, BMVC.

[12]  Homer H. Chen,et al.  Learning-Based Prediction of Visual Attention for Video Signals , 2011, IEEE Transactions on Image Processing.

[13]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Svetlana Lazebnik,et al.  Finding Things: Image Parsing with Regions and Per-Exemplar Detectors , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Hwann-Tzong Chen,et al.  Preattentive co-saliency detection , 2010, 2010 IEEE International Conference on Image Processing.

[16]  Cordelia Schmid,et al.  Learning object class detectors from weakly annotated video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Zhuwen Li,et al.  Video Co-segmentation for Meaningful Action Extraction , 2013, 2013 IEEE International Conference on Computer Vision.

[18]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[19]  Nanning Zheng,et al.  Video Object Discovery and Co-Segmentation with Extremely Weak Supervision , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Nathalie Guyader,et al.  Modelling Spatio-Temporal Saliency to Predict Gaze Direction for Short Videos , 2009, International Journal of Computer Vision.

[21]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Vladimir Kolmogorov,et al.  Object cosegmentation , 2011, CVPR 2011.

[23]  Tao Xiang,et al.  In Defence of Negative Mining for Annotating Weakly Labelled Data , 2012, ECCV.

[24]  Song-Chun Zhu,et al.  Cosegmentation and Cosketch by Unsupervised Learning , 2013, 2013 IEEE International Conference on Computer Vision.

[25]  Huchuan Lu,et al.  Bayesian Saliency via Low and mid Level Cues , 2022 .

[26]  Lizhuang Ma,et al.  Temporally Coherent Video Saliency Using Regional Dynamic Contrast , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Vittorio Ferrari,et al.  Figure-ground segmentation by transferring window masks , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Wen Gao,et al.  Probabilistic Multi-Task Learning for Visual Saliency Estimation in Video , 2010, International Journal of Computer Vision.

[29]  Bernhard Schölkopf,et al.  Ranking on Data Manifolds , 2003, NIPS.

[30]  Jean Ponce,et al.  Discriminative clustering for image co-segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Rujie Liu,et al.  Semi-supervised Learning for Large Scale Image Cosegmentation , 2013, 2013 IEEE International Conference on Computer Vision.

[32]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[33]  Jiebo Luo,et al.  iCoseg: Interactive co-segmentation with intelligent scribble guidance , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Zhi Liu,et al.  Efficient Saliency-Model-Guided Visual Co-Saliency Detection , 2015, IEEE Signal Processing Letters.

[35]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Antonio Criminisi,et al.  Object categorization by learned universal visual dictionary , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[38]  Eli Shechtman,et al.  Cosaliency: where people look when comparing images , 2010, UIST.

[39]  Jean Ponce,et al.  Multi-class cosegmentation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Tiejun Huang,et al.  Visual Saliency with Statistical Priors , 2013, International Journal of Computer Vision.

[41]  Philip H. S. Torr,et al.  BING: Binarized normed gradients for objectness estimation at 300fps , 2019, Computational Visual Media.

[42]  Tao Xiang,et al.  Looking Beyond the Image: Unsupervised Learning for Object Saliency and Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Chi-Man Pun,et al.  Image co-saliency detection by propagating superpixel affinities , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[44]  Jianbo Shi,et al.  Image Matching via Saliency Region Correspondences , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[46]  Xinlei Chen,et al.  Enriching Visual Knowledge Bases via Object Discovery and Segmentation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Xiaochun Cao,et al.  Co-Saliency Detection via Base Reconstruction , 2014, ACM Multimedia.

[48]  Takeo Kanade,et al.  Distributed cosegmentation via submodular optimization on anisotropic diffusion , 2011, 2011 International Conference on Computer Vision.

[49]  King Ngi Ngan,et al.  Object Co-Segmentation Based on Shortest Path Algorithm and Saliency Model , 2012, IEEE Transactions on Multimedia.

[50]  Jingdong Wang,et al.  Salient Object Detection: A Discriminative Regional Feature Integration Approach , 2013, International Journal of Computer Vision.

[51]  Mei Han,et al.  Category-Independent Object-Level Saliency Detection , 2013, 2013 IEEE International Conference on Computer Vision.

[52]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[53]  Xiaochun Cao,et al.  Self-Adaptively Weighted Co-Saliency Detection via Rank Constraint , 2014, IEEE Transactions on Image Processing.

[54]  Eli Shechtman,et al.  In defense of Nearest-Neighbor based image classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Gang Hua,et al.  Automatic salient object extraction with contextual cue and its applications to recognition and alpha matting , 2013, Pattern Recognit..

[56]  Chao Li,et al.  Co-saliency detection via looking deep and wide , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  King Ngi Ngan,et al.  A Co-Saliency Model of Image Pairs , 2011, IEEE Transactions on Image Processing.

[58]  Junle Wang,et al.  Computational Model of Stereoscopic 3D Visual Saliency , 2013, IEEE Transactions on Image Processing.

[59]  Ce Liu,et al.  Unsupervised Joint Object Discovery and Segmentation in Internet Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[60]  Fei-Fei Li,et al.  Co-localization in Real-World Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Shui Yu,et al.  Learning Complementary Saliency Priors for Foreground Object Segmentation in Complex Scenes , 2014, International Journal of Computer Vision.