Context Proposals for Saliency Detection

Abstract One of the fundamental properties of a salient object region is its contrast with the immediate context. The problem is that numerous object regions exist which potentially can all be salient. One way to prevent an exhaustive search over all object regions is by using object proposal algorithms. These return a limited set of regions which are most likely to contain an object. Several saliency estimation methods have used object proposals. However, they focus on the saliency of the proposal only, and the importance of its immediate context has not been evaluated. In this paper, we aim to improve salient object detection. Therefore, we extend object proposal methods with context proposals, which allow to incorporate the immediate context in the saliency computation. We propose several saliency features which are computed from the context proposals. In the experiments, we evaluate five object proposal methods for the task of saliency segmentation, and find that Multiscale Combinatorial Grouping outperforms the others. Furthermore, experiments show that the proposed context features improve performance, and that our method matches results on the FT datasets and obtains competitive results on three other datasets (PASCAL-S, MSRA-B and ECSSD).

[1]  David A. Clausi,et al.  Statistical Textural Distinctiveness for Salient Region Detection in Natural Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Laurent Itti,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Rapid Biologically-inspired Scene Classification Using Features Shared with Visual Attention , 2022 .

[3]  Thomas Deselaers,et al.  Measuring the Objectness of Image Windows , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[5]  Yizhou Yu,et al.  Deep Contrast Learning for Salient Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Zhuowen Tu,et al.  Deeply Supervised Salient Object Detection with Short Connections , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[8]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[9]  Vladlen Koltun,et al.  Geodesic Object Proposals , 2014, ECCV.

[10]  Noel E. O'Connor,et al.  Shallow and Deep Convolutional Networks for Saliency Prediction , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Ronan Collobert,et al.  Learning to Refine Object Segments , 2016, ECCV.

[12]  Yuning Jiang,et al.  FastMask: Segment Multi-scale Object Candidates in One Shot , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Meng Wang,et al.  Image saliency: From intrinsic to extrinsic context , 2011, CVPR 2011.

[16]  Pingmei Xu,et al.  TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking , 2015, ArXiv.

[17]  Lihi Zelnik-Manor,et al.  Learning Video Saliency from Human Gaze Using Candidate Selection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Derek Hoiem,et al.  Category-Independent Object Proposals with Diverse Ranking , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Xiaogang Wang,et al.  Saliency detection by multi-context deep learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  James M. Rehg,et al.  The Secrets of Salient Object Segmentation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Feng Wu,et al.  A Review of Co-saliency Detection Technique: Fundamentals, Applications, and Challenges , 2016, ArXiv.

[23]  Philip H. S. Torr,et al.  BING: Binarized normed gradients for objectness estimation at 300fps , 2019, Computational Visual Media.

[24]  Lihi Zelnik-Manor,et al.  What Makes a Patch Distinct? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Yizhou Yu,et al.  Visual saliency based on multiscale deep features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Xuelong Li,et al.  A biological inspired features based saliency map , 2012, 2012 International Conference on Computing, Networking and Communications (ICNC).

[27]  Huchuan Lu,et al.  Saliency Region Detection Based on Markov Absorption Probabilities , 2015, IEEE Transactions on Image Processing.

[28]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[29]  Huchuan Lu,et al.  Adaptive Metric Learning for Saliency Detection , 2015, IEEE Transactions on Image Processing.

[30]  Huan Wang,et al.  Regional Principal Color Based Saliency Detection , 2014, PloS one.

[31]  Yueting Zhuang,et al.  DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection , 2015, IEEE Transactions on Image Processing.

[32]  Naila Murray,et al.  Low-Level Spatiochromatic Grouping for Saliency Estimation , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Ohad Ben-Shahar,et al.  A Closer Look at Context: From Coxels to the Contextual Emergence of Object Saliency , 2014, ECCV.

[34]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Cristian Sminchisescu,et al.  Constrained parametric min-cuts for automatic object segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Huchuan Lu,et al.  Salient object detection via global and local cues , 2015, Pattern Recognit..

[39]  Stella X. Yu,et al.  Image Compression Based on Visual Saliency at Individual Scales , 2009, ISVC.

[40]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Huchuan Lu,et al.  Saliency Detection with Recurrent Fully Convolutional Networks , 2016, ECCV.

[43]  Gabriela Csurka,et al.  A framework for visual saliency detection with applications to image thumbnailing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[44]  Baoxin Li,et al.  A two-stage approach to saliency detection in images , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[45]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Daniel P. Kennedy,et al.  Atypical Visual Saliency in Autism Spectrum Disorder Quantified through Model-Based Eye Tracking , 2015, Neuron.

[47]  Peiquan Jin,et al.  An approach for image retrieval based on visual saliency , 2009, 2009 International Conference on Image Analysis and Signal Processing.

[48]  Peyman Milanfar,et al.  Static and space-time visual saliency detection by self-resemblance. , 2009, Journal of vision.

[49]  Li Xu,et al.  Hierarchical Saliency Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Jenny Benois-Pineau,et al.  Saliency Driven Object recognition in egocentric videos with deep CNN: toward application in assistance to Neuroprostheses , 2016, Comput. Vis. Image Underst..

[51]  Shuyuan Yang,et al.  Object-level saliency detection with color attributes , 2016, Pattern Recognit..

[52]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[53]  Antón García-Díaz,et al.  Decorrelation and Distinctiveness Provide with Human-Like Saliency , 2009, ACIVS.

[54]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[55]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[56]  Jingdong Wang,et al.  Salient Object Detection: A Discriminative Regional Feature Integration Approach , 2013, International Journal of Computer Vision.

[57]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[58]  Bernt Schiele,et al.  What Makes for Effective Detection Proposals? , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Qi Zhao,et al.  Learning saliency-based visual attention: A review , 2013, Signal Process..

[60]  Simone Frintrop,et al.  Traditional saliency reloaded: A good old model in new shape , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[61]  Junwei Han,et al.  DHSNet: Deep Hierarchical Saliency Network for Salient Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  J. Duncan,et al.  Visual search and stimulus similarity. , 1989, Psychological review.

[63]  Vishal S. Vaingankar,et al.  BASED ON COLOR CUES : CO-OPERATIVE EFFECTS OF TOP-DOWN & BOTTOM-UP VISUAL ATTENTION , 2003 .

[64]  Jonathan T. Barron,et al.  Multiscale Combinatorial Grouping , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  David A. Clausi,et al.  Structure-Guided Statistical Textural Distinctiveness for Salient Region Detection in Natural Images , 2015, IEEE Trans. Image Process..

[66]  Huchuan Lu,et al.  Deep networks for saliency detection via local estimation and global search , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[67]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.