How important is location information in saliency detection of natural images

Location information, i.e., the position of content in image plane, is considered as an important supplement in saliency detection. The effect of location information is usually evaluated by integrating it with the selected saliency detection methods and measuring the improvement, which is highly influenced by the selection of saliency methods. In this paper, we provide direct and quantitative analysis of the importance of location information for saliency detection in natural images. We firstly analyze the relationship between content location and saliency distribution on four public image datasets, and validate the distribution by simply treating location based Gaussian distribution as saliency map. To further validate the effectiveness of location information, we propose a location based saliency detection approach, which completely initializes saliency maps with location information and propagate saliency among patches based on color similarity, and discuss the robustness of location information’s effect. The experimental results show that location information plays a positive role in saliency detection, and the proposed method can outperform most state-of-the-art saliency detection methods and handle natural images with different object positions and multiple salient objects.

[1]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[2]  K. Madhava Krishna,et al.  Depth really Matters: Improving Visual Salient Region Detection with Depth , 2013, BMVC.

[3]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  James H. Elder,et al.  Design and perceptual validation of performance measures for salient object segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[5]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[6]  Lihi Zelnik-Manor,et al.  What Makes a Patch Distinct? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Ran Ju,et al.  Depth saliency based on anisotropic center-surround difference , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[8]  Jing Wang,et al.  Robust object removal with an exemplar-based image inpainting approach , 2014, Neurocomputing.

[9]  Arjan Durresi,et al.  A survey: Control plane scalability issues and approaches in Software-Defined Networking (SDN) , 2017, Comput. Networks.

[10]  Alberto Del Bimbo,et al.  Visual Image Retrieval by Elastic Matching of User Sketches , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Ali Borji,et al.  Analysis of Scores, Datasets, and Models in Visual Saliency Prediction , 2013, 2013 IEEE International Conference on Computer Vision.

[12]  Jian Yu,et al.  Saliency Detection by Multitask Sparsity Pursuit , 2012, IEEE Transactions on Image Processing.

[13]  Mei Han,et al.  Category-Independent Object-Level Saliency Detection , 2013, 2013 IEEE International Conference on Computer Vision.

[14]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[15]  Wenbin Zou,et al.  Saliency Tree: A Novel Saliency Detection Framework , 2014, IEEE Transactions on Image Processing.

[16]  Neil A. Dodgson,et al.  Proceedings Ninth IEEE International Conference on Computer Vision , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[17]  Tao Mei,et al.  Correlative multi-label video annotation , 2007, ACM Multimedia.

[18]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[19]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[20]  Wonjun Kim,et al.  Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  S. Yantis,et al.  Cortical mechanisms of feature-based attentional control. , 2003, Cerebral cortex.

[22]  Yang Yang,et al.  Stereo GrabCut: Interactive and Consistent Object Extraction for Stereo Images , 2013, PCM.

[23]  HongJiang Zhang,et al.  Contrast-based image attention analysis by using fuzzy growing , 2003, MULTIMEDIA '03.

[24]  Deepu Rajan,et al.  Random walks on graphs to model saliency in images , 2009, CVPR.

[25]  Evon M. O. Abu-Taieh,et al.  Comparative study , 2003, BMJ : British Medical Journal.

[26]  Bruce Barnbaum The Art of Photography: An Approach to Personal Expression , 1994 .

[27]  Yang Yang,et al.  OBSIR: Object-based stereo image retrieval , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[28]  Shuicheng Yan,et al.  Hidden-Concept Driven Multilabel Image Annotation and Label Ranking , 2012, IEEE Transactions on Multimedia.

[29]  Rainer Stiefelhagen,et al.  How the distribution of salient objects in images influences salient object detection , 2013, 2013 IEEE International Conference on Image Processing.

[30]  Shiliang Zhang,et al.  Multi-order visual phrase for scalable image search , 2013, ICIMCS '13.

[31]  Changsheng Xu,et al.  Locality discriminative coding for image classification , 2013, ICIMCS '13.

[32]  Huchuan Lu,et al.  Robust object tracking via sparsity-based collaborative model , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Gabriela Csurka,et al.  A framework for visual saliency detection with applications to image thumbnailing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[34]  Yan Liu,et al.  Soft-assigned bag of features for object tracking , 2014, Multimedia Systems.

[35]  Ronen Basri,et al.  Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Yan Liu,et al.  Image Retargeting by Combining Region Warping and Occlusion , 2013, PCM.

[38]  Ulf Assarsson,et al.  A Benchmark for , 2001 .

[39]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Ali Borji,et al.  Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study , 2013, IEEE Transactions on Image Processing.

[41]  Shi-Min Hu,et al.  SalientShape: group saliency in image collections , 2013, The Visual Computer.

[42]  Gang Wang,et al.  On the sampling of web images for learning visual concept classifiers , 2010, CIVR '10.

[43]  A. F. Adams,et al.  The Survey , 2021, Dyslexia in Higher Education.

[44]  Peng Jiang,et al.  Salient Region Detection by UFO: Uniqueness, Focusness and Objectness , 2013, 2013 IEEE International Conference on Computer Vision.

[45]  Qiang Liu,et al.  A two step salient objects extraction framework based on image segmentation and saliency detection , 2012, Multimedia Tools and Applications.

[46]  Tao Mei,et al.  Joint multi-label multi-instance learning for image classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Yan Liu,et al.  Image retargeting based on global energy optimization , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[48]  Haibin Ling,et al.  Saliency Detection on Light Field , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Shumeet Baluja,et al.  VisualRank: Applying PageRank to Large-Scale Image Search , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Yang Li,et al.  A saliency detection model using shearlet transform , 2014, Multimedia Tools and Applications.

[51]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Meng Wang,et al.  Unified Video Annotation via Multigraph Learning , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[53]  Qi Tian,et al.  Latent visual context learning for web image applications , 2011, Pattern Recognit..

[54]  Shi-Min Hu,et al.  Global Contrast Based Salient Region Detection , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[56]  Nanning Zheng,et al.  Automatic salient object segmentation based on context and shape prior , 2011, BMVC.