Saliency Detection Inspired by Topological Perception Theory

The topological perception theory claims that visual perception of a scene begins from topological properties and then exploits local details. Inspired by this theory, we defined the topological descriptor and topological complexity, and we observed, based on statistics, that the saliencies of the regions with higher topological complexities are generally higher than those of regions with lower topological complexities. We then introduced the topological complexity as a saliency prior and proposed a novel unsupervised topo-prior-guided saliency detection system (TOPS). This system is framed as a topological saliency prior (topo-prior)-guided two-level local cue processing (i.e., pixel- and regional-level cues) with a multi-scale strategy, which includes three main modules: (1) a basic computational model of the topological perception theory for extracting topological features from images, (2) a topo-prior calculation method based on the topological features, and (3) a global–local saliency combination framework guided by the topo-prior. Extensive experiments on widely used salient object detection (SOD) datasets demonstrate that our system outperforms the unsupervised state-of-the-art algorithms. In addition, the topo-prior proposed in this work can be used to boost supervised methods including the deep-learning-based ones for fixation prediction and SOD tasks.

[1]  M. Srinivasan,et al.  Global perception in small brains: Topological pattern recognition in honey bees , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Zhenjiang Miao,et al.  A Saliency Prior Context Model for Real-Time Object Tracking , 2017, IEEE Transactions on Multimedia.

[3]  Tim Fingscheidt,et al.  Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance , 2020, ECCV.

[4]  Weidong Cai,et al.  Reversion Correction and Regularized Random Walk Ranking for Saliency Detection , 2018, IEEE Transactions on Image Processing.

[5]  Weisi Lin,et al.  Saliency detection for stereoscopic images , 2013, 2013 Visual Communications and Image Processing (VCIP).

[6]  Asha Iyer,et al.  Components of bottom-up gaze allocation in natural images , 2005, Vision Research.

[7]  Wenguan Wang,et al.  Deep Visual Attention Prediction , 2017, IEEE Transactions on Image Processing.

[8]  Zhe-Ming Lu,et al.  Video abstraction based on the visual attention model and online clustering , 2013, Signal Process. Image Commun..

[9]  Huan Du,et al.  Depth-Aware Salient Object Detection and Segmentation via Multiscale Discriminative Saliency Fusion and Bootstrap Learning , 2017, IEEE Transactions on Image Processing.

[10]  Haibin Ling,et al.  Revisiting Video Saliency Prediction in the Deep Learning Era , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Huchuan Lu,et al.  Detect Globally, Refine Locally: A Novel Approach to Saliency Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[12]  Jingdong Wang,et al.  Salient Object Detection: A Discriminative Regional Feature Integration Approach , 2013, International Journal of Computer Vision.

[13]  Huchuan Lu,et al.  Multi-Source Weak Supervision for Saliency Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  David Dagan Feng,et al.  Robust saliency detection via regularized random walks ranking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Tao Xiang,et al.  Looking Beyond the Image: Unsupervised Learning for Object Saliency and Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  James M. Rehg,et al.  The Secrets of Salient Object Segmentation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Michelle R. Greene,et al.  Visual search in scenes involves selective and nonselective pathways , 2011, Trends in Cognitive Sciences.

[18]  D. C. Van Essen,et al.  Concurrent processing streams in monkey visual cortex , 1988, Trends in Neurosciences.

[19]  Ruigang Yang,et al.  Inferring Salient Objects from Human Fixations , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  R. Venkatesh Babu,et al.  DeepFix: A Fully Convolutional Neural Network for Predicting Human Eye Fixations , 2015, IEEE Transactions on Image Processing.

[22]  Ali Borji,et al.  Scene classification with a sparse set of salient regions , 2011, 2011 IEEE International Conference on Robotics and Automation.

[23]  Yuan Xie,et al.  Flow Guided Recurrent Neural Encoder for Video Salient Object Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Ming-Hsuan Yang,et al.  PiCANet: Learning Pixel-Wise Contextual Attention for Saliency Detection , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Tao Li,et al.  Structure-Measure: A New Way to Evaluate Foreground Maps , 2017, International Journal of Computer Vision.

[26]  L Chen,et al.  Topological structure in visual perception. , 1982, Science.

[27]  Li Xu,et al.  Hierarchical Saliency Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Jie Chen,et al.  Hierarchical Contour Closure-Based Holistic Salient Object Detection , 2017, IEEE Transactions on Image Processing.

[29]  Lei Wu,et al.  Saliency Detection via Topological Feature Modulated Deep Learning , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[30]  Stan Sclaroff,et al.  Saliency Detection: A Boolean Map Approach , 2013, 2013 IEEE International Conference on Computer Vision.

[31]  Philip H. S. Torr,et al.  BING: Binarized normed gradients for objectness estimation at 300fps , 2014, Computational Visual Media.

[32]  DH Hubel,et al.  Psychophysical evidence for separate channels for the perception of form, color, movement, and depth , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[33]  Yu-Jin Zhang,et al.  Water flow driven salient object detection at 180 fps , 2018, Pattern Recognit..

[34]  Nianyi Li,et al.  A weighted sparse coding framework for saliency detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Steven C. H. Hoi,et al.  Salient Object Detection With Pyramid Attention and Salient Edges , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Yan Zhuo,et al.  Contributions of the Visual Ventral Pathway to Long-Range Apparent Motion , 2003, Science.

[37]  Huchuan Lu,et al.  Saliency Detection with Recurrent Fully Convolutional Networks , 2016, ECCV.

[38]  Shao-Yi Chien,et al.  Real-Time Salient Object Detection with a Minimum Spanning Tree , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Wei Liu,et al.  Saliency propagation from simple to difficult , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Huchuan Lu,et al.  Saliency detection via Cellular Automata , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Xin Gao,et al.  Segmentation-Based Salient Object Detection , 2015, CCCV.

[42]  Jian Sun,et al.  Geodesic Saliency Using Background Priors , 2012, ECCV.

[43]  Huchuan Lu,et al.  Learning to Detect Salient Objects with Image-Level Supervision , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[45]  Bing Li,et al.  Salient Object Detection via Structured Matrix Decomposition. , 2016, IEEE transactions on pattern analysis and machine intelligence.

[46]  Lizhuang Ma,et al.  Saliency Detection via Multi-Scale Global Cues , 2019, IEEE Transactions on Multimedia.

[47]  Ben Wang,et al.  Reverse Attention for Salient Object Detection , 2018, ECCV.

[48]  Rita Cucchiara,et al.  A deep multi-level network for saliency prediction , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[49]  Yan Zhuo,et al.  Global topological dominance in the left hemisphere , 2007, Proceedings of the National Academy of Sciences.

[50]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[51]  Xuelong Hu,et al.  Discriminative saliency propagation with sink points , 2016, Pattern Recognit..

[52]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  J. Wolfe,et al.  Guided Search 2.0 A revised model of visual search , 1994, Psychonomic bulletin & review.

[54]  Rongrong Ji,et al.  RGBD Salient Object Detection: A Benchmark and Algorithms , 2014, ECCV.

[55]  Tianming Liu,et al.  Predicting eye fixations using convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[57]  Martin D. Levine,et al.  Visual Saliency Based on Scale-Space Analysis in the Frequency Domain , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  John K. Tsotsos,et al.  Saliency, attention, and visual search: an information theoretic approach. , 2009, Journal of vision.

[59]  Huchuan Lu,et al.  Bayesian Saliency via Low and mid Level Cues , 2022 .

[60]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[61]  Pablo Andrés Arbeláez,et al.  Boundary Extraction in Natural Images Using Ultrametric Contour Maps , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[62]  Jiandong Tian,et al.  RGBD Salient Object Detection via Deep Fusion , 2016, IEEE Transactions on Image Processing.

[63]  Edward H. Adelson,et al.  PYRAMID METHODS IN IMAGE PROCESSING. , 1984 .

[64]  Hui Li,et al.  A Unified Framework for Salient Structure Detection by Contour-Guided Visual Search , 2015, IEEE Transactions on Image Processing.

[65]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[66]  Ali Borji,et al.  Understanding and Visualizing Deep Visual Saliency Models , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[67]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[68]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  Huchuan Lu,et al.  Hierarchical Cellular Automata for Visual Saliency , 2017, International Journal of Computer Vision.

[70]  J. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[71]  Qingming Huang,et al.  Image Saliency Detection Video Saliency Detection Co-saliency Detection Temporal RGBD Saliency Detection Motion , 2018 .

[72]  Xiaodong Gu,et al.  Attention selection using global topological properties based on pulse coupled neural network , 2013, Comput. Vis. Image Underst..

[73]  Nanning Zheng,et al.  Salient Object Detection: A Discriminative Regional Feature Integration Approach , 2013, International Journal of Computer Vision.

[74]  Christof Koch,et al.  Image Signature: Highlighting Sparse Salient Regions , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[75]  Ali Borji,et al.  Salient object detection: A survey , 2014, Computational Visual Media.

[76]  Yizhou Yu,et al.  Visual saliency based on multiscale deep features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[77]  Stan Sclaroff,et al.  Exploiting Surroundedness for Saliency Detection: A Boolean Map Approach , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[78]  Huchuan Lu,et al.  CapSal: Leveraging Captioning to Boost Semantics for Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[79]  Yu-Jin Zhang,et al.  300-FPS Salient Object Detection via Minimum Directional Contrast , 2017, IEEE Transactions on Image Processing.

[80]  Gang Wang,et al.  Progressive Attention Guided Recurrent Network for Salient Object Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[81]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[82]  Ling Shao,et al.  An Iterative and Cooperative Top-Down and Bottom-Up Inference Network for Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[83]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[84]  Xiaowu Chen,et al.  Look, Perceive and Segment: Finding the Salient Objects in Images via Two-stream Fixation-Semantic CNNs , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[85]  Marc Pollefeys,et al.  SGM-Nets: Semi-Global Matching with Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[86]  Leon A. Gatys,et al.  Understanding Low- and High-Level Contributions to Fixation Prediction , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[87]  C. Lawrence Zitnick,et al.  Edge Boxes: Locating Object Proposals from Edges , 2014, ECCV.

[88]  Ali Borji,et al.  Saliency Prediction in the Deep Learning Era: Successes and Limitations , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[89]  Xiaodong Gu,et al.  Embedding topological features into convolutional neural network salient object detection , 2020, Neural Networks.

[90]  Ali Borji,et al.  Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study , 2013, IEEE Transactions on Image Processing.

[91]  Ting Zhao,et al.  Pyramid Feature Attention Network for Saliency Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[92]  Li Xu,et al.  Hierarchical Image Saliency Detection on Extended CSSD , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[93]  Shuhan Chen,et al.  Reverse Attention-Based Residual Network for Salient Object Detection , 2020, IEEE Transactions on Image Processing.

[94]  Esa Rahtu,et al.  Segmenting Salient Objects from Images and Videos , 2010, ECCV.

[95]  Antón García-Díaz,et al.  Saliency from hierarchical adaptation through decorrelation and variance normalization , 2012, Image Vis. Comput..

[96]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[97]  Yicong Zhou,et al.  RGB-‘D’ Saliency Detection With Pseudo Depth , 2019, IEEE Transactions on Image Processing.

[98]  Lin Chen The topological approach to perceptual organization , 2005 .

[99]  Qi Zhao,et al.  Learning saliency-based visual attention: A review , 2013, Signal Process..

[100]  Yu-Wing Tai,et al.  Salient Region Detection via High-Dimensional Color Transform and Local Spatial Support , 2014, IEEE Transactions on Image Processing.

[101]  David A. Clausi,et al.  Statistical Textural Distinctiveness for Salient Region Detection in Natural Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[102]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[103]  Zhe Wu,et al.  Cascaded Partial Decoder for Fast and Accurate Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[104]  Xiaogang Wang,et al.  Saliency detection by multi-context deep learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[105]  Tiangang Zhou,et al.  Topology-defined units in numerosity perception , 2015, Proceedings of the National Academy of Sciences.

[106]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[107]  Peng Jiang,et al.  Salient Region Detection by UFO: Uniqueness, Focusness and Objectness , 2013, 2013 IEEE International Conference on Computer Vision.

[108]  Michael Dorr,et al.  Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[109]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[110]  Dacheng Tao,et al.  Database Saliency for Fast Image Retrieval , 2015, IEEE Transactions on Multimedia.

[111]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[112]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[113]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[114]  Tieniu Tan,et al.  A Novel Visual Organization Based on Topological Perception , 2009, ACCV.