A novel dynamic graph evolution network for salient object detection

The advanced deep convolution neural networks (CNNs) based salient object detection (SOD) models still suffer from the coarse object edge. The traditional graph-based SOD models can preserve the object edge well benefitting from the superpixels, but they perform weaker in highlighting the whole object compared to recent deep learning models. To tackle this problem, we attempt to find a new way to address this issue under the framework of graph convolution networks (GCNs). Specifically, we first model the image as a set of superpixels and construct the graph structure by connecting the k nearest neighbors for each node. For the connected nodes, rather than only leveraging on the predefined edges, we propose a multi-relations edge convolution operation, expecting to learn multiple implicit relations in the pair-wise nodes and aggregate the information from their neighbors relying on the learned edges. Then, a channel-wise attention operation is also proposed to boost the intra-node massage propagation across channels within the same node. In order to optimize the graph structure from layer to layer, we learn a new metric to re-measure the distance between any pair of nodes, and the graph structure evolves dynamically by recomputing the k nearest neighbors in the saliency feature space at different layers. Finally, a residual structure is applied to enable our graph network to go as deep as CNNs models. The graph nodes (superpixels) inherently belonging to the same class will be ideally clustered together in the learned embedding space. Experiments show that this work is a good practice for designing GCNs for image SOD and achieves the comparable performance with the recent state-of-the-art deep CNNs-based models.

[1]  Yi Liu,et al.  Salient Object Detection via Two-Stage Graphs , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Wenbin Zou,et al.  Saliency Tree: A Novel Saliency Detection Framework , 2014, IEEE Transactions on Image Processing.

[3]  Huchuan Lu,et al.  Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Qingming Huang,et al.  Image Saliency Detection Video Saliency Detection Co-saliency Detection Temporal RGBD Saliency Detection Motion , 2018 .

[6]  Nanning Zheng,et al.  Salient Object Detection: A Discriminative Regional Feature Integration Approach , 2013, International Journal of Computer Vision.

[7]  Ruigang Yang,et al.  Saliency-Aware Video Object Segmentation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Ruigang Yang,et al.  A Unified Spatiotemporal Prior based on Geodesic Distance for Video Object Segmentation , 2017 .

[10]  Yu Hen Hu,et al.  Video Salient Object Detection via Robust Seeds Extraction and Multi-Graphs Manifold Propagation , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[12]  Liang-Tien Chia,et al.  Region-Based Saliency Detection and Its Application in Object Recognition , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Feng Wu,et al.  Background-Driven Salient Object Detection , 2017, IEEE Transactions on Multimedia.

[14]  Haibin Ling,et al.  Salient Object Detection in the Deep Learning Era: An In-Depth Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Huchuan Lu,et al.  Salient Object Detection with Recurrent Fully Convolutional Networks , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[17]  Mengke Huang,et al.  Saliency detection via multi-level integration and multi-scale fusion neural networks , 2019, Neurocomputing.

[18]  Moncef Gabbouj,et al.  Visual saliency by extended quantum cuts , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[19]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[20]  Ming-Ming Cheng,et al.  EGNet: Edge Guidance Network for Salient Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[21]  Hai Jin,et al.  MsLRR: A Unified Multiscale Low-Rank Representation for Image Segmentation , 2014, IEEE Transactions on Image Processing.

[22]  Shi-Min Hu,et al.  Global Contrast Based Salient Region Detection , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Ruigang Yang,et al.  Inferring Salient Objects from Human Fixations , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Tao Li,et al.  Structure-Measure: A New Way to Evaluate Foreground Maps , 2017, International Journal of Computer Vision.

[25]  Xuelong Hu,et al.  Embedding Attention and Residual Network for Accurate Salient Object Detection , 2020, IEEE Transactions on Cybernetics.

[26]  Huchuan Lu,et al.  Saliency Detection via Dense and Sparse Reconstruction , 2013, 2013 IEEE International Conference on Computer Vision.

[27]  Yizhou Yu,et al.  Visual saliency based on multiscale deep features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Xiaofei Zhou,et al.  Saliency detection using adversarial learning networks , 2020, J. Vis. Commun. Image Represent..

[29]  Huchuan Lu,et al.  Salient Object Detection With Lossless Feature Reflection and Weighted Structural Loss , 2019, IEEE Transactions on Image Processing.

[30]  Huchuan Lu,et al.  Attentive Feedback Network for Boundary-Aware Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Zhibo Chen,et al.  Full Reference Quality Assessment for Image Retargeting Based on Natural Scene Statistics Modeling and Bi-Directional Saliency Similarity , 2017, IEEE Transactions on Image Processing.

[32]  Junwei Han,et al.  DHSNet: Deep Hierarchical Saliency Network for Salient Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[35]  Yizhou Yu,et al.  Visual Saliency Detection Based on Multiscale Deep CNN Features , 2016, IEEE Transactions on Image Processing.

[36]  Sabine Süsstrunk,et al.  Superpixels and Polygons Using Simple Non-iterative Clustering , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Bo Wang,et al.  Learning Long-Term Structural Dependencies for Video Salient Object Detection , 2020, IEEE Transactions on Image Processing.

[38]  Shuyuan Yang,et al.  Salient Region Detection via Discriminative Dictionary Learning and Joint Bayesian Inference , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[39]  Huchuan Lu,et al.  Saliency Detection via Absorbing Markov Chain With Learnt Transition Probability , 2018, IEEE Transactions on Image Processing.

[40]  Gang Wang,et al.  Progressive Attention Guided Recurrent Network for Salient Object Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Weiguo Fan,et al.  Re-Caption: Saliency-Enhanced Image Captioning Through Two-Phase Learning , 2020, IEEE Transactions on Image Processing.

[42]  Ling Shao,et al.  An Iterative and Cooperative Top-Down and Bottom-Up Inference Network for Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Feiping Nie,et al.  Saliency Detection via a Multiple Self-Weighted Graph-Based Manifold Ranking , 2020, IEEE Transactions on Multimedia.

[44]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[45]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[46]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[47]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[48]  Yang Wang,et al.  Salient Object Segmentation via Effective Integration of Saliency and Objectness , 2017, IEEE Transactions on Multimedia.

[49]  Qiaosong Wang,et al.  GraB: Visual Saliency via Novel Graph Model and Background Priors , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Yi Zhang,et al.  PSANet: Point-wise Spatial Attention Network for Scene Parsing , 2018, ECCV.

[51]  Yu Hen Hu,et al.  Video Saliency Detection via Graph Clustering With Motion Energy and Spatiotemporal Objectness , 2019, IEEE Transactions on Multimedia.

[52]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Yizhou Yu,et al.  Contrast-Oriented Deep Neural Networks for Salient Object Detection , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[54]  Ling Shao,et al.  Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[55]  Li Xu,et al.  Hierarchical Saliency Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Qing Guo,et al.  Dynamic Saliency-Aware Regularization for Correlation Filter-Based Object Tracking , 2019, IEEE Transactions on Image Processing.

[57]  Mohammad Havaei,et al.  CAGNet: Content-Aware Guidance for Salient Object Detection , 2020, Pattern Recognit..

[58]  Xiaogang Wang,et al.  Saliency detection by multi-context deep learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Chenglong Li,et al.  Edge-Guided Non-Local Fully Convolutional Network for Salient Object Detection , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[60]  Xiao Lin,et al.  Image Classification with Hierarchical Multigraph Networks , 2019, BMVC.

[61]  Yueting Zhuang,et al.  DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection , 2015, IEEE Transactions on Image Processing.

[62]  Weisi Lin,et al.  Saliency Detection in the Compressed Domain for Adaptive Image Retargeting , 2012, IEEE Transactions on Image Processing.

[63]  Ting Zhao,et al.  Pyramid Feature Attention Network for Saliency Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[64]  Bin Luo,et al.  Multiple Graph Convolutional Networks for Co-Saliency Detection , 2019, 2019 IEEE International Conference on Multimedia and Expo (ICME).

[65]  Dinggang Shen,et al.  Contour Knowledge Transfer for Salient Object Detection , 2018, ECCV.

[66]  Guanbin Li,et al.  Visual Saliency Detection Based on Multiscale Deep CNN Features. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[67]  Zhibo Chen,et al.  Full Reference Quality Assessment for Image Retargeting Based on Natural Scene Statistics Modeling and Bi-Directional Saliency Similarity. , 2017, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[68]  Huchuan Lu,et al.  Learning to Detect Salient Objects with Image-Level Supervision , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[69]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[70]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[71]  Huchuan Lu,et al.  Saliency Detection via Absorbing Markov Chain With Learnt Transition Probability. , 2018, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[72]  Kate Saenko,et al.  Top-Down Visual Saliency Guided by Captions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[73]  Huchuan Lu,et al.  Saliency Detection via Absorbing Markov Chain , 2013, 2013 IEEE International Conference on Computer Vision.

[74]  Huchuan Lu,et al.  Learning Uncertain Convolutional Features for Accurate Saliency Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[75]  Huchuan Lu,et al.  Detect Globally, Refine Locally: A Novel Approach to Saliency Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[76]  James M. Rehg,et al.  The Secrets of Salient Object Segmentation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[77]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[78]  Steven C. H. Hoi,et al.  Salient Object Detection With Pyramid Attention and Salient Edges , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).