Overview of deep-learning based methods for salient object detection in videos

Abstract Video salient object detection is a challenging and important problem in computer vision domain. In recent years, deep-learning based methods have contributed to significant improvements in this domain. This paper provides an overview of recent developments in this domain and compares the corresponding methods up to date, including 1) Classification of the state-of-the-art methods and their frameworks; 2) summary of the benchmark datasets and commonly used evaluation metrics; 3) experimental comparison of the performances of the state-of-the-art methods; 4) suggestions of some promising future works for unsolved challenges.

[1]  Xia Li,et al.  SCOM: Spatiotemporal Constrained Optimization for Salient Object Detection , 2018, IEEE Transactions on Image Processing.

[2]  Truong Q. Nguyen,et al.  Improving streaming video segmentation with early and mid-level visual processing , 2014, IEEE Winter Conference on Applications of Computer Vision.

[3]  Huchuan Lu,et al.  Attentive Feedback Network for Boundary-Aware Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Huchuan Lu,et al.  Multi attention module for visual tracking , 2019, Pattern Recognit..

[5]  Bin Zhou,et al.  Primary Video Object Segmentation via Complementary CNNs and Neighborhood Reversible Flow , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Shuang Wang,et al.  Unsupervised saliency-guided SAR image change detection , 2017, Pattern Recognit..

[7]  Xia Li,et al.  Weakly Supervised Salient Object Detection With Spatiotemporal Cascade Neural Networks , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Ling Shao,et al.  Video Salient Object Detection via Fully Convolutional Networks , 2017, IEEE Transactions on Image Processing.

[9]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[10]  Kaiming He,et al.  Rethinking ImageNet Pre-Training , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[11]  Hong-Bin Shen,et al.  Saliency driven region-edge-based top down level set evolution reveals the asynchronous focus in image segmentation , 2018, Pattern Recognit..

[12]  Zhuowen Tu,et al.  Deeply Supervised Salient Object Detection with Short Connections , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Huchuan Lu,et al.  Learning to Detect Salient Objects with Image-Level Supervision , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Luc Van Gool,et al.  A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[16]  Fanlei Yan Autonomous vehicle routing problem solution based on artificial potential field with parallel ant colony optimization (ACO) algorithm , 2018, Pattern Recognit. Lett..

[17]  Junwei Han,et al.  DHSNet: Deep Hierarchical Saliency Network for Salient Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Xiaowu Chen,et al.  A Benchmark Dataset and Saliency-Guided Stacked Autoencoders for Video-Based Salient Object Detection , 2016, IEEE Transactions on Image Processing.

[19]  Luc Van Gool,et al.  One-Shot Video Object Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Karteek Alahari,et al.  Learning Motion Patterns in Videos , 2016, CVPR.

[21]  Ali Borji,et al.  Salient object detection: A survey , 2014, Computational Visual Media.

[22]  Haifeng Hu,et al.  Deep multi-path convolutional neural network joint with salient region attention for facial expression recognition , 2019, Pattern Recognit..

[23]  Josef Sivic,et al.  NetVLAD: CNN Architecture for Weakly Supervised Place Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Rongrong Ji,et al.  RGBD Salient Object Detection: A Benchmark and Algorithms , 2014, ECCV.

[25]  Trung-Nghia Le,et al.  Deeply Supervised 3D Recurrent FCN for Salient Object Detection in Videos , 2017, BMVC.

[26]  Jianmin Jiang,et al.  A Simple Pooling-Based Design for Real-Time Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Sanyuan Zhao,et al.  Pyramid Dilated Deeper ConvLSTM for Video Salient Object Detection , 2018, ECCV.

[28]  Ming-Hsuan Yang,et al.  SegFlow: Joint Learning for Video Object Segmentation and Optical Flow , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Zhaoxiang Zhang,et al.  Spatiotemporal distilled dense-connectivity network for video action recognition , 2019, Pattern Recognit..

[30]  Baoxin Li,et al.  Fusing disparate object signatures for salient object detection in video , 2017, Pattern Recognit..

[31]  Zhe Wu,et al.  Cascaded Partial Decoder for Fast and Accurate Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Karteek Alahari,et al.  Learning to Segment Moving Objects , 2017, International Journal of Computer Vision.

[33]  Peyman Milanfar,et al.  Static and space-time visual saliency detection by self-resemblance. , 2009, Journal of vision.

[34]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[35]  Yongqiang Zhang,et al.  Weakly-supervised object detection via mining pseudo ground truth bounding-boxes , 2018, Pattern Recognit..

[36]  Huchuan Lu,et al.  Hyperfusion-Net: Hyper-densely reflective feature fusion for salient object detection , 2019, Pattern Recognit..

[37]  Luc Van Gool,et al.  The 2017 DAVIS Challenge on Video Object Segmentation , 2017, ArXiv.

[38]  Dong Xu,et al.  Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey , 2018, IEEE Signal Processing Magazine.

[39]  Xuelong Li,et al.  Unsupervised image saliency detection with Gestalt-laws guided optimization and visual attention based refinement , 2018, Pattern Recognit..

[40]  Dan Su,et al.  Multi-modal fusion network with multi-scale multi-path and cross-modal interactions for RGB-D salient object detection , 2019, Pattern Recognit..

[41]  Ning Xu,et al.  Video Object Segmentation Using Space-Time Memory Networks , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[42]  Zhiming Luo,et al.  Non-local Deep Features for Salient Object Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Yuan Xie,et al.  Flow Guided Recurrent Neural Encoder for Video Salient Object Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[44]  Trung-Nghia Le,et al.  Video Salient Object Detection Using Spatiotemporal Deep Features , 2017, IEEE Transactions on Image Processing.

[45]  Xiangzhong Fang,et al.  Multimodal architecture for video captioning with memory networks and an attention mechanism , 2017, Pattern Recognit. Lett..