LFBCNet: Light Field Boundary-aware and Cascaded Interaction Network for Salient Object Detection

In light field imaging techniques, the abundance of stereo spatial information aids in improving the performance of salient object detection. In some complex scenes, however, applying the 4D light field boundary structure to discriminate salient objects from background regions is still under-explored. In this paper, we propose a light field boundary-aware and cascaded interaction network based on light field macro-EPI, named LFBCNet. Firstly, we propose a well-designed light field multi-epipolar-aware learning (LFML) module to learn rich salient boundary cues by perceiving the continuous angle changes from light field macro-EPI. Secondly, to fully excavate the correlation between salient objects and boundaries at different scales, we design multiple light field boundary interactive (LFBI) modules and cascade them to form a light field multi-scale cascade interaction decoder network. Each LFBI is assigned to predict exquisite salient objects and boundaries by interactively transmitting the salient object and boundary features. Meanwhile, the salient boundary features are forced to gradually refine the salient object features during the multi-scale cascade encoding. Furthermore, a light field multi-scale-fusion prediction (LFMP) module is developed to automatically select and integrate multi-scale salient object features for final saliency prediction. The proposed LFBCNet can accurately distinguish tiny differences between salient objects and background regions. Comprehensive experiments on large benchmark datasets prove that the proposed method achieves competitive performance over 2-D, 3-D, and 4-D salient object detection methods.

[1]  Xu Cheng,et al.  Visual Object Tracking Based on Light-Field Imaging in the Presence of Similar Distractors , 2023, IEEE Transactions on Industrial Informatics.

[2]  Huchuan Lu,et al.  PANet: Patch-Aware Network for Light Field Salient Object Detection , 2021, IEEE Transactions on Cybernetics.

[3]  S. W. Han,et al.  TRACER: Extreme Attention Guided Salient Object Tracing Network , 2021, AAAI.

[4]  Xu Cheng,et al.  Temporal Attention Convolutional Neural Network for Estimation of Icing Probability on Wind Turbine Blades , 2021, IEEE Transactions on Industrial Electronics.

[5]  Junhui Hou,et al.  Deep Spatial-Angular Regularization for Light Field Imaging, Denoising, and Super-Resolution , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Wujie Zhou,et al.  CCAFNet: Crossflow and Cross-Scale Adaptive Fusion Network for Detecting Salient Objects in RGB-D Images , 2021, IEEE Transactions on Multimedia.

[7]  Ming-Ming Cheng,et al.  MobileSal: Extremely Efficient RGB-D Salient Object Detection , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Haibin Ling,et al.  Salient Object Detection in the Deep Learning Era: An In-Depth Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Shunyu Yao,et al.  Auto-MSFNet: Search Multi-scale Fusion Network for Salient Object Detection , 2021, ACM Multimedia.

[10]  Sam Kwong,et al.  Geometry Auxiliary Salient Object Detection for Light Fields via Graph Neural Networks , 2021, IEEE Transactions on Image Processing.

[11]  Deng-Ping Fan,et al.  Specificity-preserving RGB-D saliency detection , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Qijun Zhao,et al.  Depth Quality-Inspired Feature Manipulation for Efficient RGB-D Salient Object Detection , 2021, ACM Multimedia.

[13]  Wassim Hamidouche,et al.  Learning Synergistic Attention for Light Field Salient Object Detection , 2021, BMVC.

[14]  Songyuan Li,et al.  Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Qijun Zhao,et al.  RGB-D Salient Object Detection via 3D Convolutional Neural Networks , 2021, AAAI.

[16]  Ming-Ming Cheng,et al.  RGB-D salient object detection: A survey , 2020, Computational Visual Media.

[17]  Xu Wang,et al.  A Multi-Task Collaborative Network for Light Field Salient Object Detection , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Yonghong Tian,et al.  Salient Object Detection With Purificatory Mechanism and Structural Similarity Loss , 2019, IEEE Transactions on Image Processing.

[19]  Tao Li,et al.  Structure-Measure: A New Way to Evaluate Foreground Maps , 2017, International Journal of Computer Vision.

[20]  Xu Cheng,et al.  Semantic Segmentation With Light Field Imaging and Convolutional Neural Networks , 2021, IEEE Transactions on Instrumentation and Measurement.

[21]  Yongri Piao,et al.  DUT-LFSaliency: Versatile Dataset and Light Field-to-RGB Saliency Detection , 2020, ArXiv.

[22]  Jianhuang Lai,et al.  Interactive Two-Stream Decoder for Accurate and Fast Saliency Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Huchuan Lu,et al.  Multi-Scale Interactive Network for Salient Object Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Nick Barnes,et al.  UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Yongri Piao,et al.  Exploit and Replace: An Asymmetrical Two-Stream Architecture for Versatile Light Field Saliency Detection , 2020, AAAI.

[26]  Qingming Huang,et al.  Global Context-Aware Progressive Aggregation Network for Salient Object Detection , 2020, AAAI.

[27]  Jun Zhang,et al.  Light Field Saliency Detection With Deep Convolutional Networks , 2019, IEEE Transactions on Image Processing.

[28]  Tiantian Wang,et al.  Deep Learning for Light Field Saliency Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[29]  Wei Ji,et al.  Depth-Induced Multi-Scale Recurrent Attention Network for Saliency Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[30]  Huchuan Lu,et al.  Deep Light-field-driven Saliency Detection from a Single View , 2019, IJCAI.

[31]  Yan Luo,et al.  Visual Attention in Multi-Label Image Classification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[32]  Steven C. H. Hoi,et al.  Salient Object Detection With Pyramid Attention and Salient Edges , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Xing Cai,et al.  PDNet: Prior-Model Guided Depth-Enhanced Network for Salient Object Detection , 2018, 2019 IEEE International Conference on Multimedia and Expo (ICME).

[34]  Qingming Huang,et al.  Image Saliency Detection Video Saliency Detection Co-saliency Detection Temporal RGBD Saliency Detection Motion , 2018 .

[35]  Paul E. Debevec,et al.  A system for acquiring, processing, and rendering panoramic light field stills for virtual reality , 2018, ACM Trans. Graph..

[36]  Bo Ren,et al.  Enhanced-alignment Measure for Binary Foreground Map Evaluation , 2018, IJCAI.

[37]  Meng Wang,et al.  Saliency Detection on Light Field , 2017, ACM Trans. Multim. Comput. Commun. Appl..

[38]  Jiandong Tian,et al.  RGBD Salient Object Detection via Deep Fusion , 2016, IEEE Transactions on Image Processing.

[39]  Yunchao Wei,et al.  STC: A Simple to Complex Framework for Weakly-Supervised Semantic Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Haibin Ling,et al.  Saliency Detection on Light Field , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Shao-Yi Chien,et al.  Real-Time Salient Object Detection with a Minimum Spanning Tree , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Yueting Zhuang,et al.  DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection , 2015, IEEE Transactions on Image Processing.

[44]  Seunghoon Hong,et al.  Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural Network , 2015, ICML.

[45]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Lihi Zelnik-Manor,et al.  How to Evaluate Foreground Maps , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[48]  Sabine Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.