Exploring Depth Contribution for Camouflaged Object Detection

Camouflaged object detection (COD) aims to segment camouflaged objects hiding in the environment, which is challenging due to the similar appearance of camouflaged objects and their surroundings. Research in biology suggests depth can provide useful object localization cues for camouflaged object discovery. In this paper, we study the depth contribution for camouflaged object detection, where the depth maps are generated with existing monocular depth estimation (MDE) methods. Due to the domain gap between the MDE dataset and our COD dataset, the generated depth maps are not accurate enough to be directly used. We then introduce two solutions to avoid the noisy depth maps from dominating the training process. Firstly, we present an auxiliary depth estimation branch (“ADE”), aiming to regress the depth maps. We find that “ADE” is especially necessary for our “generated depth” scenario. Secondly, we introduce a multi-modal confidenceaware loss function via a generative adversarial network to weigh the contribution of depth for camouflaged object detection. Our extensive experiments on various camouflaged object detection datasets explain that the existing “sensor depth” based RGB-D segmentation techniques work poorly with “generated depth”, and our proposed two solutions work cooperatively, achieving effective depth contribution exploration for camouflaged object detection.

[1]  M. Stevens,et al.  Background matching and disruptive coloration as habitat-specific strategies for camouflage , 2019, Scientific Reports.

[2]  Jitendra Malik,et al.  Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[4]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[5]  B. D. Todd,et al.  Hiding in plain sight: a study on camouflage and habitat selection in a slow-moving desert herbivore , 2015 .

[6]  Xueqing Li,et al.  Leveraging stereopsis for saliency analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  William T. Freeman,et al.  Learning the Depths of Moving People by Watching Frozen People , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Yongri Piao,et al.  Select, Supplement and Focus for RGB-D Saliency Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  G. Katzir,et al.  Plant coloration undermines herbivorous insect camouflage. , 2004, BioEssays : news and reviews in molecular, cellular and developmental biology.

[10]  Shuai Li,et al.  A Fusion Framework for Camouflaged Moving Foreground Detection in the Wavelet Domain , 2018, IEEE Transactions on Image Processing.

[11]  Meng Sun,et al.  Detection of People With Camouflage Pattern Via Dense Deconvolution Network , 2019, IEEE Signal Processing Letters.

[12]  Junwei Han,et al.  Learning Selective Self-Mutual Attention for RGB-D Saliency Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Olivier Penacchio,et al.  Three-Dimensional Camouflage: Exploiting Photons to Conceal Form , 2015, The American Naturalist.

[14]  Longin Jan Latecki,et al.  Semantic Segmentation of RGBD Images with Mutex Constraints , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Kilian Q. Weinberger,et al.  On Calibration of Modern Neural Networks , 2017, ICML.

[16]  Roberto Cipolla,et al.  Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Ming-Hsuan Yang,et al.  Adversarial Learning for Semi-supervised Semantic Segmentation , 2018, BMVC.

[18]  Ling Shao,et al.  Concealed Object Detection , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Ling Shao,et al.  PraNet: Parallel Reverse Attention Network for Polyp Segmentation , 2020, MICCAI.

[20]  Konrad Schindler,et al.  Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Wei Jia,et al.  Camouflage performance analysis and evaluation framework based on features fusion , 2015, Multimedia Tools and Applications.

[22]  Seungyong Lee,et al.  RDFNet: RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[23]  A. Thayer,et al.  Concealing-coloration in the animal kingdom : an exposition of the laws of disguise through color and pattern being a summary of Abbott H. Thayer's discoveries , 1909 .

[24]  Tao Li,et al.  Structure-Measure: A New Way to Evaluate Foreground Maps , 2017, International Journal of Computer Vision.

[25]  Qingming Huang,et al.  F3Net: Fusion, Feedback and Focus for Salient Object Detection , 2019, AAAI.

[26]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Yehezkel Yeshurun,et al.  Convexity-Based Visual Camouflage Breaking , 2001, Comput. Vis. Image Underst..

[28]  Jimeng Sun,et al.  SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates , 2020, ICML.

[29]  Charles Blundell,et al.  Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles , 2016, NIPS.

[30]  Xinxin Hu,et al.  ACNET: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[31]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[32]  W. Adams,et al.  Disruptive coloration and binocular disparity: breaking camouflage , 2019, Proceedings of the Royal Society B.

[33]  Chenglizhao Chen,et al.  Mutual Graph Learning for Camouflaged Object Detection , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Bo Ren,et al.  Enhanced-alignment Measure for Binary Foreground Map Evaluation , 2018, IJCAI.

[35]  Geng Chen,et al.  Towards Accurate Camouflaged Object Detection with Mixture Convolution and Interactive Fusion , 2021, ArXiv.

[36]  Minghui Wang,et al.  RGB-D Salient Object Detection via Minimum Barrier Distance Transform and Saliency Fusion , 2017, IEEE Signal Processing Letters.

[37]  Thomas W. Pike,et al.  Quantifying camouflage and conspicuousness using visual salience , 2018 .

[38]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[39]  Rongrong Ji,et al.  RGBD Salient Object Detection: A Benchmark and Algorithms , 2014, ECCV.

[40]  Trung-Nghia Le,et al.  Anabranch network for camouflaged object segmentation , 2019, Comput. Vis. Image Underst..

[41]  F. Yang,et al.  Uncertainty-Guided Transformer Reasoning for Camouflaged Object Detection , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[42]  Oisin Mac Aodha,et al.  Unsupervised Monocular Depth Estimation with Left-Right Consistency , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Trung-Nghia Le,et al.  MirrorNet: Bio-Inspired Adversarial Attack for Camouflaged Object Segmentation , 2020, ArXiv.

[44]  Mohan M. Trivedi,et al.  Models and metrics for signature strength evaluation of camouflaged targets , 1997, Defense, Security, and Sensing.

[45]  Nick Barnes,et al.  Local Background Enclosure for RGB-D Salient Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Dieter Fox,et al.  RGB-(D) scene labeling: Features and algorithms , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Nathan Silberman,et al.  Indoor scene segmentation using a structured light sensor , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[48]  Yuchao Dai,et al.  Simultaneously Localize, Segment and Rank the Camouflaged Objects , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Xiongwei Zhang,et al.  Camouflage people detection via strong semantic dilation network , 2019, ACM TUR-C.

[50]  Nicholas E. Scott-Samuel,et al.  A platform for initial testing of multiple camouflage patterns , 2020 .

[51]  Daniel Cohen-Or,et al.  Cascaded Feature Network for Semantic Segmentation of RGB-D Images , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[52]  Wei Ji,et al.  Depth-Induced Multi-Scale Recurrent Attention Network for Saliency Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[53]  Xiaopeng Wei,et al.  Camouflaged Object Segmentation with Distraction Mining , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  R. Baddeley,et al.  A review of cuttlefish camouflage and object recognition and evidence for depth perception , 2008, Journal of Experimental Biology.

[57]  Kai Zhao,et al.  Res2Net: A New Multi-Scale Backbone Architecture , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Qijun Zhao,et al.  JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Xiaochun Cao,et al.  Depth Enhanced Saliency Detection Method , 2014, ICIMCS '14.

[61]  B. Zuckerberg,et al.  An experimental translocation identifies habitat features that buffer camouflage mismatch in snowshoe hares , 2018, Conservation Letters.

[62]  Wolfram Burgard,et al.  Self-Supervised Model Adaptation for Multimodal Semantic Segmentation , 2018, International Journal of Computer Vision.

[63]  Jing Gu,et al.  Camouflage texture evaluation using a saliency map , 2014, Multimedia Systems.

[64]  Zheng Lin,et al.  Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[65]  Nick Barnes,et al.  UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[66]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  Yang Liu,et al.  Depth-aware salient object detection using anisotropic center-surround difference , 2015, Signal Process. Image Commun..

[68]  Anjith George,et al.  Cross Modal Focal Loss for RGBD Face Anti-Spoofing , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[69]  Xiaokang Chen,et al.  Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation , 2020, ECCV.

[70]  Mert R. Sabuncu,et al.  Real-Time Uncertainty Estimation in Computer Vision via Uncertainty-Aware Distribution Distillation , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[71]  P. Nagabhushan,et al.  Camouflage Defect Identification: A Novel Approach , 2006, 9th International Conference on Information Technology (ICIT'06).

[72]  Michael Ying Yang,et al.  Exploiting global priors for RGB-D saliency detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[73]  Ling Shao,et al.  BBS-Net: RGB-D Salient Object Detection with a Bifurcated Backbone Strategy Network , 2020, ECCV.

[74]  Ling Shao,et al.  Camouflaged Object Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).