Can SAM Segment Anything? When SAM Meets Camouflaged Object Detection

SAM is a segmentation model recently released by Meta AI Research and has been gaining attention quickly due to its impressive performance in generic object segmentation. However, its ability to generalize to specific scenes such as camouflaged scenes is still unknown. Camouflaged object detection (COD) involves identifying objects that are seamlessly integrated into their surroundings and has numerous practical applications in fields such as medicine, art, and agriculture. In this study, we try to ask if SAM can address the COD task and evaluate the performance of SAM on the COD benchmark by employing maximum segmentation evaluation and camouflage location evaluation. We also compare SAM's performance with 22 state-of-the-art COD methods. Our results indicate that while SAM shows promise in generic object segmentation, its performance on the COD task is limited. This presents an opportunity for further research to explore how to build a stronger SAM that may address the COD task. The results of this paper are provided in \url{https://github.com/luckybird1994/SAMCOD}.

[1]  L. Gool,et al.  Deep Gradient Learning for Efficient Camouflaged Object Detection , 2022, Machine Intelligence Research.

[2]  L. Shao,et al.  Salient Object Detection via Integrity Learning , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Chen Gong,et al.  Feature Aggregation and Propagation Network for Camouflaged Object Detection , 2022, IEEE Transactions on Image Processing.

[4]  Ge-Peng Ji,et al.  Camouflaged Object Detection via Context-Aware Cross-Level Fusion , 2022, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Hongbo Bi,et al.  TPRNet: camouflaged object detection via transformer-induced progressive refinement network , 2022, The Visual Computer.

[6]  Chenglizhao Chen,et al.  Boundary-Guided Camouflaged Object Detection , 2022, IJCAI.

[7]  Dapeng Chen,et al.  I Can Find You! Boundary-Guided Separated Attention Network for Camouflaged Object Detection , 2022, AAAI.

[8]  Shuang Wu,et al.  Detecting Camouflaged Object in Frequency Domain , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Xin Fan,et al.  Segment, Magnify and Reiterate: Detecting Camouflaged Objects the Hard Way , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Huchuan Lu,et al.  Zoom In and Out: A Mixed-scale Triplet Network for Camouflaged Object Detection , 2022, 2203.02688.

[11]  Xiankai Lu,et al.  CubeNet: X-shape connection for camouflaged object detection , 2022, Pattern Recognit..

[12]  Keren Fu,et al.  Fast Camouflaged Object Detection via Edge-based Reversible Re-calibration Network , 2021, Pattern Recognit..

[13]  Ming-Ming Cheng,et al.  Concealed Object Detection , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  F. Yang,et al.  Uncertainty-Guided Transformer Reasoning for Camouflaged Object Detection , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[15]  明明 程,et al.  Cognitive vision inspired object segmentation metric and loss function , 2021, SCIENTIA SINICA Informationis.

[16]  Michael S. Bernstein,et al.  On the Opportunities and Risks of Foundation Models , 2021, ArXiv.

[17]  Tao Zhou,et al.  Context-aware Cross-level Fusion Network for Camouflaged Object Detection , 2021, IJCAI.

[18]  Shuo Zhang,et al.  Inferring Camouflaged Objects by Texture-Aware Interactive Guidance Network , 2021, AAAI.

[19]  Ling Shao,et al.  Visual Saliency Transformer , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[20]  Xiaopeng Wei,et al.  Camouflaged Object Segmentation with Distraction Mining , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Yuchao Dai,et al.  Uncertainty-aware Joint Salient Object and Camouflaged Object Detection , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Chenglizhao Chen,et al.  Mutual Graph Learning for Camouflaged Object Detection , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Yuchao Dai,et al.  Simultaneously Localize, Segment and Rank the Camouflaged Objects , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Ilya Sutskever,et al.  Learning Transferable Visual Models From Natural Language Supervision , 2021, ICML.

[25]  Quoc V. Le,et al.  Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision , 2021, ICML.

[26]  Tao Li,et al.  Structure-Measure: A New Way to Evaluate Foreground Maps , 2017, International Journal of Computer Vision.

[27]  Ling Shao,et al.  PraNet: Parallel Reverse Attention Network for Polyp Segmentation , 2020, MICCAI.

[28]  Ling Shao,et al.  Camouflaged Object Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[30]  D.-P. Fan,et al.  Inf-Net: Automatic COVID-19 Lung Infection Segmentation From CT Images , 2020, IEEE Transactions on Medical Imaging.

[31]  Trung-Nghia Le,et al.  Anabranch network for camouflaged object segmentation , 2019, Comput. Vis. Image Underst..

[32]  Ming-Ming Cheng,et al.  IP102: A Large-Scale Benchmark Dataset for Insect Pest Recognition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Xi Cheng,et al.  Pest identification via deep residual learning in complex background , 2017, Comput. Electron. Agric..

[34]  Lihi Zelnik-Manor,et al.  How to Evaluate Foreground Maps , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.