SAM Struggles in Concealed Scenes - Empirical Study on "Segment Anything"

Segmenting anything is a ground-breaking step toward artificial general intelligence, and the Segment Anything Model (SAM) greatly fosters the foundation models for computer vision. We could not be more excited to probe the performance traits of SAM. In particular, exploring situations in which SAM does not perform well is interesting. In this report, we choose three concealed scenes, i.e., camouflaged animals, industrial defects, and medical lesions, to evaluate SAM under unprompted settings. Our main observation is that SAM looks unskilled in concealed scenes.

[1]  L. Gool,et al.  Advances in Deep Concealed Scene Understanding , 2023, ArXiv.

[2]  Ross B. Girshick,et al.  Segment Anything , 2023, 2023 IEEE/CVF International Conference on Computer Vision (ICCV).

[3]  L. Shao,et al.  Salient Object Detection via Integrity Learning , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  L. Gool,et al.  CamoFormer: Masked Separable Attention for Camouflaged Object Detection , 2022, ArXiv.

[5]  Ling Shao,et al.  High-resolution Iterative Feedback Network for Camouflaged Object Detection , 2022, AAAI.

[6]  P. Luo,et al.  PVT v2: Improved baselines with Pyramid Vision Transformer , 2021, Computational Visual Media.

[7]  Bjoern H Menze,et al.  The Medical Segmentation Decathlon , 2021, Nature Communications.

[8]  Ming-Ming Cheng,et al.  Concealed Object Detection , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  明明 程,et al.  Cognitive vision inspired object segmentation metric and loss function , 2021, SCIENTIA SINICA Informationis.

[10]  Christos Davatzikos,et al.  The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification , 2021, ArXiv.

[11]  Yuchao Dai,et al.  Simultaneously Localize, Segment and Rank the Camouflaged Objects , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Carsten Steger,et al.  The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection , 2021, International Journal of Computer Vision.

[13]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[14]  Tao Li,et al.  Structure-Measure: A New Way to Evaluate Foreground Maps , 2017, International Journal of Computer Vision.

[15]  Stephen Lin,et al.  Swin Transformer: Hierarchical Vision Transformer using Shifted Windows , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  Ling Shao,et al.  Camouflaged Object Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  D.-P. Fan,et al.  Inf-Net: Automatic COVID-19 Lung Infection Segmentation From CT Images , 2020, IEEE Transactions on Medical Imaging.

[18]  Jure Skvarč,et al.  Segmentation-based deep-learning approach for surface-defect detection , 2019, Journal of Intelligent Manufacturing.

[19]  Trung-Nghia Le,et al.  Anabranch network for camouflaged object segmentation , 2019, Comput. Vis. Image Underst..

[20]  Yibin Huang,et al.  Surface defect saliency of magnetic tile , 2018, The Visual Computer.

[21]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[22]  Lihi Zelnik-Manor,et al.  How to Evaluate Foreground Maps , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Fernando Vilariño,et al.  Towards automatic polyp detection with a polyp appearance model , 2012, Pattern Recognit..