Towards Robust Few-shot Point Cloud Semantic Segmentation

Few-shot point cloud semantic segmentation aims to train a model to quickly adapt to new unseen classes with only a handful of support set samples. However, the noise-free assumption in the support set can be easily violated in many practical real-world settings. In this paper, we focus on improving the robustness of few-shot point cloud segmentation under the detrimental influence of noisy support sets during testing time. To this end, we first propose a Component-level Clean Noise Separation (CCNS) representation learning to learn discriminative feature representations that separates the clean samples of the target classes from the noisy samples. Leveraging the well separated clean and noisy support samples from our CCNS, we further propose a Multi-scale Degree-based Noise Suppression (MDNS) scheme to remove the noisy shots from the support set. We conduct extensive experiments on various noise settings on two benchmark datasets. Our results show that the combination of CCNS and MDNS significantly improves the performance. Our code is available at https://github.com/Pixie8888/R3DFSSeg.

[1]  Andreas Møgelmose,et al.  From CAD models to soft point cloud labels: An automatic annotation pipeline for cheaply supervised 3D semantic segmentation , 2023, Remote Sensing.

[2]  Ying J. Zhu,et al.  Few-Shot Classification with Contrastive Learning , 2022, ECCV.

[3]  Zonghao Guo,et al.  Bidirectional Feature Globalization for Few-shot Semantic Segmentation of 3D Point Cloud Scenes , 2022, 2022 International Conference on 3D Vision (3DV).

[4]  Guosheng Lin,et al.  Tackling background ambiguities in multi-class few-shot point cloud semantic segmentation , 2022, Knowl. Based Syst..

[5]  Kevin J Liang,et al.  Few-shot Learning with Noisy Labels , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Abhinav Gupta,et al.  The Challenges of Continuous Self-Supervised Learning , 2022, ECCV.

[7]  Tongliang Liu,et al.  Selective-Supervised Contrastive Learning with Noisy Labels , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  C. Schmid,et al.  Learning with Neighbor Consistency for Noisy Labels , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Caiming Xiong,et al.  Learning from Noisy Data with Robust Representation Learning , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Shih-Fu Chang,et al.  Partner-Assisted Learning for Few-Shot Image Classification , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[11]  Daan Bloembergen,et al.  Automatic labelling of urban point clouds using data fusion , 2021, ArXiv.

[12]  Zhi-Fan Wu,et al.  NGC: A Unified Framework for Learning with Open-World Noisy Data , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[13]  Dongdong Chen,et al.  Learning with Noisy Labels for Robust Point Cloud Segmentation , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[14]  Jimin Xiao,et al.  Self-Guided and Cross-Guided Learning for Few-Shot Segmentation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Zhiwu Lu,et al.  Contrastive Prototype Learning with Augmented Embeddings for Few-Shot Learning , 2021, UAI.

[16]  N. O'Connor,et al.  Multi-Objective Interpolation Training for Robustness to Label Noise , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Yanning Zhang,et al.  3D Point Cloud Labeling Tool for Driving Automatically , 2020, 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).

[18]  Vinay P. Namboodiri,et al.  RNNP: A Robust Few-Shot Learning Approach , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[19]  Qixiang Ye,et al.  Prototype Mixture Models for Few-shot Semantic Segmentation , 2020, ECCV.

[20]  Hengshuang Zhao,et al.  Prior Guided Feature Enrichment Network for Few-Shot Segmentation , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  M. Kölle,et al.  EVALUATION AND OPTIMISATION OF CROWD-BASED COLLECTION OF TREES FROM 3D POINT CLOUDS , 2020 .

[22]  Xuming He,et al.  Part-aware Prototype Network for Few-shot Semantic Segmentation , 2020, ECCV.

[23]  Sheng Liu,et al.  Early-Learning Regularization Prevents Memorization of Noisy Labels , 2020, NeurIPS.

[24]  Tat-Seng Chua,et al.  Few-shot 3D Point Cloud Semantic Segmentation , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Ce Liu,et al.  Supervised Contrastive Learning , 2020, NeurIPS.

[26]  Sheng Jin,et al.  Robust Few-Shot Learning for User-Provided Data , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Lei Feng,et al.  Combating Noisy Labels by Agreement: A Joint Training Method with Co-Regularization , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Junnan Li,et al.  DivideMix: Learning with Noisy Labels as Semi-supervised Learning , 2020, ICLR.

[29]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[30]  Chi Zhang,et al.  Pyramid Graph Networks With Connection Attentions for Region-Based One-Shot Semantic Segmentation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[31]  Jiashi Feng,et al.  PANet: Few-Shot Image Semantic Segmentation With Prototype Alignment , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[32]  Christoph Stiller,et al.  PointAtMe: Efficient 3D Point Cloud Labeling in Virtual Reality , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[33]  Yang Liu,et al.  Image Detector Based Automatic 3D Data Labeling and Training for Vehicle Detection on Point Cloud , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[34]  Jae-Gil Lee,et al.  SELFIE: Refurbishing Unclean Samples for Robust Deep Learning , 2019, ICML.

[35]  Kurt Keutzer,et al.  LATTE: Accelerating LiDAR Point Cloud Annotation via Sensor Fusion, One-Click Annotation, and Tracking , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[36]  Rui Yao,et al.  CANet: Class-Agnostic Segmentation Networks With Iterative Refinement and Attentive Few-Shot Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Xingrui Yu,et al.  How does Disagreement Help Generalization against Label Corruption? , 2019, ICML.

[38]  Fei Sha,et al.  Few-Shot Learning via Embedding Adaptation With Set-to-Set Functions , 2018, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Masashi Sugiyama,et al.  Co-teaching: Robust training of deep neural networks with extremely noisy labels , 2018, NeurIPS.

[40]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[41]  Yoshua Bengio,et al.  A Closer Look at Memorization in Deep Networks , 2017, ICML.

[42]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[43]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[44]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[45]  Matthias Nießner,et al.  ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Silvio Savarese,et al.  3D Semantic Parsing of Large-Scale Indoor Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[49]  Bharath Hariharan,et al.  Low-Shot Visual Recognition by Shrinking and Hallucinating Features , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[50]  Dumitru Erhan,et al.  Training Deep Neural Networks on Noisy Labels with Bootstrapping , 2014, ICLR.

[51]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[52]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .