Complementary Pseudo Multimodal Feature for Point Cloud Anomaly Detection

Point cloud (PCD) anomaly detection steadily emerges as a promising research area. This study aims to improve PCD anomaly detection performance by combining handcrafted PCD descriptions with powerful pre-trained 2D neural networks. To this end, this study proposes Complementary Pseudo Multimodal Feature (CPMF) that incorporates local geometrical information in 3D modality using handcrafted PCD descriptors and global semantic information in the generated pseudo 2D modality using pre-trained 2D neural networks. For global semantics extraction, CPMF projects the origin PCD into a pseudo 2D modality containing multi-view images. These images are delivered to pre-trained 2D neural networks for informative 2D modality feature extraction. The 3D and 2D modality features are aggregated to obtain the CPMF for PCD anomaly detection. Extensive experiments demonstrate the complementary capacity between 2D and 3D modality features and the effectiveness of CPMF, with 95.15% image-level AU-ROC and 92.93% pixel-level PRO on the MVTec3D benchmark. Code is available on https://github.com/caoyunkang/CPMF.

[1]  Xinyu Li,et al.  Unsupervised Image Anomaly Detection and Segmentation Based on Pretrained Feature Mapping , 2023, IEEE Transactions on Industrial Informatics.

[2]  B. Rosenhahn,et al.  Asymmetric Student-Teacher Networks for Industrial Anomaly Detection , 2022, 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).

[3]  Paul Bergmann,et al.  Anomaly Detection in 3D Point Clouds using Deep Geometric Descriptors , 2022, 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).

[4]  Dagan Feng,et al.  Unsupervised Landmark Detection-Based Spatiotemporal Motion Estimation for 4-D Dynamic Medical Images , 2021, IEEE Transactions on Cybernetics.

[5]  Liang Gao,et al.  Industrial Image Anomaly Localization Based on Gaussian Clustering of Pretrained Feature , 2022, IEEE Transactions on Industrial Electronics.

[6]  Liang Gao,et al.  Informative knowledge distillation for image anomaly segmentation , 2022, Knowl. Based Syst..

[7]  Yedid Hoshen,et al.  Back to the Feature: Classical 3D Features are (Almost) All You Need for 3D Anomaly Detection , 2022, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Chen Zhao,et al.  Rotation invariant point cloud analysis: Where local geometry meets global topology , 2022, Pattern Recognit..

[9]  Xingyu Li,et al.  Anomaly Detection via Reverse Distillation from One-Class Embedding , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Carsten Steger,et al.  The MVTec 3D-AD Dataset for Unsupervised 3D Anomaly Detection and Localization , 2021, VISIGRAPP.

[11]  G. Boracchi,et al.  Deep open-set recognition for silicon wafer production monitoring , 2021, Pattern Recognit..

[12]  Xinggang Wang,et al.  Defect attention template generation cycleGAN for weakly supervised surface defect segmentation , 2021, Pattern Recognit..

[13]  Kazuki Kozuka,et al.  CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows , 2021, 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).

[14]  B. Schölkopf,et al.  Towards Total Recall in Industrial Anomaly Detection , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Jie Yang,et al.  Learning deep feature correspondence for unsupervised anomaly detection and segmentation , 2022, Pattern Recognit..

[16]  D. Skočaj,et al.  DRÆM – A discriminatively trained reconstruction embedding for surface anomaly detection , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[17]  Marcel Bengs,et al.  Three-dimensional deep learning with spatial erasing for unsupervised anomaly segmentation in brain MRI , 2021, International Journal of Computer Assisted Radiology and Surgery.

[18]  Quan Z. Sheng,et al.  A Comprehensive Survey on Graph Anomaly Detection With Deep Learning , 2021, IEEE Transactions on Knowledge and Data Engineering.

[19]  Haibin Ling,et al.  Multi-View 3D Shape Recognition via Correspondence-Aware Deep Learning , 2021, IEEE Transactions on Image Processing.

[20]  Tomas Pfister,et al.  CutPaste: Self-Supervised Learning for Anomaly Detection and Localization , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Errui Ding,et al.  Student-Teacher Feature Pyramid Matching for Anomaly Detection , 2021, BMVC.

[22]  Carsten Steger,et al.  The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection , 2021, International Journal of Computer Vision.

[23]  Zhiquan Qi,et al.  DFR: Deep Feature Reconstruction for Unsupervised Anomaly Segmentation , 2020, Neurocomputing.

[24]  Matej Kristan,et al.  Reconstruction by inpainting for visual anomaly detection , 2020, Pattern Recognit..

[25]  Xinyu Li,et al.  A novel robotic grasp detection method based on region proposal networks , 2020, Robotics Comput. Integr. Manuf..

[26]  Leonidas J. Guibas,et al.  PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding , 2020, ECCV.

[27]  Jian Sun,et al.  View-GCN: View-Based Graph Convolutional Network for 3D Shape Analysis , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Vladlen Koltun,et al.  Fully Convolutional Geometric Features , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[29]  Junwei Han,et al.  3D2SeqViews: Aggregating Sequential Views for 3D Global Feature Learning by CNN With Hierarchical Attention Aggregation , 2019, IEEE Transactions on Image Processing.

[30]  Carsten Steger,et al.  Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders , 2018, VISIGRAPP.

[31]  Vladlen Koltun,et al.  Open3D: A Modern Library for 3D Data Processing , 2018, ArXiv.

[32]  Ersin Yumer,et al.  Learning Local Shape Descriptors from Part Correspondences with Multiview Convolutional Networks , 2017, ACM Trans. Graph..

[33]  Liang Gao,et al.  Differential evolution algorithm-based range image registration for free-form surface parts quality inspection , 2017, Swarm Evol. Comput..

[34]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[35]  Liang Gao,et al.  An ensemble fruit fly optimization algorithm for solving range image registration to improve quality inspection of free-form surface parts , 2016, Inf. Sci..

[36]  José García Rodríguez,et al.  PointNet: A 3D Convolutional Neural Network for real-time object class recognition , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[37]  Nikos Komodakis,et al.  Wide Residual Networks , 2016, BMVC.

[38]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[40]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[41]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[42]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[43]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.