Visualizing point cloud classifiers by curvature smoothing

Recently, several networks that operate directly on point clouds have been proposed. There is significant utility in understanding their mechanisms to classify point clouds, which can potentially help diagnosing these networks and designing better architectures. In this paper, we propose a novel approach to visualize features important to the point cloud classifiers. Our approach is based on smoothing curved areas on a point cloud. After prominent features were smoothed, the resulting point cloud can be evaluated on the network to assess whether the feature is important to the classifier. A technical contribution of the paper is an approximated curvature smoothing algorithm, which can smoothly transition from the original point cloud to one of constant curvature, such as a uniform sphere. Based on the smoothing algorithm, we propose PCI-GOS (Point Cloud Integrated-Gradients Optimized Saliency), a visualization technique that can automatically find the minimal saliency map that covers the most important features on a shape. Experiment results revealed insights into different point cloud classifiers.

[1]  Chong Xiang,et al.  Generating 3D Adversarial Point Clouds , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Tamy Boubekeur,et al.  Point morphology , 2014, ACM Trans. Graph..

[3]  Jyh-Ming Lien.  Point-Based Minkowski Sum Boundary , 2007, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[4]  Carlo H. Séquin,et al.  Type-Constrained Direct Fitting of Quadric Surfaces , 2014 .

[5]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[6]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[7]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[8]  Anna Shcherbina,et al.  Not Just a Black Box: Learning Important Features Through Propagating Activation Differences , 2016, ArXiv.

[9]  Fuxin Li,et al.  PointConv: Deep Convolutional Networks on 3D Point Clouds , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  K. Fujiwara.  Eigenvalues of Laplacians on a closed Riemannian manifold and its nets , 1995 .

[11]  Kate Saenko,et al.  RISE: Randomized Input Sampling for Explanation of Black-box Models , 2018, BMVC.

[12]  Hao Su,et al.  Extending Adversarial Attacks and Defenses to Deep 3D Point Cloud Classifiers , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[13]  Wesley E. Snyder,et al.  Fitting a quadratic surface to three dimensional data , 1989 .

[14]  Alexander Binder,et al.  On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2015, PloS one.

[15]  Bolei Zhou,et al.  Object Detectors Emerge in Deep Scene CNNs , 2014, ICLR.

[16]  Yarin Gal,et al.  Real Time Image Saliency for Black Box Classifiers , 2017, NIPS.

[17]  Yaron Lipman,et al.  Point convolutional neural networks by extension operators , 2018, ACM Trans. Graph..

[18]  Zoltan-Csaba Marton,et al.  On Fast Surface Reconstruction Methods for Large and Noisy Datasets , 2009, IEEE International Conference on Robotics and Automation.

[19]  Mark Meyer,et al.  Implicit fairing of irregular meshes using diffusion and curvature flow , 1999, SIGGRAPH.

[20]  Marc Alexa,et al.  Point set surfaces , 2001, Proceedings Visualization, 2001. VIS '01..

[21]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[22]  Heinrich Müller,et al.  SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Vladlen Koltun,et al.  Tangent Convolutions for Dense Prediction in 3D , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Wei Wu,et al.  PointCNN: convolution on Χ -transformed points , 2018, NIPS 2018.

[25]  Wei Wu,et al.  PointCNN: Convolution On X-Transformed Points , 2018, NeurIPS.

[26]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[27]  Kehe Zhu.  Operator theory in function spaces , 1990 .

[28]  Yifan Xu,et al.  SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters , 2018, ECCV.

[29]  Ankur Taly,et al.  Axiomatic Attribution for Deep Networks , 2017, ICML.

[30]  Subhransu Maji,et al.  SPLATNet: Sparse Lattice Networks for Point Cloud Processing , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Gabriel Taubin,et al.  A signal processing approach to fair surface design , 1995, SIGGRAPH.

[32]  Fuxin Li,et al.  Visualizing Deep Networks by Optimizing with Integrated Gradients , 2019, CVPR Workshops.

[33]  Andrea Vedaldi,et al.  Interpretable Explanations of Black Boxes by Meaningful Perturbation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[34]  David Levin,et al.  The approximation power of moving least-squares , 1998, Math. Comput..

[35]  L. Velho,et al.  Robust Smoothing of Noisy Point Clouds , 2003 .

[36]  Kui Ren,et al.  PointCloud Saliency Maps , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[37]  Hans-Peter Seidel,et al.  Interactive multi-resolution modeling on arbitrary meshes , 1998, SIGGRAPH.