Pointfilter: Point Cloud Filtering via Encoder-Decoder Modeling

Point cloud filtering is a fundamental problem in geometry modeling and processing. Despite of advancement in recent years, the existing methods still suffer from two issues: 1) they are either designed without preserving sharp features or or less robust in features preservation; and 2) they usually have many parameters and require tedious parameter tuning. In this paper, we propose a novel deep learning approach that automatically and robustly filters point clouds by removing noise and preserving their sharp features. Our point-wise learning architecture consists of an encoder and a decoder. The encoder directly takes points (a point and its neighbors) as input, and learns a latent representation vector which is going through the decoder to relate the ground-truth position with a displacement vector. The trained neural network can automatically generate a set of clean points from a noisy input. Extensive experiments show that our approach outperforms the state-of-the-art deep learning techniques in terms of both visual quality and quantitative error metrics. We will make our code and dataset publicly available.

[1]  Daniel Cohen-Or,et al.  EC-Net: an Edge-aware Point set Consolidation Network , 2018, ECCV.

[2]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Lizhuang Ma,et al.  Low Rank Matrix Approximation for Geometry Filtering , 2018, ArXiv.

[4]  François Goulette,et al.  Paris-rue-Madame Database - A 3D Mobile Laser Scanner Dataset for Benchmarking Urban Detection, Segmentation and Classification Methods , 2014, ICPRAM.

[5]  Daniel Cohen-Or,et al.  Patch-Based Progressive 3D Point Set Upsampling , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Timo Ropinski,et al.  Total Denoising: Unsupervised Learning of 3D Point Cloud Cleaning , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[7]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[8]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[9]  Yuxin Liu,et al.  Guided point cloud denoising via sharp feature skeletons , 2017, The Visual Computer.

[10]  Nina Amenta,et al.  Defining point-set surfaces , 2004, ACM Trans. Graph..

[11]  Cewu Lu,et al.  Pointwise Rotation-Invariant Network with Adaptive Sampling and 3D Spherical Voxel Convolution , 2020, AAAI.

[12]  Markus H. Gross,et al.  Feature Preserving Point Set Surfaces based on Non‐Linear Kernel Regression , 2009, Comput. Graph. Forum.

[13]  Konrad Polthier,et al.  Anisotropic smoothing of point sets, , 2005, Comput. Aided Geom. Des..

[14]  David Levin,et al.  The approximation power of moving least-squares , 1998, Math. Comput..

[15]  Cohen-OrDaniel,et al.  ℓ1-Sparse reconstruction of sharp point set surfaces , 2010 .

[16]  Michael Wimmer,et al.  Continuous projection for fast L1 reconstruction , 2014, ACM Trans. Graph..

[17]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  D. Levin,et al.  Mesh-Independent Surface Interpolation , 2004 .

[19]  Daniel Cohen-Or,et al.  Parameterization-free projection for geometry reconstruction , 2007, ACM Trans. Graph..

[20]  Markus H. Gross,et al.  PointProNets: Consolidation of Point Clouds with Convolutional Neural Networks , 2018, Comput. Graph. Forum.

[21]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[22]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[23]  Dima Damen,et al.  Recognizing linked events: Searching the space of feasible explanations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Daniel Cohen-Or,et al.  Edge-aware point set resampling , 2013, ACM Trans. Graph..

[25]  Maks Ovsjanikov,et al.  PointCleanNet: Learning to Denoise and Remove Outliers from Dense Point Clouds , 2019, Comput. Graph. Forum.

[26]  Marc Alexa,et al.  Point set surfaces , 2001, Proceedings Visualization, 2001. VIS '01..

[27]  Lizhuang Ma,et al.  PRIN: Pointwise Rotation-Invariant Network , 2018, ArXiv.

[28]  Daniel Cohen-Or,et al.  PU-Net: Point Cloud Upsampling Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Wei Wu,et al.  PointCNN: Convolution On X-Transformed Points , 2018, NeurIPS.

[30]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[31]  Lizhuang Ma,et al.  Low Rank Matrix Approximation for 3D Geometry Filtering , 2020, IEEE Transactions on Visualization and Computer Graphics.

[32]  Wenping Wang,et al.  Denoising point sets via L0 minimization , 2015, Comput. Aided Geom. Des..

[33]  Daniel Cohen-Or,et al.  Consolidation of unorganized point clouds for surface reconstruction , 2009, ACM Trans. Graph..

[34]  Yifan Xu,et al.  SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters , 2018, ECCV.

[35]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[36]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[37]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[38]  Jelena Kovacevic,et al.  3D Point Cloud Denoising via Deep Neural Network Based Local Surface Estimation , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[39]  Yongwei Nie,et al.  Rolling normal filtering for point clouds , 2018, Comput. Aided Geom. Des..

[40]  Daniel Cohen-Or,et al.  ℓ1-Sparse reconstruction of sharp point set surfaces , 2010, TOGS.

[41]  Hao Su,et al.  A Point Set Generation Network for 3D Object Reconstruction from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Ravi Krishna Kolluri,et al.  Provably good moving least squares , 2005, SIGGRAPH Courses.

[43]  Xin Tong,et al.  Mesh denoising via cascaded normal regression , 2016, ACM Trans. Graph..

[44]  Marc Alexa,et al.  Computing and Rendering Point Set Surfaces , 2003, IEEE Trans. Vis. Comput. Graph..

[45]  D. Cohen-Or,et al.  Robust moving least-squares fitting with sharp features , 2005, ACM Trans. Graph..

[46]  Maks Ovsjanikov,et al.  PCPNet Learning Local Shape Properties from Raw Point Clouds , 2017, Comput. Graph. Forum.

[47]  Jun Wang,et al.  Deep Feature-preserving Normal Estimation for Point Cloud Filtering , 2020, Comput. Aided Des..

[48]  Chunxia Xiao,et al.  Efficient feature-preserving local projection operator for geometry reconstruction , 2013, Comput. Aided Des..

[49]  Matthias Zwicker,et al.  GPF: GMM-Inspired Feature-Preserving Point Set Filtering , 2018, IEEE Transactions on Visualization and Computer Graphics.

[50]  Cewu Lu,et al.  PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation , 2018, ArXiv.