Revisiting Point Cloud Simplification: A Learnable Feature Preserving Approach

The recent advances in 3D sensing technology have made possible the capture of point clouds in significantly high resolution. However, increased detail usually comes at the expense of high storage, as well as computational costs in terms of processing and visualization operations. Mesh and Point Cloud simplification methods aim to reduce the complexity of 3D models while retaining visual quality and relevant salient features. Traditional simplification techniques usually rely on solving a time-consuming optimization problem, hence they are impractical for large-scale datasets. In an attempt to alleviate this computational burden, we propose a fast point cloud simplification method by learning to sample salient points. The proposed method relies on a graph neural network architecture trained to select an arbitrary, user-defined, number of points from the input space and to re-arrange their positions so as to minimize the visual perception error. The approach is extensively evaluated on various datasets using several perceptual metrics. Importantly, our method is able to generalize to out-of-distribution shapes, hence demonstrating zero-shot capabilities.

[1]  William E. Lorensen,et al.  Decimation of triangle meshes , 1992, SIGGRAPH.

[2]  Sun-Jeong Kim,et al.  Discrete differential error metric for surface simplification , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[3]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Guillaume Lavoué,et al.  A Multiscale Metric for 3D Mesh Visual Quality Assessment , 2011, Comput. Graph. Forum.

[5]  Maks Ovsjanikov,et al.  Spectral coarsening of geometric operators , 2019, ACM Trans. Graph..

[6]  Christopher DeCoro,et al.  Real-time mesh simplification using the GPU , 2007, SI3D.

[7]  Anup Basu,et al.  Perceptual Quality Metrics for 3D Meshes: Towards an Optimal Multi-attribute Computational Model , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[8]  Shiquan Qiao,et al.  Feature-Preserved Point Cloud Simplification Based on Natural Quadric Shape Models , 2019, Applied Sciences.

[9]  Liu Xiaoli,et al.  Edge Collapse Simplification Based on Sharp Degree , 2005 .

[10]  David W. Jacobs,et al.  Mesh saliency , 2005, ACM Trans. Graph..

[11]  Stefanos Zafeiriou,et al.  Learning to Generate Customized Dynamic 3D Facial Expressions , 2020, ECCV.

[12]  Marc Alexa,et al.  Point set surfaces , 2001, Proceedings Visualization, 2001. VIS '01..

[13]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Guillaume Lavoué,et al.  A local roughness measure for 3D meshes and its application to visual masking , 2009, TAP.

[15]  Gary W. Meyer,et al.  Perceptually Guided Polygon Reduction , 2008, IEEE Transactions on Visualization and Computer Graphics.

[16]  Paolo Cignoni,et al.  A comparison of mesh simplification algorithms , 1998, Comput. Graph..

[17]  Gabriel Taubin,et al.  The ball-pivoting algorithm for surface reconstruction , 1999, IEEE Transactions on Visualization and Computer Graphics.

[18]  Kai Wang,et al.  A Curvature Tensor Distance for Mesh Visual Quality Assessment , 2012, ICCVG.

[19]  Markus H. Gross,et al.  Efficient simplification of point-sampled surfaces , 2002, IEEE Visualization, 2002. VIS 2002..

[20]  Alexander M. Bronstein,et al.  Numerical Geometry of Non-Rigid Shapes , 2009, Monographs in Computer Science.

[21]  Inderjit S. Dhillon,et al.  Weighted Graph Cuts without Eigenvectors A Multilevel Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[23]  Touradj Ebrahimi,et al.  Perceptually driven 3D distance metrics with application to watermarking , 2006, SPIE Optics + Photonics.

[24]  Paolo Cignoni,et al.  MeshLab: an Open-Source Mesh Processing Tool , 2008, Eurographics Italian Chapter Conference.

[25]  Peter Lindstrom,et al.  Out-of-core simplification of large polygonal models , 2000, SIGGRAPH.

[26]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[27]  Nallig Leal,et al.  A Linear Programming Approach for 3 D Point Cloud Simplification , 2022 .

[28]  Rémi Ronfard,et al.  Full‐range approximation of triangulated polyhedra. , 1996, Comput. Graph. Forum.

[29]  Pierre Alliez,et al.  Variational shape approximation , 2004, ACM Trans. Graph..

[30]  Jure Leskovec,et al.  Hierarchical Graph Representation Learning with Differentiable Pooling , 2018, NeurIPS.

[31]  Libor Vása,et al.  Dihedral Angle Mesh Error: a fast perception correlated distortion measure for fixed connectivity triangle meshes , 2012, Comput. Graph. Forum.

[32]  W. Jeong,et al.  LOD Generation with Discrete Curvature Error Metric , 2001 .

[33]  Simplifying surfaces with color and texture using quadric error metrics , 1998, VIS '98.

[34]  Jarek Rossignac,et al.  Multi-resolution 3D approximations for rendering complex scenes , 1993, Modeling in Computer Graphics.

[35]  Shuiwang Ji,et al.  Graph U-Nets , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  David P. Luebke,et al.  A Developer's Survey of Polygonal Simplification Algorithms , 2001, IEEE Computer Graphics and Applications.

[37]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[38]  Libor Vása,et al.  Perceptual Metrics for Static and Dynamic Triangle Meshes , 2013, Eurographics.

[39]  Yehoshua Y. Zeevi,et al.  The farthest point strategy for progressive image sampling , 1997, IEEE Trans. Image Process..

[40]  Jonathan Li,et al.  Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[41]  Luigi Maria Galantucci,et al.  A Multilevel Approach to Edge Detection in Tessellated Point Clouds , 2005 .

[42]  Thomas Lu,et al.  A single-camera system captures high-resolution 3D images in one shot , 2006 .

[43]  Wei Li,et al.  An Improved Decimation of Triangle Meshes Based on Curvature , 2014, RSKT.

[44]  Hugues Hoppe,et al.  Progressive meshes , 1996, SIGGRAPH.

[45]  Jaewoo Kang,et al.  Self-Attention Graph Pooling , 2019, ICML.

[46]  Pietro Liò,et al.  Towards Sparse Hierarchical Graph Classifiers , 2018, ArXiv.

[47]  Stefanos Zafeiriou,et al.  AvatarMe: Realistically Renderable 3D Facial Reconstruction “In-the-Wild” , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Yaser Sheikh,et al.  LBS Autoencoder: Self-Supervised Fitting of Articulated Meshes to Point Clouds , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  George Pavlidis,et al.  Methods for 3D digitization of Cultural Heritage , 2007 .

[50]  Maks Ovsjanikov,et al.  Spectral Mesh Simplification , 2020, Comput. Graph. Forum.

[51]  Massimiliano Corsini,et al.  A Comparison of Perceptually-Based Metrics for Objective Evaluation of Geometry Processing , 2010, IEEE Transactions on Multimedia.

[52]  Xie Han,et al.  Point cloud simplification with preserved edge based on normal vector , 2015 .

[53]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[54]  Pengfei Li,et al.  A perceptual quality metric for 3D triangle meshes based on spatial pooling , 2018, Frontiers of Computer Science.

[55]  Qing Liu,et al.  Adaptive simplification of point cloud using k-means clustering , 2011, Comput. Aided Des..

[56]  Weisi Lin,et al.  Perceptual Quality Assessment for 3D Triangle Mesh Based on Curvature , 2015, IEEE Transactions on Multimedia.

[57]  Craig Gotsman,et al.  Spectral compression of mesh geometry , 2000, EuroCG.

[58]  Tony DeRose,et al.  Surface reconstruction from unorganized points , 1992, SIGGRAPH.

[59]  B. Prabhakaran,et al.  Designing and Evaluating a Mesh Simplification Algorithm for Virtual Reality , 2018, ACM Trans. Multim. Comput. Commun. Appl..

[60]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[61]  Shai Avidan,et al.  Learning to Sample , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  Heinrich Müller,et al.  SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[63]  Hui Wang,et al.  Fast Mesh Simplification Method for Three-Dimensional Geometric Models with Feature-Preserving Efficiency , 2019, Sci. Program..

[64]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[65]  David Levin,et al.  Surface simplification using a discrete curvature norm , 2002, Comput. Graph..

[66]  Itai Lang,et al.  SampleNet: Differentiable Point Cloud Sampling , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[67]  L. Yao,et al.  Quadratic Error Metric Mesh Simplification Algorithm Based on Discrete Curvature , 2015 .

[68]  Hao Zhang,et al.  DR-KFS: A Differentiable Visual Similarity Metric for 3D Shape Reconstruction , 2020, ECCV.

[69]  A. Ponniah,et al.  Large Scale 3D Morphable Models , 2017, International Journal of Computer Vision.

[70]  Tolga K. Çapin,et al.  A machine learning framework for full-reference 3D shape quality assessment , 2018, The Visual Computer.

[71]  Yinda Zhang,et al.  Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).