PCN: Point Completion Network

Shape completion, the problem of estimating the complete geometry of objects from partial observations, lies at the core of many vision and robotics applications. In this work, we propose Point Completion Network (PCN), a novel learning-based approach for shape completion. Unlike existing shape completion methods, PCN directly operates on raw point clouds without any structural assumption (e.g. symmetry) or annotation (e.g. semantic class) about the underlying shape. It features a decoder design that enables the generation of fine-grained completions while maintaining a small number of parameters. Our experiments show that PCN produces dense, complete point clouds with realistic structures in the missing regions on inputs with various levels of incompleteness and noise, including cars from LiDAR scans in the KITTI dataset.

[1]  Shi-Min Hu,et al.  Structure recovery by part assembly , 2012, ACM Trans. Graph..

[2]  David Meger,et al.  Improved Adversarial Systems for 3D Object Generation and Reconstruction , 2017, CoRL.

[3]  陈宝权 GlobFit: Consistently Fitting Primitives by Discovering Global Relations , 2011 .

[4]  Subhransu Maji,et al.  SPLATNet: Sparse Lattice Networks for Point Cloud Processing , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Feng Han,et al.  Bottom-Up/Top-Down Image Parsing with Attribute Grammar , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Niloy J. Mitra,et al.  Symmetry in 3D Geometry: Extraction and Applications , 2013, Comput. Graph. Forum.

[7]  Matthias Nießner,et al.  Shape Completion Using 3D-Encoder-Predictor CNNs and Shape Synthesis , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[9]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Leonidas J. Guibas,et al.  Discovering structural regularity in 3D geometry , 2008, SIGGRAPH 2008.

[11]  Michael M. Kazhdan,et al.  Screened poisson surface reconstruction , 2013, TOGS.

[12]  Tianjia Shao,et al.  Shape Completion from a Single RGBD Image , 2017, IEEE Transactions on Visualization and Computer Graphics.

[13]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[14]  Duc Thanh Nguyen,et al.  A Field Model for Repairing 3D Shapes , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[16]  Silvio Savarese,et al.  Joint 2D-3D-Semantic Data for Indoor Scene Understanding , 2017, ArXiv.

[17]  Alexander M. Bronstein,et al.  Deformable Shape Completion with Graph Convolutional Autoencoders , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Zhen Li,et al.  Title High Resolution Shape Completion Using Deep Neural Networksfor Global Structure and Local Geometry Inference , 2017 .

[19]  Leonidas J. Guibas,et al.  Learning Representations and Generative Models for 3D Point Clouds , 2017, ICML.

[20]  Pierre Alliez,et al.  State of the Art in Surface Reconstruction from Point Clouds , 2014, Eurographics.

[21]  Wei Zhao,et al.  A Robust Hole-Filling Algorithm for Triangular Mesh , 2007, CAD/Graphics.

[22]  Hao Su,et al.  A Point Set Generation Network for 3D Object Reconstruction from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Baoquan Chen,et al.  PointCNN: Convolution On $\mathcal{X}$-Transformed Points , 2018 .

[24]  Baoquan Chen,et al.  PointCNN , 2018, NIPS 2018.

[25]  Didier Stricker,et al.  Learning Quadrangulated Patches for 3D Shape Parameterization and Completion , 2017, 2017 International Conference on 3D Vision (3DV).

[26]  Leonidas J. Guibas,et al.  Data-driven structural priors for shape completion , 2015, ACM Trans. Graph..

[27]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[28]  Steve Marschner,et al.  Filling holes in complex surfaces using volumetric diffusion , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[29]  D. Bertsekas A distributed asynchronous relaxation algorithm for the assignment problem , 1985, 1985 24th IEEE Conference on Decision and Control.

[30]  Leonidas J. Guibas,et al.  Database‐Assisted Object Retrieval for Real‐Time 3D Reconstruction , 2015, Comput. Graph. Forum.

[31]  Daniel Cohen-Or,et al.  Least-squares meshes , 2004, Proceedings Shape Modeling Applications, 2004..

[32]  Daniel Cohen-Or,et al.  SmartBoxes for interactive urban reconstruction , 2010, ACM Transactions on Graphics.

[33]  Jitendra Malik,et al.  Aligning 3D models to RGB-D images of cluttered scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Oliver Grau,et al.  VConv-DAE: Deep Volumetric Shape Learning Without Object Labels , 2016, ECCV Workshops.

[35]  Dong Tian,et al.  FoldingNet: Interpretable Unsupervised Learning on 3D Point Clouds , 2017, ArXiv.

[36]  Luc Van Gool,et al.  Bayesian Grammar Learning for Inverse Procedural Modeling , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Derek Hoiem,et al.  Completing 3D object shape from one depth image , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Leonidas J. Guibas,et al.  Example-Based 3D Scan Completion , 2005 .

[39]  Daniel Cohen-Or,et al.  Morfit: interactive surface reconstruction from incomplete point clouds with curve-driven topology and geometry control , 2014, ACM Trans. Graph..

[40]  Reinhard Klein,et al.  Completion and Reconstruction with Primitive Shapes , 2009, Comput. Graph. Forum.

[41]  Matthias Nießner,et al.  ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Marc Alexa,et al.  Laplacian mesh optimization , 2006, GRAPHITE '06.

[43]  Ligang Liu,et al.  Analysis, reconstruction and manipulation using arterial snakes , 2010, ACM Trans. Graph..

[44]  Raquel Urtasun,et al.  Deep Parametric Continuous Convolutional Neural Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[45]  Stephen DiVerdi,et al.  Learning part-based templates from large collections of 3D shapes , 2013, ACM Trans. Graph..

[46]  Jean-Philippe Pons,et al.  Robust piecewise-planar 3D reconstruction and completion from large-scale unstructured point data , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[47]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[48]  Kun Zhou,et al.  An interactive approach to semantic modeling of indoor scenes with an RGBD camera , 2012, ACM Trans. Graph..

[49]  Andreas Geiger,et al.  Learning 3D Shape Completion from Laser Scan Data with Weak Supervision , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[50]  Ke Xie,et al.  A search-classify approach for cluttered indoor scene understanding , 2012, ACM Trans. Graph..

[51]  Dong Tian,et al.  FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[52]  Bo Yang,et al.  3D Object Dense Reconstruction from a Single Depth View , 2018, ArXiv.

[53]  Paul J. Besl,et al.  Method for registration of 3-D shapes , 1992, Other Conferences.

[54]  Siddhartha Chaudhuri,et al.  A probabilistic model for component-based shape synthesis , 2012, ACM Trans. Graph..

[55]  Sebastian Thrun,et al.  Shape from symmetry , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[56]  Vladlen Koltun,et al.  Tangent Convolutions for Dense Prediction in 3D , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[57]  Thomas A. Funkhouser,et al.  Semantic Scene Completion from a Single Depth Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Chad DeChant,et al.  Shape completion enabled robotic grasping , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[59]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[60]  Tobias Schreck,et al.  Approximate Symmetry Detection in Partial 3D Meshes , 2014, Comput. Graph. Forum.

[61]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[62]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.