View-Guided Point Cloud Completion

This paper presents a view-guided solution for the task of point cloud completion. Unlike most existing methods directly inferring the missing points using shape priors, we address this task by introducing ViPC (view-guided point cloud completion) that takes the missing crucial global structure information from an extra single-view image. By leveraging a framework that sequentially performs effective cross-modality and cross-level fusions, our method achieves significantly superior results over typical existing solutions on a new large-scale dataset we collect for the view-guided point cloud completion task.

[1]  Jian Liang,et al.  Point Cloud Segmentation and Denoising via Constrained Nonlinear Least Squares Normal Estimates , 2013, Innovations for Shape Analysis, Models and Algorithms.

[2]  Matthias Nießner,et al.  Shape Completion Using 3D-Encoder-Predictor CNNs and Shape Synthesis , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Shinpei Kato,et al.  Autoware on Board: Enabling Autonomous Vehicles with Embedded Systems , 2018, 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS).

[4]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[5]  Yi Lin,et al.  LiDAR: An important tool for next-generation phenotyping technology of high potential for plant phenomics? , 2015, Comput. Electron. Agric..

[6]  Badong Chen,et al.  Robust rigid registration algorithm based on pointwise correspondence and correntropy , 2020, Pattern Recognit. Lett..

[7]  D. Bertsekas.  A distributed asynchronous relaxation algorithm for the assignment problem , 1985, 1985 24th IEEE Conference on Decision and Control.

[8]  Martial Hebert,et al.  PCN: Point Completion Network , 2018, 2018 International Conference on 3D Vision (3DV).

[9]  Duc Thanh Nguyen,et al.  A Field Model for Repairing 3D Shapes , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Silvio Savarese,et al.  3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction , 2016, ECCV.

[11]  Wei Hu,et al.  Local Frequency Interpretation and Non-Local Self-Similarity on Graph for Point Cloud Inpainting , 2018, IEEE Transactions on Image Processing.

[12]  M. Lo Brutto,et al.  Computer Vision Tools for 3D Modelling in Archaeology , 2012 .

[13]  Reinhard Klein,et al.  Completion and Reconstruction with Primitive Shapes , 2009, Comput. Graph. Forum.

[14]  Shangchen Zhou,et al.  GRNet: Gridding Residual Network for Dense Point Cloud Completion , 2020, ECCV.

[15]  Zhen Li,et al.  High-Resolution Shape Completion Using Deep Neural Networks for Global Structure and Local Geometry Inference , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Martin Rutzinger,et al.  Data infrastructure for multitemporal airborne LiDAR point cloud analysis - Examples from physical geography in high mountain environments , 2014, Comput. Environ. Urban Syst..

[17]  Siddhartha Chaudhuri,et al.  A probabilistic model for component-based shape synthesis , 2012, ACM Trans. Graph..

[18]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Hao Su,et al.  A Point Set Generation Network for 3D Object Reconstruction from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Wei Liu,et al.  Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images , 2018, ECCV.

[21]  Derek Hoiem,et al.  Completing 3D object shape from one depth image , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Mathieu Aubry,et al.  AtlasNet: A Papier-M\^ach\'e Approach to Learning 3D Surface Generation , 2018, CVPR 2018.

[24]  Stephen DiVerdi,et al.  Learning part-based templates from large collections of 3D shapes , 2013, ACM Trans. Graph..

[25]  Sebastian Thrun,et al.  Shape from symmetry , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[26]  Michael M. Kazhdan,et al.  Screened poisson surface reconstruction , 2013, TOGS.

[27]  Neil A. Dodgson,et al.  Fast Marching farthest point sampling , 2003, Eurographics.

[28]  Daniel Cohen-Or,et al.  Morfit: interactive surface reconstruction from incomplete point clouds with curve-driven topology and geometry control , 2014, ACM Trans. Graph..

[29]  Marc Alexa,et al.  Laplacian mesh optimization , 2006, GRAPHITE '06.

[30]  Leonidas J. Guibas,et al.  Example-Based 3D Scan Completion , 2005 .

[31]  Daniel Cohen-Or,et al.  Patch-Based Progressive 3D Point Set Upsampling , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Leonidas J. Guibas,et al.  Discovering structural regularity in 3D geometry , 2008, ACM Trans. Graph..

[33]  Thomas Brox,et al.  What Do Single-View 3D Reconstruction Networks Learn? , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Abderrahim Elmoataz,et al.  Partial Difference Operators on Weighted Graphs for Image Processing on Surfaces and Point Clouds , 2014, IEEE Transactions on Image Processing.

[35]  Roland Siegwart,et al.  A Review of Point Cloud Registration Algorithms for Mobile Robotics , 2015, Found. Trends Robotics.

[36]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[37]  Richard A. Newcombe,et al.  DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Paul J. Besl,et al.  Method for registration of 3-D shapes , 1992, Other Conferences.

[39]  Pierre Alliez,et al.  State of the Art in Surface Reconstruction from Point Clouds , 2014, Eurographics.

[40]  Silvio Savarese,et al.  TopNet: Structural Point Cloud Decoder , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Dong Tian,et al.  FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[42]  Jean-Philippe Pons,et al.  Robust piecewise-planar 3D reconstruction and completion from large-scale unstructured point data , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.