A Survey of Surface Reconstruction from Point Clouds

The area of surface reconstruction has seen substantial progress in the past two decades. The traditional problem addressed by surface reconstruction is to recover the digital representation of a physical shape that has been scanned, where the scanned data contain a wide variety of defects. While much of the earlier work has been focused on reconstructing a piece‐wise smooth representation of the original shape, recent work has taken on more specialized priors to address significantly challenging data imperfections, where the reconstruction can take on different representations—not necessarily the explicit geometry. We survey the field of surface reconstruction, and provide a categorization with respect to priors, data imperfections and reconstruction output. By considering a holistic view of surface reconstruction, we show a detailed characterization of the field, highlight similarities between diverse reconstruction techniques and provide directions for future work in surface reconstruction.

[1]  Dan A. Alcantara,et al.  Space-time surface reconstruction using incompressible flow , 2008, SIGGRAPH 2008.

[2]  Leonidas J. Guibas,et al.  Discovering structural regularity in 3D geometry , 2008, SIGGRAPH 2008.

[3]  Cláudio T. Silva,et al.  Provenance for Visualizations: Reproducibility and Beyond , 2007, Computing in Science & Engineering.

[4]  Holger Wendland,et al.  Scattered Data Approximation: Conditionally positive definite functions , 2004 .

[5]  Michael M. Kazhdan,et al.  Screened poisson surface reconstruction , 2013, TOGS.

[6]  Vladlen Koltun,et al.  Single-view reconstruction via joint analysis of image and shape collections , 2015, ACM Trans. Graph..

[7]  Daniel Cohen-Or,et al.  Consolidation of unorganized point clouds for surface reconstruction , 2009, ACM Trans. Graph..

[8]  Ligang Liu,et al.  Analysis, reconstruction and manipulation using arterial snakes , 2010, ACM Trans. Graph..

[9]  Wolfgang Heidrich,et al.  Globally Consistent Space‐Time Reconstruction , 2010, Comput. Graph. Forum.

[10]  Werner Purgathofer,et al.  Reconstructing shape boundaries with multimodal constraints , 2013, Comput. Graph..

[11]  Cláudio T. Silva,et al.  Robust Smooth Feature Extraction from Point Clouds , 2007, IEEE International Conference on Shape Modeling and Applications 2007 (SMI '07).

[12]  Daniel Cohen-Or,et al.  Non-local scan consolidation for 3D urban scenes , 2010, ACM Trans. Graph..

[13]  Michael Wimmer,et al.  Continuous projection for fast L1 reconstruction , 2014, ACM Trans. Graph..

[14]  Daniel Cohen-Or,et al.  Competing Fronts for Coarse–to–Fine Surface Reconstruction , 2006, Comput. Graph. Forum.

[15]  Wolfgang Straßer,et al.  Surface Reconstruction from Fitted Shape Primitives , 2008, VMV.

[16]  Joachim Giesen,et al.  Delaunay Triangulation Based Surface Reconstruction , 2006 .

[17]  N. Mitra,et al.  GlobFit: consistently fitting primitives by discovering global relations , 2011, SIGGRAPH 2011.

[18]  D. Levin,et al.  Mesh-Independent Surface Interpolation , 2004 .

[19]  Pierre Alliez,et al.  Eurographics Symposium on Geometry Processing (2007) Voronoi-based Variational Reconstruction of Unoriented Point Sets , 2022 .

[20]  Thomas A. Funkhouser,et al.  Symmetry factored embedding and distance , 2010, ACM Transactions on Graphics.

[21]  Marc Alexa,et al.  Computing and Rendering Point Set Surfaces , 2003, IEEE Trans. Vis. Comput. Graph..

[22]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[23]  Pierre Alliez,et al.  Noise-adaptive shape reconstruction from raw point sets , 2013, SGP '13.

[24]  Shang-Hong Lai,et al.  Binary Orientation Trees for Volume and Surface Reconstruction from Unoriented Point Clouds , 2010, Comput. Graph. Forum.

[25]  Leonidas J. Guibas,et al.  Estimating surface normals in noisy point cloud data , 2004, Int. J. Comput. Geom. Appl..

[26]  Gabriel Taubin,et al.  SSD: Smooth Signed Distance Surface Reconstruction , 2011, Comput. Graph. Forum.

[27]  Junjie Cao,et al.  Point Cloud Skeletons via Laplacian Based Contraction , 2010, 2010 Shape Modeling International Conference.

[28]  Alexandre Boulch,et al.  Fast and Robust Normal Estimation for Point Clouds with Sharp Features , 2012, Comput. Graph. Forum.

[29]  B. Neubert,et al.  Approximate image-based tree-modeling using particle flows , 2007, SIGGRAPH 2007.

[30]  Kun Zhou,et al.  An interactive approach to semantic modeling of indoor scenes with an RGBD camera , 2012, ACM Trans. Graph..

[31]  Florent Lafarge,et al.  LOD Generation for Urban Scenes , 2015, ACM Trans. Graph..

[32]  Hui Huang,et al.  Proactive 3D scanning of inaccessible parts , 2014, ACM Trans. Graph..

[33]  Shi-Min Hu,et al.  Adaptive partitioning of urban facades , 2011, SA '11.

[34]  I. J. Schoenberg On Hermite-Birkhoff interpolation☆ , 1966 .

[35]  Jianfei Cai,et al.  Robust surface reconstruction via dictionary learning , 2014, ACM Trans. Graph..

[36]  Leif Kobbelt,et al.  Robust reconstruction of watertight 3D models from non-uniformly sampled point clouds without normal information , 2006, SGP '06.

[37]  H. Seidel,et al.  Multi-level partition of unity implicits , 2003 .

[38]  Michael Wimmer,et al.  O-snap , 2013, ACM Trans. Graph..

[39]  Marc Pouget,et al.  Estimating differential quantities using polynomial fitting of osculating jets , 2003, Comput. Aided Geom. Des..

[40]  Niloy J. Mitra,et al.  Symmetry in 3D Geometry: Extraction and Applications , 2013, Comput. Graph. Forum.

[41]  Alexandre Boulch,et al.  Piecewise‐Planar 3D Reconstruction with Edge and Corner Regularization , 2014, Comput. Graph. Forum.

[42]  Richard K. Beatson,et al.  Reconstruction and representation of 3D objects with radial basis functions , 2001, SIGGRAPH.

[43]  Miguel López-Díaz,et al.  Ranking Star-Shaped Valued Mappings with Respect to Shape Variability , 2012, Journal of Mathematical Imaging and Vision.

[44]  Remco C. Veltkamp,et al.  Watertight Scenes from Urban LiDAR and Planar Surfaces , 2013 .

[45]  Paul A. Beardsley,et al.  Image-based reconstruction and synthesis of dense foliage , 2013, ACM Trans. Graph..

[46]  Florent Lafarge,et al.  Surface Reconstruction through Point Set Structuring , 2013, Comput. Graph. Forum.

[47]  James F. O'Brien,et al.  Spectral surface reconstruction from noisy point clouds , 2004, SGP '04.

[48]  Baoquan Chen,et al.  2D‐D Lifting for Shape Reconstruction , 2014, Comput. Graph. Forum.

[49]  Vladlen Koltun,et al.  Depth camera tracking with contour cues , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Ross T. Whitaker,et al.  A Level-Set Approach to 3D Reconstruction from Range Data , 1998, International Journal of Computer Vision.

[51]  Nathan Silberman,et al.  Instance Segmentation of Indoor Scenes Using a Coverage Loss , 2014, ECCV.

[52]  Horst Bischof,et al.  A Globally Optimal Algorithm for Robust TV-L1 Range Image Integration , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[53]  M. Goesele,et al.  Floating scale surface reconstruction , 2014, ACM Trans. Graph..

[54]  Hao Zhang,et al.  Automatic reconstruction of tree skeletal structures from point clouds , 2010, SIGGRAPH 2010.

[55]  D. Cohen-Or,et al.  Robust moving least-squares fitting with sharp features , 2005, ACM Trans. Graph..

[56]  Cláudio T. Silva,et al.  Medial Kernels , 2012, Comput. Graph. Forum.

[57]  Leonidas J. Guibas,et al.  Example-Based 3D Scan Completion , 2005 .

[58]  Dieter Fox,et al.  DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Marc Alexa,et al.  Interpolatory point set surfaces—convexity and Hermite data , 2009, TOGS.

[60]  Daniel Cohen-Or,et al.  L1-medial skeleton of point cloud , 2013, ACM Trans. Graph..

[61]  Daniel Cohen-Or,et al.  2D-3D fusion for layer decomposition of urban facades , 2011, 2011 International Conference on Computer Vision.

[62]  Tony DeRose,et al.  Surface reconstruction from unorganized points , 1992, SIGGRAPH.

[63]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[64]  Sven Oesau,et al.  Planar Shape Detection and Regularization in Tandem , 2016, Comput. Graph. Forum.

[65]  D. Cohen-Or,et al.  Curve skeleton extraction from incomplete point cloud , 2009, SIGGRAPH 2009.

[66]  W. Zhu,et al.  Feature‐Preserving Surface Reconstruction From Unoriented, Noisy Point Data , 2013, Comput. Graph. Forum.

[67]  Daniel G. Aliaga,et al.  A Survey of Urban Reconstruction , 2013, Comput. Graph. Forum.

[68]  Markus H. Gross,et al.  Feature Preserving Point Set Surfaces based on Non‐Linear Kernel Regression , 2009, Comput. Graph. Forum.

[69]  Gabriel Taubin,et al.  A benchmark for surface reconstruction , 2013, TOGS.

[70]  Marc Pollefeys,et al.  Live Metric 3D Reconstruction on Mobile Phones , 2013, 2013 IEEE International Conference on Computer Vision.

[71]  Ke Xie,et al.  A search-classify approach for cluttered indoor scene understanding , 2012, ACM Trans. Graph..

[72]  Matthew J. Sottile,et al.  Curve and surface reconstruction: algorithms with mathematical analysis by Tamal K. Dey Cambridge University Press , 2010, SIGA.

[73]  Jean-Philippe Pons,et al.  Robust piecewise-planar 3D reconstruction and completion from large-scale unstructured point data , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[74]  Lijun Yin,et al.  A Global Parity Measure for Incomplete Point Cloud Data , 2012, Comput. Graph. Forum.

[75]  Markus H. Gross,et al.  Multi‐scale Feature Extraction on Point‐Sampled Surfaces , 2003, Comput. Graph. Forum.

[76]  Hao Li,et al.  Realtime performance-based facial animation , 2011, ACM Trans. Graph..

[77]  Marc Levoy,et al.  The digital Michelangelo project: 3D scanning of large statues , 2000, SIGGRAPH.

[78]  Ghassan Hamarneh,et al.  A Survey on Shape Correspondence , 2011, Comput. Graph. Forum.

[79]  Victor S. Lempitsky,et al.  Global Optimization for Shape Fitting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[80]  Nassir Navab,et al.  Model globally, match locally: Efficient and robust 3D object recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[81]  Charlie C. L. Wang,et al.  Orienting unorganized points for surface reconstruction , 2010, Comput. Graph..

[82]  Pierre Alliez,et al.  Signing the Unsigned: Robust Surface Reconstruction from Raw Pointsets , 2010, Comput. Graph. Forum.

[83]  M. Gross,et al.  Algebraic point set surfaces , 2007, SIGGRAPH 2007.

[84]  Daniel G. Aliaga,et al.  Automatic Extraction of Manhattan-World Building Masses from 3D Laser Range Scans , 2012, IEEE Transactions on Visualization and Computer Graphics.

[85]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[86]  N. Amenta,et al.  Defining point-set surfaces , 2004, SIGGRAPH 2004.

[87]  Cláudio T. Silva,et al.  Direct (Re)Meshing for Efficient Surface Processing , 2006, Comput. Graph. Forum.

[88]  Niloy J. Mitra,et al.  RAPter , 2015, ACM Trans. Graph..

[89]  Josiah Manson,et al.  Streaming Surface Reconstruction Using Wavelets , 2008, Comput. Graph. Forum.

[90]  Ligang Liu,et al.  Analysis, reconstruction and manipulation using arterial snakes , 2010, SIGGRAPH 2010.

[91]  Marc Alexa,et al.  On Normals and Projection Operators for Surfaces Defined by Point Sets , 2004, PBG.

[92]  Daniel Cohen-Or,et al.  Eurographics Symposium on Geometry Processing (2007) Data-dependent Mls for Faithful Surface Approximation , 2022 .

[93]  Hao Wang,et al.  Bandwidth Selection and Reconstruction Quality in Point-Based Surfaces , 2009, IEEE Transactions on Visualization and Computer Graphics.

[94]  Niloy J. Mitra,et al.  Visibility of noisy point cloud data , 2010, Comput. Graph..

[95]  Paolo Cignoni,et al.  3D Models for Cultural Heritage: Beyond Plain Visualization , 2011, Computer.

[96]  Cláudio T. Silva,et al.  Provenance for Computational Tasks: A Survey , 2008, Computing in Science & Engineering.

[97]  Pierre Alliez,et al.  Noname manuscript No. (will be inserted by the editor) Feature-Preserving Surface Reconstruction and Simplification from Defect-Laden Point Sets , 2013 .

[98]  Hans-Peter Seidel,et al.  Shape Analysis with Subspace Symmetries , 2011, Comput. Graph. Forum.

[99]  James F. O'Brien,et al.  Interpolating and approximating implicit surfaces from polygon soup , 2005, SIGGRAPH Courses.

[100]  H. Tal-Ezer,et al.  Parameterization-free projection for geometry reconstruction , 2007, SIGGRAPH 2007.

[101]  Wolfgang Straßer,et al.  Patch-Graph Reconstruction for Piecewise Smooth Surfaces , 2008, VMV.

[102]  Daniel Cohen-Or,et al.  Morfit: interactive surface reconstruction from incomplete point clouds with curve-driven topology and geometry control , 2014, ACM Trans. Graph..

[103]  Reinhard Klein,et al.  Completion and Reconstruction with Primitive Shapes , 2009, Comput. Graph. Forum.

[104]  Leonidas J. Guibas,et al.  Acquiring 3D indoor environments with variability and repetition , 2012, ACM Trans. Graph..

[105]  Jean-Philippe Pons,et al.  Robust and Efficient Surface Reconstruction From Range Data , 2009, Comput. Graph. Forum.

[106]  Luiz Velho,et al.  Hermite Radial Basis Functions Implicits , 2011, Comput. Graph. Forum.

[107]  Reinhard Klein,et al.  Efficient RANSAC for Point‐Cloud Shape Detection , 2007, Comput. Graph. Forum.

[108]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[109]  Ariel Shamir,et al.  Cone carving for surface reconstruction , 2010, SIGGRAPH 2010.

[110]  Leonidas J. Guibas,et al.  Guided Real‐Time Scanning of Indoor Objects , 2013, Comput. Graph. Forum.

[111]  Yutaka Ohtake,et al.  Smoothing of Partition of Unity Implicit Surfaces for Noise Robust Surface Reconstruction , 2009, Comput. Graph. Forum.

[112]  Jean-Michel Morel,et al.  Scale Space Meshing of Raw Data Point Sets , 2011, Comput. Graph. Forum.

[113]  Bo Fu,et al.  Data-Driven Flower Petal Modeling with Botany Priors , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[114]  Ronen Basri,et al.  Direct visibility of point sets , 2007, ACM Trans. Graph..

[115]  Leonidas J. Guibas,et al.  Uncertainty and Variability in Point Cloud Surface Data , 2004, PBG.

[116]  Daniel Cohen-Or,et al.  SmartBoxes for interactive urban reconstruction , 2010, ACM Transactions on Graphics.

[117]  Fabio Pellacini,et al.  MeshFlow : interactive visualization of mesh construction sequences , 2011, SIGGRAPH 2011.

[118]  Leonidas J. Guibas,et al.  Estimating image depth using shape collections , 2014, ACM Trans. Graph..

[119]  陈宝权 GlobFit: Consistently Fitting Primitives by Discovering Global Relations , 2011 .

[120]  Daniel Cohen-Or,et al.  ℓ1-Sparse reconstruction of sharp point set surfaces , 2010, TOGS.

[121]  Yutaka Ohtake,et al.  Tomographic surface reconstruction from point cloud , 2015, Comput. Graph..

[122]  M. Goesele,et al.  Fusion of depth maps with multiple scales , 2011, ACM Trans. Graph..

[123]  B. Li,et al.  A Survey of Methods for Moving Least Squares Surfaces , 2008, VG/PBG@SIGGRAPH.

[124]  D. Cohen-Or,et al.  Interactive topology-aware surface reconstruction , 2007, ACM Trans. Graph..

[125]  Daniel G. Aliaga,et al.  Urban ecosystem design , 2011, SI3D.

[126]  Pascal Barla,et al.  Non‐Oriented MLS Gradient Fields , 2013, Comput. Graph. Forum.

[127]  Shang-Hong Lai,et al.  Bipartite Polar Classification for Surface Reconstruction , 2011, Comput. Graph. Forum.

[128]  P. Prusinkiewicz,et al.  Modeling and visualization of leaf venation patterns , 2005, SIGGRAPH 2005.

[129]  Shi-Min Hu,et al.  Structure recovery by part assembly , 2012, ACM Trans. Graph..

[130]  Andrei Sharf,et al.  Grammar-based 3D facade segmentation and reconstruction , 2012, Comput. Graph..

[131]  Jianxiong Xiao,et al.  Reconstructing the World's Museums , 2012, ECCV.

[132]  Daniel Cohen-Or,et al.  Edge-aware point set resampling , 2013, ACM Trans. Graph..

[133]  Daniel Cohen-Or,et al.  Error bounds and optimal neighborhoods for MLS approximation , 2006, SGP '06.

[134]  Daniel Cohen-Or,et al.  Quality-driven poisson-guided autoscanning , 2014, ACM Trans. Graph..

[135]  Michael M. Kazhdan,et al.  Reconstruction of solid models from oriented point sets , 2005, SGP '05.

[136]  Ghassan Hamarneh,et al.  VASE: Volume‐Aware Surface Evolution for Surface Reconstruction from Incomplete Point Clouds , 2011, Comput. Graph. Forum.

[137]  Matthias Nießner,et al.  Real-time 3D reconstruction at scale using voxel hashing , 2013, ACM Trans. Graph..

[138]  Ulrich Neumann,et al.  2.5D building modeling by discovering global regularities , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[139]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[140]  Matthias Nießner,et al.  Shading-based refinement on volumetric signed distance functions , 2015, ACM Trans. Graph..

[141]  Leonidas J. Guibas,et al.  Database‐Assisted Object Retrieval for Real‐Time 3D Reconstruction , 2015, Comput. Graph. Forum.