An Analysis of Reconstruction Algorithms Applied to 3D Building Modeling

Objectives: The process of reconstructing a city cannot be fully automated. More human intervention is required to achieve high quality mass modeling. Our objectives are to provide a guide for the 3D graphic designers and 3D modelers in developing 3D building models. Methods: We present an up-to-date survey of the reconstruction algorithms which are further classified as point cloud based reconstruction methods and structure-aware shape reconstruction methods. We introduce various existing 3D reconstruction techniques and have compared those techniques based on shape simplification, surface completion and shape modeling methodologies. Findings: We have identified the pipeline process and technical issues encountered in each phase of the reconstruction process from a given image layout. As stated, the inverse procedural reconstruction approaches are best oriented towards the reconstruction of buildings from a given input 2D image. Also, there is a lack of systems that are capable of generating procedural rules from user interaction. Improvements: We propose a new 3D reconstruction methodology and novel framework that combines component based structure capturing and rule based modeling techniques that would best be suitable for 3D building modeling, resembling the real world buildings in future systems.

[1]  Reinhard Klein,et al.  Efficient RANSAC for Point‐Cloud Shape Detection , 2007, Comput. Graph. Forum.

[2]  Peter Wonka,et al.  Interactive architectural modeling with procedural extrusions , 2011, TOGS.

[3]  Sven Oesau,et al.  Indoor Scene Reconstruction using Primitive-driven Space Partitioning and Graph-cut , 2013, UDMV.

[4]  Iasonas Kokkinos,et al.  Shape grammar parsing via Reinforcement Learning , 2011, CVPR 2011.

[5]  Daniel Cohen-Or,et al.  Surface reconstruction using local shape priors , 2007, Symposium on Geometry Processing.

[6]  Niloy J. Mitra,et al.  Symmetry in 3D Geometry: Extraction and Applications , 2013, Comput. Graph. Forum.

[7]  Daniel Cohen-Or,et al.  L1-medial skeleton of point cloud , 2013, ACM Trans. Graph..

[8]  Daniel Cohen-Or,et al.  2D-3D fusion for layer decomposition of urban facades , 2011, 2011 International Conference on Computer Vision.

[9]  Jean-Philippe Pons,et al.  Robust piecewise-planar 3D reconstruction and completion from large-scale unstructured point data , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Daniel Cohen-Or,et al.  Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering , 2011, ACM Trans. Graph..

[11]  M. V. D. Panne,et al.  Joint-aware manipulation of deformable models , 2009, SIGGRAPH 2009.

[12]  Daniel Cohen-Or,et al.  Curve skeleton extraction from incomplete point cloud , 2009, ACM Trans. Graph..

[13]  Hans-Peter Seidel,et al.  Shape Analysis with Subspace Symmetries , 2011, Comput. Graph. Forum.

[14]  Marc Alexa,et al.  Computing and Rendering Point Set Surfaces , 2003, IEEE Trans. Vis. Comput. Graph..

[15]  Leif Kobbelt,et al.  Linear Analysis of Nonlinear Constraints for Interactive Geometric Modeling , 2012, Comput. Graph. Forum.

[16]  Daniel G. Aliaga,et al.  Building reconstruction using manhattan-world grammars , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Dong-Ming Yan,et al.  Inverse procedural modeling of facade layouts , 2013, ACM Trans. Graph..

[18]  Cláudio T. Silva,et al.  Direct (Re)Meshing for Efficient Surface Processing , 2006, Comput. Graph. Forum.

[19]  Luc Van Gool,et al.  Bayesian Grammar Learning for Inverse Procedural Modeling , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Daniel Cohen-Or,et al.  ℓ1-Sparse reconstruction of sharp point set surfaces , 2010, TOGS.

[21]  Oliver Deussen,et al.  Approximate image-based tree-modeling using particle flows , 2007, ACM Trans. Graph..

[22]  Ariel Shamir,et al.  On‐the‐fly Curve‐skeleton Computation for 3D Shapes , 2007, Comput. Graph. Forum.

[23]  Ariel Shamir,et al.  A survey on Mesh Segmentation Techniques , 2008, Comput. Graph. Forum.

[24]  Cohen-OrDaniel,et al.  ℓ1-Sparse reconstruction of sharp point set surfaces , 2010 .

[25]  Pascal Müller Procedural modeling of cities , 2006, SIGGRAPH Courses.

[26]  Reinhard Klein,et al.  Completion and Reconstruction with Primitive Shapes , 2009, Comput. Graph. Forum.

[27]  Markus H. Gross,et al.  Algebraic point set surfaces , 2007, ACM Trans. Graph..

[28]  Werner Purgathofer,et al.  Reconstructing shape boundaries with multimodal constraints , 2013, Comput. Graph..

[29]  Stephen DiVerdi,et al.  Exploring collections of 3D models using fuzzy correspondences , 2012, ACM Trans. Graph..

[30]  Jean-Philippe Pons,et al.  Robust and Efficient Surface Reconstruction From Range Data , 2009, Comput. Graph. Forum.

[31]  Florent Lafarge,et al.  Surface Reconstruction through Point Set Structuring , 2013, Comput. Graph. Forum.

[32]  Pat Hanrahan,et al.  Example-based synthesis of 3D object arrangements , 2012, ACM Trans. Graph..

[33]  U Flemming,et al.  More Than the Sum of Parts: The Grammar of Queen Anne Houses , 1987 .

[34]  Ligang Liu,et al.  Analysis, reconstruction and manipulation using arterial snakes , 2010, ACM Trans. Graph..

[35]  Greg Turk,et al.  Reconstructing surfaces using anisotropic basis functions , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[36]  Hans-Peter Seidel,et al.  An algebraic model for parameterized shape editing , 2012, ACM Trans. Graph..

[37]  Ross T. Whitaker,et al.  A Level-Set Approach to 3D Reconstruction from Range Data , 1998, International Journal of Computer Vision.

[38]  Daniel Cohen-Or,et al.  Competing Fronts for Coarse–to–Fine Surface Reconstruction , 2006, Comput. Graph. Forum.

[39]  N. Mitra,et al.  Non-local scan consolidation for 3D urban scenes , 2010, ACM Trans. Graph..

[40]  Marco Attene,et al.  Characterization of 3D shape parts for semantic annotation , 2009, Comput. Aided Des..

[41]  Levent Burak Kara,et al.  Co-abstraction of shape collections , 2012, ACM Trans. Graph..

[42]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[43]  Daniel G. Aliaga,et al.  Single viewpoint model completion of symmetric objects for digital inspection , 2011, Comput. Vis. Image Underst..

[44]  Shang-Hong Lai,et al.  Binary Orientation Trees for Volume and Surface Reconstruction from Unoriented Point Clouds , 2010, Comput. Graph. Forum.

[45]  Daniel Cohen-Or,et al.  Non-homogeneous resizing of complex models , 2008, SIGGRAPH Asia '08.

[46]  Cláudio T. Silva,et al.  Medial Kernels , 2012, Comput. Graph. Forum.

[47]  Leonidas J. Guibas,et al.  Example-Based 3D Scan Completion , 2005 .

[48]  Daniel G. Aliaga,et al.  Inverse Procedural Modeling by Automatic Generation of L‐systems , 2010, Comput. Graph. Forum.

[49]  Vladlen Koltun,et al.  Joint shape segmentation with linear programming , 2011, ACM Trans. Graph..

[50]  William J. Mitchell,et al.  The Palladian Grammar , 1978 .

[51]  Yutaka Ohtake,et al.  Smoothing of Partition of Unity Implicit Surfaces for Noise Robust Surface Reconstruction , 2009, Comput. Graph. Forum.

[52]  Daniel Cohen-Or,et al.  iWIRES: an analyze-and-edit approach to shape manipulation , 2009, ACM Trans. Graph..

[53]  Thomas A. Funkhouser,et al.  Consistent segmentation of 3D models , 2009, Comput. Graph..

[54]  Cláudio T. Silva,et al.  Triangulating point set surfaces with bounded error , 2005, SGP '05.

[55]  Daniel Cohen-Or,et al.  Active co-analysis of a set of shapes , 2012, ACM Trans. Graph..

[56]  Ghassan Hamarneh,et al.  VASE: Volume‐Aware Surface Evolution for Surface Reconstruction from Incomplete Point Clouds , 2011, Comput. Graph. Forum.

[57]  Javad Sadidi,et al.  Designing 3D Semantic Model in LOD4 to Simulate Building Utility Network , 2015 .

[58]  Shimin Hu,et al.  Adaptive Partitioning of Urban Facades , 2011 .

[59]  Luc Van Gool,et al.  Procedural 3D Building Reconstruction Using Shape Grammars and Detectors , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[60]  Sam Friedman,et al.  Online facade reconstruction from dominant frequencies in structured point clouds , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[61]  Stefan Greuter,et al.  Real-time procedural generation of `pseudo infinite' cities , 2003, GRAPHITE '03.

[62]  Remco C. Veltkamp,et al.  Watertight Scenes from Urban LiDAR and Planar Surfaces , 2013 .

[63]  Claus Brenner,et al.  Application of a Formal Grammar to Facade Reconstruction in Semiautomatic and Automatic Environments , 2009 .

[64]  Luc Van Gool,et al.  Procedural 3D Reconstruction of Puuc Buildings in Xkipché , 2006, VAST.

[65]  Marshall W. Bern,et al.  Surface Reconstruction by Voronoi Filtering , 1998, SCG '98.

[66]  D. Cohen-Or,et al.  Style-content separation by anisotropic part scales , 2010, ACM Trans. Graph..

[67]  Pat Hanrahan,et al.  Characterizing structural relationships in scenes using graph kernels , 2011, ACM Trans. Graph..

[68]  Radomír Mech,et al.  Guided Procedural Modeling , 2011, Comput. Graph. Forum.

[69]  Daniel Cohen-Or,et al.  Component‐wise Controllers for Structure‐Preserving Shape Manipulation , 2011, Comput. Graph. Forum.

[70]  Andrei Sharf,et al.  Grammar-based 3D facade segmentation and reconstruction , 2012, Comput. Graph..

[71]  Horst Bischof,et al.  A Globally Optimal Algorithm for Robust TV-L1 Range Image Integration , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[72]  Ulrich Neumann,et al.  2.5D building modeling by discovering global regularities , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[73]  Daniel Cohen-Or,et al.  Smart Variations: Functional Substructures for Part Compatibility , 2013, Comput. Graph. Forum.

[74]  Matt Olson,et al.  Automatic reconstruction of tree skeletal structures from point clouds , 2010, ACM Trans. Graph..

[75]  Hans-Peter Seidel,et al.  Multi-level partition of unity implicits , 2005, SIGGRAPH Courses.

[76]  B. Li,et al.  A Survey of Methods for Moving Least Squares Surfaces , 2008, VG/PBG@SIGGRAPH.

[77]  Shang-Hong Lai,et al.  Bipartite Polar Classification for Surface Reconstruction , 2011, Comput. Graph. Forum.

[78]  R. Basri,et al.  Direct visibility of point sets , 2007, SIGGRAPH 2007.

[79]  Niloy J. Mitra,et al.  Visibility of noisy point cloud data , 2010, Comput. Graph..

[80]  Daniel Cohen-Or,et al.  GlobFit: consistently fitting primitives by discovering global relations , 2011, ACM Trans. Graph..

[81]  Nina Amenta,et al.  Defining point-set surfaces , 2004, ACM Trans. Graph..

[82]  Sebastian Thrun,et al.  Shape from symmetry , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[83]  Junjie Cao,et al.  Point Cloud Skeletons via Laplacian Based Contraction , 2010, 2010 Shape Modeling International Conference.

[84]  Luc Van Gool,et al.  Computational Symmetry in Computer Vision and Computer Graphics , 2010, Found. Trends Comput. Graph. Vis..

[85]  B. Neubert,et al.  Approximate image-based tree-modeling using particle flows , 2007, SIGGRAPH 2007.

[86]  Jan-Michael Frahm,et al.  Detecting Large Repetitive Structures with Salient Boundaries , 2010, ECCV.

[87]  Leif Kobbelt,et al.  Robust reconstruction of watertight 3D models from non-uniformly sampled point clouds without normal information , 2006, SGP '06.

[88]  Leonidas J. Guibas,et al.  Discovering structural regularity in 3D geometry , 2008, ACM Trans. Graph..

[89]  Ghassan Hamarneh,et al.  A Survey on Shape Correspondence , 2011, Comput. Graph. Forum.

[90]  A. Senthil Kumar,et al.  3D VLSI Non-Slicing Floor Planning using Modified Corner List Representation , 2015 .

[91]  Ligang Liu,et al.  Co‐Segmentation of 3D Shapes via Subspace Clustering , 2012, Comput. Graph. Forum.

[92]  Victor S. Lempitsky,et al.  Global Optimization for Shape Fitting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[93]  I. Daubechies,et al.  Symmetry factored embedding and distance , 2010, ACM Trans. Graph..

[94]  Nassir Navab,et al.  Model globally, match locally: Efficient and robust 3D object recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[95]  Thomas Brox,et al.  Point-Based 3D Reconstruction of Thin Objects , 2013, 2013 IEEE International Conference on Computer Vision.

[96]  Adam Runions,et al.  Modeling and visualization of leaf venation patterns , 2005, SIGGRAPH '05.

[97]  Daniel Cohen-Or,et al.  Upright orientation of man-made objects , 2008, ACM Trans. Graph..

[98]  D. Cohen-Or,et al.  Robust moving least-squares fitting with sharp features , 2005, ACM Trans. Graph..

[99]  Daniel Cohen-Or,et al.  Functional Substructures for Part Compatibility , 2012 .

[100]  Marc Alexa,et al.  Anisotropic point set surfaces , 2006, AFRIGRAPH '06.

[101]  Gil Shklarski,et al.  Interactive topology-aware surface reconstruction , 2007, ACM Trans. Graph..

[102]  Simon Fuhrmann,et al.  Fusion of depth maps with multiple scales , 2011, ACM Trans. Graph..

[103]  Evangelos Kalogerakis,et al.  Folding meshes: hierarchical mesh segmentation based on planar symmetry , 2006, SGP '06.

[104]  Martin E. Newell,et al.  The utilization of procedure models in digital image synthesis. , 1975 .

[105]  James F. O'Brien,et al.  Spectral surface reconstruction from noisy point clouds , 2004, SGP '04.