Neural Procedural Reconstruction for Residential Buildings

This paper proposes a novel 3D reconstruction approach, dubbed Neural Procedural Reconstruction (NPR). NPR infers a sequence of shape grammar rule applications and reconstructs CAD-quality models with procedural structure from 3D points. While most existing methods rely on low-level geometry analysis to extract primitive structures, our approach conducts global analysis of entire building structures by deep neural networks (DNNs), enabling the reconstruction even from incomplete and sparse input data. We demonstrate the proposed system for residential buildings with aerial LiDAR as the input. Our 3D models boast compact geometry and semantically segmented architectural components. Qualitative and quantitative evaluations on hundreds of houses demonstrate that the proposed approach makes significant improvements over the existing state-of-the-art.

[1]  Roberto Cipolla,et al.  A Bayesian Estimation of Building Shape Using MCMC , 2002, ECCV.

[2]  Luc Van Gool,et al.  Procedural modeling of buildings , 2006, ACM Trans. Graph..

[3]  Daniel G. Aliaga,et al.  Procedural Editing of 3D Building Point Clouds , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Wuming Zhang,et al.  Semantic Decomposition and Reconstruction of Compound Buildings with Symmetric Roofs from LiDAR Data and Aerial Imagery , 2015, Remote. Sens..

[5]  Luc Van Gool,et al.  Procedural 3D Building Reconstruction Using Shape Grammars and Detectors , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[6]  Georgios Tzimiropoulos,et al.  Human Pose Estimation via Convolutional Part Heatmap Regression , 2016, ECCV.

[7]  Jianxiong Xiao,et al.  Reconstructing the World's Museums , 2012, ECCV.

[8]  Daniel G. Aliaga,et al.  Interactive sketching of urban procedural models , 2016, ACM Trans. Graph..

[9]  Hui Lin,et al.  Semantic decomposition and reconstruction of residential scenes from LiDAR data , 2013, ACM Trans. Graph..

[10]  Long Quan,et al.  Image-based tree modeling , 2007, SIGGRAPH 2007.

[11]  Jianxiong Xiao,et al.  A Linear Approach to Matching Cuboids in RGBD Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Radomír Mech,et al.  Metropolis procedural modeling , 2011, TOGS.

[13]  Pascal Mueller,et al.  Rome Reborn 2.0: A Case Study of Virtual City Reconstruction Using Procedural Modeling Techniques , 2009 .

[14]  Richard Szeliski,et al.  Manhattan-world stereo , 2009, CVPR.

[15]  William Nguatem,et al.  Modeling Urban Scenes from Pointclouds , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Ulrich Neumann,et al.  2.5D Dual Contouring: A Robust Approach to Creating Building Models from Aerial LiDAR Point Clouds , 2010, ECCV.

[17]  Horst Bischof,et al.  Fusion of Feature- and Area-Based Information for Urban Buildings Modeling from Aerial Imagery , 2008, ECCV.

[18]  Daniel G. Aliaga,et al.  Proceduralization for Editing 3D Architectural Models , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[19]  Richard Szeliski,et al.  Piecewise planar stereo for image-based rendering , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Luc Van Gool,et al.  Image-based procedural modeling of facades , 2007, SIGGRAPH 2007.

[22]  Fei Yang,et al.  Expression flow for 3D-aware face component transfer , 2011, SIGGRAPH 2011.

[23]  Ben Taskar,et al.  Detecting and parsing architecture at city scale from range data , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Luc Van Gool,et al.  Bayesian Grammar Learning for Inverse Procedural Modeling , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Michael M. Kazhdan,et al.  Screened poisson surface reconstruction , 2013, TOGS.

[26]  Luc Van Gool,et al.  Hierarchical Co-Segmentation of Building Facades , 2014, 2014 2nd International Conference on 3D Vision.

[27]  C. Brenner,et al.  A generative statistical approach to automatic 3D building roof reconstruction from laser scanning data , 2013 .

[28]  Lutz Plümer,et al.  Model driven reconstruction of roofs from sparse LIDAR point clouds , 2013 .

[29]  Jean-Philippe Pons,et al.  Robust piecewise-planar 3D reconstruction and completion from large-scale unstructured point data , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Long Quan,et al.  Image-based plant modeling , 2006, SIGGRAPH 2006.

[31]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[32]  Pat Hanrahan,et al.  Neurally-Guided Procedural Models: Amortized Inference for Procedural Graphics Programs using Neural Networks , 2016, NIPS.

[33]  Vladlen Koltun,et al.  Computer-generated residential building layouts , 2010, ACM Trans. Graph..

[34]  Stephen DiVerdi,et al.  Learning part-based templates from large collections of 3D shapes , 2013, ACM Trans. Graph..

[35]  Jana Kosecka,et al.  3D Bounding Box Estimation Using Deep Learning and Geometry , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Leonidas J. Guibas,et al.  Shape google: Geometric words and expressions for invariant shape retrieval , 2011, TOGS.

[37]  Hang Yang,et al.  Structured Indoor Modeling , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[38]  陈宝权.  GlobFit: Consistently Fitting Primitives by Discovering Global Relations , 2011 .

[39]  Daniel G. Aliaga,et al.  Procedural Modeling of a Building from a Single Image , 2018, Comput. Graph. Forum.

[40]  Ulrich Neumann,et al.  2.5D building modeling by discovering global regularities , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.