Layout graph model for semantic façade reconstruction using laser point clouds

ABSTRACT Building façades can feature different patterns depending on the architectural style, functionality, and size of the buildings; therefore, reconstructing these façades can be complicated. In particular, when semantic façades are reconstructed from point cloud data, uneven point density and noise make it difficult to accurately determine the façade structure. When investigating façade layouts, Gestalt principles can be applied to cluster visually similar floors and façade elements, allowing for a more intuitive interpretation of façade structures. We propose a novel model for describing façade structures, namely the layout graph model, which involves a compound graph with two structure levels. In the proposed model, similar façade elements such as windows are first grouped into clusters. A down-layout graph is then formed using this cluster as a node and by combining intra- and inter-cluster spacings as the edges. Second, a top-layout graph is formed by clustering similar floors. By extracting relevant parameters from this model, we transform semantic façade reconstruction to an optimization strategy using simulated annealing coupled with Gibbs sampling. Multiple façade point cloud data with different features were selected from three datasets to verify the effectiveness of this method. The experimental results show that the proposed method achieves an average accuracy of 86.35%. Owing to its flexibility, the proposed layout graph model can deal with different types of façades and qualities of point cloud data, enabling a more robust and accurate reconstruction of façade models.

[1]  Han Hu,et al.  Interactive Correction of a Distorted Street-View Panorama for Efficient 3-D Façade Modeling , 2020, IEEE Geoscience and Remote Sensing Letters.

[2]  Jie Jiang,et al.  China’s high-resolution optical remote sensing satellites and their mapping applications , 2020, Geo spatial Inf. Sci..

[3]  Jie Wu,et al.  Building Facade Reconstruction Using Crowd- Sourced Photos and Two-Dimensional Maps , 2020 .

[4]  Jiaqiang Li,et al.  A Flexible Inference Machine for Global Alignment of Wall Openings , 2020, Remote. Sens..

[5]  Eric Ariel L. Salas,et al.  Waveform LiDAR concepts and applications for potential vegetation phenology monitoring and modeling: a comprehensive review , 2020, Geo spatial Inf. Sci..

[6]  Yusheng Xu,et al.  Registration of large-scale terrestrial laser scanner point clouds: A review and benchmark , 2020 .

[7]  Guoqing Zhou,et al.  Reconstructing facade semantic models using hierarchical topological graphs , 2020, Trans. GIS.

[8]  Qing Zhu,et al.  Fast and Regularized Reconstruction of Building Façades from Street-View Images using Binary Integer Programming , 2020, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[9]  J. Clinch Planning and Environment , 2020, Wind Turbines.

[10]  Lei Wang,et al.  Toward a unified theoretical framework for photogrammetry , 2020, Geo spatial Inf. Sci..

[11]  Henry Johan,et al.  A Data-driven Approach for Adding Facade Details to Textured LoD2 CityGML Models , 2019, VISIGRAPP.

[12]  Ioannis Stamos,et al.  A Kronecker Product Model for Repeated Pattern Detection on 2D Urban Images , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Steffen Goebbels,et al.  FACADE RECONSTRUCTION FOR TEXTURED LOD2 CITYGML MODELS BASED ON DEEP LEARNING AND MIXED INTEGER LINEAR PROGRAMMING , 2019, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[14]  Dong Lin,et al.  Fusion of thermal imagery with point clouds for building façade thermal attribute mapping , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[15]  M. Yaghoubi,et al.  The Optimum Window-to-Wall Ratio in Office Buildings for Hot‒Humid, Hot‒Dry, and Cold Climates in Iran , 2019, Environments.

[16]  Ruisheng Wang,et al.  Façade Separation in Ground-Based LiDAR Point Clouds Based on Edges and Windows , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  Silvio Savarese,et al.  Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  S. M. Iman Zolanvari,et al.  Three-dimensional building façade segmentation and opening area detection from point clouds , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[19]  Song-Chun Zhu,et al.  Attribute And-Or Grammar for Joint Parsing of Human Pose, Parts and Attributes , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Joseph Redmon,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[21]  Song-Chun Zhu,et al.  Single-View 3D Scene Reconstruction and Parsing by Attribute Grammar , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Subhransu Maji,et al.  SPLATNet: Sparse Lattice Networks for Point Cloud Processing , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Jean-Emmanuel Deschaud,et al.  Paris-Lille-3D: A large and high-quality ground-truth urban point cloud dataset for automatic segmentation and classification , 2017, Int. J. Robotics Res..

[24]  Christoph van Treeck,et al.  Facade geometry generation from low-resolution aerial photographs for building energy modeling , 2017 .

[25]  George Vogiatzis,et al.  A deep learning pipeline for semantic facade segmentation , 2017, BMVC.

[26]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[27]  Marc Pollefeys,et al.  Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark , 2017, ArXiv.

[28]  Fabian Hadiji,et al.  Statistical Relational Learning of Grammar Rules for 3D Building Reconstruction , 2017, Trans. GIS.

[29]  Filip Biljecki,et al.  GENERATION OF MULTI-LOD 3D CITY MODELS IN CITYGML WITH THE PROCEDURAL MODELLING ENGINE RANDOM3DCITY , 2016 .

[30]  F. Goia Search for the optimal window-to-wall ratio in office buildings in different European climates and the implications on total energy saving potential , 2016 .

[31]  Renaud Marlet,et al.  Efficient 2D and 3D Facade Segmentation Using Auto-Context , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Nikos Paragios,et al.  Learning Grammars for Architecture-Specific Facade Parsing , 2016, International Journal of Computer Vision.

[33]  Alexei A. Efros,et al.  What makes Paris look like Paris? , 2015, Commun. ACM.

[34]  Dani Lischinski,et al.  JumpCut , 2015, ACM Trans. Graph..

[35]  Pamela Buxton,et al.  Metric Handbook : Planning and Design Data , 2015 .

[36]  M. Jancosek,et al.  Flexible building primitives for 3D building modeling , 2015 .

[37]  Ligang Liu,et al.  Structure completion for facade layouts , 2014, ACM Trans. Graph..

[38]  William Nguatem,et al.  Localization of Windows and Doors in 3d Point Clouds of Facades , 2014 .

[39]  S. J. Oude Elberink,et al.  A graph edit dictionary for correcting errors in roof topology graphs reconstructed from point clouds , 2014 .

[40]  Marc Pollefeys,et al.  Efficient Structured Parsing of Facades Using Dynamic Programming , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Uwe Stilla,et al.  Reconstruction of façades in point clouds from multi aspect oblique ALS , 2013, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[42]  Matthias Haase,et al.  Optimizing the configuration of a façade module for office buildings by means of integrated thermal and lighting simulations in a total energy perspective , 2013 .

[43]  Daniel Cohen-Or,et al.  Layered analysis of irregular facades via symmetry maximization , 2013, ACM Trans. Graph..

[44]  Luc Van Gool,et al.  Bayesian Grammar Learning for Inverse Procedural Modeling , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Luc Van Gool,et al.  Is There a Procedural Logic to Architecture? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Ming C. Lin,et al.  Example-guided physically based modal sound synthesis , 2013, ACM Trans. Graph..

[47]  Hyung-Jo Jung,et al.  Optimization of building window system in Asian regions by analyzing solar heat gain and daylighting elements , 2013 .

[48]  Lutz Plümer,et al.  CityGML – Interoperable semantic 3D city models , 2012 .

[49]  Hayko Riemenschneider,et al.  Irregular lattices for complex shape grammar facade parsing , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Iasonas Kokkinos,et al.  Shape grammar parsing via Reinforcement Learning , 2011, CVPR 2011.

[51]  Wolfgang Förstner,et al.  Integration of conditional random fields and attribute grammars for range data interpretation of man-made objects , 2009, Ann. GIS.

[52]  Susanne Becker,et al.  Generation and application of rules for quality dependent façade reconstruction , 2009 .

[53]  Georgios Tziritas,et al.  Single view reconstruction using shape grammars for urban environments , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[54]  P. Davidson The energy performance of buildings directive , 2008 .

[55]  Luc Van Gool,et al.  Image-based procedural modeling of facades , 2007, ACM Trans. Graph..

[56]  Claus Brenner,et al.  Reconstruction of Façade Structures Using a Formal Grammar and RjMCMC , 2006, DAGM-Symposium.

[57]  Keith W. Hipel,et al.  The Graph Model for Conflict Resolution: Past, Present, and Future , 2005 .

[58]  Feng Han,et al.  Bottom-up/top-down image parsing by attribute graph grammar , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[59]  Emek Demir,et al.  A Compound Graph Layout Algorithm for Biological Pathways , 2004, GD.

[60]  Michael Wimmer,et al.  Instant architecture , 2003, ACM Trans. Graph..

[61]  David Adler,et al.  New Metric Handbook: Planning and design data , 1997 .

[62]  Kozo Sugiyama,et al.  Visualization of structural information: automatic drawing of compound digraphs , 1991, IEEE Trans. Syst. Man Cybern..

[63]  Ulrich Flemming,et al.  Syntactic structures in architecture: teaching composition with computer assistance , 1990 .

[64]  Guy Louchard,et al.  Probabilistic Analysis of Some Distributed Algorithms , 1990, Random Struct. Algorithms.

[65]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[67]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[68]  O. Reiser,et al.  Principles Of Gestalt Psychology , 1936 .

[69]  K. Aysha Jennath,et al.  Aesthetic Judgement and Visual Impact of Architectural Forms: A Study of Library Buildings☆ , 2016 .

[70]  H. Zaghloul Table (1) , 2016 .

[71]  Maria Capua Vetere Official Journal of the European Union , 2011 .

[72]  Josiane Zerubia,et al.  Structural Approach for Building Reconstruction from a Single DSM , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[73]  Claus Brenner,et al.  Application of a Formal Grammar to Facade Reconstruction in Semiautomatic and Automatic Environments , 2009 .

[74]  Frank Dellaert,et al.  A Probabilistic Approach to the Semantic Interpretation of Building Facades , 2004 .

[75]  J. Moran,et al.  Sensation and perception , 1980 .