Roof-GAN: Learning to Generate Roof Geometry and Relations for Residential Houses

This paper presents Roof-GAN, a novel generative adversarial network that generates structured geometry of residential roof structures as a set of roof primitives and their relationships. Given the number of primitives, the generator produces a structured roof model as a graph, which consists of 1) primitive geometry as raster images at each node, encoding facet segmentation and angles; 2) inter-primitive colinear/coplanar relationships at each edge; and 3) primitive geometry in a vector format at each node, generated by a novel differentiable vectorizer while enforcing the relationships. The discriminator is trained to assess the primitive raster geometry, the primitive relationships, and the primitive vector geometry in a fully end-to-end architecture. Qualitative and quantitative evaluations demonstrate the effectiveness of our approach in generating diverse and realistic roof models over the competing methods with a novel metric proposed in this paper for the task of structured geometry generation. Code and data are available at https://github.com/yi-ming-qian/roofgan.

[1]  Yasutaka Furukawa,et al.  Learning Pairwise Inter-plane Relations for Piecewise Planar Reconstruction , 2020, ECCV.

[2]  Hao Zhang,et al.  Graph2Plan , 2020, ACM Trans. Graph..

[3]  Greg Mori,et al.  House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation , 2020, ECCV.

[4]  Yasutaka Furukawa,et al.  Conv-MPN: Convolutional Message Passing Neural Network for Structured Outdoor Architecture Reconstruction , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Hao Zhang,et al.  PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Hao Zhang,et al.  BSP-Net: Generating Compact Meshes via Binary Space Partitioning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Kai Xu,et al.  Learning Generative Models of 3D Structures , 2020, Eurographics.

[8]  Rui Tang,et al.  Data-driven interior plan generation for residential buildings , 2019, ACM Trans. Graph..

[9]  Leonidas J. Guibas,et al.  StructureNet , 2019, ACM Trans. Graph..

[10]  Angel X. Chang,et al.  PlanIT: planning and instantiating indoor scenes with relation graph and spatial prior networks , 2019, ACM Trans. Graph..

[11]  Li-Yi Wei,et al.  Learning to Reconstruct 3D Manhattan Wireframes From a Single Image , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Richard A. Newcombe,et al.  DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  L. Guibas,et al.  Composite Shape Modeling via Latent Space Factorization , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[14]  Jan Kautz,et al.  PlaneRCNN: 3D Plane Detection and Reconstruction From a Single Image , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Hao Zhang,et al.  Learning Implicit Fields for Generative Shape Modeling , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Daniel Cohen-Or,et al.  GRAINS , 2018, ACM Trans. Graph..

[17]  Lin Gao SDM-NET : Deep Generative Network for Structured Deformable Mesh , 2019 .

[18]  Daniel Cohen-Or,et al.  Learning to Generate the "Unseen" via Part Synthesis and Composition , 2018, ArXiv.

[19]  Jiaye Wu,et al.  Neural Procedural Reconstruction for Residential Buildings , 2018, ECCV.

[20]  Yuting Zhang,et al.  Unsupervised Discovery of Object Landmarks as Structural Representations , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Derek Hoiem,et al.  LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Leonidas J. Guibas,et al.  Learning Representations and Generative Models for 3D Point Clouds , 2017, ICML.

[23]  Peter Wonka,et al.  PolyFit: Polygonal Surface Reconstruction from Point Clouds , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Christopher K. I. Williams,et al.  The shape variational autoencoder: A deep generative model of part‐segmented 3D objects , 2017, Comput. Graph. Forum.

[25]  Leonidas J. Guibas,et al.  GRASS: Generative Recursive Autoencoders for Shape Structures , 2017, ACM Trans. Graph..

[26]  Sepp Hochreiter,et al.  GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium , 2017, NIPS.

[27]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[28]  Hao Su,et al.  A Point Set Generation Network for 3D Object Reconstruction from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Vladimir G. Kim,et al.  Data‐Driven Shape Analysis and Processing , 2015, Comput. Graph. Forum.

[30]  Jiajun Wu,et al.  Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling , 2016, NIPS.

[31]  Abhinav Gupta,et al.  Learning a Predictable and Generative Vector Representation for Objects , 2016, ECCV.

[32]  Evangelos Kalogerakis,et al.  Eurographics Symposium on Geometry Processing 2015 Analysis and Synthesis of 3d Shape Families via Deep-learned Generative Models of Surfaces , 2022 .

[33]  Hui Lin,et al.  Semantic decomposition and reconstruction of residential scenes from LiDAR data , 2013, ACM Trans. Graph..

[34]  C. Brenner,et al.  A generative statistical approach to automatic 3D building roof reconstruction from laser scanning data , 2013 .

[35]  Alexandru Iosup,et al.  Procedural content generation for games: A survey , 2013, TOMCCAP.

[36]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[37]  Radomír Mech,et al.  Metropolis procedural modeling , 2011, TOGS.

[38]  Vladlen Koltun,et al.  Computer-generated residential building layouts , 2010, ACM Trans. Graph..

[39]  Richard Szeliski,et al.  Manhattan-world stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Luc Van Gool,et al.  Procedural modeling of buildings , 2006, ACM Trans. Graph..

[41]  Roberto Cipolla,et al.  A Bayesian Estimation of Building Shape Using MCMC , 2002, ECCV.