Graph-based generative representation learning of semantically and behaviorally augmented floorplans

Floorplans are commonly used to represent the layout of buildings. In computer aided-design (CAD) floorplans are usually represented in the form of hierarchical graph structures. Research works towards computational techniques that facilitate the design process, such as automated analysis and optimization, often use simple floorplan representations that ignore the semantics of the space and do not take into account usage related analytics. We present a floorplan embedding technique that uses an attributed graph to represent the geometric information as well as design semantics and behavioral features of the inhabitants as node and edge attributes. A Long Short-Term Memory (LSTM) Variational Autoencoder (VAE) architecture is proposed and trained to embed attributed graphs as vectors in a continuous space. A user study is conducted to evaluate the coupling of similar floorplans retrieved from the embedding space with respect to a given input (e.g., design layout). The qualitative, quantitative and user-study evaluations show that our embedding framework produces meaningful and accurate vector representations for floorplans. In addition, our proposed model is a generative model. We studied and showcased its effectiveness for generating new floorplans. We also release the dataset that we have constructed and which, for each floorplan, includes the design semantics attributes as well as simulation generated human behavioral features for further study in the community.

[1]  Nitin Gupta,et al.  DANIEL: A Deep Architecture for Automatic Analysis and Retrieval of Building Floor Plans , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[2]  Joseph M. Hellerstein,et al.  Distributed GraphLab: A Framework for Machine Learning in the Cloud , 2012, Proc. VLDB Endow..

[3]  Nikos Komodakis,et al.  GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders , 2018, ICANN.

[4]  Ann Heylighen,et al.  A case base of Case-Based Design tools for architecture , 2001, Comput. Aided Des..

[5]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[6]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[7]  Samy Bengio,et al.  Generating Sentences from a Continuous Space , 2015, CoNLL.

[8]  Max Q.-H. Meng,et al.  HouseExpo: A Large-scale 2D Indoor Layout Dataset for Learning-based Algorithms on Mobile Robots , 2019, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[9]  W. Steffen Looking Back to the Future , 2008, Ambio.

[10]  Ernest Valveny,et al.  Runlength Histogram Image Signature for Perceptual Retrieval of Architectural Floor Plans , 2013, GREC.

[11]  Yang Liu,et al.  graph2vec: Learning Distributed Representations of Graphs , 2017, ArXiv.

[12]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[13]  Vladimir Pavlovic,et al.  Laying the Foundations of Deep Long-Term Crowd Flow Prediction , 2020, ECCV.

[14]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[15]  Claire Cardie,et al.  Proceedings of the Eighteenth International Conference on Machine Learning, 2001, p. 577–584. Constrained K-means Clustering with Background Knowledge , 2022 .

[16]  Hao Zhang,et al.  Graph2Plan , 2020, ACM Trans. Graph..

[17]  Marcus Liwicki,et al.  a.SCAtch - A Sketch-Based Retrieval for Architectural Floor Plans , 2010, 2010 12th International Conference on Frontiers in Handwriting Recognition.

[18]  Vladlen Koltun,et al.  Computer-generated residential building layouts , 2010, SIGGRAPH 2010.

[19]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[20]  M. Kramer Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[21]  Greg Mori,et al.  House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation , 2020, ECCV.

[22]  Aynaz Taheri,et al.  Learning Graph Representations with Recurrent Neural Network Autoencoders , 2018 .

[23]  Chiranjoy Chattopadhyay,et al.  A unified framework for semantic matching of architectural floorplans , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[24]  Ann Heylighen,et al.  Looking Back to the Future An Updated Case Base of Case-Based Design Tools for Architecture , 2007 .

[25]  Glenn Reinman,et al.  An Open Framework for Developing, Evaluating, and Sharing Steering Algorithms , 2009, MIG.

[26]  Kun Zhou,et al.  Crowd-driven mid-scale layout design , 2016, ACM Trans. Graph..

[27]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[29]  Reynold Xin,et al.  GraphX: Graph Processing in a Distributed Dataflow Framework , 2014, OSDI.

[30]  Eugénio Rodrigues,et al.  An evolutionary strategy enhanced with a local search technique for the space allocation problem in architecture, Part 2: Validation and performance tests , 2013, Comput. Aided Des..

[31]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[32]  Chiranjoy Chattopadhyay,et al.  High-level feature aggregation for fine-grained architectural floor plan retrieval , 2018, IET Comput. Vis..

[33]  Abdolreza Mirzaei,et al.  Hierarchical graph embedding in vector space by graph pyramid , 2017, Pattern Recognit..

[34]  Georg Lambert,et al.  Line Moments and Invariants for Real Time Processing of Vectorized Contour Data , 1995, ICIAP.

[35]  Rui Tang,et al.  Data-driven interior plan generation for residential buildings , 2019, ACM Trans. Graph..

[36]  H. Howie Huang,et al.  G-Store: High-Performance Graph Store for Trillion-Edge Processing , 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

[37]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[38]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[39]  Guoyin Wang,et al.  Topic-Guided Variational Auto-Encoder for Text Generation , 2019, NAACL.

[40]  Eugénio Rodrigues,et al.  An evolutionary strategy enhanced with a local search technique for the space allocation problem in architecture, Part 1: Methodology , 2013, Comput. Aided Des..

[41]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[42]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[43]  Yizhou Sun,et al.  Unsupervised Inductive Graph-Level Representation Learning via Graph-Graph Proximity , 2019, IJCAI.

[44]  Umapada Pal,et al.  Symbol Spotting in Line Drawings through Graph Paths Hashing , 2011, 2011 International Conference on Document Analysis and Recognition.

[45]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..