Room semantics inference using random forest and relational graph convolutional networks: A case study of research building

Semantically rich maps are the foundation of indoor location‐based services. Many map providers such as OpenStreetMap and automatic mapping solutions focus on the representation and detection of geometric information (e.g., shape of room) and a few semantics (e.g., stairs and furniture) but neglect room usage. To mitigate the issue, this work proposes a general room tagging method for public buildings, which can benefit both existing map providers and automatic mapping solutions by inferring the missing room usage based on indoor geometric maps. Two kinds of statistical learning‐based room tagging methods are adopted: traditional machine learning (e.g., random forests) and deep learning, specifically relational graph convolutional networks (R‐GCNs), based on the geometric properties (e.g., area), topological relationships (e.g., adjacency and inclusion), and spatial distribution characteristics of rooms. In the machine learning‐based approach, a bidirectional beam search strategy is proposed to deal with the issue that the tag of a room depends on the tag of its neighbors in an undirected room sequence. In the R‐GCN‐based approach, useful properties of neighboring nodes (rooms) in the graph are automatically gathered to classify the nodes. Research buildings are taken as examples to evaluate the proposed approaches based on 130 floor plans with 3,330 rooms by using fivefold cross‐validation. The experiments conducted show that the random forest‐based approach achieves a higher tagging accuracy (0.85) than R‐GCN (0.79).

[1]  William J. Mitchell,et al.  The Logic of Architecture: Design, Computation, and Cognition , 1992 .

[2]  Adwait Ratnaparkhi,et al.  A Maximum Entropy Model for Part-Of-Speech Tagging , 1996, EMNLP.

[3]  Christian Ah-Soon,et al.  A complete system for the analysis of architectural drawings , 2000, International Journal on Document Analysis and Recognition.

[4]  Gibbs Sampling , 2000 .

[5]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[6]  Georg Gartner,et al.  A Survey of Mobile Indoor Navigation Systems , 2009 .

[7]  Richard Szeliski,et al.  Reconstructing building interiors from images , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[8]  Ernest Valveny,et al.  A system to detect rooms in architectural floor plan images , 2010, DAS '10.

[9]  Ernest Valveny,et al.  Notation-Invariant Patch-Based Wall Detector in Architectural Floor Plans , 2011, GREC.

[10]  Marcus Liwicki,et al.  Improved Automatic Analysis of Architectural Floor Plans , 2011, 2011 International Conference on Document Analysis and Recognition.

[11]  Patric Jensfelt,et al.  Large-scale semantic mapping and reasoning with heterogeneous modalities , 2012, 2012 IEEE International Conference on Robotics and Automation.

[12]  John Folkesson,et al.  What can we learn from 38,000 rooms? Reasoning about unexplored space in indoor environments , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Ramesh Krishnamurti,et al.  Estimating the Interior Layout of Buildings Using a Shape Grammar to Capture Building Style , 2012, J. Comput. Civ. Eng..

[14]  Moustafa Youssef,et al.  CrowdInside: automatic construction of indoor floorplans , 2012, SIGSPATIAL/GIS.

[15]  Steven M. Seitz,et al.  Capturing indoor scenes with smartphones , 2012, UIST.

[16]  Burcu Akinci,et al.  Automatic Creation of Semantically Rich 3D Building Models from Laser Scanner Data , 2013 .

[17]  Jian Zhang,et al.  Estimating the 3D Layout of Indoor Scenes and Its Clutter from Depth Sensors , 2013, 2013 IEEE International Conference on Computer Vision.

[18]  Peter,et al.  Grammar supported indoor mapping , 2013 .

[19]  Robert P. Dick,et al.  Hallway based automatic indoor floorplan construction using room fingerprints , 2013, UbiComp.

[20]  Ernest Valveny,et al.  Statistical segmentation and structural recognition for floor plan interpretation , 2013, International Journal on Document Analysis and Recognition (IJDAR).

[21]  Alberto Quattrini Li,et al.  A System for Building Semantic Maps of Indoor Environments Exploiting the Concept of Building Typology , 2013, RoboCup.

[22]  Kourosh Khoshelham,et al.  3D Modelling of interior spaces : learning the language of indoor architecture , 2014 .

[23]  Kaigui Bian,et al.  Jigsaw: indoor floor plan reconstruction via mobile crowdsensing , 2014, MobiCom.

[24]  Frank Dürr,et al.  MapGENIE: Grammar-enhanced indoor map construction from crowd-sourced data , 2014, 2014 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[25]  Markus Kattenbeck,et al.  Dissertation Abstract: Empirically Measuring Salience of Objects for Use in Pedestrian Navigation , 2017, KI - Künstliche Intelligenz.

[26]  Dieter Fritsch,et al.  GRAMMAR-SUPPORTED 3D INDOOR RECONSTRUCTION FROM POINT CLOUDS FOR “AS-BUILT” BIM , 2015 .

[27]  Moustafa Youssef,et al.  SemSense: Automatic construction of semantic indoor floorplans , 2015, 2015 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[28]  Josep Lladós,et al.  Attributed Graph Grammar for floor plan analysis , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[29]  Chunming Qiao,et al.  Crowd Map: Accurate Reconstruction of Indoor Floor Plans from Crowdsourced Sensor-Rich Videos , 2015, 2015 IEEE 35th International Conference on Distributed Computing Systems.

[30]  Hang Yang,et al.  Structured Indoor Modeling , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Charlotte Klonk New Laboratories: Historical and Critical Perspectives on Contemporary Developments , 2016 .

[32]  Silvio Savarese,et al.  3D Semantic Parsing of Large-Scale Indoor Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Sylvain Robert,et al.  Automatic reconstruction of 3D building models from scanned 2D floor plans , 2016 .

[34]  Björn Stenger,et al.  Parsing floor plan images , 2017, 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA).

[35]  Francesco Amigoni,et al.  Semantic classification by reasoning on the whole structure of buildings using statistical relational learning techniques , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[37]  Ronald Raulefs,et al.  Recent Advances in Indoor Localization: A Survey on Theoretical Approaches and Applications , 2017, IEEE Communications Surveys & Tutorials.

[38]  Gavin Smith,et al.  Data-driven estimation of building interior plans , 2017, Int. J. Geogr. Inf. Sci..

[39]  Yizhou Wang,et al.  Knitter: Fast, resilient single-user indoor floor plan construction , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[40]  Alexander Zipf,et al.  a Conceptual Framework for Indoor Mapping by Using Grammars , 2017 .

[41]  Axel Wendt,et al.  Automatic Room Segmentation From Unstructured 3-D Data of Indoor Environments , 2017, IEEE Robotics and Automation Letters.

[42]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Jan-Henrik Haunert,et al.  ROOM SHAPES AND FUNCTIONAL USES PREDICTED FROM SPARSE DATA , 2018 .

[44]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[45]  Francesco Amigoni,et al.  Predicting the global structure of indoor environments: A constructive machine learning approach , 2018, Autonomous Robots.

[46]  Alexander Zipf,et al.  Feasibility of Using Grammars to Infer Room Semantics , 2019, Remote. Sens..

[47]  Geoffrey A. Hollinger,et al.  Deep learning of structured environments for robot search , 2019, Auton. Robots.