Automatic semantic and geometric enrichment of CityGML building models using HoG-based template matching

Semantically rich 3D building models give the potential for a wealth of rich geo-spatially-enabled applications such as cultural heritage augmented reality, urban planning, radio network planning and personal navigation. However, the majority of existing building models lack much if any semantic detail. This work demonstrates a novel method for automatically locating subclasses of windows and doors, using computer vision techniques including the histogram of oriented gradient (HOG) template matching, and automatically creating enriched CityGML content for the matched windows and doors. Good results were achieved for class identification with potential for further refinement of subclasses of windows and doors and other architectural features. It is part of a wider project to bring even richer semantic content to 3D geo-spatial building models.

[1]  Milan Sonka,et al.  Image processing analysis and machine vision [2nd ed.] , 1999 .

[2]  Mani Golparvar-Fard,et al.  Mapping actual thermal properties to building elements in gbXML-based BIM for reliable building energy performance modeling , 2015 .

[3]  Qing Zhu,et al.  Research and practice in three-dimensional city modeling , 2009 .

[4]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Renaud Marlet,et al.  Image parsing with graph grammars and Markov Random Fields applied to facade analysis , 2014, IEEE Winter Conference on Applications of Computer Vision.

[6]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[7]  Yinda Zhang,et al.  FrameBreak: Dramatic Image Extrapolation by Guided Shift-Maps , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Roberto Cipolla,et al.  Modelling and Interpretation of Architecture from Several Images , 2004, International Journal of Computer Vision.

[9]  Björn Johansson,et al.  Detecting Windows in City Scenes , 2002, SVM.

[10]  Jake K. Aggarwal,et al.  Retrieval by classification of images containing large manmade objects using perceptual grouping , 2002, Pattern Recognit..

[11]  Helmut Mayer,et al.  BUILDING FACADE INTERPRETATION FROM IMAGE SEQUENCES , 2005 .

[12]  Alexei A. Efros,et al.  Urban-Scale Quantitative Visual Analysis , 2014, ERCIM News.

[13]  Nikos Paragios,et al.  High-Level Bottom-Up Cues for Top-Down Parsing of Facade Images , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[14]  Lutz Plümer,et al.  CityGML – Interoperable semantic 3D city models , 2012 .

[15]  Karl-Heinz Häfele,et al.  OGC City Geography Markup Language (CityGML) Encoding Standard , 2012 .

[16]  Thierry Pun,et al.  Fast Robust Template Matching for Affine Resistant Image Watermarks , 1999, Information Hiding.

[17]  Christopher B. Jones,et al.  City model enrichment , 2011 .

[18]  Maurice Murphy,et al.  Semi-automatic generation of as-built BIM façade geometry from laser and image data , 2014, J. Inf. Technol. Constr..

[19]  H. Sizun,et al.  Radio Wave Propagation for Telecommunication Applications , 2004 .

[20]  Paul L. Rosin,et al.  Semantic and geometric enrichment of 3D geo-spatial models with captioned photos and labelled illustrations , 2014, VL@COLING.

[21]  Sisi Zlatanova,et al.  Towards Defining a Framework for Automatic Generation of Buildings in CityGML Using Building Information Models , 2009 .

[22]  Keith D. Hampson,et al.  The global construction industry and R&D , 2014 .

[23]  Fabrizio Sebastiani,et al.  Machine learning in automated text categorization , 2001, CSUR.

[24]  Alexander Koutamanis,et al.  Computer vision in architectural design , 1993 .

[25]  Helmut Mayer,et al.  IMPLICIT SHAPE MODELS, MODEL SELECTION, AND PLANE SWEEPING FOR 3D FACADE INTERPRETATION , 2007 .

[27]  Jürgen Döllner,et al.  Enhancing 3D City Models with Heterogeneous Spatial Information: Towards 3D Land Information Systems , 2009, AGILE Conf..

[28]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[29]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Thomas H. Kolbe,et al.  Spatio-semantic coherence in the integration of 3D city models , 2007 .

[31]  Thomas H. Kolbe,et al.  Representing and Exchanging 3D City Models with CityGML , 2009 .

[32]  F. Leberl,et al.  INTERPRETATION OF 2D AND 3D BUILDING DETAILS ON FACADES AND ROOFS , 2013 .

[33]  Sisi Zlatanova,et al.  Establishing a national standard for 3D topographic data compliant to CityGML , 2013, Int. J. Geogr. Inf. Sci..

[34]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.