4D Building Reconstruction with Machine Learning and Historical Maps

The increasing importance of three-dimensional (3D) city modelling is linked to these data’s different applications and advantages in many domains. Images and Light Detection and Ranging (LiDAR) data availability are now an evident and unavoidable prerequisite, not always verified for past scenarios. Indeed, historical maps are often the only source of information when dealing with historical scenarios or multi-temporal (4D) digital representations. The paper presents a methodology to derive 4D building models in the level of detail 1 (LoD1), inferring missing height information through machine learning techniques. The aim is to realise 4D LoD1 buildings for geospatial analyses and visualisation, valorising historical data, and urban studies. Several machine learning regression techniques are analysed and employed for deriving missing height data from digitised multi-temporal maps. The implemented method relies on geometric, neighbours, and categorical attributes for height prediction. Derived elevation data are then used for 4D building reconstructions, offering multi-temporal versions of the considered urban scenarios. Various evaluation metrics are also presented for tackling the common issue of lack of ground-truth information within historical data.

[1]  Filip Biljecki,et al.  TOWARDS AUTOMATIC SEMANTIC LABELLING OF 3D CITY MODELS , 2016 .

[2]  Thomas P. Kersten,et al.  Automated Generation of an Historic 4D City Model of Hamburg and Its Visualisation with the GE Engine , 2012, EuroMed.

[3]  Jean-Michel Guldmann,et al.  Creating 3D city models with building footprints and LIDAR point cloud classification: A machine learning approach , 2019, Comput. Environ. Urban Syst..

[4]  Thomas H. Kolbe,et al.  Applications of 3D City Models for a Better Understanding of the Built Environment , 2018 .

[5]  A. Zipf,et al.  Modelling the world in 3D from VGI/Crowdsourced data , 2016 .

[6]  Yue Wu,et al.  Application of Deep Learning for 3D building generalization , 2019 .

[7]  Kavisha Kumar,et al.  CityJSON: a compact and easy-to-use encoding of the CityGML data model , 2019, Open Geospatial Data, Software and Standards.

[8]  Norbert Haala,et al.  An update on automatic 3D building reconstruction , 2010 .

[9]  Fabio Remondino RECOVERING METRIC INFORMATION FROM OLD MONOCULAR VIDEO SEQUENCES , 2003 .

[10]  Aviva Peeters,et al.  A GIS-based method for modeling urban-climate parameters using automated recognition of shadows cast by buildings , 2016, Comput. Environ. Urban Syst..

[11]  Taghi M. Khoshgoftaar,et al.  A survey on Image Data Augmentation for Deep Learning , 2019, Journal of Big Data.

[12]  Taghi M. Khoshgoftaar,et al.  CatBoost for big data: an interdisciplinary review , 2020, J. Big Data.

[13]  Filip Biljecki,et al.  Generating 3D city models without elevation data , 2017, Comput. Environ. Urban Syst..

[14]  Karl-Heinz Häfele,et al.  OGC City Geography Markup Language (CityGML) Encoding Standard , 2012 .

[15]  Thomas Blaschke,et al.  Building Extraction from Airborne Laser Scanning Data: An Analysis of the State of the Art , 2015, Remote. Sens..

[16]  Xiao Xiang Zhu,et al.  IM2HEIGHT: Height Estimation from Single Monocular Imagery via Fully Residual Convolutional-Deconvolutional Network , 2018, ArXiv.

[17]  Yunfei Shi,et al.  Creating Topologically Consistent 3D City Models of LOD+ with Extrusion , 2012, CCTA.

[18]  Rafael Bidarra,et al.  A Survey on Procedural Modelling for Virtual Worlds , 2014, Comput. Graph. Forum.

[19]  Julien Perret,et al.  A Generic Model to Exploit Urban Regulation Knowledge , 2016, ISPRS Int. J. Geo Inf..

[20]  Nguyen Viet Ha,et al.  Preliminary Result of 3D City Modelling For Hanoi, Vietnam , 2018, 2018 5th NAFOSTED Conference on Information and Computer Science (NICS).

[21]  Filip Biljecki,et al.  Applications of 3D City Models: State of the Art Review , 2015, ISPRS Int. J. Geo Inf..

[22]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[23]  Umit Isikdag,et al.  Semi-Automatic 3D City Model Generation from Large-Format Aerial Images , 2018, ISPRS Int. J. Geo Inf..

[24]  Sungwon Jung,et al.  Prediction Model Based on an Artificial Neural Network for User-Based Building Energy Consumption in South Korea , 2019, Energies.

[25]  Fionn Murtagh,et al.  Multilayer perceptrons for classification and regression , 1991, Neurocomputing.

[26]  Martin Kada,et al.  ROOFN3D: DEEP LEARNING TRAINING DATA FOR 3D BUILDING RECONSTRUCTION , 2018, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[27]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[28]  Rozenn Dahyot,et al.  IM2ELEVATION: Building Height Estimation from Single-View Aerial Imagery , 2020, Remote. Sens..

[29]  J. G. Semple,et al.  Algebraic Projective Geometry , 1953 .

[30]  Filip Biljecki,et al.  Estimating Building Age with 3D GIS , 2017 .

[31]  Filip Biljecki,et al.  An improved LOD specification for 3D building models , 2016, Comput. Environ. Urban Syst..

[32]  Nicholas C. Coops,et al.  Predicting building ages from LiDAR data with random forests for building energy modeling , 2014 .

[33]  D. Basak,et al.  Support Vector Regression , 2008 .

[34]  Muntaha Sakeena,et al.  Automatic Prediction of Building Age from Photographs , 2018, ICMR.

[35]  T. H. Kolbe,et al.  The Virtual 3D City Model of Berlin-Managing, Integrating, and Communicating Complex Urban Information , 2006 .

[36]  Y. Ghassoun,et al.  Exploring the Benefits of 3D City Models in the Field of Urban Particles Distribution Modelling—A Comparison of Model Results , 2015 .

[37]  Nan Jiang,et al.  A Full Stage Data Augmentation Method in Deep Convolutional Neural Network for Natural Image Classification , 2020 .

[38]  Martin Kada,et al.  GENERATING 3D CITY MODELS BASED ON THE SEMANTIC SEGMENTATION OF LIDAR DATA USING CONVOLUTIONAL NEURAL NETWORKS , 2019 .

[39]  Fabio Menna,et al.  MULTI-TEMPORAL ANALYSIS OF LANDSCAPES AND URBAN AREAS , 2012 .

[40]  César Benavente-Peces,et al.  Buildings Energy Efficiency Analysis and Classification Using Various Machine Learning Technique Classifiers , 2020, Energies.

[41]  Mark D. McDonnell,et al.  Understanding Data Augmentation for Classification: When to Warp? , 2016, 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[42]  Melissa M. Bilec,et al.  Application of Machine Learning for Predicting Building Energy Use at Different Temporal and Spatial Resolution under Climate Change in USA , 2020, Buildings.

[43]  S. Tripodi,et al.  Deep learning-based extraction of building contours for large-scale 3D urban reconstruction , 2019, Remote Sensing.

[44]  Ran Chen,et al.  The development of 3D city model and its applications in urban planning , 2011, 2011 19th International Conference on Geoinformatics.

[45]  Ian D. Reid,et al.  Single View Metrology , 2000, International Journal of Computer Vision.

[46]  Kumud Wasnik,et al.  Prediction of building structure age using machine learning , 2019 .

[47]  Kamal Jain,et al.  VIRTUAL 3D CITY MODELING: TECHNIQUES AND APPLICATIONS , 2013 .

[48]  Alexander Zipf,et al.  Towards Defining a Framework for the Automatic Derivation of 3D CityGML Models from Volunteered Geographic Information , 2012, Int. J. 3 D Inf. Model..

[49]  Michael F. Chiang,et al.  Introduction to Machine Learning, Neural Networks, and Deep Learning , 2020, Translational vision science & technology.

[50]  Guler Yalcin,et al.  3D City Modelling with Oblique Photogrammetry Method , 2015 .

[51]  Amol Kapoor,et al.  Nostalgin: Extracting 3D City Models from Historical Image Data , 2019, KDD.

[52]  Jantien E. Stoter,et al.  A dimension-independent extrusion algorithm using generalised maps , 2015, Int. J. Geogr. Inf. Sci..

[53]  Hugo Ledoux,et al.  Topologically consistent 3D city models obtained by extrusion , 2011, Int. J. Geogr. Inf. Sci..

[54]  Hansoo Lee,et al.  Data Augmentation Method by Applying Color Perturbation of Inverse PSNR and Geometric Transformations for Object Recognition Based on Deep Learning , 2020, Applied Sciences.

[55]  Shen Rong,et al.  The research of regression model in machine learning field , 2018 .

[56]  Nassir Navab,et al.  Manifold Exploring Data Augmentation with Geometric Transformations for Increased Performance and Robustness , 2019, IPMI.

[57]  Jan-Michael Frahm,et al.  Boundary-Aware 3D Building Reconstruction From a Single Overhead Image , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Luis Perez,et al.  The Effectiveness of Data Augmentation in Image Classification using Deep Learning , 2017, ArXiv.

[59]  Lutz Plümer,et al.  CityGML – Interoperable semantic 3D city models , 2012 .

[60]  Mathias Rothermel,et al.  Extracting 3D urban models from oblique aerial images , 2015, 2015 Joint Urban Remote Sensing Event (JURSE).