3D Virtual Urban Scene Reconstruction From a Single Optical Remote Sensing Image

This paper presents a low-cost and efficient method for 3D virtual urban scene reconstruction based on multi-source remote sensing big data and deep learning. By integrating maps, satellite optical images, and digital terrain model (DTM), the proposed method achieves a reasonable reconstructed 3D model for complex urban. The method consists of two independent convolutional neural networks (CNN) to process the land cover and the building height extraction. The proposed method is then tested on a 100 km2 scene in San Diego, USA, including about 30 000 buildings. The land cover classification achieves an overall accuracy (OA) of 80.4% for eight types of land as defined in NLCD 2011 datasets. Building height estimation achieves an average error at 1.9 meters on NYC open data, the building footprint. Furthermore, the scene reconstruction including the estimation of both land cover and building height can be finished in 10 min on a single NVidia Titan X GPU.

[1]  Kamal Jain,et al.  VIRTUAL 3D CITY MODELING: TECHNIQUES AND APPLICATIONS , 2013 .

[2]  S. Dwivedi,et al.  Obesity May Be Bad: Compressed Convolutional Networks for Biomedical Image Segmentation , 2020 .

[3]  Leif Kobbelt,et al.  Automatic Registration of Oblique Aerial Images with Cadastral Maps , 2010, ECCV Workshops.

[4]  Muhittin Gökmen,et al.  Automatic 3D Roof Reconstruction using Digital Cadastral Map, Architectural Knowledge and an Aerial Image , 2006, 2006 IEEE International Symposium on Geoscience and Remote Sensing.

[5]  Jan-Michael Frahm,et al.  Towards Urban 3D Reconstruction from Video , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[6]  Daniel G. Aliaga,et al.  A Survey of Urban Reconstruction , 2013, Comput. Graph. Forum.

[7]  Suming Jin,et al.  Completion of the 2011 National Land Cover Database for the Conterminous United States – Representing a Decade of Land Cover Change Information , 2015 .

[8]  James D. Wickham,et al.  Thematic accuracy assessment of the 2011 National Land Cover Database (NLCD). , 2017, Remote sensing of environment.

[9]  Li-Ta Hsu,et al.  3D building map reconstruction in dense urban areas by integrating airborne laser point cloud with 2D boundary map , 2015, 2015 IEEE International Conference on Vehicular Electronics and Safety (ICVES).

[10]  Jiangye Yuan,et al.  Automatic Building Extraction in Aerial Scenes Using Convolutional Networks , 2016, ArXiv.

[11]  Zhanyi Hu,et al.  Automatic building extraction from oblique aerial images , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).