Fusion of Heterogeneous Earth Observation Data for the Classification of Local Climate Zones

This paper proposes a novel framework for fusing multi-temporal, multispectral satellite images and OpenStreetMap (OSM) data for the classification of local climate zones (LCZs). Feature stacking is the most commonly used method of data fusion but does not consider the heterogeneity of multimodal optical images and OSM data, which becomes its main drawback. The proposed framework processes two data sources separately and then combines them at the model level through two fusion models (the landuse fusion model and building fusion model) that aim to fuse optical images with landuse and buildings layers of OSM data, respectively. In addition, a new approach to detecting building incompleteness of OSM data is proposed. The proposed framework was trained and tested using the data from the 2017 IEEE GRSS Data Fusion Contest and further validated on one additional test (AT) set containing test samples that are manually labeled in Munich and New York. The experimental results have indicated that compared with the feature stacking-based baseline framework, the proposed framework is effective in fusing optical images with OSM data for the classification of LCZs with high generalization capability on a large scale. The classification accuracy of the proposed framework outperforms the baseline framework by more than 6% and 2% while testing on the test set of 2017 IEEE GRSS Data Fusion Contest and the AT set, respectively. In addition, the proposed framework is less sensitive to spectral diversities of optical satellite images and thus achieves more stable classification performance than the state-of-the-art frameworks.

[1]  Gabriele Moser,et al.  Multimodal Classification of Remote Sensing Images: A Review and Future Directions , 2015, Proceedings of the IEEE.

[2]  T. Oke,et al.  Local Climate Zones for Urban Temperature Studies , 2012 .

[3]  Timothy R. Oke,et al.  Evaluation of the ‘local climate zone’ scheme using temperature observations and model simulations , 2014 .

[4]  Naoto Yokoya,et al.  Advances in Hyperspectral Image and Signal Processing: A Comprehensive Overview of the State of the Art , 2017, IEEE Geoscience and Remote Sensing Magazine.

[5]  William P. Lowry,et al.  Empirical Estimation of Urban Effects on Climate: A Problem Analysis. , 1977 .

[6]  Bertrand Le Saux,et al.  Joint Learning from Earth Observation and OpenStreetMap Data to Get Faster Better Semantic Maps , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  E. R. White,et al.  Cities of the World: World Regional Urban Development , 1992 .

[8]  Lorenzo Bruzzone,et al.  Domain Adaptation for the Classification of Remote Sensing Data: An Overview of Recent Advances , 2016, IEEE Geoscience and Remote Sensing Magazine.

[9]  Cidália Costa Fonte,et al.  Using OpenStreetMap data to assist in the creation of LCZ maps , 2017, 2017 Joint Urban Remote Sensing Event (JURSE).

[10]  Benjamin Bechtel,et al.  Local climate zones and annual surface thermal response in a Mediterranean city , 2017, 2017 Joint Urban Remote Sensing Event (JURSE).

[11]  L. S. Davis,et al.  An assessment of support vector machines for land cover classi(cid:142) cation , 2002 .

[12]  C. S. Anjos,et al.  Classification of urban environments using feature extraction and random forest , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[13]  Xiao Xiang Zhu,et al.  Feature Importance Analysis for Local Climate Zone Classification Using a Residual Convolutional Neural Network with Multi-Source Datasets , 2018, Remote. Sens..

[14]  Timothy A. Warner,et al.  Implementation of machine-learning classification in remote sensing: an applied review , 2018 .

[15]  Qihao Weng,et al.  A survey of image classification methods and techniques for improving classification performance , 2007 .

[16]  Mario Chica-Olmo,et al.  An assessment of the effectiveness of a random forest classifier for land-cover classification , 2012 .

[17]  Senén Barro,et al.  Do we need hundreds of classifiers to solve real world classification problems? , 2014, J. Mach. Learn. Res..

[18]  Naoto Yokoya,et al.  Open Data for Global Multimodal Land Use Classification: Outcome of the 2017 IEEE GRSS Data Fusion Contest , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[19]  Christian Jutten,et al.  Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects , 2015, Proceedings of the IEEE.

[20]  Jun Li,et al.  Advanced Spectral Classifiers for Hyperspectral Images: A review , 2017, IEEE Geoscience and Remote Sensing Magazine.

[21]  Naoto Yokoya,et al.  Multimodal, multitemporal, and multisource global data fusion for local climate zones classification based on ensemble learning , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[22]  Xiao Xiang Zhu,et al.  Feature Extraction and Selection of Sentinel-1 Dual-Pol Data for Global-Scale Local Climate Zone Classification , 2018, ISPRS Int. J. Geo Inf..

[23]  Frank D. Wood,et al.  Canonical Correlation Forests , 2015, ArXiv.

[24]  Kotaro Iizuka,et al.  Employing crowdsourced geographic data and multi-temporal/multi-sensor satellite imagery to monitor land cover change: A case study in an urbanizing region of the Philippines , 2017, Comput. Environ. Urban Syst..

[25]  Yong Xu,et al.  Beyond the urban mask , 2017, 2017 Joint Urban Remote Sensing Event (JURSE).

[26]  Weisheng Wang,et al.  A Study for Texture Feature Extraction of High-Resolution Satellite Images Based on a Direction Measure and Gray Level Co-Occurrence Matrix Fusion Algorithm , 2017, Sensors.

[27]  James R. Anderson,et al.  A land use and land cover classification system for use with remote sensor data , 1976 .

[28]  Wenzhong Shi,et al.  Fusion of Sentinel-2 images , 2016 .

[29]  Gabriele Moser,et al.  2017 IEEE GRSS Data Fusion Contest: Open Data for Global Multimodal Land Use Classification [Technical Committees] , 2017 .

[30]  Xiao Xiang Zhu,et al.  Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources , 2017, IEEE Geoscience and Remote Sensing Magazine.

[31]  G. Lin,et al.  Changing theoretical perspectives on urbanisation in Asian developing countries. , 1994, Third world planning review.

[32]  Iain Stewart,et al.  Mapping Local Climate Zones for a Worldwide Database of the Form and Function of Cities , 2015, ISPRS Int. J. Geo Inf..

[33]  Thomas Esch,et al.  Urban Footprint Processor—Fully Automated Processing Chain Generating Settlement Masks From Global Data of the TanDEM-X Mission , 2013, IEEE Geoscience and Remote Sensing Letters.

[34]  Yee Leung,et al.  A co-training approach to the classification of local climate zones with multi-source data , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[35]  Benjamin Bechtel,et al.  Classification of Local Climate Zones Based on Multiple Earth Observation Data , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[36]  R. Dickinson,et al.  The Footprint of Urban Areas on Global Climate as Characterized by MODIS , 2005 .

[37]  Jérôme Louradour,et al.  Multilevel ensembling for local climate zones classification , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[38]  Juan José Rodríguez Diez,et al.  Rotation Forest: A New Classifier Ensemble Method , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.