Satellite-Based Mapping of Urban Poverty With Transfer-Learned Slum Morphologies

In the course of global urbanization, poverty in cities has been observed to increase, especially in the Global South. Poverty is one of the major challenges for our society in the upcoming decades, making it one of the most important issues in the Sustainable Development Goals defined by the United Nations. Satellite-based mapping can provide valuable information about slums where insights about the location and size are still missing. Large-scale slum mapping remains a challenge, fuzzy feature spaces between formal and informal settlements, significant imbalance of slum occurrences opposed to formal settlements, and various categories of multiple morphological slum features. We propose a transfer learned fully convolutional Xception network (XFCN), which is able to differentiate between formal built-up structures and the various categories of slums in high-resolution satellite data. The XFCN is trained on a large sample of globally distributed slums, located in cities of Cape Town, Caracas, Delhi, Lagos, Medellin, Mumbai, Nairobi, Rio de Janeiro, São Paulo, and Shenzhen. Slums in these cities are greatly heterogeneous in its morphological feature space and differ to a varying degree to formal settlements. Transfer learning can help to improve segmentation results when learning on a variety of slum morphologies, with high $F$1 scores of up to $89\%$.

[1]  Monika Kuffer,et al.  Challenges of mapping the missing spaces , 2019, 2019 Joint Urban Remote Sensing Event (JURSE).

[2]  Hannes Taubenböck,et al.  How dynamic are slums? EO-based assessment of Kibera’s morphologic transformation , 2019, 2019 Joint Urban Remote Sensing Event (JURSE).

[3]  H. Taubenböck,et al.  The physical face of slums: a structural comparison of slums in Mumbai, India, based on remotely sensed data , 2014 .

[4]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[5]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[6]  Hannes Taubenböck,et al.  Spatial patterns of slums: Comparing African and Asian cities , 2017, 2017 Joint Urban Remote Sensing Event (JURSE).

[7]  Xiao Xiang Zhu,et al.  Mapping Human Settlements with Multi-seasonal Sentinel-2 Imagery and Attention-based ResNeXt , 2019, 2019 Joint Urban Remote Sensing Event (JURSE).

[8]  Xiao Xiang Zhu,et al.  Buildings Detection in VHR SAR Images Using Fully Convolution Neural Networks , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Xiao Xiang Zhu,et al.  Building segmentation through a gated graph convolutional neural network with deep structured feature embedding , 2019, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[10]  Hannes Taubenböck,et al.  Large-scale building extraction in very high-resolution aerial imagery using Mask R-CNN , 2019, 2019 Joint Urban Remote Sensing Event (JURSE).

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Ryan Engstrom,et al.  Poverty from Space: Using High-Resolution Satellite Imagery for Estimating Economic Well-Being , 2017, The World Bank Economic Review.

[13]  R. B.,et al.  The United Nations , 1947, Nature.

[14]  David Satterthwaite,et al.  Urbanization , Rural – urban Migration and Urban Poverty , 2015 .

[15]  H. Taubenböck,et al.  The similar size of slums , 2018 .

[16]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Stefano Ermon,et al.  Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping , 2015, AAAI.

[18]  Kepeng Xu,et al.  Segmentation of Building Footprints with Xception and IoUloss , 2019, 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[19]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[20]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[21]  Qiming Qin,et al.  Effective Building Extraction From High-Resolution Remote Sensing Images With Multitask Driven Deep Neural Network , 2019, IEEE Geoscience and Remote Sensing Letters.

[22]  Un-Habitat The Challenge of Slums: Global Report on Human Settlements 2003 , 2004 .

[23]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  S. Dech,et al.  Inferring floor area ratio thresholds for the delineation of city centers based on cognitive perception , 2019, Environment and Planning B: Urban Analytics and City Science.

[25]  Bistra N. Dilkina,et al.  Large Scale High-Resolution Land Cover Mapping With Multi-Resolution Data , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[27]  F. Torner,et al.  The Challenge of Slums. Global Report on Human Settlements 2003 , 2005 .

[28]  Xiao Xiang Zhu,et al.  Semantic segmentation of slums in satellite images using transfer learning on fully convolutional neural networks , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[29]  Hannes Taubenböck,et al.  A new ranking of the world's largest cities—Do administrative units obscure morphological realities? , 2019, Remote Sensing of Environment.

[30]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[31]  Uwe Stilla,et al.  Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection , 2016, ISPRS Journal of Photogrammetry and Remote Sensing.

[32]  J. D. Wegner,et al.  SEMANTIC SEGMENTATION OF AERIAL IMAGES WITH AN ENSEMBLE OF CNNS , 2016, ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences.

[33]  Begüm Demir,et al.  Bigearthnet: A Large-Scale Benchmark Archive for Remote Sensing Image Understanding , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[34]  Hannes Taubenböck,et al.  Slum mapping in polarimetric SAR data using spatial features , 2017 .

[35]  Sang Michael Xie,et al.  Combining satellite imagery and machine learning to predict poverty , 2016, Science.

[36]  Helena Titheridge,et al.  predictSLUMS: A new model for identifying and predicting informal settlements and slums in cities from street intersections using machine learning , 2018, Comput. Environ. Urban Syst..

[37]  Michael Wurm,et al.  Mapping urban villages using fully convolutional neural networks , 2020 .

[38]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Hannes Taubenböck,et al.  Exploitation of textural and morphological image features in Sentinel-2A data for slum mapping , 2017, 2017 Joint Urban Remote Sensing Event (JURSE).

[41]  Piotr Bilinski,et al.  Mapping Informal Settlements in Developing Countries using Machine Learning and Low Resolution Multi-spectral Data , 2019, AIES.

[42]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[43]  H. Taubenböck,et al.  The morphology of the Arrival City - A global categorization based on literature surveys and remotely sensed data , 2018 .

[44]  H. Taubenböck,et al.  Detecting social groups from space – Assessment of remote sensing-based mapped morphological slums using income data , 2018 .

[45]  Bertrand Le Saux,et al.  Beyond RGB: Very High Resolution Urban Remote Sensing With Multimodal Deep Networks , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[46]  Xiao Xiang Zhu,et al.  Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources , 2017, IEEE Geoscience and Remote Sensing Magazine.

[47]  Xiao Xiang Zhu,et al.  A Relation-Augmented Fully Convolutional Network for Semantic Segmentation in Aerial Scenes , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[49]  Xiao Xiang Zhu,et al.  Slum Mapping in Imbalanced Remote Sensing Datasets Using Transfer Learned Deep Features , 2019, 2019 Joint Urban Remote Sensing Event (JURSE).

[50]  Alfred Stein,et al.  Deep Fully Convolutional Networks for the Detection of Informal Settlements in VHR Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[51]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Jonathan Hersh,et al.  Evaluating the relationship between spatial and spectral features derived from high spatial resolution satellite data and urban poverty in Colombo, Sri Lanka , 2017, 2017 Joint Urban Remote Sensing Event (JURSE).

[53]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[54]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[55]  Monika Kuffer,et al.  Slums from Space - 15 Years of Slum Mapping Using Remote Sensing , 2016, Remote. Sens..