Advances in urban information extraction from high-resolution remote sensing imagery

The study of urban area is one of the hottest research topics in the field of remote sensing. With the accumulation of high-resolution (HR) remote sensing data and emerging of new satellite sensors, HR observation of urban areas has become increasingly possible, which provides us with more elaborate urban information. However, the strong heterogeneity in the spectral and spatial domain of HR imagery brings great challenges to urban remote sensing. In recent years, numerous approaches were proposed to deal with HR image interpretation over complex urban scenes, including a series of features from low level to high level, as well as state-of-the-art methods depicting not only the urban extent, but also the intra-urban variations. In this paper, we aim to summarize the major advances in HR urban remote sensing from the aspects of feature representation and information extraction. Moreover, the future trends are discussed from the perspectives of methodology, urban structure and pattern characterization, big data challenge, and global mapping.

[1]  Jon Atli Benediktsson,et al.  Morphological Attribute Profiles for the Analysis of Very High Resolution Images , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Jin Sun,et al.  An Efficient and Scalable Framework for Processing Remotely Sensed Big Data in Cloud Computing Environments , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[5]  Le Yu,et al.  Climate effects of the GlobeLand30 land cover dataset on the Beijing Climate Center climate model simulations , 2016, Science China Earth Sciences.

[6]  Rongjun Qin,et al.  A Hierarchical Building Detection Method for Very High Resolution Remotely Sensed Images Combined with DSM Using Graph Cut Optimization , 2014 .

[7]  K. Vani,et al.  Water flow based geometric active deformable model for road network , 2015 .

[8]  Deng Xiangzheng,et al.  Regional differences and determinants of built-up area expansion in China , 2007 .

[9]  Francesca Bovolo,et al.  A Novel Framework for the Design of Change-Detection Systems for Very-High-Resolution Remote Sensing Images , 2013, Proceedings of the IEEE.

[10]  Jin Chen,et al.  Global land cover mapping at 30 m resolution: A POK-based operational approach , 2015 .

[11]  Liu Hui,et al.  Integrating Multiple Textural Features for Remote Sensing Image Change Detection , 2017 .

[12]  Fang Liu,et al.  Unsupervised Deep Feature Learning for Remote Sensing Image Retrieval , 2018, Remote. Sens..

[13]  Shiyong Cui,et al.  Building Change Detection Based on Satellite Stereo Imagery and Digital Surface Models , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Çaglar Senaras,et al.  Automated Detection of Arbitrarily Shaped Buildings in Complex Environments From Monocular VHR Optical Satellite Imagery , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Johannes R. Sveinsson,et al.  Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles , 2008, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[16]  Dengsheng Lu,et al.  An EcoCity model for regulating urban land cover structure and thermal environment: Taking Beijing as an example , 2017, Science China Earth Sciences.

[17]  Weiqi Zhou,et al.  Comparing Machine Learning Classifiers for Object-Based Land Cover Classification Using Very High Resolution Imagery , 2014, Remote Sensing.

[18]  Li Yin,et al.  Mapping Urban Impervious Surface by Fusing Optical and SAR Data at the Decision Level , 2016, Remote. Sens..

[19]  Tao Zhang,et al.  Monitoring of Urban Impervious Surfaces Using Time Series of High-Resolution Remote Sensing Images in Rapidly Urbanized Areas: A Case Study of Shenzhen , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[20]  Peijun Li,et al.  Rotation Invariant Texture Measured by Local Binary Pattern for Remote Sensing Image Classification , 2010, 2010 Second International Workshop on Education Technology and Computer Science.

[21]  Jianya Gong,et al.  Angular difference feature extraction for urban scene classification using ZY-3 multi-angle high-resolution satellite imagery , 2018 .

[22]  Jon Atli Benediktsson,et al.  A Novel Automatic Change Detection Method for Urban High-Resolution Remotely Sensed Imagery Based on Multiindex Scene Representation , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Liangpei Zhang,et al.  Combining Pixel- and Object-Based Machine Learning for Identification of Water-Body Types From Urban High-Resolution Remote-Sensing Imagery , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[24]  Bo Huang,et al.  Urban land-use mapping using a deep convolutional neural network with high spatial resolution multispectral remote sensing imagery , 2018, Remote Sensing of Environment.

[25]  Chun Liu,et al.  Automatic extraction of built-up area from ZY3 multi-view satellite imagery: Analysis of 45 global cities , 2019, Remote Sensing of Environment.

[26]  Soe W. Myint,et al.  Assessing local climate zones in arid cities: The case of Phoenix, Arizona and Las Vegas, Nevada , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[27]  Xin Huang,et al.  Road centreline extraction from high‐resolution imagery based on multiscale structural features and support vector machines , 2009 .

[28]  Rongjun Qin,et al.  Change detection on LOD 2 building models with very high resolution spaceborne stereo imagery , 2014 .

[29]  Qiang Li,et al.  A method characterizing urban expansion based on land cover map at 30 m resolution , 2016, Science China Earth Sciences.

[30]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[31]  Francesca Bovolo A Multilevel Parcel-Based Approach to Change Detection in Very High Resolution Multitemporal Images , 2009, IEEE Geosci. Remote. Sens. Lett..

[32]  A. Pandey,et al.  Built-up and vegetation extraction and density mapping using WorldView-II , 2012 .

[33]  Bin Jiang,et al.  Geospatial Big Data Handling Theory and Methods: A Review and Research Challenges , 2015, ArXiv.

[34]  Wei Su,et al.  Textural and local spatial statistics for the object‐oriented classification of urban areas using high resolution imagery , 2008 .

[35]  Pierre Soille,et al.  A New European Settlement Map From Optical Remotely Sensed Data , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[36]  Chun Liu,et al.  Assessing the quality of building height extraction from ZiYuan-3 multi-view imagery , 2017 .

[37]  Zhifeng Liu,et al.  Detecting global urban expansion over the last three decades using a fully convolutional network , 2019, Environmental Research Letters.

[38]  Lorenzo Bruzzone,et al.  Extended profiles with morphological attribute filters for the analysis of hyperspectral data , 2010 .

[39]  Jianya Gong,et al.  A New Stereo Pair Disparity Index (SPDI) for Detecting Built-Up Areas from High-Resolution Stereo Imagery , 2017, Remote. Sens..

[40]  Xin Huang,et al.  A multiscale urban complexity index based on 3D wavelet transform for spectral–spatial feature extraction and classification: an evaluation on the 8-channel WorldView-2 imagery , 2012 .

[41]  Daniel Sui,et al.  Geospatial Big Data , 2022, Encyclopedia of Big Data.

[42]  Jon Atli Benediktsson,et al.  Classification and feature extraction for remote sensing images from urban areas based on morphological transformations , 2003, IEEE Trans. Geosci. Remote. Sens..

[43]  Jan Haas,et al.  Mapping and Monitoring Urban Ecosystem Services Using Multitemporal High-Resolution Satellite Data , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[44]  Jon Atli Benediktsson,et al.  Generalized Differential Morphological Profiles for Remote Sensing Image Classification , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[45]  D. Tao,et al.  A modified stochastic neighbor embedding for multi-feature dimension reduction of remote sensing images , 2013 .

[46]  P. Gong,et al.  A 30-year (1984–2013) record of annual urban dynamics of Beijing City derived from Landsat data , 2015 .

[47]  S. Liang,et al.  Urbanisation and health in China , 2010, The Lancet.

[48]  Liangpei Zhang,et al.  A Multichannel Gray Level Co-Occurrence Matrix for Multi/Hyperspectral Image Texture Representation , 2014, Remote. Sens..

[49]  Tao Zhang,et al.  Urban Building Density Estimation From High-Resolution Imagery Using Multiple Features and Support Vector Regression , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[50]  Xiao Xiang Zhu,et al.  Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources , 2017, IEEE Geoscience and Remote Sensing Magazine.

[51]  Xing Fang,et al.  A novel water index for urban high-resolution eight-band WorldView-2 imagery , 2016, Int. J. Digit. Earth.

[52]  Liangpei Zhang,et al.  A New Building Extraction Postprocessing Framework for High-Spatial-Resolution Remote-Sensing Imagery , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[53]  Iain Stewart,et al.  Mapping Local Climate Zones for a Worldwide Database of the Form and Function of Cities , 2015, ISPRS Int. J. Geo Inf..

[54]  N. Lam,et al.  Wavelets for Urban Spatial Feature Discrimination: Comparisons with Fractal, Spatial Autocorrelation, and Spatial Co-Occurrence Approaches , 2004 .

[55]  Sérgio Freire,et al.  GHS built-up grid, derived from Landsat, multitemporal (1975, 1990, 2000, 2014) , 2015 .

[56]  Joachim Denzler,et al.  Deep learning and process understanding for data-driven Earth system science , 2019, Nature.

[57]  Hui Liu,et al.  The Use of Landscape Metrics and Transfer Learning to Explore Urban Villages in China , 2017, Remote. Sens..

[58]  M. Herold,et al.  Global Mapping of Human Settlement : Experiences, Datasets, and Prospects , 2009 .

[59]  Albert Y. Zomaya,et al.  Remote sensing big data computing: Challenges and opportunities , 2015, Future Gener. Comput. Syst..

[60]  William J. Emery,et al.  Very High Resolution Multiangle Urban Classification Analysis , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[61]  Richard Lepage,et al.  Road Extraction From Very High Resolution Remote Sensing Optical Images Based on Texture Analysis and Beamlet Transform , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[62]  WangLizhe,et al.  Remote sensing big data computing , 2015 .

[63]  Charalambos Poullis,et al.  Tensor-Cuts: A simultaneous multi-type feature extractor and classifier and its application to road extraction from satellite images , 2014 .

[64]  Qihao Weng,et al.  Remote sensing of impervious surfaces in the urban areas: Requirements, methods, and trends , 2012 .

[65]  Xin Huang,et al.  A Multidirectional and Multiscale Morphological Index for Automatic Building Extraction from Multispectral GeoEye-1 Imagery , 2011 .

[66]  Francesca Bovolo,et al.  Supervised change detection in VHR images using contextual information and support vector machines , 2013, Int. J. Appl. Earth Obs. Geoinformation.

[67]  Hankui K. Zhang,et al.  Finer resolution observation and monitoring of global land cover: first mapping results with Landsat TM and ETM+ data , 2013 .

[68]  Liangpei Zhang,et al.  Classification and Extraction of Spatial Features in Urban Areas Using High-Resolution Multispectral Imagery , 2007, IEEE Geoscience and Remote Sensing Letters.

[69]  Patricia Gober,et al.  Per-pixel vs. object-based classification of urban land cover extraction using high spatial resolution imagery , 2011, Remote Sensing of Environment.

[70]  Bin Chen,et al.  Stable classification with limited sample: transferring a 30-m resolution sample set collected in 2015 to mapping 10-m resolution global land cover in 2017. , 2019, Science bulletin.

[71]  Jie Tian,et al.  Optimization in multi‐scale segmentation of high‐resolution satellite images for artificial feature recognition , 2007 .

[72]  Jon Atli Benediktsson,et al.  A Survey on Spectral–Spatial Classification Techniques Based on Attribute Profiles , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[73]  Hui Liu,et al.  Spatiotemporal Detection and Analysis of Urban Villages in Mega City Regions of China Using High-Resolution Remotely Sensed Imagery , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[74]  Fabio Del Frate,et al.  Automatic Change Detection in Very High Resolution Images With Pulse-Coupled Neural Networks , 2010, IEEE Geoscience and Remote Sensing Letters.

[75]  Jon Atli Benediktsson,et al.  Change Detection in VHR Images Based on Morphological Attribute Profiles , 2013, IEEE Geoscience and Remote Sensing Letters.

[76]  Liangpei Zhang,et al.  A pixel shape index coupled with spectral information for classification of high spatial resolution remotely sensed imagery , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[77]  Xin Pan,et al.  Joint Deep Learning for land cover and land use classification , 2019, Remote Sensing of Environment.

[78]  Y. Ouma,et al.  On the optimization and selection of wavelet texture for feature extraction from high‐resolution satellite imagery with application towards urban‐tree delineation , 2006 .

[79]  Qian Du,et al.  Local Binary Patterns and Extreme Learning Machine for Hyperspectral Imagery Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[80]  Johannes R. Sveinsson,et al.  Classification of hyperspectral data from urban areas based on extended morphological profiles , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[81]  Liangpei Zhang,et al.  Semantic Classification of Urban Trees Using Very High Resolution Satellite Imagery , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[82]  Liangpei Zhang,et al.  Morphological Building/Shadow Index for Building Extraction From High-Resolution Imagery Over Urban Areas , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[83]  William J. Emery,et al.  A neural network approach using multi-scale textural metrics from very high-resolution panchromatic imagery for urban land-use classification , 2009 .

[84]  Jefersson Alex dos Santos,et al.  Towards better exploiting convolutional neural networks for remote sensing scene classification , 2016, Pattern Recognit..

[85]  Uwe Stilla,et al.  Deep Learning Earth Observation Classification Using ImageNet Pretrained Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[86]  Ali Ozgun Ok,et al.  Automated detection of buildings from single VHR multispectral images using shadow information and graph cuts , 2013 .

[87]  Martino Pesaresi,et al.  A Robust Built-Up Area Presence Index by Anisotropic Rotation-Invariant Textural Measure , 2008, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[88]  Martino Pesaresi,et al.  Improved Textural Built-Up Presence Index for Automatic Recognition of Human Settlements in Arid Regions With Scattered Vegetation , 2011, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[89]  Jon Atli Benediktsson,et al.  A new approach for the morphological segmentation of high-resolution satellite imagery , 2001, IEEE Trans. Geosci. Remote. Sens..

[90]  Maoqiong Gong,et al.  The storage of hydrogen based on low-temperature and high-pressure method , 2019, Chinese Science Bulletin.

[91]  M. Friedl,et al.  Mapping global urban areas using MODIS 500-m data: new methods and datasets based on 'urban ecoregions'. , 2010 .

[92]  Michael Dixon,et al.  Google Earth Engine: Planetary-scale geospatial analysis for everyone , 2017 .

[93]  Shihong Du,et al.  A Linear Dirichlet Mixture Model for decomposing scenes: Application to analyzing urban functional zonings , 2015 .

[94]  Jun Zhou,et al.  Hyperspectral Image Classification Based on Structured Sparse Logistic Regression and Three-Dimensional Wavelet Texture Features , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[95]  Rongjun Qin,et al.  Multi-level monitoring of subtle urban changes for the megacities of China using high-resolution multi-view satellite imagery , 2017 .

[96]  Xin Huang,et al.  Unsupervised Deep Feature Learning for Urban Village Detection from High-Resolution Remote Sensing Images , 2017 .

[97]  William J. Emery,et al.  Exploiting SAR and VHR Optical Images to Quantify Damage Caused by the 2003 Bam Earthquake , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[98]  Hao Wu,et al.  Global mapping of artificial surfaces at 30-m resolution , 2016, Science China Earth Sciences.

[99]  Jon Atli Benediktsson,et al.  Multiple Morphological Profiles From Multicomponent-Base Images for Hyperspectral Image Classification , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[100]  Hongsheng Zhang,et al.  Improving the impervious surface estimation with combined use of optical and SAR remote sensing images , 2014 .

[101]  Wei Zhang,et al.  40-Year (1978-2017) human settlement changes in China reflected by impervious surfaces from satellite remote sensing. , 2019, Science bulletin.

[102]  Christiane Schmullius,et al.  Object-based land cover mapping and comprehensive feature calculation for an automated derivation of urban structure types at block level , 2014 .

[103]  Francesca Bovolo,et al.  Building Change Detection in Multitemporal Very High Resolution SAR Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[104]  Anne Puissant,et al.  The utility of texture analysis to improve per‐pixel classification for high to very high spatial resolution imagery , 2005 .

[105]  Xintao Hu,et al.  Weakly supervised target detection in remote sensing images based on transferred deep features and negative bootstrapping , 2016, Multidimens. Syst. Signal Process..

[106]  Liangpei Zhang,et al.  An Adaptive Multiscale Information Fusion Approach for Feature Extraction and Classification of IKONOS Multispectral Imagery Over Urban Areas , 2007, IEEE Geoscience and Remote Sensing Letters.

[107]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[108]  Liangpei Zhang,et al.  Three-Dimensional Wavelet Texture Feature Extraction and Classification for Multi/Hyperspectral Imagery , 2014, IEEE Geoscience and Remote Sensing Letters.

[109]  Weidong Li,et al.  Building block level urban land-use information retrieval based on Google Street View images , 2017 .

[110]  Kiwon Lee,et al.  Quantitative indices based on 3D discrete wavelet transform for urban complexity estimation using remotely sensed imagery , 2009 .

[111]  William J. Emery,et al.  Classification of Very High Spatial Resolution Imagery Using Mathematical Morphology and Support Vector Machines , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[112]  Jon Atli Benediktsson,et al.  Classification of remote sensing images from urban areas using a fuzzy possibilistic model , 2006, IEEE Geoscience and Remote Sensing Letters.

[113]  Raul Queiroz Feitosa,et al.  Assessment of Binary Coding Techniques for Texture Characterization in Remote Sensing Imagery , 2013, IEEE Geoscience and Remote Sensing Letters.

[114]  Xin Huang,et al.  Deep neural network for remote-sensing image interpretation: status and perspectives , 2019, National science review.

[115]  Xiaoping Liu,et al.  High-resolution multi-temporal mapping of global urban land using Landsat images based on the Google Earth Engine Platform , 2018 .

[116]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[117]  Dengsheng Lu,et al.  Remote sensing-based artificial surface cover classification in Asia and spatial pattern analysis , 2016, Science China Earth Sciences.

[118]  Huadong Guo,et al.  A Global Human Settlement Layer From Optical HR/VHR RS Data: Concept and First Results , 2013, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[119]  Jianya Gong,et al.  Development of a Framework for Stereo Image Retrieval With Both Height and Planar Features , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[120]  Benjamin Bechtel,et al.  Classification of Local Climate Zones Using SAR and Multispectral Data in an Arid Environment , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[121]  Xuefei Hu,et al.  Impervious surface area extraction from IKONOS imagery using an object-based fuzzy method , 2011 .

[122]  T. Oke,et al.  Local Climate Zones for Urban Temperature Studies , 2012 .

[123]  F. Müller,et al.  Mapping ecosystem service supply, demand and budgets , 2012 .

[124]  Martino Pesaresi,et al.  Toward Global Automatic Built-Up Area Recognition Using Optical VHR Imagery , 2011, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.