Multi-branch convolutional neural network for built-up area extraction from remote sensing image

Abstract Built-up area is one of the most important objects of remote sensing images analysis, therefore extracting built-up area from remote sensing image automatically has attracted wide attention. It is common to treat built-up area extraction as image segmentation task. However, it's hard to devise a handcrafted feature to describe built-up area since it contains many non-built-up elements, such as trees, grasslands, and small ponds. Besides, built-up area corresponds to large size local region without precise boundary in remote sensing image so that the precision of segmentation in pixel level is not reliable. To cope with the problem of built-up area extraction, a segmentation framework based on deep feature learning and graph model is proposed. The segmentation procedure comprises of three steps. Firstly, the image is divided into small patches whose deep features are extracted by the devised lightweight multi-branch convolutional neural network (LMB-CNN). Secondly, a patch-wise graph model is constructed according to the learnt features, and then is optimized to segment built-up area with patch-level precision in full frame of remote sensing image. At last, post-processing step is also adopted to make the segmentation result visually intact. The experiments verify that the proposed method shows excellent performance by achieving high overall accuracy over 98.6% on Gaofen-2 remote sensing image data with size of 10,240 × 10,240.

[1]  Tamás Szirányi,et al.  Improved Harris Feature Point Set for Orientation-Sensitive Urban-Area Detection in Aerial Images , 2013, IEEE Geoscience and Remote Sensing Letters.

[2]  Yunhong Wang,et al.  Built-up area detection based on a Bayesian saliency model , 2017, Int. J. Wavelets Multiresolution Inf. Process..

[3]  Martino Pesaresi,et al.  A Robust Built-Up Area Presence Index by Anisotropic Rotation-Invariant Textural Measure , 2008, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[4]  Sabine Süsstrunk,et al.  Superpixels and Polygons Using Simple Non-iterative Clustering , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Francisco Aparecido Rodrigues,et al.  Segmentation of Large Images with Complex Networks , 2012, 2012 25th SIBGRAPI Conference on Graphics, Patterns and Images.

[6]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[7]  Paul C. Smits,et al.  Updating land-cover maps by using texture information from very high-resolution space-borne imagery , 1999, IEEE Trans. Geosci. Remote. Sens..

[8]  Sara Bouzekri,et al.  A New Spectral Index for Extraction of Built-Up Area Using Landsat-8 Data , 2015, Journal of the Indian Society of Remote Sensing.

[9]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[10]  Wenbing Tao,et al.  Multilayer graph cuts based unsupervised color-texture image segmentation using multivariate mixed student's t-distribution and regional credibility merging , 2013, Pattern Recognit..

[11]  Yi Li,et al.  Built-Up Area Detection From Satellite Images Using Multikernel Learning, Multifield Integrating, and Multihypothesis Voting , 2015, IEEE Geoscience and Remote Sensing Letters.

[12]  Yihua Tan,et al.  Automatic extraction of built-up area based on deep convolution neural network , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[13]  Yihua Tan,et al.  Unsupervised Detection of Built-Up Areas From Multiple High-Resolution Remote Sensing Images , 2013, IEEE Geoscience and Remote Sensing Letters.

[14]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[16]  Ping Zhong,et al.  A Multiple Conditional Random Fields Ensemble Model for Urban Area Detection in Remote Sensing Optical Images , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Ying Yu,et al.  Accurate Urban Area Detection in Remote Sensing Images , 2015, IEEE Geoscience and Remote Sensing Letters.

[18]  Yihua Tan,et al.  Urban area detection using multiple Kernel Learning and graph cut , 2012, 2012 IEEE International Geoscience and Remote Sensing Symposium.

[19]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[20]  Xiaochun Cao,et al.  Unsupervised pixel-level video foreground object segmentation via shortest path algorithm , 2016, Neurocomputing.

[21]  Paul F. Whelan,et al.  Image segmentation based on the integration of colour-texture descriptors - A review , 2011, Pattern Recognit..

[22]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Jacob Goldberger,et al.  Urban-Area Segmentation Using Visual Words , 2009, IEEE Geoscience and Remote Sensing Letters.

[25]  Liangpei Zhang,et al.  Morphological Building/Shadow Index for Building Extraction From High-Resolution Imagery Over Urban Areas , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[26]  Yihua Tan,et al.  Cauchy Graph Embedding Optimization for Built-Up Areas Detection From High-Resolution Remote Sensing Images , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[27]  Dimitris Kaimaris,et al.  Identification and Area Measurement of the Built-up Area with the Built-up Index (BUI) , 2016 .

[28]  Wenjian Wang,et al.  Saliency-SVM: An automatic approach for image segmentation , 2014, Neurocomputing.

[29]  Kai Li,et al.  A Fast Large Size Image Segmentation Algorithm Based on Spectral Clustering , 2012, 2012 Fourth International Conference on Computational and Information Sciences.

[30]  Xiaojiang Chen,et al.  A novel fast image segmentation algorithm for large topographic maps , 2015, Neurocomputing.

[31]  Huchuan Lu,et al.  Superpixel level object recognition under local learning framework , 2012, 2012 19th IEEE International Conference on Image Processing.

[32]  Jianya Gong,et al.  Angular difference feature extraction for urban scene classification using ZY-3 multi-angle high-resolution satellite imagery , 2018 .

[33]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[34]  Chunhong Pan,et al.  Robust level set image segmentation via a local correntropy-based K-means clustering , 2014, Pattern Recognit..

[35]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Gang Liu,et al.  A perception-inspired building index for automatic built-up area detection in high-resolution satellite images , 2013, 2013 IEEE International Geoscience and Remote Sensing Symposium - IGARSS.

[37]  Xuelong Li,et al.  Optimized graph-based segmentation for ultrasound images , 2014, Neurocomputing.

[38]  Wei Wang,et al.  CNN based suburban building detection using monocular high resolution Google Earth images , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[39]  Rongjun Qin,et al.  Multi-level monitoring of subtle urban changes for the megacities of China using high-resolution multi-view satellite imagery , 2017 .

[40]  Xin Huang,et al.  Unsupervised Deep Feature Learning for Urban Village Detection from High-Resolution Remote Sensing Images , 2017 .