CLNet: Cross-layer convolutional neural network for change detection in optical remote sensing imagery

Abstract Change detection plays a crucial role in observing earth surface transition and has been widely investigated using deep learning methods. However, the current deep learning methods for pixel-wise change detection still suffer from limited accuracy, mainly due to their insufficient feature extraction and context aggregation. To address this limitation, we propose a novel Cross Layer convolutional neural Network (CLNet) in this paper, where the UNet structure is used as the backbone and newly designed Cross Layer Blocks (CLBs) are embedded to incorporate the multi-scale features and multi-level context information. The designed CLB starts with one input and then split into two parallel but asymmetric branches, which are leveraged to extract the multi-scale features by using different strides; and the feature maps, which come from the opposite branches but have the same size, are concatenated to incorporate multi-level context information. The designed CLBs aggregate the multi-scale features and multi-level context information so that the proposed CLNet can reuse extracted feature information and capture accurate pixel-wise change in complex scenes. Quantitative and qualitative experiments were conducted on a public very-high-resolution satellite image dataset (VHR-Dataset), a newly released building change detection dataset (LEVIR-CD Dataset) and an aerial building change detection dataset (WHU Building Dataset). The CLNet reached an F1-score of 0.921 and an overall accuracy of 98.1% with the VHR-Dataset, an F1-score of 0.900 and an overall accuracy of 98.9% with the LEVIR-CD Dataset, and an F1-score of 0.963 and an overall accuracy of 99.7% with the WHU Building Dataset. The experimental results with all the selected datasets showed that the proposed CLNet outperformed several state-of-the-art (SOTA) methods and achieved competitive accuracy and efficiency trade-offs. The code of CLNet will be released soon at: https://skyearth.org/publication/project/CLNet .

[1]  Lorenzo Bruzzone,et al.  Automatic analysis of the difference image for unsupervised change detection , 2000, IEEE Trans. Geosci. Remote. Sens..

[2]  Maoguo Gong,et al.  A Conditional Adversarial Network for Change Detection in Heterogeneous Images , 2019, IEEE Geoscience and Remote Sensing Letters.

[3]  Alexandre Boulch,et al.  High Resolution Semantic Change Detection , 2018, ArXiv.

[4]  Hao Chen,et al.  A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection , 2020, Remote. Sens..

[5]  Jian Yang,et al.  Landsat remote sensing approaches for monitoring long-term tree cover dynamics in semi-arid woodlands: Comparison of vegetation indices and spectral mixture analysis , 2012 .

[6]  Nazzareno Pierdicca,et al.  Satellite radar and optical remote sensing for earthquake damage detection: results from different case studies , 2006 .

[7]  Qihao Weng,et al.  Assessing Urban Environmental Quality Change of Indianapolis, United States, by the Remote Sensing and GIS Integration , 2011, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[8]  Meng Lu,et al.  Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data Set , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Ashish Ghosh,et al.  An unsupervised context-sensitive change detection technique based on modified self-organizing feature map neural network , 2009, Int. J. Approx. Reason..

[10]  Shuying Li,et al.  Landslide Inventory Mapping From Bitemporal Images Using Deep Convolutional Neural Networks , 2019, IEEE Geoscience and Remote Sensing Letters.

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Bo Du,et al.  Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art , 2016, IEEE Geoscience and Remote Sensing Magazine.

[13]  Tamás Szirányi,et al.  Change Detection in Optical Aerial Images by a Multilayer Conditional Mixed Markov Model , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Xuezhi Feng,et al.  Cosegmentation for Object-Based Building Change Detection From High-Resolution Remotely Sensed Images , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[16]  George Vosselman,et al.  Monitoring household upgrading in unplanned settlements with unmanned aerial vehicles , 2020, Int. J. Appl. Earth Obs. Geoinformation.

[17]  Weiqi Zhou,et al.  A new approach for land cover classification and change analysis: Integrating backdating and an object-based method , 2016 .

[18]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Chi Zhang,et al.  Detecting Large-Scale Urban Land Cover Changes from Very High Resolution Remote Sensing Images Using CNN-Based Classification , 2019, ISPRS Int. J. Geo Inf..

[20]  Yury Vizilter,et al.  CHANGE DETECTION IN REMOTE SENSING IMAGES USING CONDITIONAL ADVERSARIAL NETWORKS , 2018, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[21]  C. Homer,et al.  Updating the 2001 National Land Cover Database Impervious Surface Products to 2006 using Landsat Imagery Change Detection Methods , 2010 .

[22]  Xuehong Chen,et al.  A spectral gradient difference based approach for land cover change detection , 2013 .

[23]  Francesca Bovolo,et al.  A Theoretical Framework for Unsupervised Change Detection Based on Change Vector Analysis in the Polar Domain , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Xiao Xiang Zhu,et al.  Deep learning in remote sensing: a review , 2017, ArXiv.

[25]  Francesca Bovolo,et al.  Rayleigh-Rice Mixture Parameter Estimation via EM Algorithm for Change Detection in Multispectral Images , 2015, IEEE Transactions on Image Processing.

[26]  G. Hulley,et al.  Thermal-based techniques for land cover change detection using a new dynamic MODIS multispectral emissivity product (MOD21) , 2014 .

[27]  Jia Liu,et al.  Change detection based on deep feature representation and mapping transformation for multi-spatial-resolution remote sensing images , 2016 .

[28]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[29]  Hannes Taubenböck,et al.  Unsupervised change detection in VHR remote sensing imagery - an object-based clustering approach in a dynamic urban environment , 2017, Int. J. Appl. Earth Obs. Geoinformation.

[30]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Dongmei Chen,et al.  Change detection from remotely sensed images: From pixel-based to object-based approaches , 2013 .

[32]  Maoguo Gong,et al.  Change Detection in Synthetic Aperture Radar Images Based on Deep Neural Networks , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Monika Kuffer,et al.  The Temporal Dynamics of Slums Employing a CNN-Based Change Detection Approach , 2019, Remote. Sens..

[34]  Germán Ros,et al.  Street-view change detection with deconvolutional networks , 2016, Autonomous Robots.

[35]  Qian Du,et al.  GETNET: A General End-to-End 2-D CNN Framework for Hyperspectral Image Change Detection , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Bo Du,et al.  Asymmetric Siamese Networks for Semantic Change Detection , 2020, ArXiv.

[37]  Yongjun Zhang,et al.  Object-based change detection from satellite imagery by segmentation optimization and multi-features fusion , 2017 .

[38]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Alexandre Boulch,et al.  Fully Convolutional Siamese Networks for Change Detection , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[40]  Nima Tajbakhsh,et al.  UNet++: A Nested U-Net Architecture for Medical Image Segmentation , 2018, DLMIA/ML-CDS@MICCAI.

[41]  Maoguo Gong,et al.  A Generative Discriminatory Classified Network for Change Detection in Multispectral Imagery , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[42]  Xiao Xiang Zhu,et al.  Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Yongjun Zhang,et al.  End-to-End Change Detection for High Resolution Satellite Images Using Improved UNet++ , 2019, Remote. Sens..

[44]  Huseyin Gokhan Akcay,et al.  Building detection using directional spatial constraints , 2010, 2010 IEEE International Geoscience and Remote Sensing Symposium.

[45]  Lichao Mou,et al.  Learning a Transferable Change Rule from a Recurrent Neural Network for Land Cover Change Detection , 2016, Remote. Sens..

[46]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Sang-Eun Park,et al.  Dual-Dense Convolution Network for Change Detection of High-Resolution Panchromatic Imagery , 2018, Applied Sciences.

[48]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[49]  Bo Du,et al.  Deep Siamese Multi-scale Convolutional Network for Change Detection in Multi-temporal VHR Images , 2019, 2019 10th International Workshop on the Analysis of Multitemporal Remote Sensing Images (MultiTemp).

[50]  Patrick Bogaert,et al.  Forest change detection by statistical object-based method , 2006 .

[51]  Haiyan Guan,et al.  Unsupervised change detection method based on saliency analysis and convolutional neural network , 2019, Journal of Applied Remote Sensing.