NestNet: a multiscale convolutional neural network for remote sensing image change detection

ABSTRACT With the rapid development of remote sensing technologies, the frequency of observations of the same location is increasing, and many satellites and sensors produced a large amount of time series images. These images make long-term change detection and dynamic characteristic estimation of ground features possible. However, conventional remote sensing image change detection methods mostly rely on manual visual interpretation and supervised or unsupervised computer-aided classification. Traditional methods always face many bottlenecks when processing big and fast-growing datasets, such as low computational efficiency, low level of automation, and different identification standards and accuracies caused by different operators. With the rapid accumulation of remote sensing data, it has become an important but more challenging task to conduct change detection in a more precise, automated and standardized way. The development of geointelligent computing technologies provides a means of solving these problems and improve the accuracy and efficiency of remote sensing image change detection. In this paper, we presented a novel deep learning model called nest network(NestNet) based on a convolutional neural network to improve the accuracy of the automatic change detection task by using remotely sensed time series images. NestNet extracts the respective features of bi-temporal images using an encoding parallel module and subsequently employs absolute different operations to process the features of two images. Compared with change detection method based on U-Shaped network plus plus (UNet++), the parallel module improves the efficiency of NestNet. Finally, a decoding module is used to generate a predicted change image. This paper compares NestNet to traditional methods and state-of-the-art deep learning models on two datasets. The experimental results demonstrate that the accuracy of NestNet is better than that of state-of-the-art methods. It can be concluded that the NestNet model is a potential approach for change detection using high resolution remote sensing images.

[1]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Yongjun Zhang,et al.  End-to-End Change Detection for High Resolution Satellite Images Using Improved UNet++ , 2019, Remote. Sens..

[3]  Maoguo Gong,et al.  Feature learning and change feature classification based on deep learning for ternary change detection in SAR images , 2017 .

[4]  Allan Aasbjerg Nielsen,et al.  The Regularized Iteratively Reweighted MAD Method for Change Detection in Multi- and Hyperspectral Data , 2007, IEEE Transactions on Image Processing.

[5]  Albert Y. Zomaya,et al.  Remote sensing big data computing: Challenges and opportunities , 2015, Future Gener. Comput. Syst..

[6]  Weiqi Zhou,et al.  A new approach for land cover classification and change analysis: Integrating backdating and an object-based method , 2016 .

[7]  Liangpei Zhang,et al.  A Multifeature Tensor for Remote-Sensing Target Recognition , 2011, IEEE Geoscience and Remote Sensing Letters.

[8]  Bo Du,et al.  A post-classification change detection method based on iterative slow feature analysis and Bayesian soft fusion , 2017, Remote Sensing of Environment.

[9]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  W. Malila.  Change Vector Analysis: An Approach for Detecting Forest Changes with Landsat , 1980 .

[11]  Patrick Bogaert,et al.  Forest change detection by statistical object-based method , 2006 .

[12]  Xia Li,et al.  A three-component method for timely detection of land cover changes using polarimetric SAR images , 2015 .

[13]  C. Homer,et al.  Updating the 2001 National Land Cover Database Impervious Surface Products to 2006 using Landsat Imagery Change Detection Methods , 2010 .

[14]  Lichao Mou,et al.  Learning a Transferable Change Rule from a Recurrent Neural Network for Land Cover Change Detection , 2016, Remote. Sens..

[15]  J. Fry,et al.  Updating the 2001 National Land Cover Database land cover classification to 2006 by using Landsat imagery change detection methods , 2009 .

[16]  Molly E. Brown.  Satellite Remote Sensing in Agriculture and Food Security Assessment , 2015 .

[17]  Xiao Xiang Zhu,et al.  Long-Term Annual Mapping of Four Cities on Different Continents by Applying a Deep Information Learning Method to Landsat Data , 2018, Remote. Sens..

[18]  Alexandre Boulch,et al.  High Resolution Semantic Change Detection , 2018, ArXiv.

[19]  Jon Atli Benediktsson,et al.  A toolbox for unsupervised change detection analysis , 2016 .

[20]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[21]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Qian Du,et al.  Remote Sensing Big Data: Theory, Methods and Applications , 2018, Remote. Sens..

[23]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Turgay Çelik,et al.  Unsupervised Change Detection in Satellite Images Using Principal Component Analysis and $k$-Means Clustering , 2009, IEEE Geoscience and Remote Sensing Letters.

[25]  Junjun Jiang,et al.  Edge-Enhanced GAN for Remote Sensing Image Superresolution , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Jia Liu,et al.  Iterative feature mapping network for detecting multiple changes in multi-source remote sensing images , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[27]  Nazzareno Pierdicca,et al.  Satellite radar and optical remote sensing for earthquake damage detection: results from different case studies , 2006 .

[28]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[29]  Bo Du,et al.  Slow Feature Analysis for Change Detection in Multispectral Imagery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[30]  Michael Kampffmeyer,et al.  Semantic Segmentation of Small Objects and Modeling of Uncertainty in Urban Remote Sensing Images Using Deep Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[31]  Hanqing Lu,et al.  Fast Object-Level Change Detection for VHR Images , 2010, IEEE Geoscience and Remote Sensing Letters.

[32]  Jia Liu,et al.  Change detection based on deep feature representation and mapping transformation for multi-spatial-resolution remote sensing images , 2016 .

[33]  Christian Wietfeld,et al.  Using Public Network Infrastructures for UAV Remote Sensing in Civilian Security Operations , 2011 .

[34]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[35]  Knut Conradsen,et al.  Multivariate Alteration Detection (MAD) and MAF Postprocessing in Multispectral, Bitemporal Image Data: New Approaches to Change Detection Studies , 1998 .

[36]  Ronald Kemker,et al.  Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[37]  Lei Ma,et al.  Identifying industrial heat sources using time-series of the VIIRS Nightfire product with an object-oriented approach , 2018 .

[38]  Francesca Bovolo,et al.  Sequential Spectral Change Vector Analysis for Iteratively Discovering and Detecting Multiple Changes in Hyperspectral Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Francesca Bovolo,et al.  A Theoretical Framework for Unsupervised Change Detection Based on Change Vector Analysis in the Polar Domain , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[40]  Xiaofang Li,et al.  Assessing urban public security based on remote sensing and GIS , 2015, 2015 23rd International Conference on Geoinformatics.

[41]  Jinsong Deng,et al.  PCA‐based land‐use change detection and analysis using multitemporal and multisensor satellite data , 2008 .

[42]  Alexandre Boulch,et al.  Fully Convolutional Siamese Networks for Change Detection , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[43]  Nima Tajbakhsh,et al.  UNet++: A Nested U-Net Architecture for Medical Image Segmentation , 2018, DLMIA/ML-CDS@MICCAI.

[44]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Liu Qin,et al.  Rasterization Computing-Based Parallel Vector Polygon Overlay Analysis Algorithms Using OpenMP and MPI , 2018, IEEE Access.

[46]  Qiang Zhao,et al.  Change detection with absolute difference of multiscale deep features , 2020, Neurocomputing.

[47]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[48]  Alexandre Boulch,et al.  Urban Change Detection for Multispectral Earth Observation Using Convolutional Neural Networks , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[49]  Yury Vizilter,et al.  CHANGE DETECTION IN REMOTE SENSING IMAGES USING CONDITIONAL ADVERSARIAL NETWORKS , 2018, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[50]  Yue Zhang,et al.  Rotation-aware and multi-scale convolutional neural network for object detection in remote sensing images , 2020 .