Advances and Challenges in Deep Learning-Based Change Detection for Remote Sensing Images: A Review through Various Learning Paradigms

Change detection (CD) in remote sensing (RS) imagery is a pivotal method for detecting changes in the Earth’s surface, finding wide applications in urban planning, disaster management, and national security. Recently, deep learning (DL) has experienced explosive growth and, with its superior capabilities in feature learning and pattern recognition, it has introduced innovative approaches to CD. This review explores the latest techniques, applications, and challenges in DL-based CD, examining them through the lens of various learning paradigms, including fully supervised, semi-supervised, weakly supervised, and unsupervised. Initially, the review introduces the basic network architectures for CD methods using DL. Then, it provides a comprehensive analysis of CD methods under different learning paradigms, summarizing commonly used frameworks. Additionally, an overview of publicly available datasets for CD is offered. Finally, the review addresses the opportunities and challenges in the field, including: (a) incomplete supervised CD, encompassing semi-supervised and weakly supervised methods, which is still in its infancy and requires further in-depth investigation; (b) the potential of self-supervised learning, offering significant opportunities for Few-shot and One-shot Learning of CD; (c) the development of Foundation Models, with their multi-task adaptability, providing new perspectives and tools for CD; and (d) the expansion of data sources, presenting both opportunities and challenges for multimodal CD. These areas suggest promising directions for future research in CD. In conclusion, this review aims to assist researchers in gaining a comprehensive understanding of the CD field.

[1]  Hui Liu,et al.  A Point Cloud Segmentation Method for Dim and Cluttered Underground Tunnel Scenes Based on the Segment Anything Model , 2023, Remote. Sens..

[2]  D. Cerra,et al.  A 2D/3D multimodal data simulation approach with applications on urban semantic segmentation, building extraction and change detection , 2023, ISPRS Journal of Photogrammetry and Remote Sensing.

[3]  Gong Cheng,et al.  TransY-Net: Learning Fully Transformer Networks for Change Detection of Remote Sensing Images , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Ziyan Zhang,et al.  VcT: Visual Change Transformer for Remote Sensing Image Change Detection , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[5]  W. Gonçalves,et al.  The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot , 2023, Int. J. Appl. Earth Obs. Geoinformation.

[6]  Yulin Ding,et al.  BCE-Net: Reliable Building Footprints Change Extraction based on Historical Map and Up-to-Date Images using Contrastive Learning , 2023, ISPRS Journal of Photogrammetry and Remote Sensing.

[7]  N. Yokoya,et al.  Fourier domain structural relationship analysis for unsupervised multimodal change detection , 2023, ISPRS Journal of Photogrammetry and Remote Sensing.

[8]  W. Shi,et al.  Cross-domain landslide mapping from large-scale remote sensing images using prototype-guided domain-aware progressive representation learning , 2023, ISPRS Journal of Photogrammetry and Remote Sensing.

[9]  Guisong Xia,et al.  Detecting building changes with off-nadir aerial images , 2023, Science China Information Sciences.

[10]  Jun Pan,et al.  MTCNet: Multitask consistency network with single temporal supervision for semi-supervised building change detection , 2022, Int. J. Appl. Earth Obs. Geoinformation.

[11]  Qing Guo,et al.  Background-Mixed Augmentation for Weakly Supervised Change Detection , 2022, AAAI.

[12]  Xin-Yi Tong,et al.  MFATNet: Multi-Scale Feature Aggregation via Transformer for Remote Sensing Image Change Detection , 2022, Remote. Sens..

[13]  N. Yokoya,et al.  Unsupervised Multimodal Change Detection Based on Structural Relationship Graph Representation Learning , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Di Wu,et al.  Semi-supervised label propagation for multi-source remote sensing image change detection , 2022, Computers & Geosciences.

[15]  Lingxi Xie,et al.  A Survey on Label-Efficient Deep Image Segmentation: Bridging the Gap Between Weak Supervision and Dense Prediction , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Chengzhen Sun,et al.  SemiSANet: A Semi-Supervised High-Resolution Remote Sensing Image Change Detection Model Using Siamese Networks with Graph Attention , 2022, Remote. Sens..

[17]  Zhenwei Shi,et al.  Semantic-Aware Dense Representation Learning for Remote Sensing Image Change Detection , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[18]  D. Tapete,et al.  A domain adaptation neural network for change detection with heterogeneous optical and SAR remote sensing images , 2022, Int. J. Appl. Earth Obs. Geoinformation.

[19]  G. Mountrakis,et al.  Fusion of optical, radar and waveform LiDAR observations for land cover classification , 2022, ISPRS Journal of Photogrammetry and Remote Sensing.

[20]  D. Peng,et al.  An Overview of the Applications of Earth Observation Satellite Data: Impacts and Future Trends , 2022, Remote. Sens..

[21]  Jungbeom Lee,et al.  Anti-Adversarially Manipulated Attributions for Weakly Supervised Semantic Segmentation and Object Localization , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Zia Khan,et al.  Deep Learning-Based Change Detection in Remote Sensing Images: A Review , 2022, Remote Sensing.

[23]  Deren Li,et al.  Land-Use/Land-Cover change detection based on a Siamese global learning framework for high spatial resolution remote sensing imagery , 2022, ISPRS Journal of Photogrammetry and Remote Sensing.

[24]  Bo Du,et al.  Fully Convolutional Change Detection Framework With Generative Adversarial Network for Unsupervised, Weakly Supervised and Regional Supervised Change Detection , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Y. Zhong,et al.  ChangeMask: Deep multi-task encoder-transformer-decoder architecture for semantic change detection , 2022, ISPRS Journal of Photogrammetry and Remote Sensing.

[26]  Alexander J. Smola,et al.  Improving Semantic Segmentation via Efficient Self-Training , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Xinghua Li,et al.  A deep translation (GAN) based change detection network for optical and SAR remote sensing images , 2021 .

[28]  L. Bruzzone,et al.  Bi-Temporal Semantic Reasoning for the Semantic Change Detection in HR Remote Sensing Images , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Licheng Jiao,et al.  Adaptive Affinity Loss and Erroneous Pseudo-Label Refinement for Weakly Supervised Semantic Segmentation , 2021, ACM Multimedia.

[30]  Donghai Xie,et al.  S2Looking: A Satellite Side-Looking Dataset for Building Change Detection , 2021, Remote. Sens..

[31]  Liangpei Zhang,et al.  A Deeply Supervised Attention Metric-Based Network and an Open Aerial Image Dataset for Remote Sensing Change Detection , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Ming-Ming Cheng,et al.  LayerCAM: Exploring Hierarchical Class Activation Maps for Localization , 2021, IEEE Transactions on Image Processing.

[33]  Chunhua Shen,et al.  Twins: Revisiting the Design of Spatial Attention in Vision Transformers , 2021, NeurIPS.

[34]  L. Bruzzone,et al.  Self-Supervised Change Detection in Multiview Remote Sensing Images , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Zhenwei Shi,et al.  Remote Sensing Image Change Detection With Transformers , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Enhua Wu,et al.  Transformer in Transformer , 2021, NeurIPS.

[37]  Jinyuan Shao,et al.  SNUNet-CD: A Densely Connected Siamese Network for Change Detection of VHR Images , 2021, IEEE Geoscience and Remote Sensing Letters.

[38]  Xiaoxiang Zhu,et al.  Self-Supervised Multisensor Change Detection , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Hanlin Yin,et al.  DifUnet++: A Satellite Images Change Detection Network Based on Unet++ and Differential Pyramid , 2021, IEEE Geoscience and Remote Sensing Letters.

[40]  Lorenzo Bruzzone,et al.  SemiCDNet: A Semisupervised Convolutional Neural Network for Change Detection in High Resolution Remote-Sensing Images , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Peng Yue,et al.  A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images , 2020 .

[42]  Wenzhong Shi,et al.  Deep Multiple Instance Learning for Landslide Mapping , 2020, IEEE Geoscience and Remote Sensing Letters.

[43]  Maoguo Gong,et al.  A Semisupervised GAN-Based Multiple Change Detection Framework in Multi-Spectral Images , 2020, IEEE Geoscience and Remote Sensing Letters.

[44]  Priya Goyal,et al.  Unsupervised Learning of Visual Features by Contrasting Cluster Assignments , 2020, NeurIPS.

[45]  Pierre H. Richemond,et al.  Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning , 2020, NeurIPS.

[46]  Max Mignotte,et al.  Deep Learning for Change Detection in Remote Sensing Images: Comprehensive Review and Meta-Analysis , 2020, IEEE Access.

[47]  Tom B. Brown,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[48]  Wenzhong Shi,et al.  Change Detection Based on Artificial Intelligence: State-of-the-Art and Challenges , 2020, Remote. Sens..

[49]  Hao Chen,et al.  A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection , 2020, Remote. Sens..

[50]  Gabriele Moser,et al.  Code-Aligned Autoencoders for Unsupervised Change Detection in Multimodal Remote Sensing Images , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[51]  David Berthelot,et al.  FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence , 2020, NeurIPS.

[52]  Bo Du,et al.  Unsupervised Change Detection in Multitemporal VHR Images Based on Deep Kernel PCA Convolutional Mapping Network , 2019, IEEE Transactions on Cybernetics.

[53]  Peter J. Liu,et al.  Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer , 2019, J. Mach. Learn. Res..

[54]  Sudipan Saha,et al.  Unsupervised change-detection based on convolutional-autoencoder feature extraction , 2019, Remote Sensing.

[55]  Yee Whye Teh,et al.  Stacked Capsule Autoencoders , 2019, NeurIPS.

[56]  Yongjun Zhang,et al.  End-to-End Change Detection for High Resolution Satellite Images Using Improved UNet++ , 2019, Remote. Sens..

[57]  J. Anitha,et al.  Change detection techniques for remote sensing applications: a survey , 2019, Earth Science Informatics.

[58]  Francesca Bovolo,et al.  Unsupervised Deep Change Vector Analysis for Multiple-Change Detection in VHR Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[59]  Maoguo Gong,et al.  A Generative Discriminatory Classified Network for Change Detection in Multispectral Imagery , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[60]  Qian Du,et al.  GETNET: A General End-to-End 2-D CNN Framework for Hyperspectral Image Change Detection , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[61]  Meng Lu,et al.  Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data Set , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[62]  Bo Du,et al.  Unsupervised Deep Slow Feature Analysis for Change Detection in Multi-Temporal Remote Sensing Images , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[63]  Jia Liu,et al.  Iterative feature mapping network for detecting multiple changes in multi-source remote sensing images , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[64]  Alexandre Boulch,et al.  Multitask learning for large-scale semantic change detection , 2018, Comput. Vis. Image Underst..

[65]  Chen Chen,et al.  Deep Learning and Superpixel Feature Extraction Based on Contractive Autoencoder for Change Detection in SAR Images , 2018, IEEE Transactions on Industrial Informatics.

[66]  Chong Liu,et al.  Urban Change Detection Based on Dempster-Shafer Theory for Multitemporal Very High-Resolution Imagery , 2018, Remote. Sens..

[67]  Yury Vizilter,et al.  CHANGE DETECTION IN REMOTE SENSING IMAGES USING CONDITIONAL ADVERSARIAL NETWORKS , 2018, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[68]  Xiao Xiang Zhu,et al.  Long-Term Annual Mapping of Four Cities on Different Continents by Applying a Deep Information Learning Method to Landsat Data , 2018, Remote. Sens..

[69]  Maoguo Gong,et al.  A Deep Convolutional Coupling Network for Change Detection Based on Heterogeneous Optical and Radar Images , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[70]  Zhetao Li,et al.  Generative Adversarial Networks for Change Detection in Multispectral Imagery , 2017, IEEE Geoscience and Remote Sensing Letters.

[71]  Maoguo Gong,et al.  Feature learning and change feature classification based on deep learning for ternary change detection in SAR images , 2017 .

[72]  Evert P L van Nieuwenburg,et al.  Discriminative Cooperative Networks for Detecting Phase Transitions. , 2017, Physical review letters.

[73]  Bo Du,et al.  Kernel Slow Feature Analysis for Scene Change Detection , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[74]  Jon Atli Benediktsson,et al.  Big Data for Remote Sensing: Challenges and Opportunities , 2016, Proceedings of the IEEE.

[75]  Maoguo Gong,et al.  Feature-Level Change Detection Using Deep Representation and Feature Change Analysis for Multispectral Imagery , 2016, IEEE Geoscience and Remote Sensing Letters.

[76]  Lichao Mou,et al.  Learning a Transferable Change Rule from a Recurrent Neural Network for Land Cover Change Detection , 2016, Remote. Sens..

[77]  Bo Du,et al.  Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art , 2016, IEEE Geoscience and Remote Sensing Magazine.

[78]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[79]  Albert Y. Zomaya,et al.  Remote sensing big data computing: Challenges and opportunities , 2015, Future Gener. Comput. Syst..

[80]  Aaron C. Courville,et al.  Generative adversarial networks , 2014, Commun. ACM.

[81]  C. Woodcock,et al.  Continuous change detection and classification of land cover using all available Landsat data , 2014 .

[82]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[83]  H. Abdi,et al.  Principal component analysis , 2010 .

[84]  Turgay Çelik,et al.  Unsupervised Change Detection in Satellite Images Using Principal Component Analysis and $k$-Means Clustering , 2009, IEEE Geoscience and Remote Sensing Letters.

[85]  Tamás Szirányi,et al.  Change Detection in Optical Aerial Images by a Multilayer Conditional Mixed Markov Model , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[86]  Jinsong Deng,et al.  PCA‐based land‐use change detection and analysis using multitemporal and multisensor satellite data , 2008 .

[87]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[88]  Allan Aasbjerg Nielsen,et al.  The Regularized Iteratively Reweighted MAD Method for Change Detection in Multi- and Hyperspectral Data , 2007, IEEE Transactions on Image Processing.

[89]  P. Gong,et al.  Land-use/land-cover change detection using improved change-vector analysis , 2003 .

[90]  Lorenzo Bruzzone,et al.  Automatic analysis of the difference image for unsupervised change detection , 2000, IEEE Trans. Geosci. Remote. Sens..

[91]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[92]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[93]  M. Kramer Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[94]  Ashbindu Singh,et al.  Review Article Digital change detection techniques using remotely-sensed data , 1989 .

[95]  K. Esbensen,et al.  Principal component analysis , 1987 .

[96]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[97]  Min Zhang,et al.  STCRNet: A Semi-Supervised Network Based on Self-Training and Consistency Regularization for Change Detection in VHR Remote Sensing Images , 2024, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[98]  Yu Wang,et al.  AERNet: An Attention-Guided Edge Refinement Network and a Dataset for Remote Sensing Building Change Detection , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[99]  Lizhe Wang,et al.  Lightweight Remote Sensing Change Detection With Progressive Feature Aggregation and Supervised Attention , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[100]  C. Du,et al.  SemiBuildingChange: A Semi-Supervised High-Resolution Remote Sensing Image Building Change Detection Method With a Pseudo Bitemporal Data Generator , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[101]  D. Mo,et al.  LightCDNet: Lightweight Change Detection Network Based on VHR Images , 2023, IEEE Geoscience and Remote Sensing Letters.

[102]  Gang Li,et al.  ConvTransNet: A CNN–Transformer Network for Change Detection With Multiscale Global–Local Representations , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[103]  Jianwei Zheng,et al.  Change Detection on Remote Sensing Images Using Dual-Branch Multilevel Intertemporal Network , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[104]  Yaochu Jin,et al.  Triple Change Detection Network via Joint Multifrequency and Full-Scale Swin-Transformer for Remote Sensing Images , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[105]  Yaochu Jin,et al.  Ultralightweight Spatial–Spectral Feature Cooperation Network for Change Detection in Remote Sensing Images , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[106]  Liejun Wang,et al.  Asymmetric Cross-Attention Hierarchical Network Based on CNN and Transformer for Bitemporal Remote Sensing Images Change Detection , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[107]  Wenzhong Shi,et al.  Landslide Mapping Using Multilevel-Feature-Enhancement Change Detection Network , 2023, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[108]  Jiayi Li,et al.  Joint Self-Training and Rebalanced Consistency Learning for Semi-Supervised Change Detection , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[109]  Deren Li,et al.  AFDE-Net: Building Change Detection Using Attention-Based Feature Differential Enhancement for Satellite Imagery , 2023, IEEE Geoscience and Remote Sensing Letters.

[110]  Min Zhang,et al.  CS-WSCDNet: Class Activation Mapping and Segment Anything Model-Based Framework for Weakly Supervised Change Detection , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[111]  Jialu Liu,et al.  Self-Supervised Global–Local Contrastive Learning for Fine-Grained Change Detection in VHR Images , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[112]  Yang Zhao,et al.  Self-Supervised Pretraining via Multimodality Images With Transformer for Change Detection , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[113]  Wenzhong Shi,et al.  MSFF-CDNet: A Multiscale Feature Fusion Change Detection Network for Bi-Temporal High-Resolution Remote Sensing Image , 2023, IEEE Geoscience and Remote Sensing Letters.

[114]  Tao Chen,et al.  A Siamese Network Based U-Net for Change Detection in High Resolution Remote Sensing Images , 2022, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[115]  Jinjiang Li,et al.  Remote Sensing Image Change Detection Transformer Network Based on Dual-Feature Mixed Attention , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[116]  Q. Shi,et al.  PA-Former: Learning Prior-Aware Transformer for Remote Sensing Building Change Detection , 2022, IEEE Geoscience and Remote Sensing Letters.

[117]  Yanning Zhang,et al.  ADHR-CDNet: Attentive Differential High-Resolution Change Detection Network for Remote Sensing Images , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[118]  Guangzhao Chen,et al.  MSCDUNet: A Deep Learning Framework for Built-Up Area Change Detection Integrating Multispectral, SAR, and VHR Data , 2022, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[119]  Zhengxia Zou,et al.  Remote Sensing Image Change Captioning With Dual-Branch Transformers: A New Method and a Large Scale Dataset , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[120]  Ruofei Zhong,et al.  TransUNetCD: A Hybrid Transformer Network for Change Detection in Optical Remote-Sensing Images , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[121]  L. Bruzzone,et al.  A Self-Supervised Approach to Pixel-Level Change Detection in Bi-Temporal RS Images , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[122]  Jinjiang Li,et al.  PSTNet: Progressive Sampling Transformer Network for Remote Sensing Image Change Detection , 2022, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[123]  Richard C. Wilson,et al.  Reliable Contrastive Learning for Semi-Supervised Change Detection in Remote Sensing Images , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[124]  Haojun Deng,et al.  A CNN-Transformer Network With Multiscale Context Aggregation for Fine-Grained Cropland Change Detection , 2022, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[125]  Zhenming Peng,et al.  MSTDSNet-CD: Multiscale Swin Transformer and Deeply Supervised Network for Change Detection of the Fast-Growing Urban Regions , 2022, IEEE Geoscience and Remote Sensing Letters.

[126]  Q. Shi,et al.  Learning Token-Aligned Representations With Multimodel Transformers for Different-Resolution Change Detection , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[127]  Lianru Gao,et al.  Spectral–Spatial–Temporal Transformers for Hyperspectral Image Change Detection , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[128]  Zhenghao Chen,et al.  Digging Into Pseudo Label: A Low-Budget Approach for Semi-Supervised Semantic Segmentation , 2020, IEEE Access.

[129]  Geoffrey E. Hinton,et al.  Supporting Online Material for Reducing the Dimensionality of Data with Neural Networks , 2006 .

[130]  R. D. Johnson,et al.  Change vector analysis: A technique for the multispectral monitoring of land cover and condition , 1998 .

[131]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.