HANet: A Hierarchical Attention Network for Change Detection With Bitemporal Very-High-Resolution Remote Sensing Images

Benefiting from the developments in deep learning technology, deep-learning-based algorithms employing automatic feature extraction have achieved remarkable performance on the change detection (CD) task. However, the performance of existing deep-learning-based CD methods is hindered by the imbalance between changed and unchanged pixels. To tackle this problem, a progressive foreground-balanced sampling strategy on the basis of not adding change information is proposed in this article to help the model accurately learn the features of the changed pixels during the early training process and thereby improve detection performance. Furthermore, we design a discriminative Siamese network, hierarchical attention network (HANet), which can integrate multiscale features and refine detailed features. The main part of HANet is the HAN module, which is a lightweight and effective self-attention mechanism. Extensive experiments and ablation studies on two CD datasets with extremely unbalanced labels validate the effectiveness and efficiency of the proposed method.

[1]  Y. Zhong,et al.  Generating annual high resolution land cover products for 28 metropolises in China based on a deep super-resolution mapping network using Landsat imagery , 2022, GIScience & Remote Sensing.

[2]  Chen Wu,et al.  HyperNet: Self-Supervised Hyperspectral Spatial–Spectral Feature Understanding Network for Hyperspectral Change Detection , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Zhenwei Shi,et al.  Semantic-Aware Dense Representation Learning for Remote Sensing Image Change Detection , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Guisong Xia,et al.  An Empirical Study of Remote Sensing Pretraining , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Pedram Ghamisi,et al.  Consistency-Regularized Region-Growing Network for Semantic Segmentation of Urban Scenes With Point-Level Annotations , 2022, IEEE Transactions on Image Processing.

[6]  Qian Shi,et al.  Generating 2m fine-scale urban tree cover product over 34 metropolises in China based on deep context-aware sub-pixel mapping network , 2022, Int. J. Appl. Earth Obs. Geoinformation.

[7]  Vishal M. Patel,et al.  A Transformer-Based Siamese Network for Change Detection , 2022, IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium.

[8]  Liangpei Zhang,et al.  A Deeply Supervised Attention Metric-Based Network and an Open Aerial Image Dataset for Remote Sensing Change Detection , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Zhenwei Shi,et al.  Remote Sensing Image Change Detection With Transformers , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[10]  A. Marinoni,et al.  Super-Resolution-Based Change Detection Network With Stacked Attention Module for Images With Different Resolutions , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Jinyuan Shao,et al.  SNUNet-CD: A Densely Connected Siamese Network for Change Detection of VHR Images , 2021, IEEE Geoscience and Remote Sensing Letters.

[12]  Zhenwei Shi,et al.  Adversarial Instance Augmentation for Building Change Detection in Remote Sensing Images , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Bo Du,et al.  Deep building footprint update network: A semi-supervised method for updating existing building footprint from bi-temporal remote sensing images , 2021 .

[14]  Ruofei Zhong,et al.  Optical Remote Sensing Image Change Detection Based on Attention Mechanism and Image Difference , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Dacheng Tao,et al.  ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias , 2021, NeurIPS.

[16]  J. Guan,et al.  Accurate Few-shot Object Detection with Support-Query Mutual Guidance and Hybrid Loss , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Bo Du,et al.  Assessing the Threat of Adversarial Examples on Deep Neural Networks for Remote Sensing Scene Classification: Attacks and Defenses , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Ye Zhang,et al.  Deep Multiscale Siamese Network with Parallel Convolutional Structure and Self-Attention for Change Detection , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Xiao Xiang Zhu,et al.  On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances, and Million-AID , 2021, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[20]  Peng Yue,et al.  A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images , 2020 .

[21]  Prabuddh Kumar Mishra,et al.  Land use and land cover change detection using geospatial techniques in the Sikkim Himalaya, India , 2020, The Egyptian Journal of Remote Sensing and Space Science.

[22]  Bo Du,et al.  A Low-Rank and Sparse Matrix Decomposition- Based Dictionary Reconstruction and Anomaly Extraction Framework for Hyperspectral Anomaly Detection , 2020, IEEE Geoscience and Remote Sensing Letters.

[23]  Hao Chen,et al.  A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection , 2020, Remote. Sens..

[24]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Tim Salimans,et al.  Axial Attention in Multidimensional Transformers , 2019, ArXiv.

[26]  Yuanyuan Qin,et al.  Landslide mapping from multi-sensor data through improved change detection-based Markov random field , 2019, Remote Sensing of Environment.

[27]  Maria Vakalopoulou,et al.  Detecting Urban Changes with Recurrent Neural Networks from Multitemporal Sentinel-2 Data , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[28]  Bo Du,et al.  Change Detection in Multi-temporal VHR Images Based on Deep Siamese Multi-scale Convolutional Networks , 2019 .

[29]  Yongjun Zhang,et al.  End-to-End Change Detection for High Resolution Satellite Images Using Improved UNet++ , 2019, Remote. Sens..

[30]  Ling Wan,et al.  A Post-Classification Comparison Method for SAR and Optical Images Change Detection , 2019, IEEE Geoscience and Remote Sensing Letters.

[31]  Meng Lu,et al.  Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data Set , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Jun Fu,et al.  Dual Attention Network for Scene Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Gui-Song Xia,et al.  Learning RoI Transformer for Detecting Oriented Objects in Aerial Images , 2018, ArXiv.

[34]  Alexandre Boulch,et al.  Fully Convolutional Siamese Networks for Change Detection , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[35]  Nima Tajbakhsh,et al.  UNet++: A Nested U-Net Architecture for Medical Image Segmentation , 2018, DLMIA/ML-CDS@MICCAI.

[36]  Yury Vizilter,et al.  CHANGE DETECTION IN REMOTE SENSING IMAGES USING CONDITIONAL ADVERSARIAL NETWORKS , 2018, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[37]  Menglong Yan,et al.  Change Detection Based on Deep Siamese Convolutional Network for Optical Aerial Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[38]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[39]  Roberto Cipolla,et al.  Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Yoshua Bengio,et al.  Attention-Based Models for Speech Recognition , 2015, NIPS.

[41]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[42]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[43]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[45]  Keith Richards,et al.  Remote sensing of alpine lake water environment changes on the Tibetan Plateau and surroundings: A review , 2014 .

[46]  Bo Du,et al.  Slow Feature Analysis for Change Detection in Multispectral Imagery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[47]  C. Homer,et al.  Updating the 2001 National Land Cover Database Impervious Surface Products to 2006 using Landsat Imagery Change Detection Methods , 2010 .

[48]  Turgay Çelik,et al.  Unsupervised Change Detection in Satellite Images Using Principal Component Analysis and $k$-Means Clustering , 2009, IEEE Geoscience and Remote Sensing Letters.

[49]  Lorenzo Bruzzone,et al.  Automatic analysis of the difference image for unsupervised change detection , 2000, IEEE Trans. Geosci. Remote. Sens..

[50]  Knut Conradsen,et al.  Multivariate Alteration Detection (MAD) and MAF Postprocessing in Multispectral, Bitemporal Image Data: New Approaches to Change Detection Studies , 1998 .

[51]  Ashbindu Singh,et al.  Review Article Digital change detection techniques using remotely-sensed data , 1989 .