Deep Learning and Transfer Learning applied to Sentinel-1 DInSAR and Sentinel-2 optical satellite imagery for change detection

This paper discusses Deep Learning (DL) and Transfer Learning (TL) state of the art techniques applied to a binary classification task for change detection in satellite imagery. A blob detection algorithm is applied to a Differential Interferometric Synthetic Aperture Radar (DInSAR) generated displacement map. The blobs are classified as either positive, corresponding to uplift or subsidence, or negative, corresponding to noise. The novel dataset consists of Sentinel-1 DInSAR processed georeferenced images of displacement, phase, coherence and RGB Sentinel-2 optical satellite imagery of the blobs. TL via Feature Extraction (FE) is applied using numerous DL models with weights pretrained on the ImageNet dataset to generate feature maps after removing the last predictive layer. A Logistic Regression classifier is then applied to the features. Fine-Tuning (FT) and Random Initialisation (RI), training from scratch, are also applied to ResNet-50 and EfficientNet B4 architectures. The best performing model (85.76%) is the ResNet-50 using FE. Small ensembles of some models are also investigated. An ensemble of ResNet-50, ResNeXt-50 and EfficientNet B4 has an accuracy of 84.83%. TL via FE with the ResNet-50 has an accuracy of approx. 9% and 8% higher than when using it for TL via FT or RI respectively. The EfficientNet B4 obtained an accuracy of 82.29% for FE, 66.35% for FT and 50.00% (as good as a random guess) for RI.

[1]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[2]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  D. Hubel,et al.  Receptive fields of single neurones in the cat's striate cortex , 1959, The Journal of physiology.

[4]  Jude Shavlik,et al.  Chapter 11 Transfer Learning , 2009 .

[5]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[6]  Chen Wang,et al.  Automated Geophysical Classification of Sentinel-L Wave Mode Sar Images Through Deep-Learning , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[7]  Claus Aranha,et al.  Data Augmentation Using GANs , 2019, ArXiv.

[8]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Andreas Dengel,et al.  EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[10]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Quoc V. Le,et al.  AutoAugment: Learning Augmentation Policies from Data , 2018, ArXiv.

[12]  Frank Warmerdam,et al.  The Geospatial Data Abstraction Library , 2008 .

[13]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[16]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[17]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[18]  Xiao Xiang Zhu,et al.  Mapping Human Settlements with Multi-seasonal Sentinel-2 Imagery and Attention-based ResNeXt , 2019, 2019 Joint Urban Remote Sensing Event (JURSE).

[19]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[20]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[21]  Maria Vakalopoulou,et al.  Detecting Urban Changes with Recurrent Neural Networks from Multitemporal Sentinel-2 Data , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[22]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[23]  Unsupervised deep learning based change detection in Sentinel-2 images , 2019, 2019 10th International Workshop on the Analysis of Multitemporal Remote Sensing Images (MultiTemp).

[24]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[25]  Quoc V. Le,et al.  Do Better ImageNet Models Transfer Better? , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[27]  S. C. Kremer,et al.  Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies , 2001 .

[28]  Hong Zhang,et al.  Combining single shot multibox detector with transfer learning for ship detection using Sentinel-1 images , 2017, 2017 SAR in Big Data Era: Models, Methods and Applications (BIGSARDATA).

[29]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[30]  F. Rocca,et al.  InSAR Principles-Guidelines for SAR Interferometry Processing and Interpretation , 2007 .