Self-supervised pre-training enhances change detection in Sentinel-2 imagery

This is the preprint version of the paper published in the Pattern Recognition and Remote Sensing workshop (PRRS) 2021, held in ICPR 2020/2021 (virtual). For the postprint, please refer to the official LNCS publication of the conference proceedings. While annotated images for change detection using satellite imagery are scarce and costly to obtain, there is a wealth of unlabeled images being generated every day. In order to leverage these data to learn an image representation more adequate for change detection, we explore methods that exploit the temporal consistency of Sentinel-2 times series to obtain a usable self-supervised learning signal. For this, we build and make publicly available (https://zenodo.org/record/4280482) the Sentinel2 Multitemporal Cities Pairs (S2MTCP) dataset, containing multitemporal image pairs from 1520 urban areas worldwide. We test the results of multiple self-supervised learning methods for pre-training models for change detection and apply it on a public change detection dataset made of Sentinel-2 image pairs (OSCD).

[1]  Rob J Hyndman,et al.  Detecting trend and seasonal changes in satellite image time series , 2010 .

[2]  Alexander Kolesnikov,et al.  Revisiting Self-Supervised Visual Representation Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Menglong Yan,et al.  Change Detection Based on Deep Siamese Convolutional Network for Optical Aerial Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[4]  Nikos Komodakis,et al.  Unsupervised Representation Learning by Predicting Image Rotations , 2018, ICLR.

[5]  Jan Verbesselt,et al.  Using spatial context to improve early detection of deforestation from Landsat time series , 2016 .

[6]  Devis Tuia,et al.  When a Few Clicks Make All the Difference: Improving Weakly-Supervised Wildlife Detection in UAV Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Alexandre Boulch,et al.  Urban Change Detection for Multispectral Earth Observation Using Convolutional Neural Networks , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[8]  Matthijs Douze,et al.  Deep Clustering for Unsupervised Learning of Visual Features , 2018, ECCV.

[9]  Alexandre Boulch,et al.  Fully Convolutional Siamese Networks for Change Detection , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[10]  Alexei A. Efros,et al.  Unsupervised Visual Representation Learning by Context Prediction , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  世界大都市サミット会議事務局.  Major Cities of the World , 1991 .

[12]  Lorenzo Bruzzone,et al.  Earthquake Damage Assessment of Buildings Using VHR Optical and SAR Imagery , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Francesca Bovolo,et al.  Supervised change detection in VHR images using contextual information and support vector machines , 2013, Int. J. Appl. Earth Obs. Geoinformation.

[14]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[15]  Francesca Bovolo,et al.  Unsupervised Deep Change Vector Analysis for Multiple-Change Detection in VHR Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Francesca Bovolo,et al.  The Time Variable in Data Fusion: A Change Detection Perspective , 2015, IEEE Geoscience and Remote Sensing Magazine.

[17]  Qian Du,et al.  Multi-Modal Change Detection, Application to the Detection of Flooded Areas: Outcome of the 2009–2010 Data Fusion Contest , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[18]  Francesca Bovolo,et al.  A Novel Framework for the Design of Change-Detection Systems for Very-High-Resolution Remote Sensing Images , 2013, Proceedings of the IEEE.

[19]  Xiao Xiang Zhu,et al.  Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Yongjun Zhang,et al.  End-to-End Change Detection for High Resolution Satellite Images Using Improved UNet++ , 2019, Remote. Sens..

[21]  Pedram Ghamisi,et al.  Multispectral Change Detection With Bilinear Convolutional Neural Networks , 2020, IEEE Geoscience and Remote Sensing Letters.

[22]  Rongjun Qin,et al.  Multi-level monitoring of subtle urban changes for the megacities of China using high-resolution multi-view satellite imagery , 2017 .

[23]  Paul L. Rosin.  Unimodal thresholding , 2001, Pattern Recognit..

[24]  J. Qi,et al.  Spatio-temporal dynamics and evolution of land use change and landscape pattern in response to rapid urbanization , 2009 .

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  Xiao Xiang Zhu,et al.  Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources , 2017, IEEE Geoscience and Remote Sensing Magazine.

[27]  Francesca Bovolo,et al.  A Review of Change Detection in Multitemporal Hyperspectral Images: Current Techniques, Applications, and Challenges , 2019, IEEE Geoscience and Remote Sensing Magazine.

[28]  Alexandre Boulch,et al.  Guided Anisotropic Diffusion and Iterative Learning for Weakly Supervised Change Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).