Change Detection between Multimodal Remote Sensing Data Using Siamese CNN

Detecting topographic changes in the urban environment has always been an important task for urban planning and monitoring. In practice, remote sensing data are often available in different modalities and at different time epochs. Change detection between multimodal data can be very challenging since the data show different characteristics. Given 3D laser scanning point clouds and 2D imagery from different epochs, this paper presents a framework to detect building and tree changes. First, the 2D and 3D data are transformed to image patches, respectively. A Siamese CNN is then employed to detect candidate changes between the two epochs. Finally, the candidate patch-based changes are grouped and verified as individual object changes. Experiments on the urban data show that 86.4\% of patch pairs can be correctly classified by the model.

[1]  Xiao Xiang Zhu,et al.  Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Bo Du,et al.  A post-classification change detection method based on iterative slow feature analysis and Bayesian soft fusion , 2017, Remote Sensing of Environment.

[3]  Yunsheng Zhang,et al.  Building Change Detection Using Old Aerial Images and New LiDAR Data , 2016, Remote. Sens..

[4]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[5]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  Nikos Komodakis,et al.  Learning to compare image patches via convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Xiao Xiang Zhu,et al.  A CNN for the identification of corresponding patches in SAR and optical imagery of urban scenes , 2017, 2017 Joint Urban Remote Sensing Event (JURSE).

[11]  Rongjun Qin,et al.  An Object-Based Hierarchical Method for Change Detection Using Unmanned Aerial Vehicle Images , 2014, Remote. Sens..

[12]  Sudan Xu,et al.  Detection and Classification of Changes in Buildings from Airborne Laser Scanning Data , 2013, Remote. Sens..

[13]  Kyu-Ri Choi,et al.  A FEATURE BASED APPROACH TO AUTOMATIC CHANGE DETECTION FROM LIDAR DATA IN URBAN AREAS , 2009 .

[14]  Maoguo Gong,et al.  Superpixel-Based Difference Representation Learning for Change Detection in Multispectral Remote Sensing Images , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Bertrand Le Saux,et al.  Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks , 2016, ACCV.

[16]  Shiyong Cui,et al.  Building Change Detection Based on Satellite Stereo Imagery and Digital Surface Models , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Francesca Bovolo,et al.  Supervised change detection in VHR images using contextual information and support vector machines , 2013, Int. J. Appl. Earth Obs. Geoinformation.

[18]  Alfred Stein,et al.  Change Vector Analysis to Monitor the Changes in Fuzzy Shorelines , 2017, Remote. Sens..

[19]  Luc Van Gool,et al.  One-Shot Video Object Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[21]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Yann LeCun,et al.  Computing the stereo matching cost with a convolutional neural network , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Gregory R. Koch,et al.  Siamese Neural Networks for One-Shot Image Recognition , 2015 .

[24]  Michael Ying Yang,et al.  A patch-based method for the evaluation of dense image matching quality , 2018, Int. J. Appl. Earth Obs. Geoinformation.

[25]  Qing Zhu,et al.  Digital terrain modeling - principles and methodology , 2004 .

[26]  Peter Axelsson,et al.  Processing of laser scanner data-algorithms and applications , 1999 .

[27]  Menglong Yan,et al.  Change Detection Based on Deep Siamese Convolutional Network for Optical Aerial Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[28]  Hui Lu,et al.  A deep information based transfer learning method to detect annual urban dynamics of Beijing and Newyork from 1984–2016 , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[29]  Jia Liu,et al.  Change detection based on deep feature representation and mapping transformation for multi-spatial-resolution remote sensing images , 2016 .

[30]  M. Rothermel,et al.  SURE : PHOTOGRAMMETRIC SURFACE RECONSTRUCTION FROM IMAGER Y , 2013 .

[31]  Jan Dirk Wegner,et al.  Toward Seamless Multiview Scene Analysis From Satellite to Street Level , 2017, Proceedings of the IEEE.

[32]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Yann LeCun,et al.  Signature Verification Using A "Siamese" Time Delay Neural Network , 1993, Int. J. Pattern Recognit. Artif. Intell..

[34]  H. Murakami,et al.  Change detection of buildings using an airborne laser scanner , 1999 .

[35]  Jamie Sherrah,et al.  Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagery , 2016, ArXiv.

[36]  Ashbindu Singh,et al.  Review Article Digital change detection techniques using remotely-sensed data , 1989 .

[37]  D. Lu,et al.  Change detection techniques , 2004 .

[38]  Norbert Pfeifer,et al.  Integrated Change Detection and Classification in Urban Areas Based on Airborne Laser Scanning Point Clouds , 2018, Sensors.

[39]  George Vosselman,et al.  Contextual segment-based classification of airborne laser scanner data , 2017 .

[40]  Serge J. Belongie,et al.  Learning deep representations for ground-to-aerial geolocalization , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  G. Camps-Valls,et al.  Spectral alignment of multi-temporal cross-sensor images with automated kernel canonical correlation analysis , 2015 .

[42]  Rahul Sukthankar,et al.  MatchNet: Unifying feature and metric learning for patch-based matching , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).