Mask-CDNet: A mask based pixel change detection network

Abstract Change detection between multi-temporal images becomes a core technique widely used in various fields. But there still exist some challenging issues in the field of change detection. This thesis mainly focuses on the issue that two compared images captured at different times are hard to be precisely aligned due to the fact that the camera is mounted on a moving platform. This issue will lead to the problem that it is difficult to determine the change area between a pair of roughly registered images in pixel-level. To conquer this problem, a novel deep learning based change detection framework, consisting of two collaborative modules, is proposed to improve the estimation accuracy and computation efficiency. The first module is mainly used to roughly predict change areas and match information of two unaligned compared images in the absence of ground truth. The second module aims to refine the change areas and make results more accurate and interpretative. Because of the designed two modules, the framework could not only deal with two roughly registered images, but also be robust to uninteresting change coming from noise or arbitrary spurious differences. Although it is a two-stage method, the proposed framework is trained end-to-end. Meanwhile, the design of pure network make it efficient. Experiments evaluated on PCD-2015 dataset and AICD 2012 dataset demonstrate the method outperforms existing literatures.

[1]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Jizhou Sun,et al.  Fine-Grained Change Detection of Misaligned Scenes with Varied Illuminations , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[3]  Jizhou Sun,et al.  Learning to Detect Fine-Grained Change Under Variant Imaging Conditions , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[4]  Francesca Bovolo,et al.  A Theoretical Framework for Unsupervised Change Detection Based on Change Vector Analysis in the Polar Domain , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Ping Tan,et al.  Active Recurrence of Lighting Condition for Fine-Grained Change Detection , 2018, IJCAI.

[6]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[7]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Fabrice Heitz,et al.  Automatic change detection in multimodal serial MRI: application to multiple sclerosis lesion evolution , 2003, NeuroImage.

[9]  Lichao Mou,et al.  Learning a Transferable Change Rule from a Recurrent Neural Network for Land Cover Change Detection , 2016, Remote. Sens..

[10]  Pol Coppin,et al.  Review ArticleDigital change detection methods in ecosystem monitoring: a review , 2004 .

[11]  Yongjun Zhang,et al.  Object-Based Change Detection for VHR Images Based on Multiscale Uncertainty Analysis , 2018, IEEE Geoscience and Remote Sensing Letters.

[12]  Haoxiang Wang,et al.  A Novel VHR Image Change Detection Algorithm Based on Image Fusion and Fuzzy C-Means Clustering , 2017, ArXiv.

[13]  Hichem Sahbi,et al.  Constrained optical flow for aerial image change detection , 2011, 2011 IEEE International Geoscience and Remote Sensing Symposium.

[14]  Nicolas Dobigeon,et al.  Robust Fusion of Multiband Images With Different Spatial and Spectral Resolutions for Change Detection , 2016, IEEE Transactions on Computational Imaging.

[15]  J. Fry,et al.  Updating the 2001 National Land Cover Database land cover classification to 2006 by using Landsat imagery change detection methods , 2009 .

[16]  Raffay Hamid,et al.  Large-scale damage detection using satellite imagery , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Dongmei Chen,et al.  Change detection from remotely sensed images: From pixel-based to object-based approaches , 2013 .

[18]  Cordelia Schmid,et al.  DeepFlow: Large Displacement Optical Flow with Deep Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[19]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[20]  Tomoya Murase,et al.  Change Detection under Global Viewpoint Uncertainty , 2017, ArXiv.

[21]  Ashish Ghosh,et al.  Semi-supervised change detection using modified self-organizing feature map neural network , 2014, Appl. Soft Comput..

[22]  Joseph L. Mundy,et al.  Image-Based 4-d Reconstruction Using 3-d Change Detection , 2014, ECCV.

[23]  Gabriele Moser,et al.  A Clustering Approach to Heterogeneous Change Detection , 2017, SCIA.

[24]  Francesca Bovolo,et al.  Detection of Land-Cover Transitions in Multitemporal Remote Sensing Images With Active-Learning-Based Compound Classification , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Bo Du,et al.  A post-classification change detection method based on iterative slow feature analysis and Bayesian soft fusion , 2017, Remote Sensing of Environment.

[26]  Björn Stenger,et al.  Detecting Change for Multi-View, Long-Term Surface Inspection , 2015, BMVC.

[27]  Nobuo Kawaguchi,et al.  Dense Optical Flow based Change Detection Network Robust to Difference of Camera Viewpoints , 2017, ArXiv.

[28]  Y. Ban,et al.  Change Detection Techniques: A Review , 2016 .

[29]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Jinsong Deng,et al.  PCA‐based land‐use change detection and analysis using multitemporal and multisensor satellite data , 2008 .

[31]  T. M. Lillesand,et al.  Remote Sensing and Image Interpretation , 1980 .

[32]  Pol Coppin,et al.  Digital change detection methods in natural ecosystem monitoring: a review , 2002 .

[33]  Stephen J. Aldington,et al.  The accurate assessment of changes in retinal vessel diameter using multiple frame electrocardiograph synchronised fundus photography. , 1996, Current eye research.

[34]  Liangpei Zhang,et al.  Unsupervised Change Detection Based on Hybrid Conditional Random Field Model for High Spatial Resolution Remote Sensing Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Lorenzo Bruzzone,et al.  Automatic analysis of the difference image for unsupervised change detection , 2000, IEEE Trans. Geosci. Remote. Sens..

[36]  Bo Du,et al.  Kernel Slow Feature Analysis for Scene Change Detection , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Mohammed Bennamoun,et al.  Learning deep structured network for weakly supervised change detection , 2016, IJCAI.

[38]  Takayuki Okatani,et al.  Change Detection from a Street Image Pair using CNN Features and Superpixel Segmentation , 2015, BMVC.

[39]  Germán Ros,et al.  Street-view change detection with deconvolutional networks , 2016, Autonomous Robots.

[40]  Andreas Savakis,et al.  Unsupervised change detection using Spatial Transformer Networks , 2016, 2016 IEEE Western New York Image and Signal Processing Workshop (WNYISPW).

[41]  Ronen Basri,et al.  A Survey on Structure from Motion , 2017, ArXiv.

[42]  Marc Pollefeys,et al.  City-Scale Change Detection in Cadastral 3D Models Using Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  S. T. Seydi,et al.  A new land-cover match-based change detection for hyperspectral imagery , 2017 .

[44]  John J. Leonard,et al.  Toward lifelong object segmentation from change detection in dense RGB-D maps , 2013, 2013 European Conference on Mobile Robots.

[45]  Lin Yi.  Object-oriented classification of remote sensing data for change detection , 2008 .

[46]  Suparn Pathak.  New Change Detection Techniques to monitor land cover dynamics in mine environment , 2014 .

[47]  Ole Winther,et al.  Gaussian Processes for Classification: Mean-Field Algorithms , 2000, Neural Computation.

[48]  Yutaka Satoh,et al.  Semantic Change Detection with Hypermaps , 2016, ArXiv.

[49]  Yupeng Li,et al.  Change Detection Based on Conditional Random Field With Region Connection Constraints in High-Resolution Remote Sensing Images , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[50]  Allan Aasbjerg Nielsen,et al.  The Regularized Iteratively Reweighted MAD Method for Change Detection in Multi- and Hyperspectral Data , 2007, IEEE Transactions on Image Processing.

[51]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[52]  Sébastien Lefèvre,et al.  Coupling ground-level panoramas and aerial imagery for change detection , 2016, Geo spatial Inf. Sci..

[53]  Björn Stenger,et al.  Precise deterministic change detection for smooth surfaces , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).