Canonical Correlation Analysis for Misaligned Satellite Image Change Detection

Canonical correlation analysis (CCA) is a statistical learning method that seeks to build view-independent latent representations from multi-view data. This method has been successfully applied to several pattern analysis tasks such as image-to-text mapping and view-invariant object/action recognition. However, this success is highly dependent on the quality of data pairing (i.e., alignments) and mispairing adversely affects the generalization ability of the learned CCA representations. In this paper, we address the issue of alignment errors using a new variant of canonical correlation analysis referred to as alignment-agnostic (AA) CCA. Starting from erroneously paired data taken from different views, this CCA finds transformation matrices by optimizing a constrained maximization problem that mixes a data correlation term with context regularization; the particular design of these two terms mitigates the effect of alignment errors when learning the CCA transformations. Experiments conducted on multi-view tasks, including multi-temporal satellite image change detection, show that our AA CCA method is highly effective and resilient to mispairing errors.

[1]  Hichem Sahbi,et al.  Multi-view object matching and tracking using canonical correlation analysis , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[2]  Hichem Sahbi,et al.  Interactive Satellite Image Change Detection With Context-Aware Canonical Correlation Analysis , 2017, IEEE Geoscience and Remote Sensing Letters.

[3]  Hichem Sahbi,et al.  Nonlinear Deep Kernel Learning for Image Annotation , 2017, IEEE Transactions on Image Processing.

[4]  Hichem Sahbi,et al.  Using entropy for image and video authentication watermarks , 2006, Electronic Imaging.

[5]  Joachim M. Buhmann,et al.  Time-Series Alignment by Non-negative Multiple Generalized Canonical Correlation Analysis , 2007, WILF.

[6]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[7]  Edwin R. Hancock,et al.  Iterative Procrustes alignment with the EM algorithm , 2002, Image Vis. Comput..

[8]  Tae-Kyun Kim,et al.  Canonical Correlation Analysis of Video Volume Tensors for Action Categorization and Detection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Andrew Zisserman,et al.  Multi-task Self-Supervised Visual Learning , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[10]  Horst Bischof,et al.  Appearance models based on kernel canonical correlation analysis , 2003, Pattern Recognit..

[11]  Fernando De la Torre,et al.  Generalized Canonical Time Warping , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  C. E. M. Pearce,et al.  Some New Bounds for Singular Values and Eigenvalues of Matrix Products , 2000, Ann. Oper. Res..

[13]  Hichem Sahbi,et al.  Applying interest operators in semi-fragile video watermarking , 2005, IS&T/SPIE Electronic Imaging.

[14]  Florian Yger,et al.  Adaptive Canonical Correlation Analysis Based On Matrix Manifolds , 2012, ICML.

[15]  Wen Gao,et al.  Instance-specific canonical correlation analysis , 2015, Neurocomputing.

[16]  Pavel Krsek,et al.  Robust Euclidean alignment of 3D point sets: the trimmed iterative closest point algorithm , 2005, Image Vis. Comput..

[17]  Hichem Sahbi,et al.  Camera pose estimation using Visual Servoing for aerial video change detection , 2012, 2012 IEEE International Geoscience and Remote Sensing Symposium.

[18]  Hichem Sahbi,et al.  Superpixel-based object class segmentation using conditional random fields , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[19]  John Shawe-Taylor,et al.  Sparse canonical correlation analysis , 2009, Machine Learning.

[20]  T. W. Anderson,et al.  An Introduction to Multivariate Statistical Analysis , 1959 .

[21]  Ignacio Santamaría,et al.  A learning algorithm for adaptive canonical correlation analysis of several data sets , 2007, Neural Networks.

[22]  Hichem Sahbi,et al.  TELECOMParisTech at ImageClefphoto 2008: Bi-Modal Text and Image Retrieval with Diversity Enhancement , 2008, CLEF.

[23]  Daniela M Witten,et al.  Extensions of Sparse Canonical Correlation Analysis with Applications to Genomic Data , 2009, Statistical applications in genetics and molecular biology.

[24]  Shaogang Gong,et al.  Multi-camera activity correlation analysis , 2009, CVPR.

[25]  Hichem Sahbi.  Discriminant canonical correlation analysis for interactive satellite image change detection , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[26]  Daniel D. Lee,et al.  Semisupervised alignment of manifolds , 2005, AISTATS.

[27]  Songcan Chen,et al.  Locality preserving CCA with applications to data visualization and pose estimation , 2007, Image Vis. Comput..

[28]  Anja Vogler,et al.  An Introduction to Multivariate Statistical Analysis , 2004 .

[29]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[30]  Hichem Sahbi,et al.  Coarse-to-fine support vector classifiers for face detection , 2002, Object recognition supported by user interaction for service robots.

[31]  Hichem Sahbi,et al.  Spatio-temporal interaction for aerial video change detection , 2012, 2012 IEEE International Geoscience and Remote Sensing Symposium.

[32]  Hervé Glotin,et al.  A Comparative Study of Diversity Methods for Hybrid Text and Image Retrieval Approaches , 2008, CLEF.

[33]  Ludmila I. Kuncheva,et al.  PCA feature extraction for change detection in multidimensional unlabelled streaming data , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[34]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[35]  Badrinath Roysam,et al.  Image change detection algorithms: a systematic survey , 2005, IEEE Transactions on Image Processing.

[36]  Hichem Sahbi,et al.  Context-dependent kernel design for object matching and recognition , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Turgay Çelik,et al.  Image change detection using Gaussian mixture model and genetic algorithm , 2010, J. Vis. Commun. Image Represent..

[38]  Wanquan Liu,et al.  Low Resolution Face Recognition in Surveillance Systems , 2014 .

[39]  Ronald R. Coifman,et al.  Data Fusion and Multicue Data Matching by Diffusion Maps , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Taejung Kim,et al.  Automatic satellite image registration by combination of matching and random sample consensus , 2003, IEEE Trans. Geosci. Remote. Sens..

[41]  Sridhar Mahadevan,et al.  Manifold alignment using Procrustes analysis , 2008, ICML '08.

[42]  Peter Glöckner,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2013 .

[43]  George Trigeorgis,et al.  Deep Canonical Time Warping for Simultaneous Alignment and Representation Learning of Sequences , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Clément Mallet,et al.  INVESTIGATING THE POTENTIAL OF DEEP NEURAL NETWORKS FOR LARGE-SCALE CLASSIFICATION OF VERY HIGH RESOLUTION SATELLITE IMAGES , 2017 .

[45]  Hichem Sahbi,et al.  Relevance feedback for satellite image change detection , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[46]  Diane J. Cook,et al.  Collegial activity learning between heterogeneous sensors , 2017, Knowledge and Information Systems.

[47]  Hichem Sahbi,et al.  Coarse-to-Fine Deep Kernel Networks , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[48]  Tommy W. S. Chow,et al.  Binary- and Multi-class Group Sparse Canonical Correlation Analysis for Feature Extraction and Classification , 2013, IEEE Transactions on Knowledge and Data Engineering.

[49]  Hichem Sahbi,et al.  Constrained optical flow for aerial image change detection , 2011, 2011 IEEE International Geoscience and Remote Sensing Symposium.

[50]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  R. Manmatha,et al.  Using Maximum Entropy for Automatic Image Annotation , 2004, CIVR.

[52]  Dongmei Chen,et al.  Change detection from remotely sensed images: From pixel-based to object-based approaches , 2013 .

[53]  Vladlen Koltun,et al.  Playing for Data: Ground Truth from Computer Games , 2016, ECCV.

[54]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[55]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[56]  Hichem Sahbi,et al.  Context-Based Support Vector Machines for Interconnected Image Annotation , 2010, ACCV.

[57]  Jianyong Sun,et al.  Canonical Correlation Analysis on Data With Censoring and Error Information , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[58]  Hichem Sahbi,et al.  CNRS - TELECOM ParisTech at ImageCLEF 2013 Scalable Concept Image Annotation Task: Winning Annotations with Context Dependent SVMs , 2013, CLEF.

[59]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[60]  F. De la Torre,et al.  A UNIFICATION OF COMPONENT ANALYSIS METHODS , 2009 .

[61]  Fernando De la Torre,et al.  Canonical Time Warping for Alignment of Human Behavior , 2009, NIPS.

[62]  Hichem Sahbi,et al.  Directed Acyclic Graph Kernels for Action Recognition , 2013, 2013 IEEE International Conference on Computer Vision.

[63]  Jieping Ye,et al.  Canonical Correlation Analysis for Multilabel Classification: A Least-Squares Formulation, Extensions, and Analysis , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.