Manifold matching using shortest-path distance and joint neighborhood selection

We propose a new manifold matching method, that is superior than existing methods based on single modality.Our method is robust against noise and different types of geometry in matching.The method is particularly useful for graph and network matching. Matching datasets of multiple modalities has become an important task in data analysis. Existing methods often rely on the embedding and transformation of each single modality without utilizing any correspondence information, which often results in sub-optimal matching performance. In this paper, we propose a nonlinear manifold matching algorithm using shortest-path distance and joint neighborhood selection. Specifically, a joint nearest-neighbor graph is built for all modalities. Then the shortest-path distance within each modality is calculated from the joint neighborhood graph, followed by embedding into and matching in a common low-dimensional Euclidean space. Compared to existing algorithms, our approach exhibits superior performance for matching disparate datasets of multiple modalities.

[1]  Guillermo Sapiro,et al.  Graph Matching: Relax at Your Own Risk , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  R. Sibson Studies in the Robustness of Multidimensional Scaling: Procrustes Statistics , 1978 .

[3]  Carey E. Priebe,et al.  A consistent dot product embedding for stochastic blockmodel graphs , 2011 .

[4]  Ronald R. Coifman,et al.  Data Fusion and Multicue Data Matching by Diffusion Maps , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Alon Zakai,et al.  Manifold Learning: The Price of Normalization , 2008, J. Mach. Learn. Res..

[6]  David W. Jacobs,et al.  Generalized Multiview Analysis: A discriminative latent space , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Mario Vento,et al.  Thirty Years Of Graph Matching In Pattern Recognition , 2004, Int. J. Pattern Recognit. Artif. Intell..

[8]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[9]  D. Donoho,et al.  Hessian Eigenmaps : new locally linear embedding techniques for high-dimensional data , 2003 .

[10]  Carey E. Priebe,et al.  Fast Approximate Quadratic Programming for Graph Matching , 2015, PloS one.

[11]  Carey E. Priebe,et al.  On the Incommensurability Phenomenon , 2016, J. Classif..

[12]  Josef Kittler,et al.  Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Alex Smola,et al.  Kernel methods in machine learning , 2007, math/0701907.

[15]  Carey E. Priebe,et al.  Generalized Canonical Correlation Analysis for Disparate Data Fusion , 2013, Pattern Recognit. Lett..

[16]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[17]  John Platt,et al.  FastMap, MetricMap, and Landmark MDS are all Nystrom Algorithms , 2005, AISTATS.

[18]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[19]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[20]  W. Torgerson Multidimensional scaling: I. Theory and method , 1952 .

[21]  张振跃,et al.  Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment , 2004 .

[22]  Michael I. Jordan,et al.  A Probabilistic Interpretation of Canonical Correlation Analysis , 2005 .

[23]  Guillermo Sapiro,et al.  Robust Multimodal Graph Matching: Sparse Coding Meets Graph Matching , 2013, NIPS.

[24]  Carey E. Priebe,et al.  Discovering Relationships Across Disparate Data Modalities , 2016 .

[25]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[26]  Chong-sun Kim Canonical Analysis of Several Sets of Variables , 1973 .

[27]  Carey E. Priebe,et al.  The out-of-sample problem for classical multidimensional scaling , 2008, Comput. Stat. Data Anal..

[28]  Sridhar Mahadevan,et al.  Sparse Manifold Alignment , 2012 .

[29]  Joshua B. Tenenbaum,et al.  Global Versus Local Methods in Nonlinear Dimensionality Reduction , 2002, NIPS.

[30]  David C. Hoyle,et al.  Automatic PCA Dimension Selection for High Dimensional Data and Small Sample Sizes , 2008 .

[31]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[32]  Yair Goldberg,et al.  Theoretical Analysis of LLE Based on Its Weighting Step , 2012 .

[33]  John K. Tsotsos,et al.  Parameterless Isomap with Adaptive Neighborhood Selection , 2006, DAGM-Symposium.

[34]  Maria L. Rizzo,et al.  Brownian distance covariance , 2009, 1010.0297.

[35]  Carey E. Priebe,et al.  Seeded graph matching for correlated Erdös-Rényi graphs , 2014, J. Mach. Learn. Res..

[36]  I. Jolliffe Principal Component Analysis , 2002 .

[37]  H. Hotelling Relations Between Two Sets of Variates , 1936 .

[38]  Nicolas Le Roux,et al.  Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering , 2003, NIPS.

[39]  Rex E. Jung,et al.  MIGRAINE: MRI Graph Reliability Analysis and Inference for Connectomics , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[40]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[41]  Carey E. Priebe,et al.  A Consistent Adjacency Spectral Embedding for Stochastic Blockmodel Graphs , 2011, 1108.2228.

[42]  Carey E. Priebe,et al.  Generalized canonical correlation analysis for classification , 2013, J. Multivar. Anal..

[43]  A. Tenenhaus,et al.  Regularized Generalized Canonical Correlation Analysis , 2011, Eur. J. Oper. Res..

[44]  Zhiliang Ma,et al.  Manifold Matching: Joint Optimization of Fidelity and Commensurability , 2011, 1112.5510.

[45]  Yaacov Ritov,et al.  Local procrustes for manifold embedding: a measure of embedding quality and embedding algorithms , 2009, Machine Learning.

[46]  Tom Minka,et al.  Automatic Choice of Dimensionality for PCA , 2000, NIPS.

[47]  I. Hassan Embedded , 2005, The Cyber Security Handbook.

[48]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[49]  Maria L. Rizzo,et al.  Measuring and testing dependence by correlation of distances , 2007, 0803.4101.

[50]  Carey E. Priebe,et al.  Efficiency investigation of manifold matching for text document classification , 2013, Pattern Recognit. Lett..

[51]  Sridhar Mahadevan,et al.  Manifold alignment using Procrustes analysis , 2008, ICML '08.

[52]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[53]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[54]  R. Sibson Studies in the Robustness of Multidimensional Scaling: Perturbational Analysis of Classical Scaling , 1979 .

[55]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[56]  Petros Drineas,et al.  On the Nyström Method for Approximating a Gram Matrix for Improved Kernel-Based Learning , 2005, J. Mach. Learn. Res..

[57]  C. Priebe,et al.  A Semiparametric Two-Sample Hypothesis Testing Problem for Random Graphs , 2017 .

[58]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Hongyuan Zha,et al.  Adaptive Manifold Learning , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Mu Zhu,et al.  Automatic dimensionality selection from the scree plot via the use of profile likelihood , 2006, Comput. Stat. Data Anal..

[61]  Arthur W. Toga,et al.  Learning based coarse-to-fine image registration , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[62]  Vin de Silva,et al.  Unsupervised Learning of Curved Manifolds , 2003 .