Aligning Mixed Manifolds

Current manifold alignment methods can effectively align data sets that are drawn from a non-intersecting set of manifolds. However, as data sets become increasingly high-dimensional and complex, this assumption may not hold. This paper proposes a novel manifold alignment algorithm, low rank alignment (LRA), that uses a low rank representation (instead of a nearest neighbor graph construction) to embed and align data sets drawn from mixtures of manifolds. LRA does not require the tuning of a sensitive nearest neighbor hyperparameter or prior knowledge of the number of manifolds, both of which are common drawbacks with existing techniques. We demonstrate the effectiveness of our algorithm in two real-world applications: a transfer learning task in spectroscopy and a canonical information retrieval task.

[1]  René Vidal,et al.  A closed form solution to robust subspace estimation and clustering , 2011, CVPR 2011.

[2]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[3]  Robert L. Tokar,et al.  Pre-flight calibration and initial data processing for the ChemCam laser-induced breakdown spectroscopy instrument on the Mars Science Laboratory rover , 2013 .

[4]  Sridhar Mahadevan,et al.  Manifold alignment using Procrustes analysis , 2008, ICML '08.

[5]  Trent D. Ridder,et al.  Robust Calibration Transfer in Noninvasive Ethanol Measurements, Part I: Mathematical Basis for Spectral Distortions in Fourier Transform Near-Infrared Spectroscopy (FT-NIR) , 2014, Applied spectroscopy.

[6]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[7]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[8]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[9]  Chang Wang,et al.  Manifold Alignment Preserving Global Geometry , 2013, IJCAI.

[10]  Noah A. Smith,et al.  The Web as a Parallel Corpus , 2003, CL.

[11]  Daniel D. Lee,et al.  Semisupervised alignment of manifolds , 2005, AISTATS.

[12]  Kenneth Ward Church,et al.  A Program for Aligning Sentences in Bilingual Corpora , 1993, CL.

[13]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[14]  Steven D. Brown,et al.  Transfer of multivariate calibration models: a review , 2002 .

[15]  J. Schmee.  An Introduction to Multivariate Statistical Analysis , 1986 .

[16]  Robert Pless,et al.  Manifold clustering , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[17]  Ronald R. Coifman,et al.  Data Fusion and Multicue Data Matching by Diffusion Maps , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[19]  Yunqian Ma,et al.  Manifold Learning Theory and Applications , 2011 .

[20]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[21]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[22]  Lawrence K. Saul,et al.  Identifying suspicious URLs: an application of large-scale online learning , 2009, ICML '09.

[23]  Chang Wang,et al.  A General Framework for Manifold Alignment , 2009, AAAI Fall Symposium: Manifold Learning and Its Applications.

[24]  Yi-Zeng Liang,et al.  Calibration transfer of near‐infrared spectra for extraction of informative components from spectra with canonical correlation analysis , 2014 .

[25]  Anja Vogler,et al.  An Introduction to Multivariate Statistical Analysis , 2004 .

[26]  Philipp Koehn,et al.  Europarl: A Parallel Corpus for Statistical Machine Translation , 2005, MTSUMMIT.

[27]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[28]  Yong Wang,et al.  Multi-manifold Clustering , 2010, PRICAI.