Product Manifold Learning

We consider problems of dimensionality reduction and learning data representations for continuous spaces with two or more independent degrees of freedom. Such problems occur, for example, when observing shapes with several components that move independently. Mathematically, if the parameter space of each continuous independent motion is a manifold, then their combination is known as a product manifold. In this paper, we present a new paradigm for non-linear independent component analysis called manifold factorization. Our factorization algorithm is based on spectral graph methods for manifold learning and the separability of the Laplacian operator on product spaces. Recovering the factors of a manifold yields meaningful lower-dimensional representations and provides a new way to focus on particular aspects of the data space while ignoring others. We demonstrate the potential use of our method for an important and challenging problem in structural biology: mapping the motions of proteins and other large molecules using cryo-electron microscopy datasets.

[1]  J M Carazo,et al.  Survey of the analysis of continuous conformational variability of biological macromolecules by electron microscopy. , 2019, Acta crystallographica. Section F, Structural biology communications.

[2]  A. Grigor’yan Heat Kernel and Analysis on Manifolds , 2012 .

[3]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[4]  Dejan Slepcev,et al.  A variational approach to the consistency of spectral clustering , 2015, Applied and Computational Harmonic Analysis.

[5]  Joachim Frank,et al.  Retrieving functional pathways of biomolecules from single-particle snapshots , 2020, Nature Communications.

[6]  Alantha Newman Complex Semidefinite Programming and Max-k-Cut , 2018, SOSA@SODA.

[7]  A. Bartesaghi,et al.  Unsupervised particle sorting for high-resolution single-particle cryo-EM , 2019, Inverse Problems.

[8]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[9]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[10]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[11]  Ulrike von Luxburg,et al.  From Graphs to Manifolds - Weak and Strong Pointwise Consistency of Graph Laplacians , 2005, COLT.

[12]  J. Nash C 1 Isometric Imbeddings , 1954 .

[13]  V. Koltchinskii,et al.  Empirical graph Laplacian approximation of Laplace–Beltrami operators: Large sample results , 2006, math/0612777.

[14]  A. Singer Spectral independent component analysis , 2006 .

[15]  Ann B. Lee,et al.  A Spectral Series Approach to High-Dimensional Nonparametric Regression , 2016, 1602.00355.

[16]  E. Lindahl,et al.  Characterisation of molecular motions in cryo-EM single-particle data by multi-body refinement in RELION , 2018, bioRxiv.

[17]  A. Singer From graph to manifold Laplacian: The convergence rate , 2006 .

[18]  Yuting Zhang,et al.  Learning to Disentangle Factors of Variation with Manifold Interaction , 2014, ICML.

[19]  Jonathan Bates,et al.  The embedding dimension of Laplacian eigenfunction maps , 2014, ArXiv.

[20]  N. Kuiper,et al.  On C1-isometric imbeddings. II , 1955 .

[21]  Joakim Andén,et al.  Structural Variability from Noisy Tomographic Projections , 2017, SIAM J. Imaging Sci..

[22]  Frank D. Wood,et al.  Learning Disentangled Representations with Semi-Supervised Deep Generative Models , 2017, NIPS.

[23]  P. Schwander,et al.  Conformations of macromolecules and their complexes from heterogeneous datasets , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[24]  W. Kühlbrandt The Resolution Revolution , 2014, Science.

[25]  Tristan Bepler,et al.  Reconstructing continuous distributions of 3D protein structure from cryo-EM images , 2019, ICLR.

[26]  Ling Huang,et al.  An Analysis of the Convergence of Graph Laplacians , 2010, ICML.

[27]  Xiaoming Liu,et al.  Disentangled Representation Learning GAN for Pose-Invariant Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Bernhard Schölkopf,et al.  Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations , 2018, ICML.

[29]  Amit Singer,et al.  Computational Methods for Single-Particle Electron Cryomicroscopy. , 2020, Annual review of biomedical data science.

[30]  Denis S. Grebenkov,et al.  Geometrical Structure of Laplacian Eigenfunctions , 2012, SIAM Rev..

[31]  Alexander M. Bronstein,et al.  Functional Maps Representation On Product Manifolds , 2018, Comput. Graph. Forum.

[32]  Marina Meila,et al.  A regression approach for explaining manifold embedding coordinates , 2018, ArXiv.

[33]  Marina Meila,et al.  Selecting the independent coordinates of manifolds with large aspect ratios , 2019, NeurIPS.

[34]  J. Dubochet,et al.  Cryo-electron microscopy of vitrified specimens , 1988, Quarterly Reviews of Biophysics.

[35]  Daniel J. Arrigo,et al.  An Introduction to Partial Differential Equations , 2017, An Introduction to Partial Differential Equations.

[36]  Simone Melzi,et al.  Learning disentangled representations via product manifold projection , 2021, ICML.

[37]  John M. Lee Introduction to Smooth Manifolds , 2002 .

[38]  Aapo Hyvärinen,et al.  Variational Autoencoders and Nonlinear ICA: A Unifying Framework , 2019, AISTATS.

[39]  Andriy Mnih,et al.  Disentangling by Factorising , 2018, ICML.

[40]  Matthias Hein,et al.  Error Estimates for Spectral Convergence of the Graph Laplacian on Random Geometric Graphs Toward the Laplace–Beltrami Operator , 2018, Found. Comput. Math..

[41]  Mikhail Belkin,et al.  Towards a theoretical foundation for Laplacian-based manifold methods , 2005, J. Comput. Syst. Sci..

[42]  Joakim Andén,et al.  Cryo-EM reconstruction of continuous heterogeneity by Laplacian spectral volumes , 2019, Inverse problems.

[43]  Daniel Cressey,et al.  Cryo-electron microscopy wins chemistry Nobel , 2017, Nature.

[44]  Ronald R. Coifman,et al.  Diffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck Operators , 2005, NIPS.

[45]  Amit Singer,et al.  Earthmover-Based Manifold Learning for Analyzing Molecular Conformation Spaces , 2019, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[46]  Hstau Y Liao,et al.  Trajectories of the ribosome as a Brownian nanomachine , 2014, Proceedings of the National Academy of Sciences.

[47]  C. Chui,et al.  Article in Press Applied and Computational Harmonic Analysis a Randomized Algorithm for the Decomposition of Matrices , 2022 .

[48]  Stephen P. Boyd,et al.  CVXPY: A Python-Embedded Modeling Language for Convex Optimization , 2016, J. Mach. Learn. Res..

[49]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[50]  Ullrich Köthe,et al.  Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN) , 2020, ICLR.

[51]  Amit Singer,et al.  Single-Particle Cryo-Electron Microscopy: Mathematical Theory, Computational Challenges, and Opportunities , 2019, IEEE Signal Processing Magazine.

[52]  Amit Singer,et al.  Manifold Learning with Arbitrary Norms , 2020, Journal of Fourier Analysis and Applications.

[53]  R. Coifman,et al.  Non-linear independent component analysis with diffusion maps , 2008 .

[54]  Ronald R. Coifman,et al.  Diffusion Maps for Signal Processing: A Deeper Look at Manifold-Learning Techniques Based on Kernels and Graphs , 2013, IEEE Signal Processing Magazine.

[55]  Yochai Blau,et al.  Non-redundant Spectral Dimensionality Reduction , 2016, ECML/PKDD.

[56]  Mikhail Belkin,et al.  Consistency of spectral clustering , 2008, 0804.0678.

[57]  L. V. van Vliet,et al.  Image formation modeling in cryo-electron microscopy. , 2013, Journal of structural biology.

[58]  Carmeline J. Dsilva,et al.  Parsimonious Representation of Nonlinear Dynamical Systems Through Manifold Learning: A Chemotaxis Case Study , 2015, 1505.06118.

[59]  Mikhail Belkin,et al.  On Learning with Integral Operators , 2010, J. Mach. Learn. Res..

[60]  Joachim Frank,et al.  New Opportunities Created by Single-Particle Cryo-EM: The Mapping of Conformational Space , 2018, Biochemistry.