Generative locally linear embedding: A module for manifold unfolding and visualization

Data often have nonlinear patterns in machine learning. One can unfold the nonlinear manifold of a dataset for low-dimensional visualization and feature extraction. Locally Linear Embedding (LLE) is a nonlinear spectral method for dimensionality reduction and manifold unfolding. It embeds data using the same linear reconstruction weights as in the input space. In this paper, we propose an open source module which not only implements LLE, but also includes implementations of two generative LLE algorithms whose linear reconstruction phases are stochastic. Using this module, one can generate as many manifold unfoldings as desired for data visualization or feature extraction.

[1]  Achraf Oussidi,et al.  Deep generative models: Survey , 2018, 2018 International Conference on Intelligent Systems and Computer Vision (ISCV).

[2]  M. Naderi,et al.  Think globally... , 2004, HIV prevention plus!.

[3]  Simon Jackman,et al.  Introduction to Factor Analysis , 2020, Exploratory Factor Analysis.

[4]  Hongxun Yao,et al.  Auto-encoder based dimensionality reduction , 2016, Neurocomputing.

[5]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[6]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[7]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[8]  M F Sanner,et al.  Python: a programming language for software integration and development. , 1999, Journal of molecular graphics & modelling.

[9]  Eric Luhman,et al.  Denoising Synthesis: A module for fast image synthesis using denoising-based models , 2021, Softw. Impacts.

[10]  Fakhri Karray,et al.  Factor Analysis, Probabilistic Principal Component Analysis, Variational Inference, and Variational Autoencoder: Tutorial and Survey , 2021, ArXiv.

[11]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[12]  Kilian Q. Weinberger,et al.  Spectral Methods for Dimensionality Reduction , 2006, Semi-Supervised Learning.

[13]  Manjusha Pandey,et al.  A comprehensive survey and analysis of generative models in machine learning , 2020, Comput. Sci. Rev..

[14]  Yuhui Zheng,et al.  Recent Progress on Generative Adversarial Networks (GANs): A Survey , 2019, IEEE Access.

[15]  R. Darlington,et al.  Factor Analysis , 2008 .

[16]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[17]  Benyamin Ghojogh Data Reduction Algorithms in Machine Learning and Data Science , 2021 .