Optimal manifold neighborhood and kernel width for robust non-linear dimensionality reduction

Abstract The graph Laplacian approximation of the Laplace–Beltrami operator on the Riemann manifold requires estimation of the kernel width and the neighborhood which is usually obtained through cross-validation. In this work, we propose a method based on local tangent space alignment to determine the optimal neighborhood. The method works by selecting the most distant point with minimum acceptable misalignment in every local region to define the neighborhood size. The work also proposes to use local kernel width which is represented as a function of the number of data points in the optimal neighborhood and maximum neighborhood distance. Extensive experiments on synthetic and real-world data including brain computer interface readings confirm that the proposed method is able to extract the intrinsic representation and outperforms the related existing state-of-the-art methods in the area.

[1]  Yitian Xu,et al.  A safe screening rule for Laplacian support vector machine , 2018, Eng. Appl. Artif. Intell..

[2]  Heikki Mannila,et al.  Random projection in dimensionality reduction: applications to image and text data , 2001, KDD '01.

[3]  Martin Thoma,et al.  The HASYv2 dataset , 2017, ArXiv.

[4]  Marina Meila,et al.  Improved Graph Laplacian via Geometric Self-Consistency , 2014, NIPS.

[5]  B. Silverman Density estimation for statistics and data analysis , 1986 .

[6]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[7]  D. W. Scott,et al.  Variable Kernel Density Estimation , 1992 .

[8]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[9]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[10]  Dirk P. Kroese,et al.  Kernel density estimation via diffusion , 2010, 1011.2602.

[11]  Daniela Rodriguez,et al.  Kernel Density Estimation on Riemannian Manifolds: Asymptotic Results , 2009, Journal of Mathematical Imaging and Vision.

[12]  Jing Lin,et al.  Adaptive kernel density-based anomaly detection for nonlinear systems , 2018, Knowl. Based Syst..

[13]  Chang Xu,et al.  Scaling KNN multi-class twin support vector machine via safe instance reduction , 2018, Knowl. Based Syst..

[14]  Murat Kaya,et al.  A large electroencephalographic motor imagery dataset for electroencephalographic brain computer interfaces , 2018, Scientific Data.

[15]  Isa Yildirim,et al.  Approximate spectral clustering with utilized similarity information using geodesic based hybrid distance measures , 2015, Pattern Recognit..

[16]  Shekhar Verma,et al.  Graph Laplacian Regularization With Procrustes Analysis for Sensor Node Localization , 2017, IEEE Sensors Journal.

[17]  Xiaogang Wang,et al.  Distribution Adaptation and Manifold Alignment for complex processes fault diagnosis , 2018, Knowl. Based Syst..

[18]  Xianli Pan,et al.  Safe Screening Rules for Accelerating Twin Support Vector Machine Classification , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Hamido Fujita,et al.  A study of graph-based system for multi-view clustering , 2019, Knowl. Based Syst..

[20]  Pascal Frossard,et al.  Tangent-based manifold approximation with locally linear models , 2012, Signal Process..

[21]  Matthias Hein,et al.  Intrinsic dimensionality estimation of submanifolds in Rd , 2005, ICML.

[22]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.

[23]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[24]  Mikhail Belkin,et al.  Towards a theoretical foundation for Laplacian-based manifold methods , 2005, J. Comput. Syst. Sci..

[25]  Ronald R. Coifman,et al.  Graph Laplacian Tomography From Unknown Random Projections , 2008, IEEE Transactions on Image Processing.

[26]  C. Quesenberry,et al.  A nonparametric estimate of a multivariate density function , 1965 .

[27]  Sridhar Mahadevan,et al.  Manifold alignment using Procrustes analysis , 2008, ICML '08.

[28]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[29]  Saumik Bhattacharya,et al.  Effects of Degradations on Deep Neural Network Architectures , 2018, ArXiv.

[30]  Masayuki Karasuyama,et al.  Adaptive edge weighting for graph-based learning algorithms , 2016, Machine Learning.

[31]  Zeshui Xu,et al.  A spectral clustering algorithm based on intuitionistic fuzzy information , 2013, Knowl. Based Syst..

[32]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[33]  Xianli Pan,et al.  A safe reinforced feature screening strategy for lasso based on feasible solutions , 2019, Inf. Sci..

[34]  Frédéric Chazal,et al.  Data driven estimation of Laplace-Beltrami operator , 2016, NIPS.

[35]  Antonio Torralba,et al.  Recognizing indoor scenes , 2009, CVPR.

[36]  Chongsheng Zhang,et al.  An empirical comparison on state-of-the-art multi-class imbalance learning algorithms and a new diversified ensemble learning scheme , 2018, Knowl. Based Syst..

[37]  Sanjoy Dasgupta,et al.  An elementary proof of a theorem of Johnson and Lindenstrauss , 2003, Random Struct. Algorithms.

[38]  M. Spivak Calculus On Manifolds: A Modern Approach To Classical Theorems Of Advanced Calculus , 2019 .

[39]  R. Coifman,et al.  Non-linear independent component analysis with diffusion maps , 2008 .

[40]  Xian-Sheng Hua,et al.  Ensemble Manifold Regularization , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Nasir Saeed,et al.  Cluster Based Multidimensional Scaling for Irregular Cognitive Radio Networks Localization , 2016, IEEE Transactions on Signal Processing.

[42]  A. Singer From graph to manifold Laplacian: The convergence rate , 2006 .