Parzen Window Approximation on Riemannian Manifold

In graph motivated learning, label propagation largely depends on data affinity represented as edges between connected data points. The affinity assignment implicitly assumes even distribution of data on the manifold. This assumption may not hold and may lead to inaccurate metric assignment due to drift towards high-density regions. The drift affected heat kernel based affinity with a globally fixed Parzen window either discards genuine neighbors or forces distant data points to become a member of the neighborhood. This yields a biased affinity matrix. In this paper, the bias due to uneven data sampling on the Riemannian manifold is catered to by a variable Parzen window determined as a function of neighborhood size, ambient dimension, flatness range, etc. Additionally, affinity adjustment is used which offsets the effect of uneven sampling responsible for the bias. An affinity metric which takes into consideration the irregular sampling effect to yield accurate label propagation is proposed. Extensive experiments on synthetic and real-world data sets confirm that the proposed method increases the classification accuracy significantly and outperforms existing Parzen window estimators in graph Laplacian manifold regularization methods.

[1]  Edwin R. Hancock,et al.  Unfolding Kernel embeddings of graphs: Enhancing class separation through manifold learning , 2015, Pattern Recognit..

[2]  Martin Thoma,et al.  The HASYv2 dataset , 2017, ArXiv.

[3]  Edwin R. Hancock,et al.  Graph characteristics from the heat kernel trace , 2009, Pattern Recognit..

[4]  Hongdong Li,et al.  Kernel Methods on Riemannian Manifolds with Gaussian RBF Kernels , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[6]  Ling Li,et al.  Affinity learning via a diffusion process for subspace clustering , 2018, Pattern Recognit..

[7]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[8]  Ronald R. Coifman,et al.  Diffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck Operators , 2005, NIPS.

[9]  Mikhail Belkin,et al.  Towards a theoretical foundation for Laplacian-based manifold methods , 2005, J. Comput. Syst. Sci..

[10]  Frédéric Chazal,et al.  Data driven estimation of Laplace-Beltrami operator , 2016, NIPS.

[11]  M. Kubát An Introduction to Machine Learning , 2017, Springer International Publishing.

[12]  Jianbo Shi,et al.  Learning Segmentation by Random Walks , 2000, NIPS.

[13]  L. Rosasco,et al.  Manifold Regularization , 2007 .

[14]  Marina Meila,et al.  Improved Graph Laplacian via Geometric Self-Consistency , 2014, NIPS.

[15]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[16]  Sergei Ivanov,et al.  A graph discretization of the Laplace-Beltrami operator , 2013, 1301.2222.

[17]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[18]  Murat Kaya,et al.  A large electroencephalographic motor imagery dataset for electroencephalographic brain computer interfaces , 2018, Scientific Data.

[19]  Pascal Vincent,et al.  Non-Local Manifold Parzen Windows , 2005, NIPS.

[20]  E. Parzen On Estimation of a Probability Density Function and Mode , 1962 .

[21]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[22]  Mikhail Belkin,et al.  Convergence of Laplacian Eigenmaps , 2006, NIPS.

[23]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[24]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[25]  Antonio Torralba,et al.  Recognizing indoor scenes , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  A. Singer From graph to manifold Laplacian: The convergence rate , 2006 .

[27]  Guy L. Scott,et al.  Feature grouping by 'relocalisation' of eigenvectors of the proximity matrix , 1990, BMVC.

[28]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[29]  Pascal Vincent,et al.  Manifold Parzen Windows , 2002, NIPS.

[30]  Xinhua Zhang,et al.  Hyperparameter Learning for Graph Based Semi-supervised Learning Algorithms , 2006, NIPS.

[31]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[32]  Shekhar Verma,et al.  Graph Laplacian Regularization With Procrustes Analysis for Sensor Node Localization , 2017, IEEE Sensors Journal.

[33]  Fan Zhang,et al.  Graph spectral image smoothing using the heat kernel , 2008, Pattern Recognit..

[34]  Amit Singer,et al.  Spectral Convergence of the connection Laplacian from random samples , 2013, 1306.1587.

[35]  Kadim Tasdemir,et al.  Vector quantization based approximate spectral clustering of large datasets , 2012, Pattern Recognit..

[36]  Masayuki Karasuyama,et al.  Adaptive edge weighting for graph-based learning algorithms , 2016, Machine Learning.

[37]  Jing Lin,et al.  Adaptive kernel density-based anomaly detection for nonlinear systems , 2018, Knowl. Based Syst..

[38]  Isa Yildirim,et al.  Approximate spectral clustering with utilized similarity information using geodesic based hybrid distance measures , 2015, Pattern Recognit..

[39]  Jean-Michel Morel,et al.  Nonlocal Image and Movie Denoising , 2008, International Journal of Computer Vision.

[40]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[41]  Dirk P. Kroese,et al.  Kernel density estimation via diffusion , 2010, 1011.2602.

[42]  Miguel Á. Carreira-Perpiñán,et al.  Entropic Affinities: Properties and Efficient Numerical Computation , 2013, ICML.