Cross-covariance based affinity for graphs

The accuracy of graph based learning techniques relies on the underlying topological structure and affinity between data points, which are assumed to lie on a smooth Riemannian manifold. However, the assumption of local linearity in a neighborhood does not always hold true. Hence, the Euclidean distance based affinity that determines the graph edges may fail to represent the true connectivity strength between data points. Moreover, the affinity between data points is influenced by the distribution of the data around them and must be considered in the affinity measure. In this paper, we propose two techniques, CCGAL and CCGAN that use cross-covariance based graph affinity (CCGA) to represent the relation between data points in a local region. CCGAL also explores the additional connectivity between data points which share a common local neighborhood. CCGAN considers the influence of respective neighborhoods of the two immediately connected data points, which further enhance the affinity measure. Experimental results of manifold learning on synthetic datasets show that CCGA is able to represent the affinity measure between data points more accurately. This results in better low dimensional representation. Manifold regularization experiments on standard image dataset further indicate that the proposed CCGA based affinity is able to accurately identify and include the influence of the data points and its common neighborhood that increase the classification accuracy. The proposed method outperforms the existing state-of-the-art manifold regularization methods by a significant margin.

[1]  Yan Zhang,et al.  Deep Self-representative Concept Factorization Network for Representation Learning , 2019, SDM.

[2]  Longin Jan Latecki,et al.  Locally constrained diffusion process on locally densified distance spaces with applications to shape retrieval , 2009, CVPR.

[3]  Ruchir Gupta,et al.  A parameter-free affinity based clustering , 2015, Applied Intelligence.

[4]  Charu C. Aggarwal,et al.  On Density Based Transforms for Uncertain Data Mining , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[5]  Meng Wang,et al.  Joint Label Prediction Based Semi-Supervised Adaptive Concept Factorization for Robust Data Representation , 2019, IEEE Transactions on Knowledge and Data Engineering.

[6]  Le Song,et al.  Feature Selection via Dependence Maximization , 2012, J. Mach. Learn. Res..

[7]  Liqiang Nie,et al.  Learning robust affinity graph representation for multi-view clustering , 2021, Inf. Sci..

[8]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003 .

[9]  Larry A. Wasserman,et al.  Density-Sensitive Semisupervised Inference , 2012, ArXiv.

[10]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Ling Li,et al.  Fuzzy based affinity learning for spectral clustering , 2016, Pattern Recognit..

[12]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[13]  Yan Zhang,et al.  Semi-supervised local multi-manifold Isomap by linear embedding for feature extraction , 2018, Pattern Recognit..

[14]  Longin Jan Latecki,et al.  Affinity Learning with Diffusion on Tensor Product Graph , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Xian-Sheng Hua,et al.  Ensemble Manifold Regularization , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Antonio Torralba,et al.  Recognizing indoor scenes , 2009, CVPR.

[17]  Jennifer G. Dy,et al.  From Transformation-Based Dimensionality Reduction to Feature Selection , 2010, ICML.

[18]  Martin J. Wainwright,et al.  Kernel Feature Selection via Conditional Covariance Minimization , 2017, NIPS.

[19]  Pavel Yu. Chebotarev,et al.  On a Duality between Metrics and Σ-Proximities , 2002, ArXiv.

[20]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[21]  Nicolas Courty,et al.  Sparse Hilbert Schmidt Independence Criterion and Surrogate-Kernel-Based Feature Selection for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[23]  Min Xiao,et al.  Feature Space Independent Semi-Supervised Domain Adaptation via Kernel Matching , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Shaogang Gong,et al.  Constructing Robust Affinity Graphs for Spectral Clustering , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[26]  Miguel Nicolau,et al.  Learning Neural Representations for Network Anomaly Detection , 2019, IEEE Transactions on Cybernetics.

[27]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[28]  Murat Kaya,et al.  A large electroencephalographic motor imagery dataset for electroencephalographic brain computer interfaces , 2018, Scientific Data.

[29]  Michael I. Jordan,et al.  Learning Spectral Clustering, With Application To Speech Separation , 2006, J. Mach. Learn. Res..

[30]  Deli Zhao,et al.  Linear local tangent space alignment and application to face recognition , 2007, Neurocomputing.

[31]  Shuicheng Yan,et al.  Robust Neighborhood Preserving Projection by Nuclear/L2,1-Norm Regularization for Image Feature Extraction , 2017, IEEE Transactions on Image Processing.

[32]  Saumik Bhattacharya,et al.  Effects of Degradations on Deep Neural Network Architectures , 2018, ArXiv.

[33]  Zhuowen Tu,et al.  Learning Context-Sensitive Shape Similarity by Graph Transduction , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[35]  Vagan Y. Terziyan,et al.  Social Distance metric: from coordinates to neighborhoods , 2017, Int. J. Geogr. Inf. Sci..

[36]  P. Xie,et al.  COVID-CT-Dataset: A CT Scan Dataset about COVID-19 , 2020, ArXiv.

[37]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Zhengming Ma,et al.  Local tangent space alignment based on Hilbert–Schmidt independence criterion regularization , 2019, Pattern Analysis and Applications.

[39]  M. Pelillo,et al.  Dominant Sets and Pairwise Clustering , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Stan Matwin,et al.  A Density-Penalized Distance Measure for Clustering , 2015, Canadian Conference on AI.

[41]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[42]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[43]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[44]  P. Yu. Chebotarev,et al.  On a Duality between Metrics and Σ-Proximities , 2005, ArXiv.

[45]  Marcello Pelillo,et al.  Dominant Sets and Pairwise Clustering , 2007 .

[46]  Martin Thoma,et al.  The HASYv2 dataset , 2017, ArXiv.

[47]  Ramakrishna Kakarala,et al.  Consensus of k-NNs for Robust Neighborhood Selection on Graph-Based Manifolds , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  N. Cristianini,et al.  On Kernel-Target Alignment , 2001, NIPS.

[49]  Michael I. Jordan,et al.  Dimensionality Reduction for Supervised Learning with Reproducing Kernel Hilbert Spaces , 2004 .

[50]  Joshua B. Tenenbaum,et al.  Global Versus Local Methods in Nonlinear Dimensionality Reduction , 2002, NIPS.

[51]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .