Graph based semi-supervised classification with probabilistic nearest neighbors

Abstract Label propagation (LP) is one of the state-of-the-art graph based semi-supervised learning (GSSL) algorithm. Probability transition matrix (PTM) is the key for LP to propagate label information among samples. Conventionally, PTM is calculated based on the graph constructed in advance, and graph construction independent of PTM calculation. It leads to complex steps for acquiring PTM, and more importantly, brings about the lack of correlation between graph construction and inference. Based on adaptive neighbors-based method, probabilistic nearest neighbors (PNN) based graph construction algorithm is proposed for effective l2 norm optimization, and the solving process of the objective function is optimized by incorporating min-max normalization. The derived PNN matrix is more discriminative and directly serve as PTM for LP. It makes PTM computation more conveniently and more applicable for classification task. In addition, number of neighbors is adaptively determined on the premise of its preset value. Experimental results show that the proposed PNN algorithm specializes in reflecting probability differences of neighboring nodes in a graph, and positive results are achieved in semi-supervised classification. The average classification accuracy on synthetic data sets is 84.24%, and that on image data sets achieves 89.08%.

[1]  Jean-Yves Ramel,et al.  Efficient k-nearest neighbors search in graph space , 2020, Pattern Recognit. Lett..

[2]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Yuan Qi,et al.  Hyperparameter and Kernel Learning for Graph Based Semi-Supervised Classification , 2005, NIPS.

[4]  Nenghai Yu,et al.  Non-negative low rank and sparse graph for semi-supervised learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[6]  Gunnar Rätsch,et al.  Graph Based Semi-supervised Learning with Sharper Edges , 2006, ECML.

[7]  Edwin R. Hancock,et al.  Learning Aligned-Spatial Graph Convolutional Networks for Graph Classification , 2019, ECML/PKDD.

[8]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[9]  Wen Zhang,et al.  The linear neighborhood propagation method for predicting long non-coding RNA-protein interactions , 2018, Neurocomputing.

[10]  Alneu de Andrade Lopes,et al.  Regular graph construction for semi-supervised learning , 2014, Journal of Physics: Conference Series.

[11]  Zoubin Ghahramani,et al.  Nonparametric Transforms of Graph Kernels for Semi-Supervised Learning , 2004, NIPS.

[12]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Paramveer S. Dhillon,et al.  Inference Driven Metric Learning for Graph Construction , 2010 .

[14]  Zihan Zhou,et al.  Label Information Guided Graph Construction for Semi-Supervised Learning , 2017, IEEE Transactions on Image Processing.

[15]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[16]  Xuelong Li,et al.  Multi-View Clustering and Semi-Supervised Classification with Adaptive Neighbours , 2017, AAAI.

[17]  Chuan Zhou,et al.  Parameter k search strategy in outlier detection , 2018, Pattern Recognit. Lett..

[18]  Xiaobing Pei,et al.  Joint Sparse Representation and Embedding Propagation Learning: A Framework for Graph-Based Semisupervised Learning , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Hugo Jair Escalante,et al.  Scalable multi-label annotation via semi-supervised kernel semantic embedding , 2019, Pattern Recognit. Lett..

[20]  Hamid R. Rabiee,et al.  Supervised neighborhood graph construction for semi-supervised classification , 2012, Pattern Recognit..

[21]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Nicu Sebe,et al.  Dynamic Affinity Graph Construction for Spectral Clustering Using Multiple Features , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Yue Wang,et al.  Identifying The Most Informative Features Using A Structurally Interacting Elastic Net , 2018, Neurocomputing.

[24]  Feiping Nie,et al.  New l2, 1-Norm Relaxation of Multi-Way Graph Cut for Clustering , 2018, AAAI.

[25]  Li Ma,et al.  Probabilistic class structure regularized sparse representation graph for semi-supervised hyperspectral image classification , 2017, Pattern Recognit..

[26]  Nannan Wang,et al.  Semi-Supervised Classification With Graph Structure Similarity and Extended Label Propagation , 2019, IEEE Access.

[27]  Rong Wang,et al.  Fast semi-supervised learning with anchor graph for large hyperspectral images , 2020, Pattern Recognit. Lett..

[28]  Jasper Snoek,et al.  Spectral Representations for Convolutional Neural Networks , 2015, NIPS.

[29]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[30]  Rongrong Ji,et al.  Visual Reranking through Weakly Supervised Multi-graph Learning , 2013, 2013 IEEE International Conference on Computer Vision.

[31]  Edwin R. Hancock,et al.  A Quantum-Inspired Similarity Measure for the Analysis of Complete Weighted Graphs , 2019, IEEE Transactions on Cybernetics.

[32]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[33]  Masayuki Karasuyama,et al.  Adaptive edge weighting for graph-based learning algorithms , 2016, Machine Learning.

[34]  Xiaohui Cheng,et al.  A community discovery algorithm based on boundary nodes and label propagation , 2017, Pattern Recognit. Lett..

[35]  Zhao Kang,et al.  Robust Graph Learning for Semi-Supervised Classification , 2018, 2018 10th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC).

[36]  Bin Luo,et al.  Semi-Supervised Learning With Graph Learning-Convolutional Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Feiping Nie,et al.  Clustering and projected clustering with adaptive neighbors , 2014, KDD.

[38]  Koby Crammer,et al.  Inference Driven Metric Learning (IDML) for Graph Construction , 2010 .

[39]  Feiping Nie,et al.  Learning A Structured Optimal Bipartite Graph for Co-Clustering , 2017, NIPS.

[40]  Yanwen Chong,et al.  Class Probability Propagation of Supervised Information Based on Sparse Subspace Clustering for Hyperspectral Images , 2017, Remote. Sens..