Robust Adaptive Embedded Label Propagation With Weight Learning for Inductive Classification

We propose a robust inductive semi-supervised label prediction model over the embedded representation, termed adaptive embedded label propagation with weight learning (AELP-WL), for classification. AELP-WL offers several properties. First, our method seamlessly integrates the robust adaptive embedded label propagation with adaptive weight learning into a unified framework. By minimizing the reconstruction errors over embedded features and embedded soft labels jointly, our AELP-WL can explicitly ensure the learned weights to be joint optimal for representation and classification, which differs from most existing LP models that perform weight learning separately by an independent step before label prediction. Second, existing models usually precalculate the weights over the original samples that may contain unfavorable features and noise decreasing performance. To this end, our model adds a constraint that decomposes original data into a sparse component encoding embedded noise-removed sparse representations of samples and a sparse error part fitting noise, and then performs the adaptive weight learning over the embedded sparse representations. Third, our AELP-WL computes the projected soft labels by trading-off the manifold smoothness and label fitness errors over the adaptive weights and the embedded representations for enhancing the label estimation power. By including a regressive label approximation error for simultaneous minimization to correlate sample features with the embedded soft labels, the out-of-sample issue is naturally solved. By minimizing the reconstruction errors over features and embedded soft labels, classification error and label approximation error jointly, state-of-the-art results are delivered.

[1]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[2]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[3]  Iñaki Inza,et al.  Semisupervised Multiclass Classification Problems With Scarcity of Labeled Data: A Theoretical Study , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[5]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[6]  Songcan Chen,et al.  Safety-Aware Semi-Supervised Classification , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Ran He,et al.  Robust Principal Component Analysis Based on Maximum Correntropy Criterion , 2011, IEEE Transactions on Image Processing.

[8]  Vicki Bruce,et al.  Face Recognition: From Theory to Applications , 1999 .

[9]  Xuelong Li,et al.  Semisupervised Dimensionality Reduction and Classification Through Virtual Label Regression , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Yangyang Li,et al.  Self-representation based dual-graph regularized feature selection clustering , 2016, Neurocomputing.

[11]  Jian Yang,et al.  Can the Virtual Labels Obtained by Traditional LP Approaches Be Well Encoded in WLR? , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Ran He,et al.  Two-Stage Nonnegative Sparse Representation for Large-Scale Face Recognition , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Feiping Nie,et al.  A general graph-based semi-supervised learning with novel class discovery , 2010, Neural Computing and Applications.

[15]  Li Zhang,et al.  Projective Label Propagation by Label Embedding , 2015, CAIP.

[16]  A. Martínez,et al.  The AR face databasae , 1998 .

[17]  Jiang-She Zhang,et al.  Label propagation through sparse neighborhood and its applications , 2012, Neurocomputing.

[18]  Shuyuan Yang,et al.  Global discriminative-based nonnegative spectral clustering , 2016, Pattern Recognit..

[19]  Yi Yang,et al.  Improved Spectral Clustering via Embedded Label Propagation , 2014, ArXiv.

[20]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[21]  Fei Wang,et al.  Fast semi-supervised clustering with enhanced spectral embedding , 2012, Pattern Recognit..

[22]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Feiping Nie,et al.  The Constrained Laplacian Rank Algorithm for Graph-Based Clustering , 2016, AAAI.

[24]  Jieping Ye,et al.  Integrating Global and Local Structures: A Least Squares Framework for Dimensionality Reduction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[26]  George Michailidis,et al.  Graph-Based Semisupervised Learning , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[28]  D. B. Graham,et al.  Characterising Virtual Eigensignatures for General Purpose Face Recognition , 1998 .

[29]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[30]  Jing Liu,et al.  Robust Structured Subspace Learning for Data Representation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Feiping Nie,et al.  Semi-supervised feature selection based on label propagation and subset selection , 2010, 2010 International Conference on Computer and Information Application.

[32]  Ivor W. Tsang,et al.  Flexible Manifold Embedding: A Framework for Semi-Supervised and Unsupervised Dimension Reduction , 2010, IEEE Transactions on Image Processing.

[33]  Tommy W. S. Chow,et al.  Graph Based Constrained Semi-Supervised Learning Framework via Label Propagation over Adaptive Neighborhood , 2015, IEEE Transactions on Knowledge and Data Engineering.

[34]  Nanning Zheng,et al.  Convergence of a Fixed-Point Algorithm under Maximum Correntropy Criterion , 2015, IEEE Signal Processing Letters.

[35]  Hong Cheng,et al.  Sparsity induced similarity measure for label propagation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[36]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[37]  Shuicheng Yan,et al.  Pairwise Sparsity Preserving Embedding for Unsupervised Subspace Learning and Classification , 2013, IEEE Transactions on Image Processing.

[38]  Feiping Nie,et al.  Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence Adaptive Loss Minimization for Semi-Supervised Elastic Embedding , 2022 .

[39]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[40]  Li Zhang,et al.  Semi-Supervised Image Classification by Nonnegative Sparse Neighborhood Propagation , 2015, ICMR.

[41]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[42]  Larry S. Davis,et al.  Label Consistent K-SVD: Learning a Discriminative Dictionary for Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Zhao Zhang,et al.  Adaptive Neighborhood Propagation by Joint L2,1-Norm Regularized Sparse Coding for Representation and Classification , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[44]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Ran He,et al.  Label Propagation Algorithm Based on Non-negative Sparse Representation , 2010, LSMS/ICSEE.

[46]  V. Kshirsagar,et al.  Face recognition using Eigenfaces , 2011, 2011 3rd International Conference on Computer Research and Development.

[47]  Yousef Saad,et al.  Trace optimization and eigenproblems in dimension reduction methods , 2011, Numer. Linear Algebra Appl..

[48]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Shiliang Sun,et al.  Semisupervised Support Vector Machines With Tangent Space Intrinsic Manifold Regularization , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[50]  Hao Chen,et al.  Prior class dissimilarity based linear neighborhood propagation , 2015, Knowl. Based Syst..

[51]  Yan Zhang,et al.  Discriminative sparse flexible manifold embedding with novel graph for robust visual representation and label propagation , 2017, Pattern Recognit..

[52]  Fei Wang,et al.  Label Propagation through Linear Neighborhoods , 2008, IEEE Trans. Knowl. Data Eng..

[53]  Licheng Jiao,et al.  Fast Sparse Approximation for Least Squares Support Vector Machine , 2007, IEEE Transactions on Neural Networks.