Semi-supervised graph-based retargeted least squares regression

A semi-supervised graph-based retargeted least squares regression model is proposed for multicategory classification.Our aim is to utilize a graph regularization to restrict the regression labels of ReLSR, such that similar samples should have similar regression labels.Linear squares regression and graph construction are unified into a same framework to achieve an overall optimum. In this paper, we propose a semi-supervised graph-based retargeted least squares regression model (SSGReLSR) for multicategory classification. The main motivation behind SSGReLSR is to utilize a graph regularization to restrict the regression labels of ReLSR, such that similar samples should have similar regression labels. However, in SSGReLSR, constructing the graph structure and learning the regression matrix are two independent processes, which cant guarantee an overall optimum. To overcome this shortage of SSGReLSR, we also propose a semi-supervised graph learning retargeted least squares regression model (SSGLReLSR), where linear squares regression and graph construction are unified into a same framework to achieve an overall optimum. To optimize our proposed SSGLReLSR, an efficient iteration algorithm is proposed. Extensive experiments results confirm the effectiveness of our proposed methods.

[1]  Mohammed Bennamoun,et al.  Linear Regression for Face Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[3]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[4]  S. Wold,et al.  The Collinearity Problem in Linear Regression. The Partial Least Squares (PLS) Approach to Generalized Inverses , 1984 .

[5]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[6]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Feiping Nie,et al.  Discriminative Least Squares Regression for Multiclass Classification and Feature Selection , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Tilo Strutz,et al.  Data Fitting and Uncertainty: A practical introduction to weighted least squares and beyond , 2010 .

[9]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[10]  Jian Yang,et al.  A General Exponential Framework for Dimensionality Reduction , 2014, IEEE Transactions on Image Processing.

[11]  Chris H. Q. Ding,et al.  Multi-class protein fold recognition using support vector machines and neural networks , 2001, Bioinform..

[12]  Shuicheng Yan,et al.  Learning With $\ell ^{1}$-Graph for Image Analysis , 2010, IEEE Transactions on Image Processing.

[13]  Yuan Yan Tang,et al.  Robust Face Recognition via Minimum Error Entropy-Based Atomic Representation , 2015, IEEE Transactions on Image Processing.

[14]  Tieniu Tan,et al.  Feature Selection Based on Structured Sparsity: A Comprehensive Study , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Fatih Murat Porikli,et al.  Classification and Boosting with Multiple Collaborative Representations , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Sergio Verdu,et al.  Multiuser Detection , 1998 .

[19]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[20]  Ziyu Wang,et al.  Joint sparse model-based discriminative K-SVD for hyperspectral image classification , 2017, Signal Process..

[21]  Svetha Venkatesh,et al.  Face Recognition Using Kernel Ridge Regression , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Alioune Ngom,et al.  Nonnegative Least-Squares Methods for the Classification of High-Dimensional Biological Data , 2013, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[23]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[24]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[25]  Shiming Xiang,et al.  Retargeted Least Squares Regression Algorithm , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[26]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Ziqiang Wang,et al.  Multiple kernel local Fisher discriminant analysis for face recognition , 2013, Signal Process..

[28]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.