Globalized and localized matrix-pattern-oriented classification machine

Graphical abstractDisplay Omitted HighlightsA novel classification algorithm named GLMatMHKS is proposed.To capture more structual information through a new regularization term Rgl.To focus on both global and local view of the input matrix sample space.Effectiveness is validated by comparing it with some classic classifiers.The generalization risk bound of it is proved tighter. Inspired by the matrix-based methods used in feature extraction and selection, one matrix-pattern-oriented classification framework has been designed in our previous work and demonstrated to utilize one matrix pattern itself more effectively to improve the classification performance in practice. However, this matrix-based framework neglects the prior structural information of the whole input space that is made up of all the matrix patterns. This paper aims to overcome such flaw through taking advantage of one structure learning method named Alternative Robust Local Embedding (ARLE). As a result, a new regularization term Rgl is designed, expected to simultaneously represent the globality and the locality of the whole data domain, further boosting the existing matrix-based classification method. To our knowledge, it is the first trial to introduce both the globality and the locality of the whole data space into the matrixized classifier design. In order to validate the proposed approach, the designed Rgl is applied into the previous work matrix-pattern-oriented Ho-Kashyap classifier (MatMHKS) to construct a new globalized and localized MatMHKS named GLMatMHKS. The experimental results on a broad range of data validate that GLMatMHKS not only inherits the advantages of the matrixized learning, but also uses the prior structural information more reasonably to guide the classification machine design.

[1]  Alejandro F. Frangi,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004 .

[2]  Daoqiang Zhang,et al.  Pattern Representation in Feature Extraction and Classifier Design: Matrix Versus Vector , 2008, IEEE Transactions on Neural Networks.

[3]  Daoqiang Zhang,et al.  (2D)2PCA: Two-directional two-dimensional PCA for efficient face representation and recognition , 2005, Neurocomputing.

[4]  Ja-Chen Lin,et al.  A new LDA-based face recognition system which can solve the small sample size problem , 1998, Pattern Recognit..

[5]  Jing Peng,et al.  SVM vs regularized least squares classification , 2004, ICPR 2004.

[6]  Songcan Chen,et al.  Matrix-pattern-oriented Ho-Kashyap classifier with regularization learning , 2007, Pattern Recognit..

[7]  Dana Ron,et al.  An experimental and theoretical comparison of model selection methods , 1995, COLT '95.

[8]  Vladimir Koltchinskii,et al.  Rademacher penalties and structural risk minimization , 2001, IEEE Trans. Inf. Theory.

[9]  Jacek M. Łȩski,et al.  Ho--Kashyap classifier with generalization control , 2003 .

[10]  Peter L. Bartlett,et al.  Model Selection and Error Estimation , 2000, Machine Learning.

[11]  I. Jolliffe.  Principal Component Analysis , 2002 .

[12]  Ming Li,et al.  2D-LDA: A statistical linear discriminant analysis for image matrix , 2005, Pattern Recognit. Lett..

[13]  Yuxiao Hu,et al.  Learning a Spatially Smooth Subspace for Face Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Yi-Zeng Liang,et al.  Monte Carlo cross validation , 2001 .

[15]  Tomaso Poggio,et al.  Image Representations for Visual Learning , 1996, Science.

[16]  Andrew Beng Jin Teoh,et al.  Combining local face image features for identity verification , 2011, Neurocomputing.

[17]  Xuan Li,et al.  Local and global structure preserving based feature selection , 2012, Neurocomputing.

[18]  Songcan Chen,et al.  A Novel Matrix-Pattern-Oriented Ho-Kashyap Classifier with Locally Spatial Smoothness , 2009, ISNN.

[19]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[20]  Narendra Ahuja,et al.  Rank-R approximation of tensors using image-as-matrix representation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[21]  Xiangyang Xue,et al.  An efficient Kernel-based matrixized least squares support vector machine , 2011, Neural Computing and Applications.

[22]  Hui Xue,et al.  Alternative robust local embedding , 2007, 2007 International Conference on Wavelet Analysis and Pattern Recognition.

[23]  David Zhang,et al.  Ensemble of local and global information for finger-knuckle-print recognition , 2011, Pattern Recognit..

[24]  Zhongliang Jing,et al.  Local structure based supervised feature extraction , 2006, Pattern Recognit..

[25]  Hui Xu,et al.  Two-dimensional supervised local similarity and diversity projection , 2010, Pattern Recognit..

[26]  Palaiahnakote Shivakumara,et al.  (2D)2LDA: An efficient approach for face recognition , 2006, Pattern Recognit..

[27]  R. L. Kashyap,et al.  An Algorithm for Linear Inequalities and its Applications , 1965, IEEE Trans. Electron. Comput..

[28]  Songcan Chen,et al.  New Least Squares Support Vector Machines Based on Matrix Patterns , 2007, Neural Processing Letters.

[29]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[30]  Ethem Alpaydin,et al.  Localized algorithms for multiple kernel learning , 2013, Pattern Recognit..

[31]  Jieping Ye,et al.  Generalized Low Rank Approximations of Matrices , 2005, Machine Learning.

[32]  Hui Xue,et al.  Glocalization pursuit support vector machine , 2011, Neural Computing and Applications.

[33]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[34]  Songcan Chen,et al.  A novel multi-view learning developed from single-view patterns , 2011, Pattern Recognit..

[35]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[36]  Jian Yang,et al.  Sparse two-dimensional local discriminant projections for feature extraction , 2011, Neurocomputing.