Matrixized Learning Machine with Feature-Clustering Interpolation

The existing matrixized learning machines (MLMs) use bilateral weight vectors on both side of one pattern as the constraints to manipulate matrix-based datasets directly. However, MLM might be challenged while the input pattern is a vector whose features are independent from each others. The traditional solution is to transform the vector into its all corresponding matrix forms, which is not only irrational, but also requiring extra computation in preprocessing. To overcome the problem, this paper proposes a novel matrixized learning model that utilizes an efficient clustering-based interpolation strategy to mapping the original vector-based pattern to an unique matrix. The proposed method first selects the most typical features defined as $$candidate\hbox {s}$$candidates from each dimension of all patterns in the same class through a fast clustering method, and then measures the relationship between the features of each training pattern and the $$candidate\hbox {s}$$candidates to generate a new matrix named the candidate-matrix in turn. Afterwards, the candidate-matrix is combined with the pattern to form the corresponding final matrix. At last, all final matrices are collected to form the new training set for the subsequent matrixized classifier. Named FCIMLM for short, the proposed matrixized method is proved more effective and efficient than the traditional MLM under the same structural risk minimum framework through the designed experiments on 21 vector-based benchmark datasets from UCI repository. The main contributions of this paper are: (1) proposing a new matrixized learning model with a more efficient matrixization process using a feature-based fast clustering strategy; (2) combining the feature-clustering-based interpolation to the matrix-pattern-oriented classifier for the first time; (3) extending the existing MLM design techniques.

[1]  David G. Stork,et al.  Pattern Classification , 1973 .

[2]  Qijun Zhao,et al.  Feature extraction using two-dimensional neighborhood margin and variation embedding , 2013, Comput. Vis. Image Underst..

[3]  Daoqiang Zhang,et al.  (2D)2PCA: Two-directional two-dimensional PCA for efficient face representation and recognition , 2005, Neurocomputing.

[4]  Ben Taskar,et al.  The Pairwise Piecewise-Linear Embedding for Efficient Non-Linear Classification , 2013, ICML.

[5]  Zhe Wang,et al.  IMAT: matrix learning machine with interpolation mapping , 2014 .

[6]  Sean Hughes,et al.  Clustering by Fast Search and Find of Density Peaks , 2016 .

[7]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[8]  Javier Pérez-Rodríguez,et al.  OligoIS: Scalable Instance Selection for Class-Imbalanced Data Sets , 2013, IEEE Transactions on Cybernetics.

[9]  Jacek M. Łȩski,et al.  Ho--Kashyap classifier with generalization control , 2003 .

[10]  Ming Li,et al.  Feature extraction using two-dimensional maximum embedding difference , 2014, Inf. Sci..

[11]  Jonathan E. Fieldsend,et al.  Multi-class ROC analysis from a multi-objective optimisation perspective , 2006, Pattern Recognit. Lett..

[12]  Alejandro F. Frangi,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004 .

[13]  María José del Jesús,et al.  Multi-class Imbalanced Data-Sets with Linguistic Fuzzy Rule Based Classification Systems Based on Pairwise Learning , 2010, IPMU.

[14]  Tian Yongjun and Chen Songcan,et al.  Matrix-Pattern-Oriented Ho-Kashyap Classifier with Regularization Learning , 2005 .

[15]  Ming Li,et al.  2D-LDA: A statistical linear discriminant analysis for image matrix , 2005, Pattern Recognit. Lett..

[16]  Yi-Zeng Liang,et al.  Monte Carlo cross validation , 2001 .

[17]  Svetha Venkatesh,et al.  Multi-class Pattern Classification in Imbalanced Data , 2010, 2010 20th International Conference on Pattern Recognition.

[18]  Bir Bhanu,et al.  Face image super-resolution using 2D CCA , 2014, Signal Process..

[19]  José Hernández-Orallo,et al.  Volume under the ROC Surface for Multi-class Problems , 2003, ECML.

[20]  Zhi-Hua Zhou,et al.  ON MULTI‐CLASS COST‐SENSITIVE LEARNING , 2006, Comput. Intell..

[21]  Francisco Herrera,et al.  A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting-, and Hybrid-Based Approaches , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[22]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[23]  Palaiahnakote Shivakumara,et al.  (2D)2LDA: An efficient approach for face recognition , 2006, Pattern Recognit..

[24]  Xinrong Li,et al.  Two-dimensional nearest neighbor classification for agricultural remote sensing , 2014, Neurocomputing.

[25]  R. L. Kashyap,et al.  An Algorithm for Linear Inequalities and its Applications , 1965, IEEE Trans. Electron. Comput..

[26]  Feiping Nie,et al.  Multiple rank multi-linear SVM for matrix data classification , 2014, Pattern Recognit..

[27]  Songcan Chen,et al.  Matrix-pattern-oriented Ho-Kashyap classifier with regularization learning , 2007, Pattern Recognit..

[28]  Céline Rouveirol,et al.  Machine Learning: ECML-98 , 1998, Lecture Notes in Computer Science.

[29]  Jiwen Lu,et al.  Multi-feature multi-manifold learning for single-sample face recognition , 2014, Neurocomputing.