Double-fold localized multiple matrix learning machine with Universum

Matrix learning, multiple-view learning, Universum learning, and local learning are four hot spots of present research. Matrix learning aims to design feasible machines to process matrix patterns directly. Multiple-view learning takes pattern information from multiple aspects, i.e., multiple-view information into account. Universum learning can reflect priori knowledge about application domain and improve classification performances. A good local learning approach is important to the finding of local structures and pattern information. Our previous proposed learning machine, double-fold localized multiple matrix learning machine is a one with multiple-view information, local structures, and matrix learning. But this machine does not take Universum learning into account. Thus, this paper proposes a double-fold localized multiple matrix learning machine with Universum (Uni-DLMMLM) so as to improve the performance of a learning machine. Experimental results have validated that Uni-DLMMLM (1) makes full use of the domain knowledge of whole data distribution as well as inherits the advantages of matrix learning; (2) combines Universum learning with matrix learning so as to capture more global knowledge; (3) has a good ability to process different kinds of data sets; (4) has a superior classification performance and leads to a low empirical generation risk bound.

[1]  Daoqiang Zhang,et al.  Feature extraction approaches based on matrix pattern: MatPCA and MatFLDA , 2005, Pattern Recognit. Lett..

[2]  Fumin Shen,et al.  {\cal U}Boost: Boosting with the Universum , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[4]  Vladimir Koltchinskii,et al.  Rademacher penalties and structural risk minimization , 2001, IEEE Trans. Inf. Theory.

[5]  Wuyang Dai,et al.  Empirical Study of the Universum SVM Learning for High-Dimensional Data , 2009, ICANN.

[6]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .

[7]  Songcan Chen,et al.  New Least Squares Support Vector Machines Based on Matrix Patterns , 2007, Neural Processing Letters.

[8]  Gang Qian,et al.  View-Invariant Pose Recognition Using Multilinear Analysis and the Universum , 2008, ISVC.

[9]  Xiang Feng,et al.  Double-fold localized multiple matrixized learning machine , 2015, Inf. Sci..

[10]  V. Koltchinskii,et al.  Rademacher Processes and Bounding the Risk of Function Learning , 2004, math/0405338.

[11]  David G. Stork,et al.  Pattern Classification , 1973 .

[12]  Gavin C. Cawley,et al.  On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation , 2010, J. Mach. Learn. Res..

[13]  Xiang Feng,et al.  Multi-kernel classification machine with reduced complexity , 2014, Knowl. Based Syst..

[14]  Jin Xu,et al.  Multiple empirical kernel learning based on local information , 2012, Neural Computing and Applications.

[15]  Peter L. Bartlett,et al.  Model Selection and Error Estimation , 2000, Machine Learning.

[16]  Zhe Wang,et al.  Three-fold structured classifier design based on matrix pattern , 2013, Pattern Recognit..

[17]  Bo Du,et al.  Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding , 2015, Pattern Recognit..

[18]  Xuelong Li,et al.  Single-image super-resolution via local learning , 2011, Int. J. Mach. Learn. Cybern..

[19]  Yong Shi,et al.  Self-Universum support vector machine , 2014, Personal and Ubiquitous Computing.

[20]  Yong Luo,et al.  Tensor Canonical Correlation Analysis for Multi-View Dimension Reduction , 2015, IEEE Trans. Knowl. Data Eng..

[21]  Jingdong Wang,et al.  Online Robust Non-negative Dictionary Learning for Visual Tracking , 2013, 2013 IEEE International Conference on Computer Vision.

[22]  Shahar Mendelson,et al.  Rademacher averages and phase transitions in Glivenko-Cantelli classes , 2002, IEEE Trans. Inf. Theory.

[23]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[24]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[25]  Changshui Zhang,et al.  Selecting Informative Universum Sample for Semi-Supervised Learning , 2009, IJCAI.

[26]  Patrick P. K. Chan,et al.  Radial Basis Function network learning using localized generalization error bound , 2009, Inf. Sci..

[27]  Songcan Chen,et al.  A novel multi-view learning developed from single-view patterns , 2011, Pattern Recognit..

[28]  Robert Sabourin,et al.  “One Against One” or “One Against All”: Which One is Better for Handwriting Recognition with SVMs? , 2006 .

[29]  Guiqiang Ni,et al.  One-Class Support Vector Machines Based on Matrix Patterns , 2011 .

[30]  Wenwen Liu,et al.  Multi-view learning with Universum , 2014, Knowl. Based Syst..

[31]  Narendra Ahuja,et al.  Rank-R approximation of tensors using image-as-matrix representation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[32]  Hui Xue,et al.  Universum linear discriminant analysis , 2012 .

[33]  Jin Xu,et al.  Regularized multi-view learning machine based on response surface technique , 2012, Neurocomputing.

[34]  Songcan Chen,et al.  Matrix-pattern-oriented Ho-Kashyap classifier with regularization learning , 2007, Pattern Recognit..

[35]  Dacheng Tao,et al.  Multi-View Intact Space Learning , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Jieping Ye,et al.  Generalized Low Rank Approximations of Matrices , 2004, Machine Learning.

[37]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[38]  William Stafford Noble,et al.  Support vector machine , 2013 .

[39]  T. C. Edwin Cheng,et al.  Single-machine scheduling with sum-of-logarithm-processing-times-based learning considerations , 2009, Inf. Sci..

[40]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[41]  Jacek M. Łȩski,et al.  Ho--Kashyap classifier with generalization control , 2003 .

[42]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[43]  Ming Li,et al.  2D-LDA: A statistical linear discriminant analysis for image matrix , 2005, Pattern Recognit. Lett..

[44]  Alejandro F. Frangi,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004 .

[45]  Dit-Yan Yeung,et al.  Learning a Deep Compact Image Representation for Visual Tracking , 2013, NIPS.

[46]  Sankar K. Pal,et al.  Incorporating local image structure in normalized cut based graph partitioning for grouping of pixels , 2013, Inf. Sci..

[47]  Patrick P. K. Chan,et al.  Dynamic fusion method using Localized Generalization Error Model , 2012, Inf. Sci..

[48]  Dan Zhang,et al.  Document clustering with universum , 2011, SIGIR.

[49]  Abhisek Ukil,et al.  Support Vector Machine , 2007 .

[50]  Rameswar Debnath,et al.  A decision based one-against-one method for multi-class support vector machine , 2004, Pattern Analysis and Applications.