A Tensor Factorization Based Least Squares Support Tensor Machine for Classification

In the fields of machine learning, image processing, and pattern recognition, the existing least squares support tensor machine for tensor classification involves a non-convex optimization problem and needs to be solved by the iterative technique. Obviously, it is very time-consuming and may suffer from local minima. In order to overcome these two shortcomings, in this paper, we present a tensor factorization based least squares support tensor machine (TFLS-STM) for tensor classification. In TFLS-STM, we combine the merits of least squares support vector machine (LS-SVM) and tensor rank-one decomposition. Theoretically, TFLS-STM is an extension of the linear LS-SVM to tensor patterns. When the input patterns are vectors, TFLS-STM degenerates into the standard linear LS-SVM. A set of experiments is conducted on six second-order face recognition datasets to illustrate the performance of TFLS-STM. The experimental results show that compared with the alternating projection LS-STM (APLS-STM) and LS-SVM, the training speed of TFLS-STM is faster than those of APLS-STM and LS-SVM. In term of testing accuracy, TFLS-STM is comparable with LS-SVM and is superiors to APLS-STM.

[1]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[2]  Salah Bourennane,et al.  Dimensionality Reduction Based on Tensor Modeling for Classification Methods , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Jirí Vomlel,et al.  Exploiting tensor rank-one decomposition in probabilistic inference , 2007, Kybernetika.

[4]  Narendra Ahuja,et al.  Rank-R approximation of tensors using image-as-matrix representation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Lior Wolf,et al.  Modeling Appearances with Low-Rank SVM , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  S. Keerthi,et al.  SMO Algorithm for Least-Squares SVM Formulations , 2003, Neural Computation.

[7]  Yueting Zhuang,et al.  Active post-refined multimodality video semantic concept detection with tensor representation , 2008, ACM Multimedia.

[8]  Haiping Lu,et al.  MPCA: Multilinear Principal Component Analysis of Tensor Objects , 2008, IEEE Transactions on Neural Networks.

[9]  L. Lathauwer,et al.  Signal Processing based on Multilinear Algebra , 1997 .

[10]  Sudeep Sarkar,et al.  The humanID gait challenge problem: data sets, performance, and analysis , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Xuelong Li,et al.  Multitraining Support Vector Machine for Image Retrieval , 2006, IEEE Transactions on Image Processing.

[12]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[13]  Munchurl Kim,et al.  Moving object segmentation in video sequences by user interaction and automatic object tracking , 2001, Image Vis. Comput..

[14]  Pooran Singh Negi,et al.  3-D Discrete Shearlet Transform and Video Processing , 2012, IEEE Transactions on Image Processing.

[15]  Caroline Fossati,et al.  Improvement of Classification for Hyperspectral Images Based on Tensor Modeling , 2010, IEEE Geoscience and Remote Sensing Letters.

[16]  Issarachai Ngamroo,et al.  A heuristic training-based least squares support vector machines for power system stabilization by SMES , 2011, Expert Syst. Appl..

[17]  Dong Xu,et al.  Multilinear Discriminant Analysis for Face Recognition , 2007, IEEE Transactions on Image Processing.

[18]  Johan A. K. Suykens,et al.  Optimal control by least squares support vector machines , 2001, Neural Networks.

[19]  Gene H. Golub,et al.  Rank-One Approximation to High Order Tensors , 2001, SIAM J. Matrix Anal. Appl..

[20]  Irene Kotsia,et al.  Support tucker machines , 2011, CVPR 2011.

[21]  Zhi-Hua Zhou,et al.  Face Image Modeling by Multilinear Subspace Analysis With Missing Values , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  Dong Xu,et al.  Discriminant analysis with tensor representation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Xuelong Li,et al.  Supervised Tensor Learning , 2005, ICDM.

[24]  Sabine Van Huffel,et al.  A combined MRI and MRSI based multiclass system for brain tumour recognition using LS-SVMs with class probabilities and feature selection , 2007, Artif. Intell. Medicine.

[25]  Jiawei Han,et al.  Learning with Tensor Representation , 2006 .

[26]  K. Plataniotis,et al.  Color Image Processing and Applications , 2000 .

[27]  Mohamed Cheriet,et al.  Model selection for the LS-SVM. Application to handwriting recognition , 2009, Pattern Recognit..

[28]  Francesca Bovolo,et al.  A Novel Technique for Subpixel Image Classification Based on Support Vector Machine , 2010, IEEE Transactions on Image Processing.

[29]  Tamir Hazan,et al.  Sparse image coding using a 3D non-negative tensor factorization , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[30]  Kunle Olukotun,et al.  Map-Reduce for Machine Learning on Multicore , 2006, NIPS.

[31]  Jiawei Han,et al.  Subspace Learning Based on Tensor Analysis , 2005 .

[32]  Haiping Lu,et al.  Uncorrelated Multilinear Discriminant Analysis With Regularization and Aggregation for Tensor Object Recognition , 2009, IEEE Transactions on Neural Networks.

[33]  Satarupa Banerjee,et al.  Text classification: A least square support vector machine approach , 2007, Appl. Soft Comput..

[34]  Charless C. Fowlkes,et al.  Bilinear classifiers for visual recognition , 2009, NIPS.

[35]  Haiping Lu,et al.  A survey of multilinear subspace learning for tensor data , 2011, Pattern Recognit..