A Low-Rank Approximation-Based Transductive Support Tensor Machine for Semisupervised Classification

In the fields of machine learning, pattern recognition, image processing, and computer vision, the data are usually represented by the tensors. For the semisupervised tensor classification, the existing transductive support tensor machine (TSTM) needs to resort to iterative technique, which is very time-consuming. In order to overcome this shortcoming, in this paper, we extend the concave-convex procedure-based transductive support vector machine (CCCP-TSVM) to the tensor patterns and propose a low-rank approximation-based TSTM, in which the tensor rank-one decomposition is used to compute the inner product of the tensors. Theoretically, concave-convex procedure-based TSTM (CCCP-TSTM) is an extension of the linear CCCP-TSVM to tensor patterns. When the input patterns are vectors, CCCP-TSTM degenerates into the linear CCCP-TSVM. A set of experiments is conducted on 23 semisupervised classification tasks, which are generated from seven second-order face data sets, three third-order gait data sets, and two third-order image data sets, to illustrate the performance of the CCCP-TSTM. The results show that compared with CCCP-TSVM and TSTM, CCCP-TSTM provides significant performance gain in terms of test accuracy and training speed.

[1]  B. R. Rajakumar,et al.  Impact of static and adaptive mutation techniques on the performance of Genetic Algorithm , 2013, Int. J. Hybrid Intell. Syst..

[2]  C. Martin,et al.  The rank of a 2 × 2 × 2 tensor , 2011 .

[3]  Salah Bourennane,et al.  Dimensionality Reduction Based on Tensor Modeling for Classification Methods , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Philip S. Yu,et al.  DuSK: A Dual Structure-preserving Kernel for Supervised Tensor Learning with Applications to Neuroimages , 2014, SDM.

[5]  Ying Wang,et al.  TWin support tensor machines for MCs detection , 2009 .

[6]  Songcan Chen,et al.  New Least Squares Support Vector Machines Based on Matrix Patterns , 2007, Neural Processing Letters.

[7]  Andrzej Cichocki,et al.  Tensor Decompositions: A New Concept in Brain Data Analysis? , 2013, ArXiv.

[8]  Ling Guan,et al.  Quantifying and recognizing human movement patterns from monocular video images-part II: applications to biometrics , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Munchurl Kim,et al.  Moving object segmentation in video sequences by user interaction and automatic object tracking , 2001, Image Vis. Comput..

[10]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[11]  Jiawei Han,et al.  Learning with Tensor Representation , 2006 .

[12]  Alexander Zien,et al.  Semi-Supervised Classification by Low Density Separation , 2005, AISTATS.

[13]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[14]  Yueting Zhuang,et al.  Tensor-Based Transductive Learning for Multimodality Video Semantic Concept Detection , 2009, IEEE Transactions on Multimedia.

[15]  Kwang Y. Lee,et al.  Optimal reactive power planning using evolutionary algorithms: a comparative study for evolutionary programming, evolutionary strategy, genetic algorithm, and linear programming , 1998 .

[16]  Jieping Ye,et al.  GPCA: an efficient dimension reduction scheme for image compression and retrieval , 2004, KDD.

[17]  Yuan-Hai Shao,et al.  A GA-based model selection for smooth twin parametric-margin support vector machine , 2013, Pattern Recognit..

[18]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[19]  S. Sathiya Keerthi,et al.  Improvements to Platt's SMO Algorithm for SVM Classifier Design , 2001, Neural Computation.

[20]  S. Picek,et al.  Comparison of a crossover operator in binary-coded genetic algorithms , 2010 .

[21]  Jirí Vomlel,et al.  Exploiting tensor rank-one decomposition in probabilistic inference , 2007, Kybernetika.

[22]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[23]  Wutthipong Chinnasri,et al.  Comparison of performance between different selection strategies on genetic algorithm with course timetabling problem , 2010, 2010 IEEE International Conference on Advanced Management Science(ICAMS 2010).

[24]  Pooran Singh Negi,et al.  3-D Discrete Shearlet Transform and Video Processing , 2012, IEEE Transactions on Image Processing.

[25]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[26]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[27]  Ying Wu,et al.  Abnormalities in Resting-State Functional Connectivity in Early Human Immunodeficiency Virus Infection , 2011, Brain Connect..

[28]  M. Kilmer,et al.  Factorization strategies for third-order tensors , 2011 .

[29]  Xuelong Li,et al.  Supervised tensor learning , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[30]  Brett W. Bader,et al.  The TOPHITS Model for Higher-Order Web Link Analysis∗ , 2006 .

[31]  Dong Xu,et al.  Multilinear Discriminant Analysis for Face Recognition , 2007, IEEE Transactions on Image Processing.

[32]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[33]  Yueting Zhuang,et al.  Active post-refined multimodality video semantic concept detection with tensor representation , 2008, ACM Multimedia.

[34]  Juliane Junker,et al.  Color Image Processing And Applications , 2016 .

[35]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[36]  Sudeep Sarkar,et al.  The humanID gait challenge problem: data sets, performance, and analysis , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Haiping Lu,et al.  Uncorrelated Multilinear Discriminant Analysis With Regularization and Aggregation for Tensor Object Recognition , 2009, IEEE Transactions on Neural Networks.

[38]  Haiping Lu,et al.  MPCA: Multilinear Principal Component Analysis of Tensor Objects , 2008, IEEE Transactions on Neural Networks.

[39]  Zhi-Hua Zhou,et al.  Face Image Modeling by Multilinear Subspace Analysis With Missing Values , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[40]  Haiping Lu,et al.  Regularized common spatial patterns with generic learning for EEG signal classification , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[41]  Xiaowei Yang,et al.  A GA-based feature selection and parameter optimization for linear support higher-order tensor machine , 2014, Neurocomputing.

[42]  Narendra Ahuja,et al.  Compact representation of multidimensional data using tensor rank-one decomposition , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[43]  Cheng-Lung Huang,et al.  A GA-based feature selection and parameters optimizationfor support vector machines , 2006, Expert Syst. Appl..

[44]  Xiaowei Yang,et al.  A Linear Support Higher-Order Tensor Machine for Classification , 2013, IEEE Transactions on Image Processing.

[45]  V. Kshirsagar,et al.  Face recognition using Eigenfaces , 2011, 2011 3rd International Conference on Computer Research and Development.

[46]  Lieven De Lathauwer,et al.  An enhanced line search scheme for complex-valued tensor decompositions. Application in DS-CDMA , 2008, Signal Process..

[47]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[48]  Jason Weston,et al.  Large Scale Transductive SVMs , 2006, J. Mach. Learn. Res..

[49]  L. Lathauwer,et al.  Signal Processing based on Multilinear Algebra , 1997 .

[50]  Weiwei Guo,et al.  Tensor Learning for Regression , 2012, IEEE Transactions on Image Processing.

[51]  Moutaz Saleh Mustafa Saleh,et al.  Evaluating the Effectiveness of Mutation Operators on the Behavior of Genetic Algorithms Applied to Non-deterministic Polynomial Problems , 2011, Informatica.

[52]  R. Harshman,et al.  PARAFAC: parallel factor analysis , 1994 .

[53]  O. Mangasarian,et al.  Semi-Supervised Support Vector Machines for Unlabeled Data Classification , 2001 .

[54]  Jun Zhang,et al.  Comparison of Performance between Different Selection Strategies on Simple Genetic Algorithms , 2005, International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC'06).

[55]  Gene H. Golub,et al.  Rank-One Approximation to High Order Tensors , 2001, SIAM J. Matrix Anal. Appl..

[56]  Liangpei Zhang,et al.  A Multifeature Tensor for Remote-Sensing Target Recognition , 2011, IEEE Geoscience and Remote Sensing Letters.

[57]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[58]  Rama Chellappa,et al.  Recognition of Humans and Their Activities Using Video , 2005, Recognition of Humans and Their Activities Using Video.

[59]  Keith C. C. Chan,et al.  Tensor-based locally maximum margin classifier for image and video classification , 2011, Comput. Vis. Image Underst..