Multi-task least squares twin support vector machine for classification

Abstract With the bloom of machine learning, pattern recognition plays an important role in many aspects. However, traditional pattern recognition mainly focuses on single task learning (STL), and the multi-task learning (MTL) has largely been ignored. Compared to STL, MTL can improve the performance of learning methods through the shared information among all tasks. Inspired by the recently proposed directed multi-task twin support vector machine (DMTSVM) and the least squares twin support vector machine (LSTWSVM), we put forward a novel multi-task least squares twin support vector machine (MTLS-TWSVM). Instead of two dual quadratic programming problems (QPPs) solved in DMTSVM, our algorithm only needs to deal with two smaller linear equations. This leads to simple solutions, and the calculation can be effectively accelerated. Thus, our proposed model can be applied to the large scale datasets. In addition, it can deal with linear inseparable samples by using kernel trick. Experiments on three popular multi-task datasets show the effectiveness of our proposed methods. Finally, we apply it to two popular image datasets, and the experimental results also demonstrate the validity of our proposed algorithm.

[1]  Weitong Chen,et al.  Multi-task support vector machines for feature selection with shared knowledge discovery , 2016, Signal Process..

[2]  Yong Shi,et al.  ν-Nonparallel support vector machine for pattern classification , 2014, Neural Computing and Applications.

[3]  Wen Gao,et al.  Multi-Task Learning with Low Rank Attribute Embedding for Multi-Camera Person Re-Identification , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Qiang Yang,et al.  An Overview of Multi-task Learning , 2018 .

[5]  Shiliang Sun,et al.  Multitask centroid twin support vector machines , 2015, Neurocomputing.

[6]  Shiliang Sun.  Multitask learning for EEG-based biometrics , 2008, 2008 19th International Conference on Pattern Recognition.

[7]  Peter Xiang Gao.  Facial age estimation using Clustered Multi-task Support Vector Regression Machine , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[8]  Ping Zhong,et al.  The aLS-SVM based multi-task learning classifiers , 2017, Applied Intelligence.

[9]  Yuan-Hai Shao,et al.  Least squares recursive projection twin support vector machine for classification , 2012, Pattern Recognit..

[10]  Vladimir Cherkassky,et al.  Implementation and comparison of SVM-based Multi-Task Learning methods , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[11]  Rich Caruana,et al.  Multitask Learning , 1997, Machine Learning.

[12]  Ya Zhang,et al.  Boosted multi-task learning , 2010, Machine Learning.

[13]  Vinay Jayaram,et al.  Multi-task logistic regression in brain-computer interfaces , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[14]  Michael R. Lyu,et al.  Multi-task Learning for one-class classification , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[15]  Hongxun Yao,et al.  Multi-modal microblog classification via multi-task learning , 2014, Multimedia Tools and Applications.

[16]  Shuo Xu,et al.  Multi-task least-squares support vector machines , 2014, Multimedia Tools and Applications.

[17]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[18]  Pierre Beauseroy,et al.  Multi-task learning for one-class SVM with additional new features , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[19]  Yu Zhang,et al.  A Survey on Multi-Task Learning , 2017, IEEE Transactions on Knowledge and Data Engineering.

[20]  He Yan,et al.  L1-Norm GEPSVM Classifier Based on an Effective Iterative Algorithm for Classification , 2017, Neural Processing Letters.

[21]  José Ragot,et al.  Multi-task learning with one-class SVM , 2014, Neurocomputing.

[22]  Jiayu Zhou,et al.  Interactive Multi-task Relationship Learning , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[23]  Jieping Ye,et al.  Multi-stage multi-task feature learning , 2012, J. Mach. Learn. Res..

[24]  Chao Yang,et al.  Multi-Task Joint Sparse and Low-Rank Representation for the Scene Classification of High-Resolution Remote Sensing Image , 2016, Remote. Sens..

[25]  Dacheng Tao,et al.  On Better Exploring and Exploiting Task Relationships in Multitask Learning: Joint Model and Feature Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[27]  Jieping Ye,et al.  Learning incoherent sparse and low-rank patterns from multiple tasks , 2010 .

[28]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[30]  Subramanian Ramanathan,et al.  Multitask Linear Discriminant Analysis for View Invariant Action Recognition , 2014, IEEE Transactions on Image Processing.

[31]  He Yan,et al.  Least squares twin bounded support vector machines based on L1-norm distance metric for classification , 2018, Pattern Recognit..

[32]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[33]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Massimiliano Pontil,et al.  Regularized multi--task learning , 2004, KDD.

[35]  Shiliang Sun,et al.  Multitask multiclass support vector machines: Model and experiments , 2013, Pattern Recognit..

[36]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[37]  Xuelong Li,et al.  Calibrated Multi-Task Learning , 2018, KDD.

[38]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[39]  Yitian Xu,et al.  A weighted twin support vector regression , 2012, Knowl. Based Syst..

[40]  Luis Gómez-Chova,et al.  Multitask Remote Sensing Data Classification , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Gunnar Rätsch,et al.  Efficient Training of Graph-Regularized Multitask SVMs , 2012, ECML/PKDD.

[42]  Jiayu Zhou,et al.  Asynchronous Multi-task Learning , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[43]  Dacheng Tao,et al.  Multi-task proximal support vector machine , 2015, Pattern Recognit..

[44]  Xianli Pan,et al.  K-nearest neighbor based structural twin support vector machine , 2015, Knowl. Based Syst..

[45]  Kristian Kersting,et al.  Multi-task Learning with Task Relations , 2011, 2011 IEEE 11th International Conference on Data Mining.

[46]  Jiayu Zhou,et al.  Clustered Multi-Task Learning Via Alternating Structure Optimization , 2011, NIPS.

[47]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[48]  Liyan Zhang,et al.  A Feature Selection Method for Projection Twin Support Vector Machine , 2017, Neural Processing Letters.

[49]  Tom Heskes,et al.  Task Clustering and Gating for Bayesian Multitask Learning , 2003, J. Mach. Learn. Res..

[50]  Shiliang Sun,et al.  Multitask Twin Support Vector Machines , 2012, ICONIP.

[51]  Xianli Pan,et al.  A Novel Twin Support-Vector Machine With Pinball Loss , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[52]  Anton Schwaighofer,et al.  Learning Gaussian processes from multiple tasks , 2005, ICML.