Twin Support Vector Machine: A review from 2007 to 2014

Abstract Twin Support Vector Machine (TWSVM) is an emerging machine learning method suitable for both classification and regression problems. It utilizes the concept of Generalized Eigen-values Proximal Support Vector Machine (GEPSVM) and finds two non-parallel planes for each class by solving a pair of Quadratic Programming Problems. It enhances the computational speed as compared to the traditional Support Vector Machine (SVM). TWSVM was initially constructed to solve binary classification problems; later researchers successfully extended it for multi-class problem domain. TWSVM always gives promising empirical results, due to which it has many attractive features which enhance its applicability. This paper presents the research development of TWSVM in recent years. This study is divided into two main broad categories - variant based and multi-class based TWSVM methods. The paper primarily discusses the basic concept of TWSVM and highlights its applications in recent years. A comparative analysis of various research contributions based on TWSVM is also presented. This is helpful for researchers to effectively utilize the TWSVM as an emergent research methodology and encourage them to work further in the performance enhancement of TWSVM.

[1]  Koby Crammer,et al.  On the Learnability and Design of Output Codes for Multiclass Problems , 2002, Machine Learning.

[2]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[3]  Kyung-shik Shin,et al.  An application of support vector machines in bankruptcy prediction model , 2005, Expert Syst. Appl..

[4]  Xinsheng Zhang,et al.  Boosting Twin Support Vector Machine Approach for MCs Detection , 2009, 2009 Asia-Pacific Conference on Information Processing.

[5]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[6]  Xinbo Gao,et al.  Twin support vector machines and subspace learning methods for microcalcification clusters detection , 2012, Eng. Appl. Artif. Intell..

[7]  Shangbing Gao,et al.  1-Norm least squares twin support vector machines , 2011, Neurocomputing.

[8]  Shi-Jinn Horng,et al.  A novel intrusion detection system based on hierarchical clustering and support vector machines , 2011, Expert Syst. Appl..

[9]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[10]  Yuan-Hai Shao,et al.  A novel margin-based twin support vector machine with unity norm hyperplanes , 2012, Neural Computing and Applications.

[11]  Kemal Polat,et al.  Breast cancer diagnosis using least square support vector machine , 2007, Digit. Signal Process..

[12]  Divya Tomar,et al.  Prediction of software defects using Twin Support Vector Machine , 2014, 2014 International Conference on Information Systems and Computer Networks (ISCON).

[13]  Joon Beom Seo,et al.  Fast and efficient lung disease classification using hierarchical one-against-all support vector machine and cost-sensitive feature selection , 2012, Comput. Biol. Medicine.

[14]  Kyo Kageura,et al.  Virtual relevant documents in text categorization with support vector machines , 2007, Inf. Process. Manag..

[15]  Tai-Yue Wang,et al.  Solving multi-label text categorization problem using support vector machine approach with membership function , 2011, Neurocomputing.

[16]  Yitian Xu,et al.  A weighted twin support vector regression , 2012, Knowl. Based Syst..

[17]  Guangrong Ji,et al.  Multi-class LSTSVM classifier based on optimal directed acyclic graph , 2010, 2010 The 2nd International Conference on Computer and Automation Engineering (ICCAE).

[18]  Jiri Matas,et al.  Support vector machines for face authentication , 2002, Image Vis. Comput..

[19]  Divya Tomar,et al.  A Feature Selection Based Model for Software Defect Prediction , 2014 .

[20]  Joseph Picone,et al.  Applications of support vector machines to speech recognition , 2004, IEEE Transactions on Signal Processing.

[21]  Divya Tomar,et al.  Weighted Least Square Twin Support Vector Machine for Imbalanced Dataset , 2014 .

[22]  Jun He,et al.  Intrusion detection model with twin support vector machines , 2014, Journal of Shanghai Jiaotong University (Science).

[23]  Xinjun Peng,et al.  Building sparse twin support vector machine classifiers in primal space , 2011, Inf. Sci..

[24]  Divya Tomar,et al.  Feature Selection based Least Square Twin Support Vector Machine for Diagnosis of Heart Disease , 2014, BSBT 2014.

[25]  Juanying Xie,et al.  Using support vector machines with a novel hybrid feature selection method for diagnosis of erythemato-squamous diseases , 2011, Expert Syst. Appl..

[26]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[27]  Rui Guo,et al.  An improved ν-twin support vector machine , 2013, Applied Intelligence.

[28]  Kyoung-jae Kim,et al.  Financial time series forecasting using support vector machines , 2003, Neurocomputing.

[29]  Yuan-Hai Shao,et al.  Multiple birth support vector machine for multi-class classification , 2012, Neural Computing and Applications.

[30]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[31]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[32]  Yuan-Hai Shao,et al.  Least squares recursive projection twin support vector machine for classification , 2012, Pattern Recognit..

[33]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[34]  Gang Wang,et al.  A new hybrid method based on local fisher discriminant analysis and support vector machines for hepatitis disease diagnosis , 2011, Expert Syst. Appl..

[35]  Jalal A. Nasiri,et al.  Least squares twin multi-class classification support vector machine , 2015, Pattern Recognit..

[36]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[37]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[38]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[39]  Dong Xu,et al.  Bi-density twin support vector machines for pattern recognition , 2013, Neurocomputing.

[40]  Zhichao Li,et al.  High Efficient Intrusion Detection Methodology with Twin Support Vector Machines , 2008, 2008 International Symposium on Information Science and Engineering.

[41]  Xinjun Peng,et al.  A nu-twin support vector machine (nu-TSVM) classifier and its geometric algorithms , 2010, Inf. Sci..

[42]  Jian Yang,et al.  Recursive projection twin support vector machine via within-class variance minimization , 2011, Pattern Recognit..

[43]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Ning Ye,et al.  Twin Support Vector Machines via Fast Generalized Newton Refinement , 2010, 2010 Second International Conference on Intelligent Human-Machine Systems and Cybernetics.

[45]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[46]  Xiangyang Wang,et al.  Image denoising using nonsubsampled shearlet transform and twin support vector machines , 2014, Neural Networks.

[47]  B. Venkataramani,et al.  Design of a real time automatic speech recognition system using Modified One Against All SVM classifier , 2011, Microprocess. Microsystems.

[48]  Madan Gopal,et al.  Knowledge based Least Squares Twin support vector machines , 2010, Inf. Sci..

[49]  Yong Shi,et al.  Structural twin support vector machine for classification , 2013, Knowl. Based Syst..

[50]  Tai-Yue Wang,et al.  Fuzzy support vector machine for multi-class text categorization , 2007, Inf. Process. Manag..

[51]  Nasser Hassan Sweilam,et al.  Support vector machine for diagnosis cancer disease: A comparative study , 2010 .

[52]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Guodong Guo,et al.  Support vector machines for face recognition , 2001, Image Vis. Comput..

[54]  Jing Zhao,et al.  Twin least squares support vector regression , 2013, Neurocomputing.

[55]  Siyang Zhang,et al.  A novel hybrid KPCA and SVM with GA model for intrusion detection , 2014, Appl. Soft Comput..

[56]  Sonali Agarwal,et al.  SVM based context awareness using body area sensor network for pervasive healthcare monitoring , 2010, IITM '10.

[57]  Amitava Chatterjee,et al.  Support vector machines employing cross-correlation for emotional speech recognition , 2009 .

[58]  Dong Xu,et al.  A twin projection support vector machine for data regression , 2014, Neurocomputing.

[59]  Li Juelong,et al.  A new model for software defect prediction using Particle Swarm Optimization and support vector machine , 2013, 2013 25th Chinese Control and Decision Conference (CCDC).

[60]  Yitian Xu,et al.  K-nearest neighbor-based weighted twin support vector regression , 2014, Applied Intelligence.

[61]  Xinjun Peng,et al.  TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition , 2011, Pattern Recognit..

[62]  Dong Xu,et al.  Twin Mahalanobis distance-based support vector machines for pattern recognition , 2012, Inf. Sci..

[63]  Xinjun Peng,et al.  Improvements on twin parametric-margin support vector machine , 2015, Neurocomputing.

[64]  Xinbo Gao,et al.  MCs detection approach using Bagging and Boosting based twin support vector machine , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[65]  Seungmin Rho,et al.  SMERS: Music Emotion Recognition Using Support Vector Regression , 2009, ISMIR.

[66]  Ulrich H.-G. Kreßel,et al.  Pairwise classification and support vector machines , 1999 .

[67]  Guangrong Ji,et al.  Weighted least squares twin support vector machines for pattern classification , 2010, 2010 The 2nd International Conference on Computer and Automation Engineering (ICCAE).

[68]  Zhendong Wu,et al.  Study to Multi-Twin Support Vector Machines and Its Applications in Speaker Recognition , 2009, 2009 International Conference on Computational Intelligence and Software Engineering.

[69]  Yuan-Hai Shao,et al.  Probabilistic outputs for twin support vector machines , 2012, Knowl. Based Syst..

[70]  Divya Tomar,et al.  An Emotion Detection System Based on Multi Least Squares Twin Support Vector Machine , 2014, Adv. Artif. Intell..

[71]  Dayou Liu,et al.  A support vector machine classifier with rough set-based feature selection for breast cancer diagnosis , 2011, Expert Syst. Appl..

[72]  Rui Guo,et al.  A Twin Multi-Class Classification Support Vector Machine , 2012, Cognitive Computation.

[73]  Juan Manuel Górriz,et al.  Early diagnosis of Alzheimer's disease based on partial least squares, principal component analysis and support vector machine using segmented MRI images , 2015, Neurocomputing.

[74]  Tai-Yue Wang,et al.  One-against-one fuzzy support vector machine classifier: An approach to text categorization , 2009, Expert Syst. Appl..

[75]  Yinhui Li,et al.  An efficient intrusion detection system based on support vector machines and gradually feature removal method , 2012, Expert Syst. Appl..

[76]  Ingoo Han,et al.  Hybrid genetic algorithms and support vector machines for bankruptcy prediction , 2006, Expert Syst. Appl..

[77]  Yuan-Hai Shao,et al.  The Best Separating Decision Tree Twin Support Vector Machine for Multi-Class Classification , 2013, ITQM.

[78]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[79]  C. Jack,et al.  Alzheimer's Disease Neuroimaging Initiative , 2008 .

[80]  Jalal A. Nasiri,et al.  Energy-based model of least squares twin Support Vector Machines for human action recognition , 2014, Signal Process..

[81]  Yuan-Hai Shao,et al.  A GA-based model selection for smooth twin parametric-margin support vector machine , 2013, Pattern Recognit..

[82]  Sonali Agarwal,et al.  Predictive Model for Diabetic Patients using Hybrid Twin Support Vector Machine , 2014 .

[83]  Elif Derya íbeyli.  Multiclass support vector machines for diagnosis of erythemato-squamous diseases , 2008 .

[85]  Suresh Chandra,et al.  Reduced twin support vector regression , 2011, Neurocomputing.

[86]  Karim O. Elish,et al.  Predicting defect-prone software modules using support vector machines , 2008, J. Syst. Softw..

[87]  Ning Ye,et al.  Localized Multi-plane TWSVM Classifier via Manifold Regularization , 2010, 2010 Second International Conference on Intelligent Human-Machine Systems and Cybernetics.