On-line twin independent support vector machines

The success of SVM in solving pattern recognition problems has encouraged researcher to extend the development of different versions. They are well-known for their robustness and good generalization performance. In many real-world applications, the data to be trained are available on-line in a sequential fashion and because of space and time requirements, batch training methods are not suitable.This paper proposes a new fast on-line algorithm called OTWISVM. It defines two optimization problems and incremental learning is done based of them. Two hyperplanes are generated as decision functions thus each of them is closer to one of the two classes and is as far as possible from the other. The solution is constructed via two subsets of linearly independent samples seen so far, and is always bounded. Good accuracy and notable speed of the method was tested and affirmed both on ordinary and noisy data sets as opposed to similar algorithms. On-line twin independent support vector machine (OTWISVM) is proposed.OTWISWM uses way two nonparallel hyper planes as decision functions.The hyperplanes are described by SVs and are updated when new sample is received.The SVs are constructed as a set of linearly independent samples iteratively.Assessments confirm effectiveness of OTWISVM on conventional and noisy data.

[1]  Jason Weston,et al.  Fast Kernel Classifiers with Online and Active Learning , 2005, J. Mach. Learn. Res..

[2]  S. Sathiya Keerthi,et al.  Building Support Vector Machines with Reduced Classifier Complexity , 2006, J. Mach. Learn. Res..

[3]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[4]  S. Sathiya Keerthi,et al.  A Modified Finite Newton Method for Fast Solution of Large Scale Linear SVMs , 2005, J. Mach. Learn. Res..

[5]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[6]  Tu Bao Ho,et al.  An efficient method for simplifying support vector machines , 2005, ICML.

[7]  Shie Mannor,et al.  The kernel recursive least-squares algorithm , 2004, IEEE Transactions on Signal Processing.

[8]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[10]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[11]  Tom Downs,et al.  Exact Simplification of Support Vector Solutions , 2002, J. Mach. Learn. Res..

[12]  Ichiro Takeuchi,et al.  Multiple Incremental Decremental Learning of Support Vector Machines , 2009, IEEE Transactions on Neural Networks.

[13]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[14]  Shouyang Wang,et al.  Forecasting stock market movement direction with support vector machine , 2005, Comput. Oper. Res..

[15]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[16]  Xindong Wu,et al.  Eliminating Class Noise in Large Datasets , 2003, ICML.

[17]  Klaus-Robert Müller,et al.  Incremental Support Vector Learning: Analysis, Implementation and Applications , 2006, J. Mach. Learn. Res..

[18]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[19]  David E. Goldberg,et al.  Substructrual surrogates for learning decomposable classification problems: implementation and first results , 2007, GECCO '07.

[20]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[21]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[22]  Dimitrios Gunopulos,et al.  Incremental support vector machine construction , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[23]  Dale Schuurmans,et al.  implicit Online Learning with Kernels , 2006, NIPS.

[24]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[25]  Kyung-shik Shin,et al.  An application of support vector machines in bankruptcy prediction model , 2005, Expert Syst. Appl..

[26]  Shigeo Abe.  Support Vector Machines for Pattern Classification , 2010, Advances in Pattern Recognition.

[27]  Cheng Wen,et al.  A New Incremental Support Vector Machine Algorithm , 2012 .

[28]  Bernhard Schölkopf,et al.  Support vector channel selection in BCI , 2004, IEEE Transactions on Biomedical Engineering.

[29]  Alexander J. Smola,et al.  Online learning with kernels , 2001, IEEE Transactions on Signal Processing.

[30]  M. Friedman.  The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[31]  Giulio Sandini,et al.  On-line independent support vector machines , 2010, Pattern Recognit..

[32]  Nicolás García-Pedrajas,et al.  Nonlinear Boosting Projections for Ensemble Construction , 2007, J. Mach. Learn. Res..

[33]  Stefan Rüping,et al.  Incremental Learning with Support Vector Machines , 2001, ICDM.

[34]  Jason Weston,et al.  Online (and Offline) on an Even Tighter Budget , 2005, AISTATS.

[35]  F. Diebold,et al.  Comparing Predictive Accuracy , 1994, Business Cycles.

[36]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.