Support vector regression based on data shifting

In this article, we provide some preliminary theoretical analysis and extended practical experiments of a novel regression method proposed recently which is based on representing regression problems as classification ones with duplicated and shifted data. The main results regard partial equivalency of Bayes solutions for regression problems and the transformed classification ones, and improved Vapnik-Chervonenkis bounds for the proposed method compared to Support Vector Machines. We conducted experiments comparing the proposed method with @e-insensitive Support Vector Regression (@e-SVR) on various synthetic and real world data sets. The results indicate that the new method can achieve comparable generalization performance as @e-SVR with significantly improved the number of support vectors.

[1]  Ibrahim A. Albidewi,et al.  Novel ensemble methods for regression via classification problems , 2012, Expert Syst. Appl..

[2]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[3]  Marcin Orchel,et al.  Regression Based on Support Vector Classification , 2011, ICANNGA.

[4]  Vojislav Kecman,et al.  Adaptive Local Hyperplane for regression tasks , 2009, 2009 International Joint Conference on Neural Networks.

[5]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[6]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[7]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[8]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[9]  Marcin Orchel,et al.  Incorporating Detractors into SVM Classification , 2009, ICMMI.

[10]  Amir Ahmad,et al.  A Novel Ensemble Method for Regression via Classification Problems , 2011 .

[11]  Marcin Orchel.  Support Vector Regression as a Classification Problem with a Priori Knowledge in the Form of Detractors , 2011, ICMMI.

[12]  Chengqi Zhang,et al.  Testing Adaptive Local Hyperplane for multi-class classification by double cross-validation , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[13]  Bernhard Schölkopf,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[14]  Sholom M. Weiss,et al.  Solving regression problems with rule-based ensemble classifiers , 2001, KDD '01.

[15]  Hong-Bing Liu,et al.  An Improved Support Vector Regression Based on Classification , 2007, 2007 International Conference on Multimedia and Ubiquitous Engineering (MUE'07).

[16]  Fuming Lin,et al.  A novel support vector machine algorithm for solving nonlinear regression problems based on symmetrical points , 2010, 2010 2nd International Conference on Computer Engineering and Technology.