Support vector regression based on optimal training subset and adaptive particle swarm optimization algorithm

Abstract Support vector regression (SVR) has become very promising and popular in the field of machine learning due to its attractive features and profound empirical performance for small sample, nonlinearity and high dimensional data application. However, most existing support vector regression learning algorithms are limited to the parameters selection and slow learning for large sample. This paper considers an adaptive particle swarm optimization (APSO) algorithm for the parameters selection of support vector regression model. In order to accelerate its training process while keeping high accurate forecasting in each parameters selection step of APSO iteration, an optimal training subset (OTS) method is carried out to choose the representation data points of the full training data set. Furthermore, the optimal parameters setting of SVR and the optimal size of OTS are studied preliminary. Experimental results of an UCI data set and electric load forecasting in New South Wales show that the proposed model is effective and produces better generalization performance.

[1]  Chih-Chou Chiu,et al.  Financial time series forecasting using independent component analysis and support vector regression , 2009, Decis. Support Syst..

[2]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[3]  Yanning Zhang,et al.  A Modified Particle Swarm Optimization Algorithm for Support Vector Machine Training , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[4]  Jiaxin Wang,et al.  Non-flat function estimation with a multi-scale support vector regression , 2006, Neurocomputing.

[5]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[6]  Jianguo Sun,et al.  Robust support vector regression in the primal , 2008, Neural Networks.

[7]  Sancho Salcedo-Sanz,et al.  Short term wind speed prediction based on evolutionary support vector regression algorithms , 2011, Expert Syst. Appl..

[8]  Cheng-Lung Huang,et al.  A distributed PSO-SVM hybrid system with feature selection and parameter optimization , 2008, Appl. Soft Comput..

[9]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[10]  Sancho Salcedo-Sanz,et al.  Improving the training time of support vector regression algorithms through novel hyper-parameters search space reductions , 2009, Neurocomputing.

[11]  Yiqiang Chen,et al.  Building Sparse Multiple-Kernel SVM Classifiers , 2009, IEEE Transactions on Neural Networks.

[12]  Weiguo Gong,et al.  Multi-objective uniform design as a SVM model selection tool for face recognition , 2011, Expert Syst. Appl..

[13]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[14]  Zne-Jung Lee,et al.  Parameter determination of support vector machine and feature selection using simulated annealing approach , 2008, Appl. Soft Comput..

[15]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[16]  Ping-Feng Pai,et al.  Support Vector Machines with Simulated Annealing Algorithms in Electricity Load Forecasting , 2005 .

[17]  Shih-Wei Lin,et al.  Particle swarm optimization for parameter determination and feature selection of support vector machines , 2008, Expert Syst. Appl..

[18]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[19]  Jianzhou Wang,et al.  An adaptive fuzzy combination model based on self-organizing map and support vector regression for electric load forecasting , 2012 .

[20]  Wei-Chiang Hong,et al.  Chaotic particle swarm optimization algorithm in a support vector regression electric load forecasting model , 2009 .

[21]  Mingtian Zhou,et al.  Feature selection and parameter optimization for support vector machines: A new approach based on genetic algorithm with feature chromosomes , 2011, Expert Syst. Appl..

[22]  Wei-Chiang Hong,et al.  Electric load forecasting by seasonal recurrent SVR (support vector regression) with chaotic artific , 2011 .

[23]  D. Rubinfeld,et al.  Hedonic housing prices and the demand for clean air , 1978 .

[24]  Shih-Wei Lin,et al.  Parameter tuning, feature selection and weight assignment of features for case-based reasoning by artificial immune system , 2011, Appl. Soft Comput..

[25]  Zhao Lu,et al.  Non-Mercer hybrid kernel for linear programming support vector regression in nonlinear systems identification , 2009, Appl. Soft Comput..

[26]  Wei Xiong,et al.  Fuzzy relevance vector machine for learning from unbalanced data and noise , 2008, Pattern Recognit. Lett..

[27]  Andrew J. Chipperfield,et al.  Simplifying Particle Swarm Optimization , 2010, Appl. Soft Comput..

[28]  JinXing Che,et al.  Optimal training subset in a support vector regression electric load forecasting model , 2012, Appl. Soft Comput..

[29]  Mehmet Karaköse,et al.  A multi-objective artificial immune algorithm for parameter optimization in support vector machine , 2011, Appl. Soft Comput..

[30]  M. Aizerman,et al.  Theoretical Foundations of the Potential Function Method in Pattern Recognition Learning , 1964 .