An Efficient Feature Weighting Method for Support Vector Regression

Support vector regression (SVR) is a powerful kernel-based method which has been successfully applied in regression problems. Regarding the feature-weighted SVR algorithms, its contribution to model output has been taken into account. However, the performance of the model is subject to the feature weights and the time consumption on training. In the paper, an efficient feature-weighted SVR is proposed. Firstly, the value constraint of each weight is obtained according to the maximal information coefficient which reveals the relationship between each input feature and output. Then, the constrained particle swarm optimization (PSO) algorithm is employed to optimize the feature weights and the hyperparameters simultaneously. Finally, the optimal weights are used to modify the kernel function. Simulation experiments were conducted on four synthetic datasets and seven real datasets by using the proposed model, classical SVR, and some state-of-the-art feature-weighted SVR models. The results show that the proposed method has the superior generalization ability within acceptable time.

[1]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[2]  R. Yu,et al.  Variable-weighted least-squares support vector machine for multivariate spectral analysis. , 2010, Talanta.

[3]  Zhenzhou Lu,et al.  Active learning Bayesian support vector regression model for global approximation , 2021, Inf. Sci..

[4]  Yusuf Yaslan,et al.  Empirical mode decomposition based denoising method with support vector regression for time series prediction: A case study for electricity load forecasting , 2017 .

[5]  Yan Gao,et al.  A support vector machine with maximal information coefficient weighted kernel functions for regression , 2014, The 2014 2nd International Conference on Systems and Informatics (ICSAI 2014).

[6]  E. Hadavandi,et al.  Support vector regression modeling of coal flotation based on variable importance measurements by mutual information method , 2018 .

[7]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[8]  Andrew P. Witkin,et al.  Uniqueness of the Gaussian Kernel for Scale-Space Filtering , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Lili Xie,et al.  A Feature-Weighted SVR Method Based on Kernel Space Feature , 2018, Algorithms.

[10]  Guangzhong Dong,et al.  Remaining Useful Life Prediction and State of Health Diagnosis for Lithium-Ion Batteries Using Particle Filter and Support Vector Regression , 2018, IEEE Transactions on Industrial Electronics.

[11]  Hai-Long Wu,et al.  Adaptive variable-weighted support vector machine as optimized by particle swarm optimization algorithm with application of QSAR studies. , 2011, Talanta.

[12]  Vladimir Vapnik,et al.  Principles of Risk Minimization for Learning Theory , 1991, NIPS.

[13]  Hong-Yan Zou,et al.  Multiplex protein pattern unmixing using a non-linear variable-weighted support vector machine as optimized by a particle swarm optimization algorithm. , 2016, Talanta.

[14]  Yanhe Zhu,et al.  Continuous Estimation of Elbow Joint Angle by Multiple Features of Surface Electromyographic Using Grey Features Weighted Support Vector Machine , 2017 .

[15]  Xin-Ping Guan,et al.  Silicon content prediction and industrial analysis on blast furnace using support vector regression combined with clustering algorithms , 2017, Neural Computing and Applications.

[16]  James Nga-Kwok Liu,et al.  Application of feature-weighted Support Vector regression using grey correlation degree to stock price forecasting , 2012, Neural Computing and Applications.

[17]  Lili Xie,et al.  Decoupling Control of Permanent Magnet Synchronous Motor With Support Vector Regression Inverse System Method , 2020, IEEE Access.

[18]  Jiangjun Ruan,et al.  Real-Time Temperature Estimation of Three-Core Medium-Voltage Cable Joint Based on Support Vector Regression , 2018, Energies.