Robust Support Vector Regression for Uncertain Input and Output Data

In this paper, a robust support vector regression (RSVR) method with uncertain input and output data is studied. First, the data uncertainties are investigated under a stochastic framework and two linear robust formulations are derived. Linear formulations robust to ellipsoidal uncertainties are also considered from a geometric perspective. Second, kernelized RSVR formulations are established for nonlinear regression problems. Both linear and nonlinear formulations are converted to second-order cone programming problems, which can be solved efficiently by the interior point method. Simulation demonstrates that the proposed method outperforms existing RSVRs in the presence of both input and output data uncertainties.

[1]  Ivor W. Tsang,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 1 Domain Adaptation from Multiple Sources: A Domain- , 2022 .

[2]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[3]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[4]  Alexander J. Smola,et al.  Second Order Cone Programming Approaches for Handling Missing and Uncertain Data , 2006, J. Mach. Learn. Res..

[5]  Gene H. Golub,et al.  Some modified matrix eigenvalue problems , 1973, Milestones in Matrix Computation.

[6]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[7]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[8]  Sanjay Mehrotra,et al.  On the Implementation of a Primal-Dual Interior Point Method , 1992, SIAM J. Optim..

[9]  Chiranjib Bhattacharyya,et al.  Chance constrained uncertain classification via robust optimization , 2011, Math. Program..

[10]  Theodore B. Trafalis,et al.  Robust support vector machines for classification and computational issues , 2007, Optim. Methods Softw..

[11]  Phil Diamond,et al.  Fuzzy least squares , 1988, Inf. Sci..

[12]  Eduardo Bayro-Corrochano,et al.  Clifford Support Vector Machines for Classification, Regression, and Recurrence , 2010, IEEE Transactions on Neural Networks.

[13]  Dug Hun Hong,et al.  Extended fuzzy regression models using regularization method , 2004, Inf. Sci..

[14]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[15]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[16]  Les E. Atlas,et al.  Recurrent neural networks and robust time series prediction , 1994, IEEE Trans. Neural Networks.

[17]  Shun-Feng Su,et al.  Support vector interval regression networks for interval regression analysis , 2003, Fuzzy Sets Syst..

[18]  Dug Hun Hong,et al.  Support vector interval regression machine for crisp input and output data , 2006, Fuzzy Sets Syst..

[19]  Frank Plastria,et al.  Kernel Support Vector Regression with imprecise output , 2008 .

[20]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[21]  Shun-Feng Su,et al.  Robust support vector regression networks for function approximation with outliers , 2002, IEEE Trans. Neural Networks.

[22]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[23]  I. Olkin,et al.  Multivariate Chebyshev Inequalities , 1960 .

[24]  Sheng-De Wang,et al.  Training algorithms for fuzzy support vector machines with noisy data , 2003, 2003 IEEE XIII Workshop on Neural Networks for Signal Processing (IEEE Cat. No.03TH8718).

[25]  Alexander J. Smola,et al.  A Second Order Cone programming Formulation for Classifying Missing Data , 2004, NIPS.

[26]  Theodore B. Trafalis,et al.  Support vector regression with noisy data: a second order cone programming approach , 2007, Int. J. Gen. Syst..

[27]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[28]  Dug Hun Hong,et al.  Support vector fuzzy regression machines , 2003, Fuzzy Sets Syst..

[29]  Jian-Bo Yang,et al.  Feature Selection Using Probabilistic Prediction of Support Vector Regression , 2011, IEEE Transactions on Neural Networks.

[30]  Ramesh C. Jain,et al.  A robust backpropagation learning algorithm for function approximation , 1994, IEEE Trans. Neural Networks.

[31]  Lisa Turner,et al.  Applications of Second Order Cone Programming , 2012 .

[32]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[33]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[34]  Frank Plastria,et al.  Support Vector Regression for imprecise data , 2007 .

[35]  Gene H. Golub,et al.  An analysis of the total least squares problem , 1980, Milestones in Matrix Computation.

[36]  Theodore B. Trafalis,et al.  Robust classification and regression using support vector machines , 2006, Eur. J. Oper. Res..

[37]  José R. Dorronsoro,et al.  Simple Proof of Convergence of the SMO Algorithm for Different SVM Variants , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[38]  José Luis Rojo-Álvarez,et al.  Robust support vector regression for biophysical variable estimation from remotely sensed images , 2006, IEEE Geoscience and Remote Sensing Letters.

[39]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[40]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.