Support Vector Regression

Instead of minimizing the observed training error, Support Vector Regression (SVR) attempts to minimize the generalization error bound so as to achieve generalized performance. The idea of SVR is based on the computation of a linear regression function in a high dimensional feature space where the input data are mapped via a nonlinear function. SVR has been applied in various fields - time series and financial (noisy and risky) prediction, approximation of complex engineering analyses, convex quadratic programming and choices of loss functions, etc. In this paper, an attempt has been made to review the existing theory, methods, recent developments and scopes of SVR.

[1]  V. Vapnik.  Pattern recognition using generalized portrait method , 1963 .

[2]  V. Vapnik,et al.  A note one class of perceptrons , 1964 .

[3]  Shun-ichi Amari,et al.  A Theory of Pattern Recognition , 1968 .

[4]  Carl E. Rasmussen,et al.  In Advances in Neural Information Processing Systems , 2011 .

[5]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[6]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[7]  Christopher K. I. Williams.  Regression with Gaussian processes , 1997 .

[8]  Paul W. Goldberg,et al.  Regression with Input-dependent Noise: A Gaussian Process Treatment , 1997, NIPS.

[9]  Christopher K. I. Williams,et al.  An upper bound on the Bayesian error bars for generalized linear regression , 1997 .

[10]  Bernhard Schölkopf,et al.  On a Kernel-Based Method for Pattern Recognition, Regression, Approximation, and Operator Inversion , 1998, Algorithmica.

[11]  Christopher K. I. Williams,et al.  Discovering Hidden Features with Gaussian Processes Regression , 1998, NIPS.

[12]  R. C. Williamson,et al.  Support vector regression with automatic accuracy control. , 1998 .

[13]  Christopher K. I. Williams.  Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond , 1999, Learning in Graphical Models.

[14]  B. Schölkopf,et al.  General cost functions for support vector regression. , 1998 .

[15]  J. C. BurgesChristopher.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998 .

[16]  Bernhard Schölkopf,et al.  Shrinking the Tube: A New Support Vector Regression Algorithm , 1998, NIPS.

[17]  Jyrki Taskinen,et al.  Aqueous Solubility Prediction of Drugs Based on Molecular Topology and Neural Network Modeling , 1998, J. Chem. Inf. Comput. Sci..

[18]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[19]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[20]  Bernhard Schölkopf,et al.  Choosing /spl nu/ in support vector regression with different noise models-theory and experiments , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[21]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[22]  Jarmo Huuskonen,et al.  Estimation of Aqueous Solubility for a Diverse Set of Organic Compounds Based on Molecular Topology , 2000, J. Chem. Inf. Comput. Sci..

[23]  G. Wahba.  An introduction to model building with repro-ducing kernel hilbert spaces , 2000 .

[24]  Chris J. Harris,et al.  Regression models for classification to enhance interpretability , 2001 .

[25]  Samy Bengio,et al.  SVMTorch: Support Vector Machines for Large-Scale Regression Problems , 2001, J. Mach. Learn. Res..

[26]  Bernhard Schölkopf,et al.  Statistical Learning and Kernel Methods , 2001, Data Fusion and Perception.

[27]  James T. Kwok,et al.  Bayesian Support Vector Regression , 2001, AISTATS.

[28]  Deepak K. Agarwal,et al.  Shrinkage estimator generalizations of Proximal Support Vector Machines , 2002, KDD.

[29]  Jinbo Bi,et al.  Prediction of Protein Retention Times in Anion-Exchange Chromatography Systems Using Support Vector Regression , 2002, J. Chem. Inf. Comput. Sci..

[30]  Chung-Hui Kuo,et al.  Gloss Patch Selection Based on Support Vector Regression , 2002, PICS.

[31]  Osamu Watanabe,et al.  Provably Fast Support Vector Regression Using Random Sampling ∗ , 2002 .

[32]  M. Khalid,et al.  Machine Learning Using Support Vector Machines , 2002 .

[33]  Hsuan-Tien Lin,et al.  A Note on the Decomposition Methods for Support Vector Regression , 2001, Neural Computation.

[34]  Chih-Jen Lin,et al.  Training v-Support Vector Regression: Theory and Algorithms , 2002, Neural Computation.

[35]  Aníbal R. Figueiras-Vidal,et al.  Cyclosporine concentration prediction using clustering and support vector regression methods , 2002 .

[36]  John Reinitz,et al.  Support vector regression applied to the determination of the developmental age of a Drosophila embryo from its segmentation gene expression patterns , 2002, ISMB.

[37]  Gavin C. Cawley,et al.  Heteroscedastic regularised kernel regression for prediction of episodes of poor air quality , 2002, The European Symposium on Artificial Neural Networks.

[38]  Lai-Wan Chan,et al.  Support Vector Machine Regression for Volatile Stock Market Prediction , 2002, IDEAL.

[39]  Shie Mannor,et al.  Sparse Online Greedy Support Vector Regression , 2002, ECML.

[40]  Junbin Gao,et al.  SVM regression through variational methods and its sequential implementation , 2003, Neurocomputing.

[41]  Antonio J. Serrano,et al.  Dosage individualization of erythropoietin using a profile-dependent support vector regression , 2003, IEEE Transactions on Biomedical Engineering.

[42]  Timothy W. Simpson,et al.  Analysis of support vector regression for approximation of complex engineering analyses , 2003, DAC 2003.

[43]  Robert T. Schultz,et al.  Nonlinear Estimation and Modeling of fMRI Data Using Spatio-temporal Support Vector Regression , 2003, IPMI.

[44]  S. Keerthi,et al.  SMO Algorithm for Least-Squares SVM Formulations , 2003, Neural Computation.

[45]  S. Sathiya Keerthi,et al.  A simple and efficient algorithm for gene selection using sparse logistic regression , 2003, Bioinform..

[46]  Sinan Gezici,et al.  A New Approach to Mobile Position Tracking , 2003 .

[47]  J.T. Kwok,et al.  Linear Dependency betweenand the Input Noise in -Support Vector Regression , 2001 .

[48]  Yunqian Ma,et al.  Comparison of Model Selection for Regression , 2003, Neural Computation.

[49]  James Theiler,et al.  Accurate On-line Support Vector Regression , 2003, Neural Computation.

[50]  Jinbo Bi,et al.  Dimensionality Reduction via Sparse Support Vector Machines , 2003, J. Mach. Learn. Res..

[51]  Junbin Gao,et al.  Mean field method for the support vector machine regression , 2003, Neurocomputing.

[52]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[53]  D. T. Lee,et al.  Travel-time prediction with support vector regression , 2004, IEEE Transactions on Intelligent Transportation Systems.

[54]  Wenjian Wang,et al.  A heuristic training for support vector regression , 2004, Neurocomputing.

[55]  Wei Chu,et al.  Bayesian support vector regression using a unified loss function , 2004, IEEE Transactions on Neural Networks.

[56]  Andreas Zell,et al.  Towards Optimal Descriptor Subset Selection with Support Vector Machines in Classification and Regression , 2004 .

[57]  Gavin C. Cawley,et al.  Heteroscedastic kernel ridge regression , 2004, Neurocomputing.

[58]  Chih-Jen Lin,et al.  Simple Probabilistic Predictions for Support Vector Regression , 2004 .

[59]  Andrew H. Sung,et al.  Intrusion Detection Systems Using Adaptive Regression Splines , 2004, ICEIS.

[60]  Dug,et al.  2004 Proceedings of the Autumn Conference , Korean Statistical Society-67-Support Vector Machine for Interval Regression , 2004 .

[61]  Andrew H. Sung,et al.  Intrusion Detection Systems Using Adaptive Regression Splines , 2004, ICEIS.

[62]  Jun Wang,et al.  A support vector machine with a hybrid kernel and minimal Vapnik-Chervonenkis dimension , 2004, IEEE Transactions on Knowledge and Data Engineering.

[63]  Antonio J. Serrano,et al.  Profiled support vector machines for antisense oligonucleotide efficacy prediction , 2004, BMC Bioinformatics.

[64]  David R. Musicant,et al.  Active set support vector regression , 2004, IEEE Transactions on Neural Networks.

[65]  Lai-Wan Chan,et al.  Outliers Treatment in Support Vector Regression for Financial Time Series Prediction , 2004, ICONIP.

[66]  César Caballero-Gaudes,et al.  Robust blind identification of SIMO channels: a support vector regression approach , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[67]  Gavin C. Cawley,et al.  Reduced Rank Kernel Ridge Regression , 2002, Neural Processing Letters.

[68]  I. Sprinkhuizen-Kuyper,et al.  Equity style timing using support vector regressions , 2004 .

[69]  Sheng Chen,et al.  An approach for constructing parsimonious generalized Gaussian kernel regression models , 2004, Neurocomputing.

[70]  Nipon Theera-Umpon,et al.  Leeway Prediction of Oceanic Disastrous Target via Support Vector Regression , 2004, Journal of Advanced Computational Intelligence and Intelligent Informatics.

[71]  Yunqian Ma,et al.  Practical selection of SVM parameters and noise estimation for SVM regression , 2004, Neural Networks.

[72]  Christopher K. I. Williams,et al.  Using the Equivalent Kernel to Understand Gaussian Process Regression , 2004, NIPS.

[73]  Johan A. K. Suykens,et al.  Componentwise Least Squares Support Vector Machines , 2005, ArXiv.

[74]  A. Micheli,et al.  A preliminary empirical comparison of recursive neural networks and tree kernel methods on regression tasks for tree structured domains , 2005, Neurocomputing.

[75]  Wei Chu,et al.  New approaches to support vector ordinal regression , 2005, ICML.

[76]  Ivor W. Tsang,et al.  Core Vector Regression for very large regression problems , 2005, ICML.

[77]  Masayuki Kobayashi,et al.  Position Control of Ultrasonic Motor using Support Vector Regression , 2005, International Conference on Climate Informatics.

[78]  A. Zell,et al.  Efficient parameter selection for support vector machines in classification and regression via model-based global optimization , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[79]  Stephan Didas,et al.  Relations Between Higher Order TV Regularization and Support Vector Regression , 2005, Scale-Space.

[80]  Ingo Steinwart,et al.  Consistency and robustness of kernel based regression , 2005 .

[81]  Renato Campanini,et al.  Support vector regression filtering for reduction of false positives in a mass detection cad scheme: preliminary results , 2005 .

[82]  Ming-Wei Chang,et al.  Leave-One-Out Bounds for Support Vector Regression Model Selection , 2005, Neural Computation.

[83]  I. Sprinkhuizen-Kuyper,et al.  Equity style timing using support vector regressions , 2004 .

[84]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .

[85]  박찬규.  Support Vector Regression을 이용한 소프트웨어 개발비 예측 , 2006 .

[86]  Johan A. K. Suykens,et al.  LS-SVMlab : a MATLAB / C toolbox for Least Squares Support Vector Machines , 2007 .

[87]  S. Sathiya Keerthi,et al.  A Fast Dual Algorithm for Kernel Logistic Regression , 2002, 2007 International Joint Conference on Neural Networks.

[88]  Iain Murray,et al.  Introduction to Gaussian Processes , 2008 .

[89]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.