Nonparametric Regression via StatLSSVM

We present a new MATLAB toolbox under Windows and Linux for nonparametric regression estimation based on the statistical library for least squares support vector machines (StatLSSVM). The StatLSSVM toolbox is written so that only a few lines of code are necessary in order to perform standard nonparametric regression, regression with correlated errors and robust regression. In addition, construction of additive models and pointwise or uniform confidence intervals are also supported. A number of tuning criteria such as classical cross-validation, robust cross-validation and cross-validation for correlated errors are available. Also, minimization of the previous criteria is available without any user interaction.

[1]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[2]  H. Akaike.  Statistical predictor identification , 1970 .

[3]  N. L. Johnson,et al.  Discrete Multivariate Distributions , 1998 .

[4]  M. C. Jones,et al.  Generalized jackknifing and higher order kernels , 1993 .

[5]  Lipo Wang.  Support vector machines : theory and applications , 2005 .

[6]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machines , 2002 .

[7]  Igor Vajda,et al.  Generalized piecewise linear histograms , 2002 .

[8]  李幼升,et al.  Ph , 1989 .

[9]  P. Burman.  A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods , 1989 .

[10]  Johan A. K. Suykens,et al.  Approximate Confidence and Prediction Intervals for Least Squares Support Vector Regression , 2011, IEEE Transactions on Neural Networks.

[11]  László Györfi,et al.  On piecewise linear density estimators , 1999 .

[12]  Matt P. Wand,et al.  Non-Standard Semiparametric Regression via BRugs , 2010 .

[13]  P. Vieu,et al.  Nonparametric Functional Data Analysis: Theory and Practice (Springer Series in Statistics) , 2006 .

[14]  A. Meister.  Deconvolution Problems in Nonparametric Statistics , 2009 .

[15]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[16]  D. Ruppert,et al.  Measurement Error in Nonlinear Models , 1995 .

[17]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[18]  J. Simonoff.  Smoothing Methods in Statistics , 1998 .

[19]  F. J. Anscombe,et al.  THE TRANSFORMATION OF POISSON, BINOMIAL AND NEGATIVE-BINOMIAL DATA , 1948 .

[20]  C. Loader,et al.  Simultaneous Confidence Bands for Linear Regression and Smoothing , 1994 .

[21]  Ingo Steinwart,et al.  Consistency and robustness of kernel-based regression in convex risk minimization , 2007, 0709.0626.

[22]  P. Hall.  On Bootstrap Confidence Intervals in Nonparametric Regression , 1992 .

[23]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[24]  Ingo Steinwart,et al.  Estimating conditional quantiles with the help of the pinball loss , 2011, 1102.2101.

[25]  Johan A. K. Suykens,et al.  Robustness of reweighted Least Squares Kernel Based Regression , 2010, J. Multivar. Anal..

[26]  Johan A. K. Suykens,et al.  Optimized fixed-size kernel models for large data sets , 2010, Comput. Stat. Data Anal..

[27]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[28]  A. Bowman.  An alternative method of cross-validation for the smoothing of density estimates , 1984 .

[29]  R. Fildes.  Journal of the Royal Statistical Society (B): Gary K. Grunwald, Adrian E. Raftery and Peter Guttorp, 1993, “Time series of continuous proportions”, 55, 103–116.☆ , 1993 .

[30]  V. Yohai,et al.  Robust Statistics: Theory and Methods , 2006 .

[31]  Jianqing Fan,et al.  Local polynomial modelling and its applications , 1994 .

[32]  S. Lahiri,et al.  On bandwidth choice in nonparametric regression with both short- and long-range dependent errors , 1995 .

[33]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[34]  Johan A. K. Suykens,et al.  Kernel Regression in the Presence of Correlated Errors , 2011, J. Mach. Learn. Res..

[35]  Jianqing Fan.  Test of Significance Based on Wavelet Thresholding and Neyman's Truncation , 1996 .

[36]  Johan A. K. Suykens,et al.  Robustness of Kernel Based Regression: A Comparison of Iterative Weighting Schemes , 2009, ICANN.

[37]  T. Kneib,et al.  BayesX: Analyzing Bayesian Structural Additive Regression Models , 2005 .

[38]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[39]  Jing Hu,et al.  Model Selection via Bilevel Optimization , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[40]  Jana Jurečková,et al.  Robust Statistical Methods with R , 2005 .

[41]  Johan A. K. Suykens,et al.  Weighted least squares support vector machines: robustness and sparse approximation , 2002, Neurocomputing.

[42]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[43]  Yuhong Yang,et al.  Nonparametric Regression with Correlated Errors , 2001 .

[44]  James Stephen Marron,et al.  Local minima in cross validation functions , 1991 .

[45]  E. Sochett,et al.  Factors affecting and patterns of residual insulin secretion during the first year of Type 1 (insulin-dependent) diabetes mellitus in children , 1987, Diabetologia.

[46]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[47]  R. Courant,et al.  Methods of Mathematical Physics , 1962 .

[48]  D. H. Leung,et al.  Cross-validation in nonparametric regression with outliers , 2005 .

[49]  Jeffrey C. Lagarias,et al.  Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions , 1998, SIAM J. Optim..

[50]  Hans Edner,et al.  Locally weighted least squares kernel regression and statistical evaluation of LIDAR measurements , 1996 .

[51]  Kurt Hornik,et al.  kernlab - An S4 Package for Kernel Methods in R , 2004 .

[52]  Johan A. K. Suykens,et al.  On Robustness in Kernel Based Regression , 2010, NIPS 2010.

[53]  Werner A. Stahel,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[54]  Johan A. K. Suykens,et al.  Componentwise Least Squares Support Vector Machines , 2005, ArXiv.

[55]  P. J. Huber.  Robust Estimation of a Location Parameter , 1964 .

[56]  Johan A. K. Suykens,et al.  Coupled Simulated Annealing , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[57]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[58]  Ingo Steinwart,et al.  Consistency and robustness of kernel based regression , 2005 .

[59]  Igor Vajda,et al.  Nonnegative piecewise linear histograms , 2001 .

[60]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[61]  Jing Hu,et al.  Bilevel Model Selection for Support Vector Machines , 2007 .

[62]  J. Marron,et al.  Comparison of Two Bandwidth Selectors with Dependent Errors , 1991 .

[63]  J. Hart.  Kernel regression estimation with time series errors , 1991 .

[64]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[65]  M. Rudemo.  Empirical Choice of Histograms and Kernel Density Estimators , 1982 .

[66]  R. Tibshirani,et al.  Generalized Additive Models , 1991 .

[67]  Andreas Brezger,et al.  Generalized structured additive regression based on Bayesian P-splines , 2006, Comput. Stat. Data Anal..