Kernel Ridge Regression

This chapter discusses the method of Kernel Ridge Regression, which is a very simple special case of Support Vector Regression. The main formula of the method is identical to a formula in Bayesian statistics, but Kernel Ridge Regression has performance guarantees that have nothing to do with Bayesian assumptions. I will discuss two kinds of such performance guarantees: those not requiring any assumptions whatsoever, and those depending on the assumption of randomness.

[1]  Sham M. Kakade,et al.  Online Bounds for Bayesian Algorithms , 2004, NIPS.

[2]  Vladimir Vovk,et al.  A tutorial on conformal prediction , 2007, J. Mach. Learn. Res..

[3]  Kei Takeuchi,et al.  MATHEMATICAL ENGINEERING TECHNICAL REPORTS Sequential Optimizing Strategy in Multi-dimensional Bounded Forecasting Games , 2009, 0911.3933.

[4]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[5]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[6]  Manfred K. Warmuth,et al.  Relative Loss Bounds for On-Line Density Estimation with the Exponential Family of Distributions , 1999, Machine Learning.

[7]  Ingo Steinwart,et al.  On the Influence of the Kernel on the Consistency of Support Vector Machines , 2002, J. Mach. Learn. Res..

[8]  L. Wasserman.  Frequentist Bayes is objective (comment on articles by Berger and by Goldstein) , 2006 .

[9]  David A. Freedman,et al.  Rejoinder: On the Consistency of Bayes Estimates , 1986 .

[10]  Larry Wasserman.  Frasian Inference , 2012 .

[11]  Alexander Gammerman,et al.  Ridge Regression Learning Algorithm in Dual Variables , 1998, ICML.

[12]  Larry A. Wasserman,et al.  Distribution Free Prediction Bands , 2012, ArXiv.

[13]  Vladimir Vovk,et al.  Ridge Regression Confidence Machine , 2001, International Conference on Machine Learning.

[14]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[15]  N. Cressie.  The origins of kriging , 1990 .

[16]  Alexander Gammerman,et al.  Learning by Transduction , 1998, UAI.

[17]  V. Vovk.  Competitive On‐line Statistics , 2001 .

[18]  Vladimir Vovk,et al.  Competing with Gaussian linear experts , 2009, ArXiv.

[19]  Yuri Kalnishkan,et al.  An identity for kernel ridge regression , 2010, Theor. Comput. Sci..

[20]  G. Shafer,et al.  Algorithmic Learning in a Random World , 2005 .