Regression and Classification with Regularization

The purpose of this chapter is to present a theoretical framework for the problem of learning from examples. Learning from examples can be regarded [13] as the problem of approximating a multivariate function from sparse data2. The function can be real valued as in regression or binary valued as in classification. The problem of approximating a function from sparse data is ill-posed and a classical solution is regularization theory [19].

[1]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[2]  A. Kolmogorov,et al.  Entropy and "-capacity of sets in func-tional spaces , 1961 .

[3]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[4]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[5]  D. Pollard.  Convergence of stochastic processes , 1984 .

[6]  M. Bertero,et al.  Ill-posed problems in early vision , 1988, Proc. IEEE.

[7]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[8]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[9]  Tomaso A. Poggio,et al.  Extensions of a Theory of Networks for Approximation and Learning , 1990, NIPS.

[10]  Tarun Khanna,et al.  Foundations of neural networks , 1990 .

[11]  R. Dudley,et al.  Uniform and universal Glivenko-Cantelli classes , 1991 .

[12]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[13]  H. Triebel,et al.  Function Spaces, Entropy Numbers, Differential Operators: Function Spaces , 1996 .

[14]  Noga Alon,et al.  Scale-sensitive dimensions, uniform convergence, and learnability , 1997, JACM.

[15]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[16]  André Elisseeff,et al.  Algorithmic Stability and Generalization Performance , 2000, NIPS.

[17]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[18]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[19]  Ji Zhu,et al.  Kernel Logistic Regression and the Import Vector Machine , 2001, NIPS.

[20]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[21]  André Elisseeff,et al.  Stability and Generalization , 2002, J. Mach. Learn. Res..