Learning a function from noisy samples at a finite sparse set of points

In learning theory the goal is to reconstruct a function defined on some (typically high dimensional) domain @W, when only noisy values of this function at a sparse, discrete subset @[email protected][email protected] are available. In this work we use Koksma-Hlawka type estimates to obtain deterministic bounds on the so-called generalization error. The resulting estimates show that the generalization error tends to zero when the noise in the measurements tends to zero and the number of sampling points tends to infinity sufficiently fast.

[1]  Vladimir N. Temlyakov,et al.  Nonlinear Methods of Approximation , 2003, Found. Comput. Math..

[2]  S. Smale,et al.  Shannon sampling and function reconstruction from point values , 2004 .

[3]  Irwin W. Sandberg,et al.  A note on error bounds for approximation in inner product spaces , 1996 .

[4]  J. Lions,et al.  Non-homogeneous boundary value problems and applications , 1972 .

[5]  C. Vogel,et al.  Asymptotic theory of filtering for linear operator equations with discrete noisy data , 1987 .

[6]  H. W. ENGLz.  Regularized Data-Driven Construction of Fuzzy Controllers , 2002 .

[7]  T. Hohage,et al.  Consistency and rates of convergence of nonlinear Tikhonov regularization with random noise , 2004 .

[8]  D. Hinton,et al.  Weighted Interpolation Inequalities of Sum and Product form in Rn , 1988 .

[9]  Camillo Melzi.  Computing potentials on a periodic bidimensional grid , 1996 .

[10]  M. A. Lukas.  Comparisons of parameter choice methods for regularization with discrete noisy data , 1998 .

[11]  Harald Niederreiter,et al.  Random number generation and Quasi-Monte Carlo methods , 1992, CBMS-NSF regional conference series in applied mathematics.

[12]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[13]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[14]  Ronald A. DeVore,et al.  Approximation Methods for Supervised Learning , 2006, Found. Comput. Math..

[15]  G. Wahba,et al.  Convergence rates of approximate least squares solutions of linear integral and operator equations of the first kind , 1974 .

[16]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[17]  Federico Girosi,et al.  Generalization bounds for function approximation from scattered noisy data , 1999, Adv. Comput. Math..

[18]  Henryk Wozniakowski,et al.  When Are Quasi-Monte Carlo Algorithms Efficient for High Dimensional Integrals? , 1998, J. Complex..

[19]  V. N. Temli︠a︡kov.  Approximation of periodic functions , 1993 .

[20]  V. Temlyakov.  Approximation in Learning Theory , 2008 .

[21]  Martin Burger,et al.  Error Bounds for Approximation with Neural Networks , 2001, J. Approx. Theory.

[22]  L. Jones.  A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network Training , 1992 .

[23]  S. Chua.  Weighted Sobolev interpolation inequalities on product spaces , 1999 .

[24]  Andreas Hofinger.  Nonlinear function approximation: Computing smooth solutions with an adaptive greedy algorithm , 2006, J. Approx. Theory.

[25]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[26]  S. C. Zaremba.  Some applications of multidimensional integration by parts , 1968 .

[27]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[28]  C. Darken,et al.  Constructive Approximation Rates of Convex Approximation in Non-hilbert Spaces , 2022 .

[29]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[30]  Edmund Hlawka.  Über die Diskrepanz mehrdimensionaler Folgen mod. 1 , 1961 .

[31]  Martin Burger,et al.  Regularized Greedy Algorithms for Network Training with Data Noise , 2005, Computing.

[32]  Martin Burger,et al.  Regularized data-driven construction of fuzzy controllers , 2002 .

[33]  Fred J. Hickernell,et al.  On tractability of weighted integration over bounded and unbounded regions in Reals , 2004, Math. Comput..

[34]  Friedrich Pillichshammer,et al.  Bounds for the weighted Lp discrepancy and tractability of integration , 2003, J. Complex..