Occam's razor for functions

Occam approximation is an algorithm that takes as . . input a set of samples of a function and a tolerance 6, and produces as output a compact representation of a function that is within c of the given samples. We show that the existence of an Om approximation is sufficient to guarantee the probably approximate learnability of classes of functions on the reals even in the presence of arbkwily large but random additive noise. An important consequence of our results is a general technique for the design and analysis of nonlinear faltering and reconstruction systems in digital signal processing.

[1]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[2]  A. Jazwinski.  Stochastic Processes and Filtering Theory , 1970 .

[3]  D. Pollard.  Convergence of stochastic processes , 1984 .

[4]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[5]  Hiroshi Imai,et al.  An optimal algorithm for approximating a piecewise linear function , 1986 .

[6]  Michael Ben-Or,et al.  A deterministic algorithm for sparse multivariate polynomial interpolation , 1988, STOC '88.

[7]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[8]  Subhash Suri,et al.  On some link distance problems in a simple polygon , 1990, IEEE Trans. Robotics Autom..

[9]  Marek Karpinski,et al.  Fast Parallel Algorithms for Sparse Multivariate Polynomial Interpolation over Finite Fields , 1988, SIAM J. Comput..

[10]  Balas K. Natarajan,et al.  Machine Learning: A Theoretical Approach , 1992 .

[11]  Martin Anthony,et al.  Computational learning theory: an introduction , 1992 .

[12]  Ronitt Rubinfeld,et al.  Reconstructing algebraic functions from mixed data , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[13]  Robert H. Sloan,et al.  Corrigendum to types of noise in data for concept learning , 1988, COLT '92.

[14]  David Haussler,et al.  Decision Theoretic Generalizations of the PAC Model for Neural Net and Other Learning Applications , 1992, Inf. Comput..

[15]  Ming Li,et al.  Learning in the Presence of Malicious Errors , 1993, SIAM J. Comput..

[16]  Balas K. Natarajan.  Filtering random noise via data compression , 1993, [Proceedings] DCC `93: Data Compression Conference.

[17]  Robert E. Schapire,et al.  Efficient distribution-free learning of probabilistic concepts , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.