A Better Approximation for Balls

Unexpectedly accurate and parsimonious approximations for balls in R^d and related functions are given using half-spaces. Instead of a polytope (an intersection of half-spaces) which would require exponentially many half-spaces (of order (1@e)^d) to have a relative accuracy @e, we use T=c(d^2/@e^2) pairs of indicators of half-spaces and threshold a linear combination of them. In neural network terminology, we are using a single hidden layer perceptron approximation to the indicator of a ball. A special role in the analysis is played by probabilistic methods and approximation of Gaussian functions. The result is then applied to functions that have variation V"f with respect to a class of ellipsoids. Two hidden layer feedforward sigmoidal neural nets are used to approximate such functions. The approximation error is shown to be bounded by a constant times V"f/T^1^/^2"1+V"fd/T^1^/^4"2, where T"1 is the number of nodes in the outer layer and T"2 is the number of nodes in the inner layer of the approximation f"T"""1"," "T"""2.

[1]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[2]  L. Tóth.  Lagerungen in der Ebene auf der Kugel und im Raum , 1953 .

[3]  P. Gruber.  Approximation of convex bodies , 1983 .

[4]  R. Dudley.  Metric Entropy of Some Classes of Sets with Differentiable Boundaries , 1974 .

[5]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[6]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[7]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[8]  D. Pollard.  Empirical Processes: Theory and Applications , 1990 .

[9]  D. Pollard.  Convergence of stochastic processes , 1984 .

[10]  P. Gruber,et al.  Approximation of convex bodies by polytopes , 1982 .

[11]  Rolf Schneider,et al.  Approximation of Convex Bodies by Polytopes , 1981 .

[12]  Halbert White,et al.  Sup-norm approximation bounds for networks through probabilistic methods , 1995, IEEE Trans. Inf. Theory.

[13]  L. Jones.  A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network Training , 1992 .

[14]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[15]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[16]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[17]  Gerald H. L. Cheang.  Neural network approximation and estimation of functions , 1994, Proceedings of 1994 Workshop on Information Theory and Statistics.

[18]  David Haussler,et al.  Decision Theoretic Generalizations of the PAC Model for Neural Net and Other Learning Applications , 1992, Inf. Comput..