Rates of approximation of real-valued boolean functions by neural networks

We give upper bounds on rates of approximation of real-valued functions of d Boolean variables by one-hidden-layer perceptron networks. Our bounds are of the form c/n where c depends on certain norms of the function being approximated and n is the number of hidden units. We describe sets of functions where these norms grow either polynomially or exponentially with d.

[1]  Jehoshua Bruck,et al.  Harmonic Analysis of Polynomial Threshold Functions , 1990, SIAM J. Discret. Math..

[2]  Vladik Kreinovich,et al.  Estimates of the Number of Hidden Units and Variation with Respect to Half-Spaces , 1997, Neural Networks.

[3]  Charles A. Micchelli,et al.  Dimension-independent bounds on the degree of approximation by neural networks , 1994, IBM J. Res. Dev..

[4]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.

[5]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[6]  Jehoshua Bruck,et al.  Polynomial threshold functions, AC functions and spectrum norms , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[7]  Věra Kůrková,et al.  Dimension-Independent Rates of Approximation by Neural Networks , 1997 .

[8]  L. Jones.  A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network Training , 1992 .

[9]  Petr Savický.  On the Bent Boolean Functions That are Symmetric , 1994, Eur. J. Comb..

[10]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce , 1987 .

[11]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[12]  Jehoshua Bruck,et al.  On the Power of Threshold Circuits with Small Weights , 1991, SIAM J. Discret. Math..

[13]  Eduardo D. Sontag,et al.  Rate of approximation results motivated by robust neural network learning , 1993, COLT '93.

[14]  Pavel Pudlák,et al.  Threshold circuits of bounded depth , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[15]  H. Weaver.  Applications of Discrete and Continuous Fourier Analysis , 1983 .

[16]  Jooyoung Park,et al.  Approximation and Radial-Basis-Function Networks , 1993, Neural Computation.

[17]  Eyal Kushilevitz,et al.  Learning decision trees using the Fourier spectrum , 1991, STOC '91.

[18]  C. Micchelli,et al.  Approximation by superposition of sigmoidal and radial basis functions , 1992 .

[19]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.