Rate of Convergence in Density Estimation Using Neural Networks

Given N i.i.d. observations XiNi=1 taking values in a compact subset of Rd, such that p denotes their common probability density function, we estimate p from an exponential family of densities based on single hidden layer sigmoidal networks using a certain minimum complexity density estimation scheme. Assuming that p possesses a certain exponential representation, we establish a rate of convergence, independent of the dimension d, for the expected Hellinger distance between the proposed minimum complexity density estimator and the true underlying density p.

[1]  B. Silverman,et al.  On the Estimation of a Probability Density Function by the Maximum Penalized Likelihood Method , 1982 .

[2]  E. Masry,et al.  Minimum complexity regression estimation with weakly dependent observations , 1996, Proceedings of 1994 Workshop on Information Theory and Statistics.

[3]  Andrew R. Barron,et al.  Minimum complexity density estimation , 1991, IEEE Trans. Inf. Theory.

[4]  R. Tapia,et al.  Nonparametric Probability Density Estimation , 1978 .

[5]  D. W. Scott,et al.  Nonparametric Estimation of Probability Densities and Regression Curves , 1988 .

[6]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[7]  Halbert White,et al.  Parametric Statistical Estimation with Artificial Neural Networks: A Condensed Discussion , 1994 .

[8]  A. Barron,et al.  APPROXIMATION OF DENSITY FUNCTIONS BY SEQUENCES OF EXPONENTIAL FAMILIES , 1991 .

[9]  C. J. Stone,et al.  Asymptotics for Doubly Flexible Logspline Response Models , 1991 .

[10]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[11]  Jorma Rissanen,et al.  Density estimation by stochastic complexity , 1992, IEEE Trans. Inf. Theory.

[12]  J. Friedman,et al.  PROJECTION PURSUIT DENSITY ESTIMATION , 1984 .

[13]  Yeshaiahu Fainman,et al.  A learning law for density estimation , 1994, IEEE Trans. Neural Networks.

[14]  C. J. Stone,et al.  Large-Sample Inference for Log-Spline Models , 1990 .

[15]  C. J. Stone,et al.  The Use of Polynomial Splines and Their Tensor Products in Multivariate Function Estimation , 1994 .

[16]  Luc Devroye,et al.  Nonparametric Density Estimation , 1985 .

[17]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[18]  L. Devroye,et al.  Nonparametric density estimation : the L[1] view , 1987 .

[19]  L. Jones.  A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network Training , 1992 .

[20]  A. Barron.  Approximation and Estimation Bounds for Artificial Neural Networks , 1991, COLT '91.

[21]  Robin Sibson,et al.  What is projection pursuit , 1987 .