About the Asymptotic Accuracy of Barron Density Estimates

By extending the information-theoretic arguments of previous papers dealing with the Barron-type density estimates, and their consistency in information divergence and chi-square divergence, the problem of consistency in Csiszar's /spl phi/-divergence is motivated for general convex functions /spl phi/. The problem of consistency in /spl phi/-divergence is solved for all /spl phi/ with /spl phi/(0)</spl infin/ and /spl phi/(t)=O(t ln t) when t/spl rarr//spl infin/. The problem of consistency in the expected /spl phi/-divergence is solved for all /spl phi/ with t/spl phi/(1/t)+/spl phi/(t)=O(t/sup 2/) when t/spl rarr//spl infin/. Various stronger versions of these asymptotic restrictions are considered too. Assumptions about the model needed for the consistency are shown to depend on how strong these restrictions are.

[1]  J. Simonoff.  Multivariate Density Estimation , 1996 .

[2]  Martin de Prycker,et al.  Asynchronous Transfer Mode, Solution for Broadband Isdn , 1991 .

[3]  Imre Csiszár.  Generalized cutoff rates and Renyi's information measures , 1995, IEEE Trans. Inf. Theory.

[4]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[5]  Igor Vajda,et al.  About distances of discrete distributions satisfying the data processing theorem of information theory , 1997, IEEE Trans. Inf. Theory.

[6]  L. I. Galtchouk,et al.  Extremal problems in sequential testing homogeneity , 1995 .

[7]  Joseph Yu Hui,et al.  Switching and Traffic Theory for Integrated Broadband Networks , 1990 .

[8]  I. Vajda,et al.  Convex Statistical Distances , 2018, Statistical Inference for Engineers and Data Scientists.

[9]  László Györfi,et al.  Asymptotic Normality of L 1-Error in Density Estimation , 1995 .

[10]  L. Györfi,et al.  Minimum kolmogorov distance estimates of parameters and parametrized distributions , 1996 .

[11]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[12]  Andrew R. Barron,et al.  Information-theoretic asymptotics of Bayes methods , 1990, IEEE Trans. Inf. Theory.

[13]  László Györfi,et al.  Distribution estimation consistent in total variation and in two types of information divergence , 1992, IEEE Trans. Inf. Theory.

[14]  Saab Abou-Jaoudé.  Conditions nécessaires et suffisantes de convergence L1 en probabilité de l'histogramme pour une densité , 1976 .

[15]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[16]  L. Györfi,et al.  Distribution Estimates Consistent in χ2-Divergence , 1998 .

[17]  Friedrich Pukelsheim,et al.  INSTRUCTIONAL CONFERENCE ON THE THEORY OF STOCHASTIC PROCESSES: Necessary and sufficient conditions for contiguity and entire asymptotic separation of probability measures , 1982 .

[18]  Andrew R. Barron,et al.  A bound on the financial value of information , 1988, IEEE Trans. Inf. Theory.

[19]  Ferdinand Österreicher,et al.  Statistical information and discrimination , 1993, IEEE Trans. Inf. Theory.

[20]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[21]  Contiguity of Probability Measures , 1982 .

[22]  K. Matusita.  Distance and decision rules , 1964 .

[23]  I. Vajda.  On thef-divergence and singularity of probability measures , 1972 .

[24]  A. Barron,et al.  APPROXIMATION OF DENSITY FUNCTIONS BY SEQUENCES OF EXPONENTIAL FAMILIES , 1991 .

[25]  L. Devroye,et al.  Nonparametric density estimation : the L[1] view , 1987 .

[26]  Jianhua Lin,et al.  Divergence measures based on the Shannon entropy , 1991, IEEE Trans. Inf. Theory.

[27]  I. Vajda.  Theory of statistical inference and information , 1989 .

[28]  Lee D. Davisson,et al.  Universal noiseless coding , 1973, IEEE Trans. Inf. Theory.

[29]  I. Vajda,et al.  Asymptotic distributions of φ‐divergences of hypothetical and observed frequencies on refined partitions , 1998 .

[30]  Ferdinand Österreicher.  On a class of perimeter-type distances of probability distributions , 1996, Kybernetika.