RATES OF CONVERGENCE FOR THE GAUSSIAN MIXTURE SIEVE

Gaussian mixtures provide a convenient method of densityestimation that lies somewhere between parametric models and kernel densityestimators. When the number of components of the mixture is allowed to increase as sample size increases, the model is called a mixture sieve. We establish a bound on the rate of convergence in Hellinger distance for densityestimation using the Gaussian mixture sieve assuming that the true density is itself a mixture of Gaussians; the underlying mixing measure of the true densityis not necessarilyassumed to have finite support. Computing the rate involves some delicate calculations since the size of the sieve—as measured bybracketing entropy—and the saturation rate, cannot be found using standard methods. When the mixing measure has compact support, using kn ∼ n 2/3 /� log n� 1/3 components in the mixture yields a rate of order � log n� � 1+η� /6/n1/6 for every η> 0� The rates depend heavilyon the tail behavior of the true density. The sensitivity to the tail behavior is diminished byusing a robust sieve which includes a long-tailed component in the

[1]  S. Geman,et al.  Nonparametric Maximum Likelihood Estimation by the Method of Sieves , 1982 .

[2]  P. Hall On Kullback-Leibler loss and density estimation , 1987 .

[3]  Geoffrey J. McLachlan,et al.  Mixture models : inference and applications to clustering , 1989 .

[4]  K. Roeder Semiparametric Estimation of Normal Mixture Densities , 1992 .

[5]  A. Raftery,et al.  Model-based Gaussian and non-Gaussian clustering , 1993 .

[6]  Jiahua Chen Optimal Rate of Convergence for Finite Mixture Models , 1995 .

[7]  W. Wong,et al.  Probability inequalities for likelihood ratios and convergence rates of sieve MLEs , 1995 .

[8]  S. Richardson,et al.  Mixtures of distributions: inference and estimation , 1995 .

[9]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[10]  B. Lindsay Mixture models : theory, geometry, and applications , 1995 .

[11]  C. Robert Mixtures of Distributions: Inference and Estimation , 1996 .

[12]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[13]  Sara van de Geer,et al.  Rates of convergence for the maximum likelihood estimator in mixture models , 1996 .

[14]  L. Wasserman,et al.  Practical Bayesian Density Estimation Using Mixtures of Normals , 1997 .

[15]  Yuhong Yang,et al.  An Asymptotic Property of Model Selection Criteria , 1998, IEEE Trans. Inf. Theory.

[16]  Andrew R. Barron,et al.  Mixture Density Estimation , 1999, NIPS.

[17]  A. V. D. Vaart,et al.  Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities , 2001 .