The local geometry of finite mixtures

We introduce a technique to obtain local (bracketing) metric en- tropy bounds for subsets of a normed vector space from global entropy bounds. Using this method, we establish that for q � 1, the class of convex combina- tions of q translates of a probability density has finite local doubling dimension under a smoothness assumption. The proof requires a detailed investigation of the local geometry of mixture classes, which is of independent interest.

[1]  P. Massart,et al.  Concentration inequalities and model selection , 2007 .

[2]  Kim C. Border,et al.  Infinite dimensional analysis , 1994 .

[3]  A. Kolmogorov,et al.  Entropy and "-capacity of sets in func-tional spaces , 1961 .

[4]  S. Ghosal.  Convergence rates for density estimation with Bernstein polynomials , 2001 .

[5]  Fuchang Gao.  Entropy of Absolute Convex Hulls in Hilbert Spaces , 2004 .

[6]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[7]  S. Geer.  Applications of empirical process theory , 2000 .

[8]  J. Rousseau,et al.  On the asymptotic behaviour of the posterior distribution in hidden Markov Models with unknown number of states , 2014 .

[9]  Y. Shao,et al.  Asymptotics for likelihood ratio tests under loss of identifiability , 2003 .

[10]  E. Gassiat,et al.  Testing the order of a model using locally conic parametrization : population mixtures and stationary ARMA processes , 1999 .

[11]  Ramon van Handel,et al.  Consistent Order Estimation and Minimal Penalties , 2010, IEEE Transactions on Information Theory.

[12]  Fuchang Gao.  Metric entropy of convex hulls , 2001 .

[13]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[14]  J. Heinonen.  Lectures on Analysis on Metric Spaces , 2000 .

[15]  A. V. D. Vaart,et al.  Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities , 2001 .

[16]  Gabriela Ciuperca.  Likelihood Ratio Statistic for Exponential Mixtures , 2002 .

[17]  J. Wellner,et al.  How many Laplace transforms of probability measures are there? , 2010, Proceedings of the American Mathematical Society. American Mathematical Society.

[18]  P. Assouad.  Plongements lipschitziens dans Rn , 2003 .

[19]  Fuchang Gao,et al.  Metric entropy of high dimensional distributions , 2007 .

[20]  W. Marsden.  I and J , 2012 .

[21]  Patrice Assouad.  Plongements lipschitziens dans ${\mathbb {R}}^n$ , 1983 .

[22]  B. Carl,et al.  Metric Entropy of Convex Hulls in Banach Spaces , 1999 .

[23]  Cathy Maugis,et al.  A non asymptotic penalized criterion for Gaussian mixture model selection , 2011 .

[24]  L. Wasserman,et al.  RATES OF CONVERGENCE FOR THE GAUSSIAN MIXTURE SIEVE , 2000 .