On fine properties of mixtures with respect to concentration of measure and Sobolev type inequalities

Mixtures are convex combinations of laws. Despite this simple definition, a mixture can be far more subtle than its mixed components. For instance, mixing Gaussian laws may produce a potential with multiple deep wells. We study in the present work fine properties of mixtures with respect to concentration of measure and Sobolev type functional inequalities. We provide sharp Laplace bounds for Lipschitz functions in the case of generic mixtures, involving a transportation cost diameter of the mixed family. Additionally, our analysis of Sobolev type inequalities for two-component mixtures reveals natural relations with some kind of band isoperimetry and support constrained interpolation via mass transportation. We show that the Poincar\'e constant of a two-component mixture may remain bounded as the mixture proportion goes to 0 or 1 while the logarithmic Sobolev constant may surprisingly blow up. This counter-intuitive result is not reducible to support disconnections, and appears as a reminiscence of the variance-entropy comparison on the two-point space. As far as mixtures are concerned, the logarithmic Sobolev inequality is less stable than the Poincar\'e inequality and the sub-Gaussian concentration for Lipschitz functions. We illustrate our results on a gallery of concrete two-component mixtures. This work leads to many open questions.

[1]  A. J. Stam.  Some Inequalities Satisfied by the Quantities of Information of Fisher and Shannon , 1959, Inf. Control..

[2]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[3]  R. Phelps.  Lectures on Choquet's Theorem , 1966 .

[4]  V. Sudakov,et al.  Geometric Problems in the Theory of Infinite-dimensional Probability Distributions , 1979 .

[5]  A. Cohen,et al.  Finite Mixture Distributions , 1982 .

[6]  P. Meyer,et al.  Sur les inegalites de Sobolev logarithmiques. I , 1982 .

[7]  M. Gromov,et al.  A topological application of the isoperimetric inequality , 1983 .

[8]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[9]  Katalin Marton,et al.  A simple proof of the blowing-up lemma , 1986, IEEE Trans. Inf. Theory.

[10]  D. Stroock,et al.  Logarithmic Sobolev inequalities and stochastic Ising models , 1987 .

[11]  Geoffrey J. McLachlan,et al.  Mixture models : inference and applications to clustering , 1989 .

[12]  Jim Freeman.  Probability Metrics and the Stability of Stochastic Models , 1991 .

[13]  E. Carlen.  Superadditivity of Fisher's information and logarithmic Sobolev inequalities , 1991 .

[14]  P. Diaconis,et al.  LOGARITHMIC SOBOLEV INEQUALITIES FOR FINITE MARKOV CHAINS , 1996 .

[15]  M. Talagrand.  Transportation cost for Gaussian and other product measures , 1996 .

[16]  K. Marton.  Bounding $\bar{d}$-distance by informational divergence: a method to prove measure concentration , 1996 .

[17]  Feng-Yu Wang,et al.  Logarithmic Sobolev inequalities on noncompact Riemannian manifolds , 1997 .

[18]  L. Saloff-Coste,et al.  Lectures on finite Markov chains , 1997 .

[19]  S. Aida.  Uniform Positivity Improving Property, Sobolev Inequalities, and Spectral Gaps , 1998 .

[20]  S. Bobkov,et al.  Exponential Integrability and Transportation Cost Related to Logarithmic Sobolev Inequalities , 1999 .

[21]  M. Ledoux.  Concentration of measure and logarithmic Sobolev inequalities , 1999 .

[22]  C. Villani,et al.  Generalization of an Inequality by Talagrand and Links with the Logarithmic Sobolev Inequality , 2000 .

[23]  Spectral Gaps for Spin Systems: Some Non-convex Phase Examples , 2000 .

[24]  Luis A. Caffarelli,et al.  Monotonicity Properties of Optimal Transportation¶and the FKG and Related Inequalities , 2000 .

[25]  M. Ledoux.  The concentration of measure phenomenon , 2001 .

[26]  R. Lata.  On Some Inequalities for Gaussian Measures , 2002 .

[27]  Convergence of the Poincaré Constant , 2002, math/0206227.

[28]  D. Randall,et al.  Markov chain decomposition for convergence rate analysis , 2002 .

[29]  B. Helffer.  Semiclassical Analysis, Witten Laplacians, and Statistical Mechanics , 2002 .

[30]  F. Barthe,et al.  Sobolev inequalities for probability measures on the real line , 2003 .

[31]  C. Villani.  Topics in Optimal Transportation , 2003 .

[32]  G. Burton.  TOPICS IN OPTIMAL TRANSPORTATION (Graduate Studies in Mathematics 58) By CÉDRIC VILLANI: 370 pp., US$59.00, ISBN 0-8218-3312-X (American Mathematical Society, Providence, RI, 2003) , 2004 .

[33]  F. Barthe,et al.  Interpolated inequalities between exponential and Gaussian, Orlicz hypercontractivity and isoperimetry , 2004, math/0407219.

[34]  A. Guillin,et al.  Transportation cost-information inequalities and applications to random dynamical systems and diffusions , 2004, math/0410172.

[35]  O. Johnson.  CONVERGENCE OF THE POINCAR ´ E CONSTANT ∗ , 2004 .

[36]  Mokshay M. Madiman,et al.  Entropy, compound Poisson approximation, log-Sobolev inequalities and measure concentration , 2004, Information Theory Workshop.

[37]  Eric Vigoda,et al.  Elementary bounds on Poincaré and log-Sobolev constants for decomposable Markov chains , 2004, math/0503537.

[38]  Sergey G. Bobkov.  Concentration of normalized sums and a central limit theorem for noncorrelated random variables , 2004 .

[39]  Ioannis Kontoyiannis,et al.  Measure Concentration for Compound Poisson Distributions , 2005, Electronic Communications in Probability.

[40]  B. Zegarliński,et al.  Entropy Bounds and Isoperimetry , 2005 .

[41]  C. Villani,et al.  Weighted Csiszár-Kullback-Pinsker inequalities and applications to transportation inequalities , 2005 .

[42]  Sergey G. Bobkov,et al.  Generalized Symmetric Polynomials and an Approximate De Finetti Representation , 2005 .

[43]  William G. Faris.  Diffusion, quantum theory, and radically elementary mathematics , 2006 .

[44]  Sylvia Frühwirth-Schnatter,et al.  Finite Mixture and Markov Switching Models , 2006 .

[45]  Maria G. Reznikoff,et al.  A new criterion for the logarithmic Sobolev inequality and two applications , 2007 .

[46]  M. Ledoux,et al.  Perturbations of functional inequalities using growth conditions , 2007 .

[47]  S. Chatterjee.  Spin glasses and Stein’s method , 2007, 0706.3500.

[48]  F. Barthe,et al.  Mass Transport and Variants of the Logarithmic Sobolev Inequality , 2007, 0709.3890.

[49]  L. Miclo.  Quand est-ce que des bornes de Hardy permettent de calculer une constante de Poincaré exacte sur la droite ? , 2008 .

[50]  P. Nurmi.  Mixture Models , 2008 .

[51]  P. Deb.  Finite Mixture Models , 2008 .

[52]  Asuka Takatsu,et al.  On Wasserstein geometry of the space of Gaussian measures , 2008 .

[53]  N. Gozlan.  A characterization of dimension free concentration in terms of transportation inequalities , 2008, 0804.3089.