Nearly Optimal Learning and Sparse Covers for Sums of Independent Integer Random Variables

For k ∈ Z+, a k-SIIRV of order n ∈ Z+ is the discrete probability distribution of the sum of n mutually independent random variables each supported on {0,1,...,k −1}. We denote by Sn,k the set of all k-SIIRV’s of order n. In this paper we prove two main results: • We give a near-sample optimal and computationally efficient algorithm f or learning kSIIRVs from independent samples under the total variation distance (L1 distance). Our algorithm uses e

[1]  Paul W. Goldberg,et al.  Evolutionary Trees Can be Learned in Polynomial Time in the Two-State General Markov Model , 2001, SIAM J. Comput..

[2]  Sariel Har-Peled.  Geometric Approximation Algorithms , 2011 .

[3]  B. Carl,et al.  Entropy, Compactness and the Approximation of Operators , 1990 .

[4]  A. Izenman.  Recent Developments in Nonparametric Density Estimation , 1991 .

[5]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[6]  Yishay Mansour,et al.  Estimating a mixture of two product distributions , 1999, COLT '99.

[7]  Siméon-Denis Poisson.  Recherches sur la probabilité des jugements en matière criminelle et en matiére civile, précédées des règles générales du calcul des probabilités , 1837 .

[8]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[9]  L. Devroye,et al.  Nonparametric density estimation : the L[1] view , 1987 .

[10]  Peter E. Latham,et al.  Mutual Information , 2006 .

[11]  Luc Devroye,et al.  Combinatorial methods in density estimation , 2001, Springer series in statistics.

[12]  J. Feldman,et al.  Learning mixtures of product distributions over discrete domains , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[13]  Xi Chen,et al.  On the Complexity of Nash Equilibria in Anonymous Games , 2014, STOC.

[14]  Ronitt Rubinfeld,et al.  On the learnability of discrete distributions , 1994, STOC '94.

[15]  M. Blonski.  Anonymous Games with Binary Actions , 1999 .

[16]  Luc Devroye,et al.  Nonparametric Density Estimation , 1985 .

[17]  Adityanand Guntuboyina,et al.  Covering Numbers for Convex Functions , 2012, IEEE Transactions on Information Theory.

[18]  M. Blonski.  The women of Cairo Equilibria in large anonymous games , 2005 .

[19]  M. Fowler,et al.  Function Spaces , 2022 .

[20]  Rocco A. Servedio,et al.  Explorer Efficient Density Estimation via Piecewise Polynomial Approximation , 2013 .

[21]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[22]  Ryan O'Donnell,et al.  Learning Sums of Independent Integer Random Variables , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[23]  Christos H. Papadimitriou,et al.  On oblivious PTAS's for nash equilibrium , 2009, STOC '09.

[24]  Rocco A. Servedio,et al.  Learning k-Modal Distributions via Testing , 2012, Theory Comput..

[25]  W. Rudin.  Principles of mathematical analysis , 1964 .

[26]  D. Haussler,et al.  MUTUAL INFORMATION, METRIC ENTROPY AND CUMULATIVE RELATIVE ENTROPY RISK , 1997 .

[27]  C. Papadimitriou,et al.  Computing Equilibria in Anonymous Games , 2007, FOCS 2007.

[28]  I. Ibragimov,et al.  On density estimation in the view of Kolmogorov's ideas in approximation theory , 1990 .

[29]  G. Lorentz.  Metric entropy and approximation , 1966 .

[30]  S. Sheather.  Density Estimation , 2004 .

[31]  Yuly Makovoz.  On the Kolmogorov complexity of functions of finite smoothness , 1986, J. Complex..

[32]  A. Kolmogorov,et al.  Entropy and "-capacity of sets in func-tional spaces , 1961 .

[33]  J. Kruopis,et al.  Precision of approximation of the generalized binomial distribution by convolutions of Poisson measures , 1986 .

[34]  Jun S. Liu,et al.  STATISTICAL APPLICATIONS OF THE POISSON-BINOMIAL AND CONDITIONAL BERNOULLI DISTRIBUTIONS , 1997 .

[35]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[36]  Ankur Moitra,et al.  Settling the Polynomial Learnability of Mixtures of Gaussians , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[37]  J. M. Bremner,et al.  Statistical Inference under Restrictions , 1973 .

[38]  Mikhail Belkin,et al.  Polynomial Learning of Distribution Families , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[39]  Christos H. Papadimitriou,et al.  Approximate Nash equilibria in anonymous games , 2015, J. Econ. Theory.

[40]  Adam Tauman Kalai,et al.  Efficiently learning mixtures of two Gaussians , 2010, STOC '10.

[41]  L. Birge,et al.  On estimating a density using Hellinger distance and some other strange facts , 1986 .

[42]  Devdatt P. Dubhashi,et al.  Concentration of Measure for the Analysis of Randomized Algorithms: Contents , 2009 .

[43]  R. Dudley.  Metric Entropy of Some Classes of Sets with Differentiable Boundaries , 1974 .

[44]  Sanjeev Arora,et al.  Learning mixtures of arbitrary gaussians , 2001, STOC '01.

[45]  Santosh S. Vempala,et al.  A spectral algorithm for learning mixtures of distributions , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[46]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[47]  Martin Raič,et al.  Normal Approximation by Stein ’ s Method , 2003 .

[48]  Christos H. Papadimitriou,et al.  Sparse covers for sums of indicators , 2013, ArXiv.

[49]  H. Triebel,et al.  Function Spaces, Entropy Numbers, Differential Operators: Function Spaces , 1996 .

[50]  E. L. Presman,et al.  Approximation of Binomial Distributions by Infinitely Divisible Ones , 1984 .

[51]  I. Milchtaich,et al.  Congestion Games with Player-Specific Payoff Functions , 1996 .

[52]  Yuhong Yang,et al.  Information-theoretic determination of minimax rates of convergence , 1999 .

[53]  Fuchang Gao,et al.  Metric entropy of high dimensional distributions , 2007 .

[54]  Y. Yatracos.  Rates of Convergence of Minimum Distance Estimators and Kolmogorov's Entropy , 1985 .

[55]  P. Assouad.  Deux remarques sur l'estimation , 1983 .

[56]  Rocco A. Servedio,et al.  Learning Poisson Binomial Distributions , 2011, STOC '12.

[57]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[58]  A. Barbour,et al.  Poisson Approximation , 1992 .