In this paper, we are concerned with obtaining distribution-free concentration inequalities for mixture of independent Bernoulli variables that incorporate a notion of variance. Missing mass is the total probability mass associated to the outcomes that have not been seen in a given sample which is an important quantity that connects density estimates obtained from a sample to the population for discrete distributions. Therefore, we are specifically motivated to apply our method to study the concentration of missing mass - which can be expressed as a mixture of Bernoulli - in a novel way.
We not only derive - for the first time - Bernstein-like large deviation bounds for the missing mass whose exponents behave almost linearly with respect to deviation size, but also sharpen McAllester and Ortiz (2003) and Berend and Kontorovich (2013) for large sample sizes in the case of small deviations which is the most interesting case in learning theory. In the meantime, our approach shows that the heterogeneity issue introduced in McAllester and Ortiz (2003) is resolvable in the case of missing mass in the sense that one can use standard inequalities but it may not lead to strong results. Thus, we postulate that our results are general and can be applied to provide potentially sharp Bernstein-like bounds under some constraints.
[1]
B. Efron,et al.
The Jackknife Estimate of Variance
,
1981
.
[2]
Colin McDiarmid,et al.
Surveys in Combinatorics, 1989: On the method of bounded differences
,
1989
.
[3]
Gaston H. Gonnet,et al.
On the LambertW function
,
1996,
Adv. Comput. Math..
[4]
Lawrence K. Saul,et al.
Large Deviation Methods for Approximate Probabilistic Inference
,
1998,
UAI.
[5]
Luis E. Ortiz,et al.
Concentration Inequalities for the Missing Mass and for Histogram Rule Error
,
2003,
J. Mach. Learn. Res..
[6]
Henry Tirri,et al.
Generalization to Unseen Cases
,
2005,
BNAIC.
[7]
Imre Csiszár,et al.
Axiomatic Characterizations of Information Measures
,
2008,
Entropy.
[8]
D. Berend,et al.
On the concentration of the missing mass
,
2012,
1210.3248.
[9]
Gábor Lugosi,et al.
Concentration Inequalities - A Nonasymptotic Theory of Independence
,
2013,
Concentration Inequalities.