A Statistical Mechanics Approach to Large Deviation Theorems
暂无分享,去创建一个
Chernoo bounds and related large deviation bounds have a wide variety of applications in statistics and learning theory. This paper proves that for any real-valued random variable X the probability of a deviation to value x is bounded by e S(x) where S(x) is the entropy at energy x of a physical system corresponding to the variable X. It is a well known fact of statistical mechanics that entropy is equal to a double integral of the reciprocal of energy variance. So we get a general bound on large deviation probabilities in terms of variance over a range of temperatures. This greatly simpliies the derivation of Chernoo and Hoeeding bounds and leads immediately to a variety of apparently new large deviation theorems.
[1] Thomas M. Cover,et al. Elements of Information Theory , 2005 .
[2] David A. McAllester,et al. On the Convergence Rate of Good-Turing Estimators , 2000, COLT.