Lower bounds on Information Divergence

In this paper we establish lower bounds on informa- tion divergence from a distribution to certain important classes of distributions as Gaussian, exponential, Gamma, Poisson, ge- ometric, and binomial. These lower bounds are tight and for several convergence theorems where a rate of convergence can be computed, this rate is determined by the lower bounds proved in this paper. General techniques for getting lower bounds in terms of moments are developed. I. I NTRODUCTION AND NOTATIONS In 2004, O. Johnson and A. Barron have proved (1) that the rate of convergence in the information theoretic Central Limit Theorem is upper bounded by c=n under suitable conditions. P. Harremoes extended this work in (2) based on a maximum entropy approach. Similar results have been obtained for the convergence of binomial distributions to Poisson distributions. Finally the rate of convergence of convolutions of distribu tions on the unit circle toward the uniform distribution can be bounded. In each of these cases lower bounds on information divergence in terms of moments of orthogonal polynomials or trigonometric functions give lower bounds on the rate of convergence. In this paper, we provide more lower bounds on information divergence using mainly orthogonal polynomials and the related exponential families. We will identify x! with ( x + 1) even when x is not an integer. Similarly the generalized binomial coefficient x � equals x(x − 1)���(x − n + 1)=n! when x is not an integer. We useas short for 2� .