Information Topologies with Applications

Topologies related to information divergence are introduced. The conditional limit theorem is taken as motivating example, and simplified proofs of the relevant theorems are given. Continuity properties of entropy and information divergence are discussed.

[1]  G. Hardy,et al.  The general theory of Dirichlet's series , 1916, The Mathematical Gazette.

[2]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[3]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[4]  J. Linnik.  An Information-Theoretic Proof of the Central Limit Theorem with Lindeberg Conditions , 1959 .

[5]  J. Kisyński.  Convergence du type L , 1960 .

[6]  A. Rényi.  On Measures of Entropy and Information , 1961 .

[7]  David G. Kendall.  Information theory and the limit-theorem for Markov chains and processes with a countable infinity of states , 1963 .

[8]  I. Csiszár.  $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[9]  Flemming Topsøe,et al.  Information-theoretical optimization techniques , 1979, Kybernetika.

[10]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[11]  P. Ney.  Dominating Points and the Asymptotics of Large Deviations for Random Walk on $\mathbb{R}^d$ , 1983 .

[12]  I. Csiszár.  Sanov Property, Generalized $I$-Projection and a Conditional Limit Theorem , 1984 .

[13]  A. Barron.  ENTROPY AND THE CENTRAL LIMIT THEOREM , 1986 .

[14]  Seiji Takano.  CONVERGENCE OF ENTROPY IN THE CENTRAL LIMIT THEOREM , 1987 .

[15]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[16]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[17]  Imre Csisźar,et al.  The Method of Types , 1998, IEEE Trans. Inf. Theory.

[18]  R. Dudley.  Consistency of M-Estimators and One-Sided Bracketing , 1998 .

[19]  A. Barron.  Limits of information, Markov chains, and projection , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).

[20]  Peter Harremoës,et al.  Binomial and Poisson distributions as maximum entropy distributions , 2001, IEEE Trans. Inf. Theory.

[21]  U. Schmock,et al.  LARGE DEVIATIONS FOR PRODUCTS OF EMPIRICAL MEASURES OF DEPENDENT SEQUENCES , 2001 .

[22]  Peter Harremoës,et al.  Maximum Entropy Fundamentals , 2001, Entropy.

[23]  Shun-ichi Amari,et al.  Information geometry on hierarchy of probability distributions , 2001, IEEE Trans. Inf. Theory.

[24]  Oliver Johnson.  Entropy and a generalisation of “Poincaré's Observation” , 2003, Mathematical Proceedings of the Cambridge Philosophical Society.

[25]  Flemming Topsøe,et al.  Information Theory at the Service of Science , 2007 .