The Pólya information divergence

Extensions of Sanov's Theorem and the Conditional Limit Theorem (CoLT) are established for a multicolor Polya-Eggenberger (PE) urn sampling scheme, giving the Polya information divergence and a Polya extension to the Maximum Relative Entropy (MaxEnt) method. Polya MaxEnt includes the standard MaxEnt, as well as its variants used in Bose-Einstein, Fermi-Dirac and intermediate (Acharya-Swamy) statistics, as special cases. In the PE setting, standard MaxEnt is, in general, asymptotically inconsistent.

[1]  Andrew K. C. Wong,et al.  Entropy and Distance of Random Graphs with Application to Structural Pattern Recognition , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Robert K. Niven.  Exact Maxwell-Boltzmann, Bose-Einstein and Fermi-Dirac Statistics , 2004, ArXiv.

[3]  E. T. Jaynes,et al.  Papers on probability, statistics and statistical physics , 1983 .

[4]  S. Kullback,et al.  Information Theory and Statistics , 1959 .

[5]  Jean-François Bercher,et al.  On minimum Fisher information distributions with restricted support and fixed variance , 2009, Inf. Sci..

[6]  P. N. Swamy,et al.  Statistical mechanics of anyons , 1994 .

[7]  M. M. Mayoral,et al.  Simulation study of the tests of uniform association based on the power-divergence , 2007, Inf. Sci..

[8]  Gholamhossein Yari,et al.  Some properties of Rényi entropy and Rényi entropy rate , 2009, Inf. Sci..

[9]  Donald E. Brown,et al.  A weak law of large numbers for rare events , 1986 .

[10]  R. Niven.  Origins of the Combinatorial Basis of Entropy , 2007, 0708.1861.

[11]  John Skilling,et al.  Maximum Entropy and Bayesian Methods , 1989 .

[12]  I. Csiszár.  Sanov Property, Generalized $I$-Projection and a Conditional Limit Theorem , 1984 .

[13]  Imre Csisźar,et al.  The Method of Types , 1998, IEEE Trans. Inf. Theory.

[14]  I. Csiszár.  Information theoretic methods in probability and statistics , 1997, Proceedings of IEEE International Symposium on Information Theory.

[15]  I. N. Sanov.  On the probability of large deviations of random variables , 1958 .

[16]  Indistinguishability of particles or independence of the random variables? , 1997 .

[17]  Imre Csiszár,et al.  Information Theory and Statistics: A Tutorial , 2004, Found. Trends Commun. Inf. Theory.

[18]  Robert K. Niven,et al.  Combinatorial Information Theory: I. Philosophical Basis of Cross-Entropy and Entropy , 2005, ArXiv.

[19]  U. von Toussaint,et al.  Bayesian inference and maximum entropy methods in science and engineering , 2004 .

[20]  R. Niven.  Non-asymptotic thermodynamic ensembles , 2008, 0807.4160.

[21]  Jan M. Van Campenhout,et al.  Maximum entropy and conditional probability , 1981, IEEE Trans. Inf. Theory.

[22]  Armend Sh. Shabani.  Some inequalities for the gamma function. , 2007 .

[23]  Hosam M. Mahmoud,et al.  Polya Urn Models , 2008 .

[24]  Oldrich A. Vasicek.  A Conditional Law of Large Numbers , 1980 .

[25]  H. S. Steyn.  On Discrete Multivariate Probability Functions , 1951 .

[26]  Marian Grendar.  What is the question that MaxEnt answers? A probabilsitic interpretation , 2001 .

[27]  M. Grendár.  Maximum Probability and Relative Entropy Maximization. Bayesian Maximum Probability and Empirical Likelihood , 2008, 0804.3926.

[28]  Jean-François Bercher,et al.  On some entropy functionals derived from Rényi information divergence , 2008, Inf. Sci..

[29]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[30]  J. Lin,et al.  A NEW DIRECTED DIVERGENCE MEASURE AND ITS CHARACTERIZATION , 1990 .

[31]  N. L. Johnson,et al.  Discrete Multivariate Distributions , 1998 .

[32]  I. Csiszár.  Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems , 1991 .

[33]  I. Csiszár.  Maxent, Mathematics, and Information Theory , 1996 .

[34]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[35]  R. Niven.  Combinatorial entropies and statistics , 2009, 0902.3038.

[36]  M. Grendár,et al.  The Polya Urn: Limit Theorems, Polya Divergence, Maximum Entropy and Maximum Probability , 2006, cond-mat/0612697.

[37]  G. Pólya,et al.  Über die Statistik verketteter Vorgänge , 1923 .

[38]  David Taniar,et al.  Adaptive estimated maximum-entropy distribution model , 2007, Inf. Sci..

[39]  Mateu Sbert,et al.  Viewpoint-based simplification using f-divergences , 2008, Inf. Sci..

[40]  Jr.,et al.  Maximum Entropy method with non‐linear moment constraints: challenges , 2003 .

[41]  Cost of s-fold decisions in exact Maxwell–Boltzmann, Bose–Einstein and Fermi–Dirac statistics , 2005, cond-mat/0510128.

[42]  Robert K. Niven,et al.  Generalized classical, quantum and intermediate statistics and the Pólya urn model , 2009 .

[43]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .