Statistical exponential families: A digest with flash cards

This document describes concisely the ubiquitous class of exponential family distributions met in statistics. The first part recalls definitions and summarizes main properties and duality with Bregman divergences (all proofs are skipped). The second part lists decompositions and related formula of common exponential family distributions. We recall the Fisher-Rao-Riemannian geometries and the dual affine connection information geometries of statistical manifolds. It is intended to maintain and update this document and catalog by adding new distribution items.

[1]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[2]  Susan A. Murphy,et al.  Monographs on statistics and applied probability , 1990 .

[3]  Arindam Banerjee,et al.  An Analysis of Logistic Models: Exponential Family Connections and Online Performance , 2007, SDM.

[4]  M. C. Chaki.  ON STATISTICAL MANIFOLDS , 2000 .

[5]  Alexander J. Smola,et al.  Kernel methods and the exponential family , 2006, ESANN.

[6]  甘利 俊一.  Differential geometry in statistical inference , 1987 .

[7]  Frank Nielsen,et al.  Sided and Symmetrized Bregman Centroids , 2009, IEEE Transactions on Information Theory.

[8]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[9]  Thomas Hofmann,et al.  Exponential Families for Conditional Random Fields , 2004, UAI.

[10]  N. Čencov.  Statistical Decision Rules and Optimal Inference , 2000 .

[11]  M. Grasselli,et al.  On the Uniqueness of the Chentsov Metric in Quantum Information Geometry , 2000, math-ph/0006030.

[12]  L. Brown.  Fundamentals of statistical exponential families: with applications in statistical decision theory , 1986 .

[13]  M. Murray,et al.  Differential Geometry and Statistics , 1993 .

[14]  Josep M. Oller,et al.  Computing the Rao distance for gamma distributions , 2003 .

[15]  Sueli I. Rodrigues Costa,et al.  Fisher information matrix and hyperbolic geometry , 2005, IEEE Information Theory Workshop, 2005..

[16]  R. Kass,et al.  Geometrical Foundations of Asymptotic Inference , 1997 .

[17]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[18]  Guy Lebanon.  Axiomatic geometry of conditional models , 2005, IEEE Transactions on Information Theory.

[19]  Ted Chang.  Geometrical foundations of asymptotic inference , 2002 .