Computational information geometry: theory and practice

This paper lays the foundations for a unified framework for numerically and computationally applying methods drawn from a range of currently distinct geometrical approaches to statistical modelling. In so doing, it extends information geometry from a manifold based approach to one where the simplex is the fundamental geometrical object, thereby allowing applications to models which do not have a fixed dimension or support. Finally, it starts to build a computational framework which will act as a proxy for the 'space of all distributions' that can be used, in particular, to investigate model selection and model uncertainty. A varied set of substantive running examples is used to illustrate theoretical and practical aspects of the discussion. Further developments are briefly indicated.

[1]  Stephen E. Fienberg,et al.  Maximum likelihood estimation in log-linear models , 2011, 1104.3618.

[2]  Stephen E. Fienberg,et al.  Maximum Likelihood Estimation in Log-Linear Models: Theory and Algorithms , 2011 .

[3]  Piotr Zwiernik,et al.  Tree cumulants and the geometry of binary tree models , 2010, 1004.4360.

[4]  Henry P. Wynn,et al.  Algebraic and geometric methods in statistics , 2009 .

[5]  Piotr Zwiernik,et al.  Implicit inequality constraints in a binary tree model , 2009, 0904.1980.

[6]  A. Rinaldo,et al.  On the geometry of discrete exponential families with application to exponential random graph models , 2008, 0901.0026.

[7]  P. Marriott,et al.  Local mixture models of exponential families , 2007, 0709.0447.

[8]  I. Csiszár,et al.  Closures of exponential families , 2005, math/0503653.

[9]  On the global geometry of parametric models and information recovery , 2004 .

[10]  Paul Marriott,et al.  On the Geometry of Censored Models , 2002 .

[11]  Paul Marriott,et al.  On the local geometry of mixture models , 2002 .

[12]  Ted Chang.  Geometrical foundations of asymptotic inference , 2002 .

[13]  D. Geiger,et al.  Stratified exponential families: Graphical models and model selection , 2001 .

[14]  H. Wynn,et al.  Algebraic Statistics: Computational Commutative Algebra in Statistics , 2000 .

[15]  Michael I. Jordan.  Graphical Models , 1998 .

[16]  R. Kass,et al.  Geometrical Foundations of Asymptotic Inference , 1997 .

[17]  B. Lindsay.  Mixture models : theory, geometry, and applications , 1995 .

[18]  D. Cox,et al.  Inference and Asymptotics , 1994 .

[19]  David J. Hand,et al.  A Handbook of Small Data Sets , 1993 .

[20]  Paul Marriott,et al.  Preferred Point Geometry and Statistical Manifolds , 1993 .

[21]  M. Murray,et al.  Differential Geometry and Statistics , 1993 .

[22]  J. Kalbfleisch,et al.  An Algorithm for Computing the Nonparametric MLE of a Mixing Distribution , 1992 .

[23]  D. Cox,et al.  Asymptotic techniques for use in statistics , 1989 .

[24]  P. McCullagh.  Tensor Methods in Statistics , 1987 .

[25]  Herbert Edelsbrunner,et al.  Algorithms in Combinatorial Geometry , 1987, EATCS Monographs in Theoretical Computer Science.

[26]  L. Brown.  Fundamentals of statistical exponential families: with applications in statistical decision theory , 1986 .

[27]  Shun-ichi Amari,et al.  Differential-geometrical methods in statistics , 1985 .

[28]  Ole E. Barndorff-Nielsen,et al.  Exponential Models with Affine Dual Foliations , 1983 .

[29]  O. Barndorff-Nielsen.  Information and Exponential Families in Statistical Theory , 1980 .

[30]  L L Kupper,et al.  The use of a correlated binomial model for the analysis of certain toxicological experiments. , 1978, Biometrics.

[31]  B. Efron.  Defining the Curvature of a Statistical Problem (with Applications to Second Order Efficiency) , 1975 .

[32]  J. Cooper.  TOTAL POSITIVITY, VOL. I , 1970 .

[33]  Alston S. Householder,et al.  The Theory of Matrices in Numerical Analysis , 1964 .

[34]  H. Piaggio.  Mathematical Analysis , 1955, Nature.