Reference duality and representation duality in information geometry

Classical information geometry prescribes, on the parametric family of probability functions Mθ: (i) a Riemannian metric given by the Fisher information; (ii) a pair of dual connections (giving rise to the family of α-connections) that preserve the metric under parallel transport by their joint actions; and (iii) a family of (non-symmetric) divergence functions (α-divergence) defined on Mθ × Mθ, which induce the metric and the dual connections. The role of α parameter, as used in α-connection and in α-embedding, is not commonly differentiated. For instance, the case with α = ±1 may refer either to dually-flat (e- or m-) connections or to exponential and mixture families of density functions. Here we illuminate that there are two distinct types of duality in information geometry, one concerning the referential status of a point (probability function, normalized or denormalized) expressed in the divergence function ("reference duality") and the other concerning the representation of probability functions un...

[1]  Shun-ichi Amari,et al.  Geometry of q-Exponential Family of Probability Distributions , 2011, Entropy.

[2]  S. Amari.  Differential Geometry of Curved Exponential Families-Curvatures and Information Loss , 1982 .

[3]  M. C. Chaki.  ON STATISTICAL MANIFOLDS , 2000 .

[4]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[5]  Jan Naudts,et al.  Generalised Exponential Families and Associated Entropy Functions , 2008, Entropy.

[6]  C. R. Rao,et al.  Differential metrics in probability spaces , 1984 .

[7]  Takafumi Kanamori,et al.  Information Geometry of U-Boost and Bregman Divergence , 2004, Neural Computation.

[8]  H. Shima,et al.  Geometry of Hessian manifolds , 1997 .

[9]  Shun-ichi Amari,et al.  Geometry of deformed exponential families: Invariant, dually-flat and conformal geometries , 2012 .

[10]  P. Marriott,et al.  On preferred point geometry in statistics , 2002 .

[11]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[12]  Jun Zhang,et al.  Nonparametric Information Geometry: From Divergence Function to Referential-Representational Biduality on Statistical Manifolds , 2013, Entropy.

[13]  O. Barndorff-Nielsen.  Parametric statistical models and likelihood , 1988 .

[14]  Udo Simon,et al.  Introduction to the affine differential geometry of hypersurfaces , 1991 .

[15]  Shun-ichi Amari,et al.  Differential-geometrical methods in statistics , 1985 .

[16]  S. Eguchi.  A differential geometric approach to statistical inference on the basis of contrast functionals , 1985 .

[17]  野水 克己,et al.  Affine differential geometry : geometry of affine immersions , 1994 .

[18]  B. Wei,et al.  Some notes on preferred point α-geometry and α-divergence function , 1997 .

[19]  Hirohiko Shima.  Compact locally Hessian manifolds , 1978 .

[20]  S. Eguchi.  Geometry of minimum contrast , 1992 .

[21]  B. Wei,et al.  Preferred point a-manifold and Amari's a-connections , 1997 .

[22]  Shun-ichi Amari,et al.  A dually flat structure on the space of escort distributions , 2010 .

[23]  甘利 俊一.  Differential geometry in statistical inference , 1987 .

[24]  Paul Marriott,et al.  Preferred Point Geometry and the Local Differential Geometry of the Kullback-Leibler Divergence , 1994 .

[25]  Sergio Cruces,et al.  Generalized Alpha-Beta Divergences and Their Application to Robust Nonnegative Matrix Factorization , 2011, Entropy.

[26]  Keiko Uohashi,et al.  On \alpha-conformal equivalence of statistical submanifolds , 2002 .

[27]  Andrzej Cichocki,et al.  Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities , 2010, Entropy.

[28]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .

[29]  Jun Zhang,et al.  Divergence Function, Duality, and Convex Analysis , 2004, Neural Computation.

[30]  S. Eguchi.  Second Order Efficiency of Minimum Contrast Estimators in a Curved Exponential Family , 1983 .

[31]  M. C. Jones,et al.  Robust and efficient estimation by minimising a density power divergence , 1998 .

[32]  Paul Marriott,et al.  Preferred Point Geometry and Statistical Manifolds , 1993 .