Fisher information distance: a geometrical reading?

This paper presents a geometrical approach to the Fisher distance, which is a measure of dissimilarity between two probability distribution functions. The Fisher distance, as well as other divergence measures, is also used in many applications to establish a proper data average. The main purpose is to widen the range of possible interpretations and relations of the Fisher distance and its associated geometry for the prospective applications. It focuses on statistical models of the normal probability distribution functions and takes advantage of the connection with the classical hyperbolic geometry to derive closed forms for the Fisher distance in several cases. Connections with the well-known Kullback-Leibler divergence measure are also devised.

[1]  Yannick Berthoumieu,et al.  K-Centroids-Based Supervised Classification of Texture Images Using the SIRV Modeling , 2013, GSI.

[2]  Jesús Angulo,et al.  Morphological Processing of Univariate Gaussian Distribution-Valued Images Based on Poincaré Upper-Half Plane Representation , 2014 .

[3]  Amir Dembo,et al.  Information theoretic inequalities , 1991, IEEE Trans. Inf. Theory.

[4]  Frank Nielsen,et al.  Statistical exponential families: A digest with flash cards , 2009, ArXiv.

[5]  Kevin M. Carter,et al.  Dimensionality reduction on statistical manifolds , 2009 .

[6]  Frank Nielsen,et al.  Sided and Symmetrized Bregman Centroids , 2009, IEEE Transactions on Information Theory.

[7]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[8]  Frank Nielsen,et al.  Matrix Information Geometry , 2012 .

[9]  Axthonv G. Oettinger,et al.  IEEE Transactions on Information Theory , 1998 .

[10]  Anand Rangarajan,et al.  Information Geometry for Landmark Shape Analysis: Unifying Shape Representation and Deformation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[12]  A. Hero,et al.  LEARNING ON STATISTICAL MANIFOLDS FOR CLUSTERING AND VISUALIZATION , 2007 .

[13]  Josep M. Oller,et al.  A distance between multivariate normal distributions based in an embedding into the Siegel group , 1990 .

[14]  R. Fisher,et al.  On the Mathematical Foundations of Theoretical Statistics , 1922 .

[15]  Sueli I. Rodrigues Costa,et al.  Fisher information matrix and hyperbolic geometry , 2005, IEEE Information Theory Workshop, 2005..

[16]  L. Skovgaard.  A Riemannian geometry of the multivariate normal model , 1984 .

[17]  J. M. Oller,et al.  AN EXPLICIT SOLUTION OF INFORMATION GEODESIC EQUATIONS FOR THE MULTIVARIATE NORMAL MODEL , 1991 .

[18]  Miroslav Lovric,et al.  Multivariate Normal Distributions Parametrized as a Riemannian Symmetric Space , 2000 .

[19]  T. Cover,et al.  IEEE TRANSACTIONSON INFORMATIONTHEORY,VOL. IT-30,N0. 6,NOVEmER1984 Correspondence On the Similarity of the Entropy Power Inequality The preceeding equations allow the entropy power inequality and the Brunn-Minkowski Inequality to be rewritten in the equiv , 2022 .

[20]  C. Atkinson.  Rao's distance measure , 1981 .

[21]  I. Holopainen.  Riemannian Geometry , 1927, Nature.

[22]  Leo Liberti,et al.  The interval Branch-and-Prune algorithm for the discretizable molecular distance geometry problem with inexact distances , 2011, Journal of Global Optimization.

[23]  Leo Liberti,et al.  Euclidean Distance Geometry and Applications , 2012, SIAM Rev..

[24]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[25]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[26]  A. Beardon.  The Geometry of Discrete Groups , 1995 .

[27]  Daniel Silk,et al.  Optimizing Threshold - Schedules for Approximate Bayesian Computation Sequential Monte Carlo Samplers: Applications to Molecular Systems , 2012 .

[28]  S. Stigler.  Fisher in 1921 , 2005 .

[29]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[30]  Frank Nielsen,et al.  Total Bregman divergence and its applications to shape retrieval , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  S. Filippi,et al.  Information Geometry and Sequential Monte Carlo , 2012, 1212.0764.

[32]  Guido Van Oost,et al.  Multivariate Texture Discrimination Based on Geodesics to Class Centroids on a Generalized Gaussian Manifold , 2013, GSI.

[33]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .