Nonparametric Fisher Geometry with Application to Density Estimation

It is well known that the Fisher information induces a Riemannian geometry on parametric families of probability density functions. Following recent work, we consider the nonparametric generalization of the Fisher geometry. The resulting nonparametric Fisher geometry is shown to be equivalent to a familiar, albeit infinite-dimensional, geometric object— the sphere. By shifting focus away from density functions and toward square-root density functions, one may calculate theoretical quantities of interest with ease. More importantly, the sphere of square-root densities is much more computationally tractable. As discussed here, this insight leads to a novel Bayesian nonparametric density estimation model.

[1]  B. Efron THE GEOMETRY OF EXPONENTIAL FAMILIES , 1978 .

[2]  Anuj Srivastava,et al.  Functional and Shape Data Analysis , 2016 .

[3]  D. Cox Some Statistical Methods Connected with Series of Events , 1955 .

[4]  C. Antoniak Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[5]  K. Chung Lectures from Markov processes to Brownian motion , 1982 .

[6]  Sebastian Kurtek,et al.  Bayesian sensitivity analysis with the Fisher–Rao metric , 2015 .

[7]  Radford M. Neal Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[8]  Andrew M. Stuart,et al.  Geometric MCMC for infinite-dimensional inverse problems , 2016, J. Comput. Phys..

[9]  Roman Garnett,et al.  Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature , 2014, NIPS.

[10]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[11]  H. Jeffreys An invariant form for the prior probability in estimation problems , 1946, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[12]  Mitsuhiro Itoh,et al.  Geometry of Fisher Information Metric and the Barycenter Map , 2015, Entropy.

[13]  L. L. Cam,et al.  Asymptotic Methods In Statistical Decision Theory , 1986 .

[14]  Anuj Srivastava,et al.  Riemannian Analysis of Probability Density Functions with Applications in Vision , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Dirk Brockmann,et al.  Massive Parallelization Boosts Big Bayesian Multidimensional Scaling , 2019, J. Comput. Graph. Stat..

[16]  Carl E. Rasmussen,et al.  In Advances in Neural Information Processing Systems , 2011 .

[17]  Brani Vidakovic,et al.  Bayesian Inference with Wavelets: Density Estimation , 1998 .

[18]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .

[19]  Junbin Gao,et al.  A Fast Algorithm to Estimate the Square Root of Probability Density Function , 2016, SGAI Conf..

[20]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[21]  Ryan P. Adams,et al.  The Gaussian Process Density Sampler , 2008, NIPS.

[22]  Rory A. Fisher,et al.  Theory of Statistical Estimation , 1925, Mathematical Proceedings of the Cambridge Philosophical Society.

[23]  Brani Vidakovic,et al.  Estimating the square root of a density via compactly supported wavelets , 1997 .

[24]  Stephen J. Maybank,et al.  Fisher-Rao Metric , 2014, Computer Vision, A Reference Guide.

[25]  A. Genz,et al.  The Supremum of Chi-Square Processes , 2014 .

[26]  Christian Gourieroux,et al.  Statistics and econometric models , 1995 .

[27]  B. Shahbaba,et al.  A Geometric View of Posterior Approximation , 2015, 1510.00861.

[28]  N. Longford A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects , 1987 .

[29]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[30]  M. Girolami,et al.  Geodesic Monte Carlo on Embedded Manifolds , 2013, Scandinavian journal of statistics, theory and applications.

[31]  M. Hutchings,et al.  Standing crop and pattern in pure stands of Mercurialis perennis and Rubus fruticosus in mixed deciduous woodland , 1978 .

[32]  Limin Wang,et al.  Karhunen-Loeve expansions and their applications , 2008 .

[33]  Mark Moyou,et al.  The Geometry of Orthogonal-Series, Square-Root Density Estimators: Applications in Computer Vision and Model Selection , 2017 .

[34]  G. Roberts,et al.  MCMC Methods for Functions: ModifyingOld Algorithms to Make Them Faster , 2012, 1202.0709.

[35]  Babak Shahbaba,et al.  Split Hamiltonian Monte Carlo , 2011, Stat. Comput..

[36]  Babak Shahbaba,et al.  Spherical Hamiltonian Monte Carlo for Constrained Target Distributions , 2013, ICML.

[37]  A. Stuart,et al.  The Bayesian Approach to Inverse Problems , 2013, 1302.6989.

[38]  I. Dryden Statistical analysis on high-dimensional spheres and shape spaces , 2005, math/0508279.