Interpretable Stein Goodness-of-fit Tests on Riemannian Manifold

In many applications, we encounter data on Riemannian manifolds such as torus and rotation groups. Standard statistical procedures for multivariate data are not applicable to such data. In this study, we develop goodness-of-fit testing and interpretable model criticism methods for general distributions on Riemannian manifolds, including those with an intractable normalization constant. The proposed methods are based on extensions of kernel Stein discrepancy, which are derived from Stein operators on Riemannian manifolds. We discuss the connections between the proposed tests with existing ones and provide a theoretical analysis of their asymptotic Bahadur efficiency. Simulation results and real data applications show validity and usefulness of the proposed methods.

[1]  Ing Rj Ser.  Approximation Theorems of Mathematical Statistics , 1980 .

[2]  T. Gneiting.  Strictly and non-strictly positive definite functions on spheres , 2011, 1111.7077.

[3]  Louis H. Y. Chen,et al.  An Introduction to Stein's Method , 2005 .

[4]  Akimichi Takemura,et al.  Properties and applications of Fisher distribution on the rotation group , 2011, J. Multivar. Anal..

[5]  Ben Calderhead,et al.  Riemannian Manifold Hamiltonian Monte Carlo , 2009, 0907.1100.

[6]  Wittawat Jitkrittum,et al.  Large sample analysis of the median heuristic , 2017, 1707.07269.

[7]  M. Spivak.  Calculus On Manifolds: A Modern Approach To Classical Theorems Of Advanced Calculus , 2019 .

[8]  G. Reinert,et al.  Stein's method for comparison of univariate distributions , 2014, 1408.2998.

[9]  Takeru Matsuda,et al.  A Stein Goodness-of-fit Test for Directional Distributions , 2020, AISTATS.

[10]  K. Mardia,et al.  Maximum Likelihood Estimators for the Matrix Von Mises-Fisher and Bingham Distributions , 1979 .

[11]  Søren Hauberg,et al.  Geodesic exponential kernels: When curvature and linearity conflict , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Heishiro Kanagawa,et al.  Testing Goodness of Fit of Conditional Density Models with Kernels , 2020, UAI.

[13]  Chang Liu,et al.  Riemannian Stein Variational Gradient Descent for Bayesian Inference , 2017, AAAI.

[14]  Arthur Gretton,et al.  A Kernel Test of Goodness of Fit , 2016, ICML.

[15]  Arthur Gretton,et al.  A Wild Bootstrap for Degenerate Kernel Tests , 2014, NIPS.

[16]  Qiang Liu,et al.  Goodness-of-fit Testing for Discrete Distributions via Stein Discrepancy , 2018, ICML.

[17]  Peter D. Hoff,et al.  Simulation of the Matrix Bingham–von Mises–Fisher Distribution, With Applications to Multivariate and Relational Data , 2007, 0712.4166.

[18]  Diana Adler,et al.  Differential Forms With Applications To The Physical Sciences , 2016 .

[19]  Linda R Petzold,et al.  General Bayesian Inference over the Stiefel Manifold via the Givens Representation , 2017 .

[20]  Gesine Reinert,et al.  A Stein Goodness-of-test for Exponential Random Graph Models , 2021, AISTATS.

[21]  Kenji Fukumizu,et al.  A Kernel Stein Test for Comparing Latent Variable Models , 2019, Journal of the Royal Statistical Society Series B: Statistical Methodology.

[22]  Qiang Liu,et al.  A Kernelized Stein Discrepancy for Goodness-of-fit Tests , 2016, ICML.

[23]  C. Carmeli,et al.  Vector valued reproducing kernel Hilbert spaces and universality , 2008, 0807.1659.

[24]  Bernhard Schölkopf,et al.  A Kernel Method for the Two-Sample-Problem , 2006, NIPS.

[25]  Aapo Hyvärinen,et al.  Noise-Contrastive Estimation of Unnormalized Statistical Models, with Applications to Natural Image Statistics , 2012, J. Mach. Learn. Res..

[26]  Oluwasanmi Koyejo,et al.  Examples are not enough, learn to criticize! Criticism for Interpretability , 2016, NIPS.

[27]  Kenji Fukumizu,et al.  A Linear-Time Kernel Goodness-of-Fit Test , 2017, NIPS.

[28]  Harshinder Singh,et al.  Probabilistic model for two dependent circular variables , 2002 .

[29]  D. Flinn.  Orientation Statistics , 1967, Nature.

[30]  V. Pawlowsky-Glahn,et al.  Compositional data analysis : theory and applications , 2011 .

[31]  Alexander J. Smola,et al.  Hilbert space embeddings of conditional distributions with applications to dynamical systems , 2009, ICML '09.

[32]  Christophe Ley,et al.  Modern Directional Statistics , 2017 .

[33]  Huiling Le,et al.  A diffusion approach to Stein's method on Riemannian manifolds , 2020, 2003.11497.

[34]  E. Miller,et al.  TORUS GRAPHS FOR MULTIVARIATE PHASE COUPLING ANALYSIS. , 2019, The annals of applied statistics.

[35]  P. Jupp.  Data-driven Sobolev tests of uniformity on compact Riemannian manifolds , 2008, 0806.2939.

[36]  K. Mardia,et al.  Score matching estimators for directional distributions , 2016, 1604.08470.

[37]  Bernhard Schölkopf,et al.  Kernel Mean Embedding of Distributions: A Review and Beyonds , 2016, Found. Trends Mach. Learn..

[38]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[39]  HyvärinenAapo,et al.  Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics , 2012 .

[40]  Alexander J. Smola,et al.  Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy , 2016, ICLR.

[41]  Tim Schmitz,et al.  The Foundations Of Differential Geometry , 2016 .

[42]  K. Nomizu,et al.  Foundations of Differential Geometry, Volume I. , 1965 .

[43]  Vinayak A. Rao,et al.  A Stein-Papangelou Goodness-of-Fit Test for Point Processes , 2019, AISTATS.

[44]  Arthur Gretton,et al.  Kernelized Stein Discrepancy Tests of Goodness-of-fit for Time-to-Event Data , 2020, ICML.

[45]  M. Girolami,et al.  A Riemannian-Stein Kernel method , 2018 .

[46]  Zoubin Ghahramani,et al.  Statistical Model Criticism using Kernel Two Sample Tests , 2015, NIPS.

[47]  K SriperumbudurBharath,et al.  Universality, Characteristic Kernels and RKHS Embedding of Measures , 2011 .

[48]  Peter E. Jupp,et al.  A test of uniformity on shape spaces , 2004 .

[49]  Tianqi Chen,et al.  A Complete Recipe for Stochastic Gradient MCMC , 2015, NIPS.

[50]  P. Jupp,et al.  Measures of goodness of fit obtained by canonical transformations on Riemannian manifolds , 2018, 1811.04866.

[51]  Y. Chikuse.  Statistics on special manifolds , 2003 .

[52]  M. Giné,et al.  Invariant Tests for Uniformity on Compact Riemannian Manifolds Based on Sobolev Norms , 1975 .

[53]  P. Jupp.  Sobolev tests of goodness of fit of distributions on compact Riemannian manifolds , 2005, math/0603135.

[54]  Y. Chikuse.  Concentrated matrix Langevin distributions , 2003 .

[55]  Aapo Hyvärinen,et al.  Estimation of Non-Normalized Statistical Models by Score Matching , 2005, J. Mach. Learn. Res..

[56]  Masatoshi Uehara,et al.  Imputation estimators for unnormalized models with missing data , 2019, AISTATS.

[57]  R. R. Bahadur.  Stochastic comparison of tests , 1960 .

[58]  Zaïd Harchaoui,et al.  A Fast, Consistent Kernel Two-Sample Test , 2009, NIPS.

[59]  Martin Raič,et al.  Normal Approximation by Stein ’ s Method , 2003 .

[60]  Christopher K. I. Williams,et al.  Model Criticism in Latent Space , 2017, Bayesian Analysis.

[61]  Bernhard Schölkopf,et al.  Informative Features for Model Comparison , 2018, NeurIPS.

[62]  Sivaraman Balakrishnan,et al.  Optimal kernel choice for large-scale two-sample tests , 2012, NIPS.

[63]  Arthur Gretton,et al.  Interpretable Distribution Features with Maximum Testing Power , 2016, NIPS.

[64]  Estimating Density Models with Complex Truncation Boundaries , 2019, ArXiv.

[65]  Lester W. Mackey,et al.  Measuring Sample Quality with Stein's Method , 2015, NIPS.