A Kernel Test of Goodness of Fit

We propose a nonparametric statistical test for goodness-of-fit: given a set of samples, the test determines how likely it is that these were generated from a target density function. The measure of goodness-of-fit is a divergence constructed via Stein's method using functions from a Reproducing Kernel Hilbert Space. Our test statistic is based on an empirical estimate of this divergence, taking the form of a V-statistic in terms of the log gradients of the target density and the kernel. We derive a statistical test, both for i.i.d. and non-i.i.d. samples, where we estimate the null distribution quantiles using a wild bootstrap procedure. We apply our test to quantifying convergence of approximate Markov Chain Monte Carlo methods, statistical model criticism, and evaluating quality of fit vs model complexity in nonparametric density estimation.

[1]  AN Kolmogorov-Smirnov,et al.  Sulla determinazione empírica di uma legge di distribuzione , 1933 .

[2]  N. Smirnov.  Table for Estimating the Goodness of Fit of Empirical Distributions , 1948 .

[3]  C. Stein.  A bound for the error in the normal approximation to the distribution of a sum of dependent random variables , 1972 .

[4]  R. Serfling.  Approximation Theorems of Mathematical Statistics , 1980 .

[5]  E. B. Andersen,et al.  Information Science and Statistics , 1986 .

[6]  N. Henze,et al.  A consistent test for multivariate normality based on the empirical characteristic function , 1988 .

[7]  A. Barron.  Uniformly Powerful Goodness of Fit Tests , 1989 .

[8]  L. Györfi,et al.  A Consistent Goodness of Fit Test Based on the Total Variation Distance , 1991 .

[9]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[10]  A. Bowman,et al.  Adaptive Smoothing and Density-Based Tests of Multivariate Normality , 1993 .

[11]  N. H. Anderson,et al.  Two-sample test statistics for measuring discrepancies between two multivariate probability density functions using kernel-based density estimates , 1994 .

[12]  G. Lugosi,et al.  On the asymptotic normality of the L1‐ and L2‐errors in histogram density estimation , 1994 .

[13]  R. Zamar,et al.  A multivariate Kolmogorov-Smirnov test of goodness of fit , 1997 .

[14]  A. Müller.  Integral Probability Metrics and Their Generating Classes of Functions , 1997, Advances in Applied Probability.

[15]  L. Györfi,et al.  Asymptotic distributions for goodness-of-fit statistics in a sequence of multinomial models , 2002 .

[16]  J. Dedecker,et al.  New dependence coefficients. Examples and applications to statistics , 2005 .

[17]  R. C. Bradley.  Basic properties of strong mixing conditions. A survey and some open questions , 2005, math/0511078.

[18]  Maria L. Rizzo,et al.  A new test for multivariate normality , 2005 .

[19]  Le Song,et al.  A Kernel Statistical Test of Independence , 2007, NIPS.

[20]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[21]  P. Doukhan,et al.  Weak Dependence: With Examples and Applications , 2007 .

[22]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[23]  C. Carmeli,et al.  Vector valued reproducing kernel Hilbert spaces and universality , 2008, 0807.1659.

[24]  Maria L. Rizzo,et al.  New Goodness-of-Fit Tests for Pareto Distributions , 2009 .

[25]  X. Shao,et al.  The Dependent Wild Bootstrap , 2010 .

[26]  Arthur Gretton,et al.  Consistent Nonparametric Tests of Independence , 2010, J. Mach. Learn. Res..

[27]  Bernhard Schölkopf,et al.  Hilbert Space Embeddings and Metrics on Probability Measures , 2009, J. Mach. Learn. Res..

[28]  Yee Whye Teh,et al.  Bayesian Learning via Stochastic Gradient Langevin Dynamics , 2011, ICML.

[29]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[30]  A. Leucht,et al.  Degenerate U- and V-statistics under weak dependence : Asymptotic theory and bootstrap consistency , 2012, 1205.1892.

[31]  Matthieu Lerasle,et al.  Kernels Based Tests with Non-asymptotic Bootstrap Approaches for Two-sample Problems , 2012, COLT.

[32]  Kenji Fukumizu,et al.  Equivalence of distance-based and RKHS-based statistics in hypothesis testing , 2012, ArXiv.

[33]  Max Welling,et al.  Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget , 2013, ICML 2014.

[34]  Anne Leucht,et al.  Dependent wild bootstrap for degenerate U- and V-statistics , 2013, J. Multivar. Anal..

[35]  Arthur Gretton,et al.  A Kernel Independence Test for Random Processes , 2014, ICML.

[36]  N. Chopin,et al.  Control functionals for Monte Carlo integration , 2014, 1410.2392.

[37]  Arnaud Doucet,et al.  Towards scaling up Markov chain Monte Carlo: an adaptive subsampling approach , 2014, ICML.

[38]  Arthur Gretton,et al.  A Wild Bootstrap for Degenerate Kernel Tests , 2014, NIPS.

[39]  Arthur Gretton,et al.  Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families , 2015, NIPS.

[40]  Lester W. Mackey,et al.  Measuring Sample Quality with Stein's Method , 2015, NIPS.

[41]  Zoubin Ghahramani,et al.  Statistical Model Criticism using Kernel Two Sample Tests , 2015, NIPS.

[42]  Arthur Gretton,et al.  Fast Two-Sample Testing with Analytic Representations of Probability Measures , 2015, NIPS.

[43]  Qiang Liu,et al.  A Kernelized Stein Discrepancy for Goodness-of-fit Tests , 2016, ICML.

[44]  Aapo Hyvärinen,et al.  Density Estimation in Infinite Dimensional Exponential Families , 2013, J. Mach. Learn. Res..