New insights into Approximate Bayesian Computation

Approximate Bayesian Computation (ABC for short) is a family of computational techniques which offer an almost automated solution in situations where evaluation of the posterior likelihood is computationally prohibitive, or whenever suitable likelihoods are not available. In the present paper, we analyze the procedure from the point of view of k-nearest neighbor theory and explore the statistical properties of its outputs. We discuss in particular some asymptotic features of the genuine conditional density estimate associated with ABC, which is an interesting hybrid between a k-nearest neighbor and a kernel method.

[1]  M. Blum.  Approximate Bayesian Computation: A Nonparametric Perspective , 2009, 0904.0635.

[2]  J. Koláček,et al.  Nonparametric Conditional Density Estimation , 2013 .

[3]  Christian P Robert,et al.  Lack of confidence in approximate Bayesian computation model choice , 2011, Proceedings of the National Academy of Sciences.

[4]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[5]  P. Donnelly,et al.  Inferring coalescence times from DNA sequence data. , 1997, Genetics.

[6]  D. Rubin.  Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician , 1984 .

[7]  Olivier P. Faugeras,et al.  A quantile-copula approach to conditional density estimation , 2007, J. Multivar. Anal..

[8]  G. S. Watson,et al.  Smooth regression analysis , 1964 .

[9]  Arnaud Guyader,et al.  Nearest neighbor classification in infinite dimension , 2006 .

[10]  Rob J. Hyndman,et al.  Bandwidth selection for kernel conditional density estimation , 2001 .

[11]  Thomas M. Cover,et al.  Estimation by the nearest neighbor rule , 1968, IEEE Trans. Inf. Theory.

[12]  László Györfi,et al.  Nonparametric Estimation of Conditional Distributions , 2007, IEEE Transactions on Information Theory.

[13]  E. Nadaraya.  On Estimating Regression , 1964 .

[14]  W. Li,et al.  Estimating the age of the common ancestor of a sample of DNA sequences. , 1997, Molecular biology and evolution.

[15]  Jianqing Fan,et al.  A crossvalidation method for estimating conditional densities , 2004 .

[16]  Jeffrey S. Racine,et al.  Cross-Validation and the Estimation of Conditional Probability Densities , 2004 .

[17]  R. Wilkinson.  Approximate Bayesian computation (ABC) gives exact results under the assumption of model error , 2008, Statistical applications in genetics and molecular biology.

[18]  Paul Marjoram,et al.  Statistical Applications in Genetics and Molecular Biology Approximately Sufficient Statistics and Bayesian Computation , 2011 .

[19]  J. Yackel,et al.  Large Sample Properties of Nearest Neighbor Density Function Estimators , 1977 .

[20]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[21]  J. Cooper.  SINGULAR INTEGRALS AND DIFFERENTIABILITY PROPERTIES OF FUNCTIONS , 1973 .

[22]  D. Balding,et al.  Approximate Bayesian computation in population genetics. , 2002, Genetics.

[23]  R. Fefferman.  Review: Miguel de Guzmán, Differentiation of integrals in $R^n$ , 1977 .

[24]  Adam Krzyzak,et al.  New Multivariate Product Density Estimators , 2002 .

[25]  A. Zygmund,et al.  Measure and integral : an introduction to real analysis , 1977 .

[26]  L. Breiman,et al.  Variable Kernel Estimates of Multivariate Densities , 1977 .

[27]  J. Yackel,et al.  Consistency Properties of Nearest Neighbor Density Function Estimators , 1977 .

[28]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[29]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[30]  Brian D. Ripley,et al.  Stochastic Simulation , 2005 .

[31]  Rolf-Dieter Reiss,et al.  On conditional distributions of nearest neighbors , 1992 .

[32]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[33]  T. Broadbent.  Measure and Integral , 1957, Nature.

[34]  M. C. Jones.  Variable kernel density estimates and variable kernel density estimates , 1990 .

[35]  R. Jackson.  Inequalities , 2007, Algebra for Parents.

[36]  M. Feldman,et al.  Population growth of human Y chromosomes: a study of Y chromosome microsatellites. , 1999, Molecular biology and evolution.

[37]  Ian Abramson.  On Bandwidth Variation in Kernel Estimates-A Square Root Law , 1982 .

[38]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[39]  Rob J Hyndman,et al.  Estimating and Visualizing Conditional Densities , 1996 .

[40]  Antoni Zygmund,et al.  Note on the differentiability of multiple integrals , 1934 .

[41]  Jean-Michel Marin,et al.  Bayesian Core: A Practical Approach to Computational Bayesian Statistics , 2010 .

[42]  Paul Fearnhead,et al.  Constructing summary statistics for approximate Bayesian computation: semi‐automatic approximate Bayesian computation , 2012 .

[43]  M. Rosenblatt,et al.  Multivariate k-nearest neighbor density estimates , 1979 .

[44]  Mark M. Tanaka,et al.  Sequential Monte Carlo without likelihoods , 2007, Proceedings of the National Academy of Sciences.

[45]  L. Devroye.  Necessary and sufficient conditions for the pointwise convergence of nearest neighbor regression function estimates , 1982 .

[46]  C. Quesenberry,et al.  A nonparametric estimate of a multivariate density function , 1965 .

[47]  Arnaud Guyader,et al.  On the Rate of Convergence of the Bagged Nearest Neighbor Estimate , 2010, J. Mach. Learn. Res..

[48]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[49]  W. Marsden.  I and J , 2012 .

[50]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[51]  P. Heywood.  Trigonometric Series , 1968, Nature.

[52]  J.-M. Marin,et al.  Relevant statistics for Bayesian model choice , 2011, 1110.4700.

[53]  Jean-Michel Marin,et al.  Approximate Bayesian computational methods , 2011, Statistics and Computing.

[54]  E. Nadaraya.  On Non-Parametric Estimates of Density Functions and Regression Curves , 1965 .

[55]  James Stephen Marron,et al.  Variable window width kernel estimates of probability densities , 1992 .