Learning Functions and Approximate Bayesian Computation Design: ABCD

A general approach to Bayesian learning revisits some classical results, which study which functionals on a prior distribution are expected to increase, in a preposterior sense. The results are applied to information functionals of the Shannon type and to a class of functionals based on expected distance. A close connection is made between the latter and a metric embedding theory due to Schoenberg and others. For the Shannon type, there is a connection to majorization theory for distributions. A computational method is described to solve generalized optimal experimental design problems arising from the learning framework based on a version of the well-known approximate Bayesian computation (ABC) method for carrying out the Bayesian analysis based on Monte Carlo simulation. Some simple examples are given.

[1]  D. Lindley.  On a Measure of the Information Provided by an Experiment , 1956 .

[2]  Josh H. McDermott.  The cocktail party problem , 2009, Current Biology.

[3]  J. Gower.  Some distance properties of latent root and vector methods used in multivariate analysis , 1966 .

[4]  J. Berger.  The case for objective Bayesian analysis , 2006 .

[5]  D. Blackwell.  Comparison of Experiments , 1951 .

[6]  J. V. Ryff.  Orbits of ¹-functions under doubly stochastic transformations , 1965 .

[7]  A. Goldman,et al.  Results on inquiry and truth possession , 1991 .

[8]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[9]  A. Rényi.  On Measures of Entropy and Information , 1961 .

[10]  K. Chaloner,et al.  Bayesian Experimental Design: A Review , 1995 .

[11]  Anthony N Pettitt,et al.  Bayesian Experimental Design for Models with Intractable Likelihoods , 2013, Biometrics.

[12]  Markus Hainy,et al.  Approximate Bayesian Computation Design (ABCD), an Introduction , 2013 .

[13]  Paola Sebastiani,et al.  Coherent dispersion criteria for optimal experimental design , 1999 .

[14]  N. Chater,et al.  The Probability Heuristics Model of Syllogistic Reasoning , 1999, Cognitive Psychology.

[15]  J. Gower.  Euclidean Distance Geometry , 1982 .

[16]  Paul Marjoram,et al.  Markov chain Monte Carlo without likelihoods , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[17]  M. Degroot,et al.  Comparison of Experiments and Information Measures , 1979 .

[18]  Luc Pronzato,et al.  Beyond space-filling: An illustrative case , 2011 .

[19]  Mark A. McComb.  Comparison Methods for Stochastic Models and Risks , 2003, Technometrics.

[20]  M. Degroot,et al.  Comparing Probability Forecasters: Basic Binary Concepts and Multivariate Extensions , 1983 .

[21]  Josep Ginebra,et al.  On the measure of the information in a statistical experiment , 2007 .

[22]  C. Tsallis.  Possible generalization of Boltzmann-Gibbs statistics , 1988 .

[23]  G. Crooks.  On Measures of Entropy and Information , 2015 .

[24]  Joseph L. Zinnes,et al.  Theory and Methods of Scaling. , 1958 .

[25]  I. J. Schoenberg,et al.  Metric spaces and positive definite functions , 1938 .

[26]  Helga Wagner,et al.  Likelihood-free Simulation-based Optimal Design , 2013 .

[27]  H. Wynn,et al.  Maximum entropy sampling and optimal Bayesian experimental design , 2000 .

[28]  D. Fallis,et al.  Further results on inquiry and truth possession , 2002 .

[29]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .