Bayesian inference in processing experimental data: principles and basic applications

This paper introduces general ideas and some basic methods of the Bayesian probability theory applied to physics measurements. Our aim is to make the reader familiar, through examples rather than rigorous formalism, with concepts such as the following: model comparison (including the automatic Ockham's Razor filter provided by the Bayesian approach); parametric inference; quantification of the uncertainty about the value of physical quantities, also taking into account systematic effects; role of marginalization; posterior characterization; predictive distributions; hierarchical modelling and hyperparameters; Gaussian approximation of the posterior and recovery of conventional methods, especially maximum likelihood and chi-square fits under well-defined conditions; conjugate priors, transformation invariance and maximum entropy motivated priors; and Monte Carlo (MC) estimates of expectation, including a short introduction to Markov Chain MC methods.

[1]  Jeffrey D. Scargle,et al.  Statistical challenges in modern astronomy II , 1997 .

[2]  P. Astone,et al.  Bayesian model comparison applied to the Explorer–Nautilus 2001 coincidence data , 2003, gr-qc/0304096.

[3]  W. T. Grandy.  Maximum entropy in action: Buck, Brian and Macaulay, Vincent A., 1991, 220 pp., Clarendon Press, Oxford, £30 pb, ISBN 0-19-8539630 , 1995 .

[4]  G. D'Agostini,et al.  On the Higgs boson mass from direct searches and precision measurements , 1999 .

[5]  J. Skilling.  Quantified Maximum Entropy , 1990 .

[6]  R. Cousins.  Why isn’t every physicist a Bayesian? , 1995 .

[7]  Christian P. Robert,et al.  The Bayesian choice , 1994 .

[8]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[9]  J. Berger,et al.  The application of robust Bayesian analysis to hypothesis testing and Occam's Razor , 1992 .

[10]  Peter Urbach,et al.  Scientific Reasoning: The Bayesian Approach , 1989 .

[11]  Dave Higdon,et al.  Estimation of the Head Sensitivity Function in Scanning Magnetoresistance Microscopy , 2001 .

[12]  Refik Soyer,et al.  Bayesian Methods for Nonlinear Classification and Regression , 2004, Technometrics.

[13]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[14]  Volker Dose,et al.  Energy resolution enhancement in ion beam experiments with Bayesian probability theory , 1998 .

[15]  David H. Sharp,et al.  Prediction and the quantification of uncertainty , 1999 .

[16]  G. D'Agostini,et al.  Overcoming priors anxiety , 1999 .

[17]  B. Efron.  Why Isn't Everyone a Bayesian? , 1986 .

[18]  Gutti Jogesh Babu,et al.  Statistical Challenges in Modern Astronomy , 1992 .

[19]  G. D'Agostini.  Sceptical combination of experimental results: General considerations and application to epsilon-prime/epsilon , 1999 .

[20]  C. Guidorzi,et al.  Search for correlation between GRB's detected by BeppoSAX and gravitational wave detectors EXPLORER and NAUTILUS , 2002 .

[21]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[22]  Volker Dose,et al.  Outlier Tolerant Parameter Estimation , 1999 .

[23]  M. Ciuchini,et al.  2000 CKM-triangle analysis a critical review with updated experimental inputs and theoretical parameters , 2000, hep-ph/0012308.

[24]  M. Tribus.  Rational descriptions, decisions, and designs , 1969 .

[25]  Christopher Holmes,et al.  Bayesian Methods for Nonlinear Classification and Regressing , 2002 .

[26]  G. D'Agostini.  Teaching statistics in the physics curriculum: Unifying and clarifying role of subjective probability , 1999 .

[27]  K. Hanson,et al.  Three-dimensional reconstructions from low-count SPECT data using deformable models with smooth interior intensity variations , 1998 .

[28]  P. Astone,et al.  Inferring the intensity of Poisson processes at the limit of the detector sensitivity (with a case study on gravitational wave burst search) , 1999, hep-ex/9909047.

[29]  Anthony O'Hagan,et al.  Kendall's Advanced Theory of Statistics: Vol. 2B, Bayesian Inference. , 1996 .

[30]  G C Dean,et al.  An Introduction to Kalman Filters , 1986 .

[31]  H. Jeffreys,et al.  Theory of probability , 1896 .

[32]  E. Jaynes.  The well-posed problem , 1973 .

[33]  Ken D. Sauer,et al.  A generalized Gaussian image model for edge-preserving MAP estimation , 1993, IEEE Trans. Image Process..

[34]  Thomas J. Loredo,et al.  Bayesian analysis of neutrinos observed from supernova SN-1987A , 2002 .

[35]  Volker Dose,et al.  Enhancement of the energy resolution in ion-beam experimentswith the maximum-entropy method , 1997 .

[36]  W. Jefferys.  Sharpening Ockham ' s Razor on a Bayesian Strop ( Key terms : Bayes ' theorem ; Ockham ' s razor ) , 1991 .

[37]  E. T. Jaynes,et al.  Papers on probability, statistics and statistical physics , 1983 .

[38]  S. James Press,et al.  Subjective and objective Bayesian statistics : principles, models, and applications , 2003 .

[39]  P. Diaconis.  Bayesian Numerical Analysis , 1988 .

[40]  P. C. Gregory,et al.  Bayesian Periodic Signal Detection. I. Analysis of 20 Years of Radio Flux Measurements of the X-Ray Binary LS I +61°303 , 1999 .

[41]  A. Wald,et al.  Probability, statistics and truth , 1939 .

[42]  Barry N. Taylor,et al.  Guidelines for Evaluating and Expressing the Uncertainty of Nist Measurement Results , 2017 .

[43]  Volker Dose,et al.  Spline-based adaptive resolution image reconstruction , 1996 .

[44]  A. O'Hagan,et al.  Kendall's Advanced Theory of Statistics, Vol. 2b: Bayesian Inference. , 1996 .

[45]  D Malakoff,et al.  Bayes Offers a 'New' Way to Make Sense of Numbers , 1999, Science.

[46]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[47]  Kenneth M. Hanson,et al.  Introduction to Bayesian image analysis , 1993 .

[48]  D. J. M. Kester,et al.  Pyramid maximum entropy images of IRAS survey data. , 1994 .

[49]  E. Iso,et al.  Measurement Uncertainty and Probability: Guide to the Expression of Uncertainty in Measurement , 1995 .

[50]  Günter Zech.  Frequentist and Bayesian confidence intervals , 2001 .

[51]  F. Lad.  Operational Subjective Statistical Methods: A Mathematical, Philosophical, and Historical Introduction , 1996 .

[52]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[53]  T. Loredo,et al.  A new method for the detection of a periodic signal of unknown shape and period , 1992 .

[54]  John Skilling,et al.  Data analysis : a Bayesian tutorial , 1996 .

[55]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[56]  R. Scozzafava,et al.  Probabilistic Logic in a Coherent Setting , 2002 .

[57]  G. D'Agostini,et al.  Constraints on the Higgs boson mass from direct searches and precision measurements , 1999 .

[58]  G. D'Agostini,et al.  Confidence limits: what is the problem? Is there the solution? , 2000, hep-ex/0002055.

[59]  P. C. Gregory,et al.  Bayesian Periodic Signal Detection: Analysis of ROSAT Observations of PSR 0540-693 , 1996 .

[60]  G. D'Agostini.  Bayesian reasoning versus conventional statistics in High Energy Physics , 1998 .

[61]  G. D'Agostini.  Minimum bias legacy of search results , 2002 .

[62]  K. Hanson,et al.  Three dimensional reconstructions from low-count SPECT data using deformable models , 1997, 1997 IEEE Nuclear Science Symposium Conference Record.

[63]  G. D'Agostini,et al.  Uncertainties due to imperfect knowledge of systematic effects: general considerations and approximate formulae , 2000 .

[64]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo in Practice: A Roundtable Discussion , 1998 .

[65]  A. Lewis,et al.  Cosmological parameters from CMB and other data: A Monte Carlo approach , 2002, astro-ph/0205436.

[66]  L. Joseph,et al.  Bayesian Statistics: An Introduction , 1989 .

[67]  Kenneth M. Hanson,et al.  Model-based image reconstruction from time-resolved diffusion data , 1997, Medical Imaging.

[68]  Marvin H. J. Guber.  Bayesian Spectrum Analysis and Parameter Estimation , 1988 .

[69]  B. M. Hill,et al.  Theory of Probability , 1990 .

[70]  Giulio D'Agostini,et al.  BAYESIAN REASONING IN HIGH-ENERGY PHYSICS: PRINCIPLES AND APPLICATIONS , 1999 .

[71]  W. von der Linden.  Maximum-entropy data analysis , 1995 .

[72]  R. T. Cox.  Probability, frequency and reasonable expectation , 1990 .

[73]  M. P. Hobson,et al.  Combining cosmological data sets: hyperparameters and Bayesian evidence , 2002 .

[74]  J. V. Narlikar,et al.  Comparison of cosmological models using Bayesian theory , 2002 .

[75]  Jarrell,et al.  Quantum Monte Carlo simulations and maximum entropy: Dynamics from imaginary-time data. , 1991, Physical review. B, Condensed matter.

[76]  D. V. Lindley,et al.  [Why Isn't Everyone a Bayesian?]: Comment , 1986 .

[77]  D. Ankerst,et al.  Kendall's Advanced Theory of Statistics, Vol. 2B: Bayesian Inference , 2005 .

[78]  S. James Press,et al.  Subjective and Objective Bayesian Statistics , 2002 .

[79]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[80]  D. Schum,et al.  A Probabilistic Analysis of the Sacco and Vanzetti Evidence , 1996 .

[81]  William H. Press.  Understanding Data Better with Bayesian and Global Statistical Methods , 1996 .

[82]  G. Zech,et al.  Frequentist and Bayesian Confidence Limits , 2001 .

[83]  Jose M. Bernardo.  Non-informative priors do not exist A dialogue with , 2003 .

[84]  V. Dose,et al.  Background estimation in experimental spectra , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[85]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.