Bayesian Probability Theory: Applications in the Physical Sciences

From the basics to the forefront of modern research, this book presents all aspects of probability theory, statistics and data analysis from a Bayesian perspective for physicists and engineers. The book presents the roots, applications and numerical implementation of probability theory, and covers advanced topics such as maximum entropy distributions, stochastic processes, parameter estimation, model selection, hypothesis testing and experimental design. In addition, it explores state-of-the art numerical techniques required to solve demanding real-world problems. The book is ideal for students and researchers in physical sciences and engineering.

[1]  Donath,et al.  Unbiased access to exchange splitting of magnetic bands using the maximum entropy method. , 1993, Physical review letters.

[2]  L. Ballentine.  Quantum mechanics : a modern development , 1998 .

[3]  Ulrich Müller-Funk,et al.  The value of the last digit: statistical fraud detection with digit analysis , 2009, Adv. Data Anal. Classif..

[4]  S. Luttrell.  The use of transinformation in the design of data sampling schemes for inverse problems , 1985 .

[5]  R. T. Cox.  The Algebra of Probable Inference , 1962 .

[6]  T. Bayes.  An essay towards solving a problem in the doctrine of chances , 2003 .

[7]  Kevin H. Knuth,et al.  Designing Intelligent Instruments , 2007, ArXiv.

[8]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[9]  F. Hiai,et al.  The semicircle law, free random variables, and entropy , 2006 .

[10]  H. Wynn,et al.  Maximum entropy sampling and optimal Bayesian experimental design , 2000 .

[11]  Rory A. Fisher,et al.  Statistical methods and scientific inference. , 1957 .

[12]  André I. Khuri,et al.  Response surface methodology: 1966–1988 , 1989 .

[13]  Daniele Passeri,et al.  Energy loss measurement for charged particles in very thin silicon layers , 2011 .

[14]  Y Fukuda,et al.  Appearance Potential Spectroscopy (APS): Old Method, but Applicable to Study of Nano-structures , 2010, Analytical sciences : the international journal of the Japan Society for Analytical Chemistry.

[15]  Ronald L. Wasserstein,et al.  Monte Carlo: Concepts, Algorithms, and Applications , 1997 .

[16]  R. Bellman.  Dynamic programming. , 1957, Science.

[17]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[18]  F. Pukelsheim.  Optimal Design of Experiments , 1993 .

[19]  G. L. Bretthorst.  Bayesian analysis. I. Parameter estimation using quadrature NMR models , 1990 .

[20]  Wolfgang von der Linden,et al.  A quantum Monte Carlo approach to many-body physics , 1992 .

[21]  Ning Xiang,et al.  Using Nested Sampling in the Analysis of Multi‐Rate Sound Energy Decay in Acoustically Coupled Rooms , 2005 .

[22]  H. Jeffreys,et al.  Theory of probability , 1896 .

[23]  E. Jaynes.  The well-posed problem , 1973 .

[24]  S. Goodman.  Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy , 1999, Annals of Internal Medicine.

[25]  Volker Dose.  Bayesian estimate of the Newtonian constant of gravitation , 2007 .

[26]  A. Saltelli,et al.  Sensitivity Anaysis as an Ingredient of Modeling , 2000 .

[27]  Frank Kelly,et al.  Reversibility and Stochastic Networks , 1979 .

[28]  Kevin H. Knuth,et al.  Foundations of Inference , 2010, Axioms.

[29]  Donald A. Berry,et al.  Simulation-based sequential Bayesian design , 2007 .

[30]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[31]  David Lindley,et al.  Bayesian Statistics, a Review , 1987 .

[32]  Volker Dose,et al.  A Bayesian approach to the global confinement time scaling in W7-AS , 1996 .

[33]  Eleftherios N. Economou,et al.  Green's functions in quantum physics , 1979 .

[34]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[35]  John Skilling,et al.  Bayesian maximum entropy image reconstruction , 1991 .

[36]  John N. Tsitsiklis,et al.  Introduction to Probability , 2002 .

[37]  Hanke,et al.  Quasiparticle dispersion of the 2D Hubbard model: From an insulator to a metal. , 1995, Physical review letters.

[38]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[39]  Ch. Linsmeier,et al.  Auger electron spectroscopy , 1994 .

[40]  John Skilling,et al.  Data analysis : a Bayesian tutorial , 1996 .

[41]  M. Degroot.  Uncertainty, Information, and Sequential Experiments , 1962 .

[42]  Volker Dose,et al.  The use of maximum entropy and Bayesian techniques in nuclear microprobe applications , 1997 .

[43]  A. Galle.  Über die Geodätischen Arbeiten von Gauss , 1924 .

[44]  S. Gull,et al.  Image reconstruction from incomplete and noisy data , 1978, Nature.

[45]  Paul M. Goggans,et al.  Electromagnetic Induction Landmine Detection Using Bayesian Model Comparison , 2006 .

[46]  John S. J. Hsu,et al.  Bayesian Methods: An Analysis for Statisticians and Interdisciplinary Researchers , 1999 .

[47]  Volker Dose,et al.  Improved resolution in HREELS using maximum-entropy deconvolution: CO on PtxNi1−x(111) , 1997 .

[48]  P. Prescott,et al.  Monte Carlo Methods , 1964, Computational Statistical Physics.

[49]  Volker Dose,et al.  Tokamak Edge Profile Analysis Employing Bayesian Statistics , 2001 .

[50]  T Morita,et al.  Note on the lattice Green's function for the simple cubic lattice , 1975 .

[51]  Stefano Tarantola,et al.  Sensitivity Analysis as an Ingredient of Modeling , 2000 .

[52]  Barry N. Taylor,et al.  THE 1986 ADJUSTMENT OF THE FUNDAMENTAL PHYSICAL CONSTANTS: A REPORT OF THE CODATA TASK GROUP ON FUNDAMENTAL CONSTANTS , 1987 .

[53]  Thomas J. Loredo,et al.  Computational Technology for Bayesian Inference , 2010 .

[54]  G. Parisi,et al.  Simulated tempering: a new Monte Carlo scheme , 1992, hep-lat/9205018.

[55]  Michael I. Jordan,et al.  PEGASUS: A policy search method for large MDPs and POMDPs , 2000, UAI.

[56]  H. Dreier,et al.  Data adaptive control parameter estimation for scaling laws for magnetic fusion devices , 2008 .

[57]  G. Dueck.  New optimization heuristics , 1993 .

[58]  P. Laplace.  A Philosophical Essay On Probabilities , 1902 .

[59]  Philip C. Gregory,et al.  Bayesian Logical Data Analysis for the Physical Sciences: Acknowledgements , 2005 .

[60]  Luc Pronzato,et al.  Sequential Design and Active Control , 1998 .

[61]  J. Bernardo.  Expected Information as Expected Utility , 1979 .

[62]  G G Emch,et al.  On Klauder's Path: a Field Trip , 1994 .

[63]  Wolfhard Janke,et al.  Make Life Simple: Unleash the Full Power of the Parallel Tempering Algorithm , 2008 .

[64]  Volker Dose,et al.  Dimensionally exact form-free energy confinement scaling in W7-AS , 1999 .

[65]  Rainer Fischer,et al.  Bayesian Experimental Design — Studies for Fusion Diagnostics , 2004 .

[66]  P. Müller,et al.  Optimal Bayesian Design by Inhomogeneous Markov Chain Simulation , 2004 .

[67]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[68]  E. Lehmann.  The Fisher, Neyman-Pearson Theories of Testing Hypotheses: One Theory or Two? , 1993 .

[69]  D. S. Sivia,et al.  A Bayesian approach to extracting structure‐factor amplitudes from powder diffraction data , 1994 .

[70]  G. Larry Bretthorst,et al.  Physics and Probability: On the Difference in Means , 1993 .

[71]  Patricia Pepple Williamson,et al.  Detecting Fraud in Data Sets Using Benford's Law , 2004 .

[72]  Radford M. Neal.  Regression and Classification Using Gaussian Process Priors , 2009 .

[73]  W. Jacob,et al.  Surface reactions during growth and erosion of hydrocarbon films , 1998 .

[74]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[75]  Hanke,et al.  Spectral properties of the one-dimensional Hubbard model. , 1994, Physical review letters.

[76]  A. O'Hagan,et al.  Probabilistic sensitivity analysis of complex models: a Bayesian approach , 2004 .

[77]  Blaza Toman,et al.  Bayesian Experimental Design , 2006 .

[78]  D. R. Wolf,et al.  Alpha, Evidence, and the Entropic Prior , 1993 .

[79]  J. N. Kapur,et al.  Entropy optimization principles with applications , 1992 .

[80]  R. Mattuck,et al.  A guide to Feynman diagrams in the many-body problem , 1976 .

[81]  M. Stone.  Application of a Measure of Information to the Design and Comparison of Regression Experiments , 1959 .

[82]  R. Fisher.  Statistical methods for research workers , 1927, Protoplasma.

[83]  A. Garrett,et al.  Ockham’s Razor , 1991 .

[84]  Joseph B. Kadane,et al.  A Gridding Method for Bayesian Sequential Decision Problems , 2003 .

[85]  Volker Dose,et al.  Evaluation of chemical erosion data for carbon materials at high ion fluxes using Bayesian probability theory , 2001 .

[86]  W von der Linden,et al.  Analysis of multicomponent mass spectra applying Bayesian probability theory. , 2001, Journal of mass spectrometry : JMS.

[87]  Irene A. Stegun,et al.  Handbook of Mathematical Functions. , 1966 .

[88]  T. J. Mitchell,et al.  Bayesian Prediction of Deterministic Functions, with Applications to the Design and Analysis of Computer Experiments , 1991 .

[89]  H. Dreier,et al.  Bayesian experimental design of a multichannel interferometer for Wendelstein 7-X. , 2008, The Review of scientific instruments.

[90]  W. von der Linden,et al.  Perfect Tempering , 2005 .

[91]  Mark New,et al.  Surface air temperature and its changes over the past 150 years , 1999 .

[92]  W. Linden,et al.  Maximum entropy based reconstruction of soft X-ray emissivity profiles in W7-AS , 1996 .

[93]  H. F. Martz,et al.  Applications of quantum entropy to statistics , 1994 .

[94]  Edwin T. Jaynes.  Prior Probabilities , 2010, Encyclopedia of Machine Learning.

[95]  H. Dreier,et al.  Bayesian Design of Diagnostics: Case Studies for Wendelstein 7-X , 2006 .

[96]  John Odentrantz,et al.  Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues , 2000, Technometrics.

[97]  J. Dieudonne,et al.  Encyclopedic Dictionary of Mathematics , 1979 .

[98]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[99]  W. T. Grandy.  Maximum entropy in action: Buck, Brian and Macaulay, Vincent A., 1991, 220 pp., Clarendon Press, Oxford, £30 pb, ISBN 0-19-8539630 , 1995 .

[100]  Volker Dose,et al.  The inversion of autoconvolution integrals , 1981 .

[101]  Anthony C. Atkinson,et al.  Optimum Experimental Designs, with SAS , 2007 .

[102]  Rodney W. Johnson,et al.  Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy , 1980, IEEE Trans. Inf. Theory.

[103]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[104]  Flemming Besenbacher,et al.  A note on the 3He + D nuclear-reaction cross section , 1980 .

[105]  M. R. Haas,et al.  Kepler Mission Design, Realized Photometric Performance, and Early Science , 2010, 1001.0268.

[106]  David M. Borth,et al.  A Total Entropy Criterion for the Dual Problem of Model Discrimination and Parameter Estimation , 1975 .

[107]  R. T. Cox.  Probability, frequency and reasonable expectation , 1990 .

[108]  Lev Davidovich Landau,et al.  On the energy loss of fast particles by ionization , 1944 .

[109]  W. G. Hunter,et al.  Experimental Design: Review and Comment , 1984 .

[110]  M. Tribus.  Rational descriptions, decisions, and designs , 1969 .

[111]  J. Doob.  Stochastic processes , 1953 .

[112]  K. Hukushima,et al.  Exchange Monte Carlo Method and Application to Spin Glass Simulations , 1995, cond-mat/9512035.

[113]  G. Amsel,et al.  Nuclear Reaction Techniques in Materials Analysis , 1984 .

[114]  J. Skilling.  Nested Sampling’s Convergence , 2009 .

[115]  Luc Pronzato,et al.  Optimal experimental design and some related control problems , 2008, Autom..

[116]  H. Akaike.  A new look at the statistical model identification , 1974 .

[117]  G. Larry Bretthorst,et al.  Excerpts from Bayesian Spectrum Analysis and Parameter Estimation , 1988 .

[118]  Vijay P. Singh,et al.  Entropy Theory and its Application in Environmental and Water Engineering: Singh/Entropy Theory and its Application in Environmental and Water Engineering , 2013 .

[119]  Szu Hui Ng,et al.  Joint criterion for factor identification and parameter estimation , 2002, Proceedings of the Winter Simulation Conference.

[120]  Walter Dittrich,et al.  Classical and Quantum Dynamics , 1992 .

[121]  Håkan Hjalmarsson,et al.  From experiment design to closed-loop control , 2005, Autom..

[122]  J. Roth,et al.  Differential cross-section of the D(3He, p)4He nuclear reaction and depth profiling of deuterium up to large depths , 2005 .

[123]  U. Toussaint,et al.  Bayesian inference in physics , 2011 .

[124]  C. H. Skinner,et al.  Recent Advances on Hydrogen Retention in ITER’s Plasma-Facing Materials: Beryllium, Carbon, and Tungsten , 2008 .

[125]  E. Marinari,et al.  Optimized monte carlo methods , 1996, cond-mat/9612010.

[126]  D. Titterington.  General structure of regularization procedures in image reconstruction , 1985 .

[127]  Volker Dose,et al.  On the importance of α marginalization in maximum entropy , 1996 .

[128]  B. Taylor,et al.  CODATA recommended values of the fundamental physical constants: 2006 | NIST , 2007, 0801.0028.

[129]  B. Roy Frieden,et al.  Probability, Statistical Optics, And Data Testing , 1982 .

[130]  H. Dreier,et al.  Bayesian design of plasma diagnostics , 2006 .

[131]  Rainer Fischer,et al.  Integrated Bayesian Experimental Design , 2005 .

[132]  Marvin H. J. Guber.  Bayesian Spectrum Analysis and Parameter Estimation , 1988 .

[133]  W. von der Linden.  Maximum-entropy data analysis , 1995 .

[134]  G. Larry Bretthorst.  Generalizing the Lomb-Scargle periodogram , 2001 .

[135]  David W. Lewis,et al.  Matrix theory , 1991 .

[136]  R. Wolpert,et al.  Likelihood Principle , 2022, The SAGE Encyclopedia of Research Design.

[137]  K. Chaloner,et al.  Bayesian Experimental Design: A Review , 1995 .

[138]  R. A. Dory,et al.  SPECIAL TOPIC: Energy confinement scaling from the international stellarator database , 1995 .

[139]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[140]  William H. Press.  Understanding Data Better with Bayesian and Global Statistical Methods , 1996 .

[141]  V. Dose,et al.  Background estimation in experimental spectra , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[142]  D. Lindley.  On a Measure of the Information Provided by an Experiment , 1956 .

[143]  W von der Linden,et al.  Signal and background separation. , 1999, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[144]  David J. C. MacKay,et al.  Information-Based Objective Functions for Active Data Selection , 1992, Neural Computation.

[145]  J. Skilling.  Nested sampling for general Bayesian computation , 2006 .

[146]  R. Stanley.  What Is Enumerative Combinatorics , 1986 .

[147]  C. Murray,et al.  Solar System Dynamics: Expansion of the Disturbing Function , 1999 .

[148]  E. Jaynes.  Probability theory : the logic of science , 2003 .

[149]  William H. Press,et al.  Numerical recipes in C , 2002 .

[150]  K. Binder.  Monte Carlo methods in statistical physics , 1979 .

[151]  H. Chernoff.  Sequential Analysis and Optimal Design , 1987 .

[152]  Hanns Ludwig Harney.  Bayesian Inference: Parameter Estimation and Decisions , 2010 .

[153]  Rainer Fischer,et al.  Comparative studies to the design of the interferometer at W7-X with respect to technical boundary conditions , 2008 .

[154]  Stephen F. Gull,et al.  Bayesian Data Analysis: Straight-line fitting , 1989 .

[155]  K. J. Ryan,et al.  Estimating Expected Information Gains for Experimental Designs With Application to the Random Fatigue-Limit Model , 2003 .

[156]  O. Penrose.  Foundations of statistical mechanics , 1969 .

[157]  Udo V Toussaint,et al.  Bayesian neural-networks-based evaluation of binary speckle data. , 2004, Applied optics.

[158]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[159]  P. J. Paris,et al.  X-ray tomography on the TCV tokamak , 1996 .

[160]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[161]  Volker Dose,et al.  Bayesian PIXE background subtraction , 1999 .

[162]  T. Loredo.  Bayesian Adaptive Exploration , 2004, astro-ph/0409386.

[163]  Volker Dose.  Analysis of rare-event time series with application to Caribbean hurricane data , 2009 .

[164]  Annette Menzel,et al.  Bayesian analysis of climate change impacts in phenology , 2004 .

[165]  David Salsburg.  The Lady Tasting Tea , 2002 .

[166]  Vinay Ambegaokar,et al.  Estimating errors reliably in Monte Carlo simulations of the Ehrenfest model , 2010 .

[167]  Volker Dose,et al.  Probabilistic evaluation of growth models based on time-dependent Auger signals , 1998 .

[168]  Michael Werman,et al.  A Bayesian Method for Fitting Parametric and Nonparametric Models to Noisy Data , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[169]  Stephen F. Gull,et al.  Developments in Maximum Entropy Data Analysis , 1989 .

[170]  Bill Bryson.  At Home: A Short History of Private Life , 2010 .

[171]  John Skilling,et al.  Maximum entropy method in image processing , 1984 .

[172]  B. Toman.  Bayesian Experimental Design for Multiple Hypothesis Testing , 1996 .

[173]  P. Protter.  Stochastic integration and differential equations , 1990 .

[174]  C. Robert,et al.  Properties of nested sampling , 2008, 0801.3887.

[175]  M. Hobson,et al.  Efficient Bayesian inference for multimodal problems in cosmology , 2007, astro-ph/0701867.

[176]  Robert A. Gatenby,et al.  Exploratory Data Analysis Using Fisher Information , 2006 .

[177]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[178]  U. von Toussaint,et al.  Optimizing Nuclear Reaction Analysis (NRA) using Bayesian Experimental Design , 2008, 0812.3789.

[179]  H. Dreier,et al.  Design of multichannel laser interferometry for W7-X , 2006 .

[180]  Ward Edwards,et al.  Bayesian statistical inference for psychological research. , 1963 .

[181]  P. Müller,et al.  Numerical Evaluation of Information- Theoretic Measures , 1996 .

[182]  A. Howard,et al.  EFFICIENT FITTING OF MULTIPLANET KEPLERIAN MODELS TO RADIAL VELOCITY AND ASTROMETRY DATA , 2009, 0904.3725.

[183]  J. Skilling.  Quantified Maximum Entropy , 1990 .

[184]  A. N. Kolmogorov,et al.  Foundations of the theory of probability , 1960 .