Comparison Methods for Stochastic Models and Risks

scientiŽ c enterprise is reviewed. The author claims that pure objectivity does not exist in science; rather, the scientist always brings some background beliefs and a priori expectations for any type of experimental procedure to his or her observations. The author does an admirable job of explaining the differences between Bayesian probability and the frequentist notion of probability, showing that, philosophically, only the Bayesian makes sense. Chapter 2 explains subjective probability, that is, a person’s degree of belief about an event. The author suggests that this is a more proper deŽ nition of probability than the frequentist notion. He then discusses various axiom systems for probability and argues for the Renyi axiom system as the best choice for probability. Chapter 3 is short, but important. It explains the role of the likelihood function in Bayesian theory. The main points are (1) that the likelihood function is deŽ ned not uniquely, but rather only up to a proportionality constant, and (2) that the Bayesian procedure should not violate the likelihood principal and should depend only on the data. Chapter 4, the book’s central chapter, explains Bayes’s theorem for both discreet and continuous data and discreet and continuous parameters. Also treated are prior and posterior distributions. Chapter 5 discusses both subjective and objective prior distributions. Prior distributions characterize information about the underlying process prior to collecting data. Objective priors have little to say about the nature of the process under investigation. Subjective priors, on the other, hand introduce one’s beliefs about the underlying process into the Bayes equation. At the end the chapter, the author makes an important distinction concerning wrong prior distributions. He states that a prior for some unknown quantity cannot be wrong because it represents a degree of believe about the quantity; however, a belief about the phenomenon could be wrong. Part II, comprising Chapters 6 and 7, discusses numerical implementation of the Bayesian paradigm. Since the publication of the Ž rst edition, computers have enabled the use of numerical modeling techniques for solving Bayesian formulas involving complex integration. Chapter 6, written by Siddhartha Chib, introduces Markov chain Monte Carlo (MCMC) methods, including the important Metropolis–Hastings algorithm, for computing posterior densities. The chapter concludes with a tutorial (by George Woodworth) on using the computer software WinBUGS to perform MCMC computations for many types of Bayesian models. This free software is available at Mention should also be made of another free software package called R (available as a free download from, which is also quite helpful in doing computational Bayesian analysis as well as classical statistical inference procedures. Chapter 7 presents a summary of other major methods (including simulation) used to address the computational problems involved in implementing Bayesian inference. Part III, comprising Chapters 8–11, discusses Bayesian statistical inference and decision making. Chapter 8 deals with Bayesian estimation for unknown quantities occurring in probability distributions. The quantities can be either points or intervals, and the distributions can be either univariate or multivariate. Chapter 9 takes up hypothesis testing and model comparison using the Bayesian approach. Included is a very interesting historical summary (from Karl Pearson to Harold Jeffreys) and philosophical discussions of the logic of scientiŽ c inquiry on methods for testing scientiŽ c hypotheses. The author argues that the Bayesian approach to decision rules and hypothesis testing (as formulated by Jeffreys) is much superior to the frequentist approach. It seems to this reviewer that the philosophical underpinnings of this chapter and, indeed, of the entire book, are very similar to the American philosophical school of pragmatism (Pierce, Dewey, James, etc.). In Chapter 10 the author explains the idea of Bayesian predictive distribution and makes a distinction between prediction and Ž tting previous data. He claims that analysis should depend only on observables. He then discusses exchangeability and de Finetti’s theorem (connecting unobservables with observables) and de Finetti’s transform (the operation that connects the prior distributions and the predictive distribution in a one-to-one relationship). Other topics touched on include Bayesian neural networks, hidden variables, and directed acyclic graphs. Chapter 11 treats decision making under uncertainty from a Bayesian perspective. Utility functions and loss functions are discussed, and the author notes that these procedures now have wide application in Ž elds as diverse as engineering and law. Part IV, comprising Chapters 12–16, covers models and their applications from a Bayesian standpoint. Chapter 12 treats Bayesian inference in the general linear model, both univariate and multivariate, models of regression, and analysis of variance and covariance. Chapter 13 covers Bayesian model averaging to account for model uncertainty. This is necessary because different approaches to model building and selection result in different conclusions as to which might be the best model. Usually only one of the possible models is published, and this gives the false impression that the published model is the only one that accounts for the data. This can lead overconŽ dent inferences and riskier predictions. Bayesian model averaging offers a way to overcome this problem. Chapter 14 discusses Bayesian hierarchical modeling. Bayesian procedures permit estimation of general parameters, that is, parameters characterizing the entire population, as well as parameters pertaining to individuals. Chapter 15 covers Bayesian factor analysis. The Bayesian approach uses a more general covariance matrix, and these lead to a model that overcomes the indeterminacies in traditional factor analysis models. The Bayesian approach permits the analyst to bring prior information to bear on his problem. Chapter 16 describes how to use Bayesian predictive distribution procedures for classiŽ cation and discrimination. Additional application of Bayesian classiŽ cation methods to clustering procedures, contextual classiŽ cation procedures, data mining, and Bayesian neural nets are also mentioned. In conclusion, I Ž nd this book comprehensive and up-to-date in its treatment of the theory and application of Bayesian statistics. The historical references sprinkled throughout the book are helpful in gaining an overview of the development of statistics in general and Bayesian statistics in particular. The author’s frank philosophical discussions of the pros and cons of the assumptions underlying Bayesian statistics are quite informative. His inclusion of a short introduction telling what he’s going to do in each chapter, and then a summary at the end of the chapter of what he did do are very helpful in preventing the reader from becoming lost in the mathematical details. This book should be a welcome addition to the library of any practicing statistician, not only as a thorough and readable text on Bayesian statistics, but also as a rich source of reference material for understanding the historical development of the subject.