Getting the Right Mix of Experts

The Bayesian approach to combining expert opinions is well developed, providing a decision maker's posterior beliefs after receiving advice from people with deep knowledge in a given subject. A necessary part of these models is the inclusion of dependencies between the experts' judgments, often justified by an overlap in the information on which the experts base their judgments. In this paper, we propose a hierarchical structure different than those previously proposed, where the mixing distribution is treated nonparametrically with a Dirichlet process. This makes our overall model a Dirichlet process mixture and allows for experts' model parameters to be equal in the mixture. We apply this approach to published expert judgment data, demonstrating that the decision maker's posterior distributions on the quantities of interest are not restricted to specific parametric forms, even allowing for multiple modes, and are thus more intuitively appealing.

[1]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[2]  Peter A. Morris,et al.  An Axiomatic Approach to Expert Resolution , 1983 .

[3]  A. Gelfand,et al.  Dirichlet Process Mixed Generalized Linear Models , 1997 .

[4]  S. MacEachern.  Estimating normal means with a conjugate style dirichlet process prior , 1994 .

[5]  Peter A. Morris,et al.  Combining Expert Judgments: A Bayesian Approach , 1977 .

[6]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[7]  Robert T. Clemen,et al.  Copula Models for Aggregating Expert Opinions , 1996, Oper. Res..

[8]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[9]  Ali Mosleh,et al.  A critique of current practice for the use of expert opinions in probabilistic risk assessment , 1988 .

[10]  Simon French,et al.  Consensus of opinion , 1981 .

[11]  Lynn Kuo,et al.  Bayesian semiparametric inference for the accelerated failure‐time model , 1997 .

[12]  W. Gilks,et al.  Adaptive Rejection Sampling for Gibbs Sampling , 1992 .

[13]  Steven N. MacEachern,et al.  Efficient MCMC Schemes for Robust Model Extensions Using Encompassing Dirichlet Process Mixture Models , 2000 .

[14]  Peter A. Morris,et al.  Decision Analysis Expert Use , 1974 .

[15]  L. Kuo.  Computations of mixtures of dirichlet processes , 1986 .

[16]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[17]  Simon French,et al.  Updating of Belief in the Light of Someone Else's Opinion , 1980 .

[18]  Dani Gamerman,et al.  Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference, Second Edition , 2006 .

[19]  Adrian F. M. Smith,et al.  Simple conditions for the convergence of the Gibbs sampler and Metropolis-Hastings algorithms , 1994 .

[20]  D. Lindley,et al.  Bayes Estimates for the Linear Model , 1972 .

[21]  Robert L. Winkler,et al.  Combining Probability Distributions From Experts in Risk Analysis , 1999 .

[22]  Steven N. MacEachern,et al.  Computational Methods for Mixture of Dirichlet Process Models , 1998 .

[23]  Dennis V. Lindley,et al.  Reconciliation of Probability Distributions , 1983, Oper. Res..

[24]  R. L. Winkler.  Combining Probability Distributions from Dependent Information Sources , 1981 .

[25]  R. Clemen,et al.  Correlations and Copulas for Decision and Risk Analysis , 1999 .

[26]  Giovanni Parmigiani,et al.  Combining Expert Judgment by Hierarchical Modeling: An Application to Physician Staffing , 1998 .

[27]  Robert T. Clemen,et al.  Combining Overlapping Information , 1987 .