Eliciting subjective probability distributions from groups

Uncertainty analysis has become an increasingly important part of risk assessments and operations research models, and the role of expert judgment in providing information for decision making has become more useful. Decision makers often have access to more than one expert, and it is common to make the decisions on the basis of the expertise of several experts, which leads to the problem of how to combine or aggregate the experts' judgments. A number of approaches have been proposed as to how to elicit, and how to synthesize, the different experts' knowledge. We discuss a variety of models that lead to specific combination methods. The output of these methods is a “combined probability distribution,” which can be viewed as representing a summary of the current state of information regarding the uncertainty of interest. We also briefly review the psychology literature relating to group decision making, which, obviously, is relevant to behavioral aggregation.