Group Elicitation of Probability Distributions: Are Many Heads Better Than One?

Individual and group-consensus probability distributions were obtained from two independent groups of experts who considered an uncertain quantity relevant to a risk analysis of a proposed nuclear waste facility. A facilitated, structured process helped participants to generate the consensus distribution, and individual distributions were obtained at the start of the process, part way through and at the end. Individual differences in assessed probability distributions at the start of the process were different by about three orders of magnitude, but these converged substantially by the end, showing the effect of information exchange and persuasive arguments. The variance of individual distributions increased as the process progressed, indicating that a loosening of individual anchors was taking place. Statistical averages of the individual distributions were different from the consensus distributions, demonstrating that a statistical average of individual judgments does not represent the shared, constructed social reality of a consensus distribution. Surprisingly, statistical average distributions and consensus distributions of the two groups differed by three orders of magnitude; this was traced to differences in the conditioning assumptions made by the two groups. Implications of these findings for practice are suggested.