Eliciting and Combining Subjective Judgments About Uncertainty

Frequently, health care decisions must be made before compelling and unequivocal evidence is available about the benefits, risks, and costs of a proposed new health technology. It is common for decision makers to seek the advice and opinions of experts to supplement the available evidence. This article reviews some of the methods used or recommended for eliciting the opinions of experts about uncertain events and for combining these opinions with those of others and with available empirical evidence.

[1]  Simon French,et al.  Consensus of opinion , 1981 .

[2]  A. Tversky,et al.  Rational choice and the framing of decisions , 1990 .

[3]  Robert L. Winkler,et al.  Calibrating and Combining Precipitation Probability Forecasts , 1987 .

[4]  Robert L. Winkler,et al.  Expert resolution , 1986 .

[5]  Peter A. Morris,et al.  An Axiomatic Approach to Expert Resolution , 1983 .

[6]  M. Degroot.  Reaching a Consensus , 1974 .

[7]  Dennis V. Lindley,et al.  Another Look at an Axiomatic Approach to Expert Resolution , 1986 .

[8]  Robert L. Winkler,et al.  The Consensus of Subjective Probability Distributions , 1968 .

[9]  Robin M. Hogarth,et al.  Cognitive Processes and the Assessment of Subjective Probability Distributions , 1975 .

[10]  A. Tversky,et al.  Extensional versus intuitive reasoning: the conjunction fallacy in probability judgment , 1983 .

[11]  David Lindley.  The Improvement of Probability Judgements , 1982 .

[12]  Wayne S. Smith,et al.  Adaptive Forecasting Models Based on Predictive Distributions , 1978 .

[13]  P. Fishburn.  Subjective expected utility: A review of normative theories , 1981 .

[14]  Daniel Kahneman,et al.  Availability: A heuristic for judging frequency and probability , 1973 .

[15]  A. H. Murphy,et al.  “Good” Probability Assessors , 1968 .

[16]  W. Edwards,et al.  Decision Analysis and Behavioral Research , 1986 .

[17]  R. L. Winkler,et al.  Combining Economic Forecasts , 1986 .

[18]  Christian Genest,et al.  Combining Probability Distributions: A Critique and an Annotated Bibliography , 1986 .

[19]  Simon French,et al.  Updating of Belief in the Light of Someone Else's Opinion , 1980 .

[20]  K. Chaloner,et al.  Assessment of a Beta Prior Distribution: PM Elicitation , 1983 .

[21]  Mark J. Schervish,et al.  Comments on some axioms for combining expert judgments , 1986 .

[22]  S. French,et al.  Calibration and the Expert Problem , 1986 .

[23]  Bruno de Finetti,et al.  Probability, induction and statistics , 1972 .

[24]  R. L. Keeney,et al.  Decisions with Multiple Objectives: Preferences and Value Trade-Offs , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[25]  Robert T. Clemen,et al.  Calibration and the aggregation of probabilities , 1986 .

[26]  R. L. Winkler.  Combining Probability Distributions from Dependent Information Sources , 1981 .

[27]  Wayne S. Smith,et al.  Interactive Elicitation of Opinion for a Normal Linear Model , 1980 .

[28]  M. W. Merkhofer,et al.  Quantifying judgmental uncertainty: Methodology, experiences, and insights , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[29]  N. Dalkey,et al.  An Experimental Application of the Delphi Method to the Use of Experts , 1963 .

[30]  Reinhard Viertl,et al.  Probability and Bayesian Statistics , 1987 .

[31]  M. Degroot,et al.  Bayesian Statistics 2. , 1987 .

[32]  Carl-Axel S. Staël von Holstein,et al.  Exceptional Paper---Probability Encoding in Decision Analysis , 1975 .

[33]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[34]  Robert L. Winkler,et al.  Limits for the Precision and Value of Information from Dependent Sources , 1985, Oper. Res..

[35]  A. Tversky,et al.  On the Reconciliation of Probability Assessments , 1979 .

[36]  Peter A. Morris,et al.  Decision Analysis Expert Use , 1974 .

[37]  J. Zidek.  Group decision analysis and its application to combining opinions , 1988 .

[38]  R. L. Winkler,et al.  Averages of Forecasts: Some Empirical Results , 1983 .

[39]  H. Raiffa,et al.  Applied Statistical Decision Theory. , 1961 .

[40]  Peter A. Morris,et al.  Observations on expert aggregation , 1986 .