Experiences in elicitation

Summary. Elicitation of expert opinion is becoming increasingly important in the elicitation of prior distributions. In this paper, the psychology of elicitation and the currently available methods are briefly reviewed, but the primary discussion is on the distinction between 'general' elicitation methods for a class of problems and 'application-specific' methods which are useful only once. Examples of both types of elicitation are given, along with a discussion about general versus application-specific methods, and predictive versus structural elicitation.

[1]  L. J. Savage,et al.  The Foundations of Statistics , 1955 .

[2]  Captain Y. B. Nusfield.  Public Health , 1906, Canadian Medical Association journal.

[3]  R. L. Winkler.  The Assessment of Prior Distributions in Bayesian Analysis , 1967 .

[4]  R. L. Winkler.  The Quantification of Judgment: Some Methodological Suggestions , 1967 .

[5]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[6]  Vic Barnett.  Comparative Statistical Inference , 1975 .

[7]  Robin M. Hogarth,et al.  Cognitive Processes and the Assessment of Subjective Probability Distributions: Rejoinder , 1975 .

[8]  A. H. Murphy,et al.  Reliability of Subjective Probability Forecasts of Precipitation and Temperature , 1977 .

[9]  A. Tversky,et al.  On the Reconciliation of Probability Assessments , 1979 .

[10]  J. Dickey.  Beliefs about beliefs, a theory for stochastic assessments of subjective probabilities , 1980 .

[11]  Wayne S. Smith,et al.  Interactive Elicitation of Opinion for a Normal Linear Model , 1980 .

[12]  Simon French,et al.  Updating of Belief in the Light of Someone Else's Opinion , 1980 .

[13]  M. Degroot,et al.  Assessing Probability Assessors: Calibration and Refinement. , 1981 .

[14]  D. V. Gokhale,et al.  Assessment of a Prior Distribution for the Correlation Coefficient in a Bivariate Normal Distribution , 1982 .

[15]  Joseph B. Kadane,et al.  A Subjectivist View of Calibration. , 1982 .

[16]  H. Raiffa,et al.  Judgment under uncertainty: A progress report on the training of probability assessors , 1982 .

[17]  Stephen B. Vardeman,et al.  Calibration, sufficiency, and domination considerations for Bayesian probability assessors , 1983 .

[18]  D. Spiegelhalter,et al.  The Assessment of the Subjective Opinion and its Use in Relation to Stopping Rules for Clinical Trials , 1983 .

[19]  K. Chaloner,et al.  Assessment of a Beta Prior Distribution: PM Elicitation , 1983 .

[20]  David V. Budescu,et al.  Encoding subjective probabilities: A psychological and psychometric review , 1983 .

[21]  Frederick Hayes-Roth,et al.  Knowledge-Based Expert Systems , 1984, Computer.

[22]  A. H. Murphy,et al.  Probability Forecasting in Meteorology , 1984 .

[23]  Teddy Seidenfeld,et al.  Calibration, Coherence, and Scoring Rules , 1985, Philosophy of Science.

[24]  Douglas Merritt Logan,et al.  The value of probability assessment , 1985 .

[25]  Theresa M. Mullin,et al.  Understanding and supporting the process of probabilistic estimation , 1986 .

[26]  S. French,et al.  Calibration and the Expert Problem , 1986 .

[27]  D. Spiegelhalter,et al.  A predictive approach to selecting the size of a clinical trial, based on subjective clinical opinion. , 1986, Statistics in medicine.

[28]  P. Perron,et al.  Trends and random walks in macroeconomic time series : Further evidence from a new approach , 1988 .

[29]  Nozer D. Singpurwalla,et al.  The Analysis of Weibull Lifetime Data Incorporating Expert Opinion , 1987 .

[30]  Sylvia Schnatter,et al.  Bayesian Estimation of Design Floods under Regional and Subjective Prior Information , 1987 .

[31]  George T. Duncan,et al.  Some properties of the dirichlet-multinomial distribution and its use in prior elicitation , 1987 .

[32]  Umesh Gavasakar,et al.  A Comparison of Two Elicitation Methods for a Prior Distribution for a Binomial Parameter , 1988 .

[33]  E F Cook,et al.  A computer protocol to predict myocardial infarction in emergency department patients with chest pain. , 1988, The New England journal of medicine.

[34]  Paul H. Garthwaite,et al.  Quantifying Expert Opinion in Linear Regression Problems , 1988 .

[35]  G. Bennett,et al.  The assessment. , 1989, Health visitor.

[36]  R. Wolpert,et al.  Eliciting and Combining Subjective Judgments About Uncertainty , 1989, International Journal of Technology Assessment in Health Care.

[37]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[38]  R. Cooke.  Experts in Uncertainty: Opinion and Subjective Probability in Science , 1991 .

[39]  Paul H. Garthwaite,et al.  Elicitation of Prior Distributions for Variable-Selection Problems in Regression , 1992 .

[40]  D J Spiegelhalter,et al.  Application of Bayesian statistics to decision making during a clinical trial. , 1992, Statistics in medicine.

[41]  D J Spiegelhalter,et al.  Applying Bayesian ideas in drug development and clinical trials. , 1993, Statistics in medicine.

[42]  Thomas A. Louis,et al.  Graphical Elicitation of a Prior Distribution for a Clinical Trial , 1993 .

[43]  Nancy Flournoy,et al.  A Clinical Experiment in Bone Marrow Transplantation: Estimating a Percentage Point of a Quantal Response Curve , 1993 .

[44]  Paul H. Garthwaite,et al.  Assessment of prior distributions for regression models: an experimental study , 1994 .

[45]  Joseph B. Kadane,et al.  An application of robust Bayesian analysis to a medical experiment , 1994 .

[46]  Arriaga Ee,et al.  Population analysis with microcomputers. , 1994 .

[47]  Alyson G. Wilson,et al.  Cognitive Factors Affecting Subjective Probability Assessment , 1994 .

[48]  Adrian E. Raftery,et al.  Inference from a Deterministic Population Dynamics Model for Bowhead Whales , 1995 .

[49]  Joseph B. Kadane,et al.  Priors for unit root models , 1996 .

[50]  D. Schum,et al.  A Probabilistic Analysis of the Sacco and Vanzetti Evidence , 1996 .

[51]  J. Kadane,et al.  Bayesian methods and ethics in a clinical trial design , 1997 .

[52]  J. Kadane,et al.  Bayesian demography: projecting the Iraqi Kurdish population, 1977-1990. , 1997, Journal of the American Statistical Association.