Assessing Dependence: Some Experimental Results

Constructing decision- and risk-analysis probability models often requires measures of dependence among variables. Although data are sometimes available to estimate such measures, in many applications they must be obtained by means of subjective judgment by experts. We discuss two experimental studies that compare the accuracy of six different methods for assessing dependence. Our results lead to several conclusions: First, simply asking experts to report a correlation is a reasonable approach. Direct estimation is more accurate than the other methods studied, is not prone to mathematically inconsistent responses (as are some other measures), and is judged to be less difficult than alternate methods. In addition, directly assessed correlations showed less variability than the correlations derived from other assessment methods. Our results also show that experience with the variables can improve performance somewhat, as can training in a given assessment method. Finally, if a judge uses several different assessment methods, an average of the resulting estimates can also lead to better performance.

[1]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[2]  R. Clemen,et al.  Correlations and Copulas for Decision and Risk Analysis , 1999 .

[3]  D. V. Gokhale,et al.  Assessment of a Prior Distribution for the Correlation Coefficient in a Bivariate Normal Distribution , 1982 .

[4]  J S Evans,et al.  The effect of neglecting correlations when propagating uncertainty and estimating the population distribution of risk. , 1992, Risk analysis : an official publication of the Society for Risk Analysis.

[5]  Robert L. Winkler,et al.  An Assessment of the Risk of Chronic Lung Injury Attributable to Long-Term Ozone Exposure , 1995, Oper. Res..

[6]  L. Ross,et al.  Human Inference: Strategies and Shortcomings of Social Judgment. , 1981 .

[7]  G. A. Miller,et al.  Book Review Nisbett, R. , & Ross, L.Human inference: Strategies and shortcomings of social judgment.Englewood Cliffs, N.J.: Prentice-Hall, 1980. , 1982 .

[8]  W. Kruskal Ordinal Measures of Association , 1958 .

[9]  Ruth Beyth-Marom,et al.  Perception of correlation reexamined , 1982 .

[10]  Brian H. Bornstein,et al.  Effects of expectancy on assessing covariation in data: "Prior belief" versus "meaning" , 1992 .

[11]  Emiliano A. Valdez,et al.  Understanding Relationships Using Copulas , 1998 .

[12]  S. Ratneshwar,et al.  Consumer Covariation Judgments: Theory or Data Driven? , 1992 .

[13]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[14]  A. Tversky,et al.  On the psychology of prediction , 1973 .

[15]  Harriet Shaklee,et al.  Development of Rule Use in Judgments of Covariation between Events. , 1981 .

[16]  R. Nisbett,et al.  The psychometrics of everyday life , 1986, Cognitive Psychology.

[17]  J. Frank Yates,et al.  Judgment and Decision Making , 1990 .

[18]  C. Genest,et al.  Statistical Inference Procedures for Bivariate Archimedean Copulas , 1993 .

[19]  J. Smedslund THE CONCEPT OF CORRELATION IN ADULTS , 1963 .

[20]  Robert L. Winkler,et al.  Limits for the Precision and Value of Information from Dependent Sources , 1985, Oper. Res..

[21]  Robert T. Clemen,et al.  Copula Models for Aggregating Expert Opinions , 1996, Oper. Res..

[22]  R. L. Winkler,et al.  Averages of Forecasts: Some Empirical Results , 1983 .

[23]  Teresa M. Amabile,et al.  Judgment under uncertainty: Informal covariation assessment: Data-based versus theory-based judgments , 1982 .

[24]  B. Schweizer,et al.  On Nonparametric Measures of Dependence for Random Variables , 1981 .

[25]  A. Tversky,et al.  Extensional versus intuitive reasoning: the conjunction fallacy in probability judgment , 1983 .