Combining Individual Judgments

It is often assumed that n heads are better than one, that a judgment obtained from a group will be of higher quality than could be expected from an individual. This chapter considers the effectiveness of methods that have been proposed for combining individual quantitative judgments into a group judgment. For the most part, it will be found that n heads are, indeed, better than one, and at least one investigator has concluded that it does not much matter how they are combined. But the potential for improving performance is so great and the problems of achieving it so subtle that a clear understanding of the issues is essential.

[1]  E. Ghiselli.  Theory of psychological measurement , 1964 .

[2]  Ira Solomon,et al.  PROBABILITY ASSESSMENT BY INDIVIDUAL AUDITORS AND AUDIT TEAMS - AN EMPIRICAL-INVESTIGATION , 1982 .

[3]  Robin M. Hogarth,et al.  Quality of Group Judgment , 1977 .

[4]  Charles Vlek,et al.  Utility, Probability, and Human Decision Making , 1975 .

[5]  Wilfred C. Uecker.  The Quality Of Group-Performance In Simplified Information Evaluation , 1982 .

[6]  Peter A. Morris,et al.  Combining Expert Judgments: A Bayesian Approach , 1977 .

[7]  J. Armstrong,et al.  The Seer-Sucker Theory: The Value of Experts in Forecasting , 2005 .

[8]  Jay Hall,et al.  The Effects of a Normative Intervention on Group Decision-Making Performance , 1970 .

[9]  Albert N. Badre,et al.  On the Precision of Adjectives Which Denote Fuzzy Sets , 1974 .

[10]  Peter A. Morris,et al.  Decision Analysis Expert Use , 1974 .

[11]  A. Kaufman,et al.  Introduction to the Theory of Fuzzy Subsets. , 1977 .

[12]  William J. Mackinnon,et al.  Development of the SPAN Technique for Making Decisions in Human Groups , 1966 .

[13]  Howard Raiffa,et al.  Decision analysis: introductory lectures on choices under uncertainty. 1968. , 1969, M.D.Computing.

[14]  D. A. Seaver,et al.  Eliciting subjective probability distributions on continuous variables , 1978 .

[15]  William J. Mackinnon,et al.  Span Decision Making in Established Groups , 1969 .

[16]  S. Holstein,et al.  Assessment and evaluation of subjective probability distributions , 1970 .

[17]  Horace W. Brock,et al.  The Problem of "Utility Weights" in Group Preference Aggregation , 1980, Oper. Res..

[18]  C. E. Agnew.  Multiple Probability Assessments by Dependent Experts , 1985 .

[19]  M. Degroot.  Reaching a Consensus , 1974 .

[20]  Gardner Murphy,et al.  Experimental social psychology. , 1934 .

[21]  R. Keeney.  A Group Preference Axiomatization with Cardinal Utility , 1976 .

[22]  Ralph L. Keeney,et al.  Group Decision Making Using Cardinal Social Welfare Functions , 1975 .

[23]  G. William Walster,et al.  A comparative study of differences in subjective likelihood estimates made by individuals, interacting groups, Delphi groups, and nominal groups☆ , 1973 .

[24]  R. L. Winkler.  Combining Probability Distributions from Dependent Information Sources , 1981 .

[25]  J. Davitz,et al.  A survey of studies contrasting the quality of group performance and individual performance, 1920-1957. , 1958, Psychological bulletin.

[26]  William R. Ferrell,et al.  A model of calibration for subjective probabilities , 1980 .

[27]  R. John,et al.  A criterion validation of multiattribute utility analysis and of group communication strategy , 1980 .

[28]  Michael Olinick,et al.  An Introduction to Mathematical Models in the Social and Life Sciences , 1978 .

[29]  R. Hogarth,et al.  Unit weighting schemes for decision making , 1975 .

[30]  J. M. Bates,et al.  The Combination of Forecasts , 1969 .

[31]  Murray Turoff,et al.  The Delphi Method: Techniques and Applications , 1976 .

[32]  Norman Crolee Dalkey,et al.  An experimental study of group opinion , 1969 .

[33]  A. V. D. Ven,et al.  Group Techniques for Program Planning , 1975 .

[34]  Robin M. Hogarth,et al.  A note on aggregating opinions , 1978 .

[35]  R. Bordley.  The Combination of Forecasts: a Bayesian Approach , 1982 .

[36]  James O. Hicks,et al.  A Fuzzy Set Approach to Aggregating Internal Control Judgments , 1983 .

[37]  W. R. Ferrell,et al.  A model of subjective probabilities from small groups , 1982 .

[38]  Brian R. Gaines,et al.  The Fuzzy Decade: A Bibliography of Fuzzy Systems and Closely Related Topics , 1977, Int. J. Man Mach. Stud..

[39]  G. W. Fischer.  When oracles fail—A comparison of four procedures for aggregating subjective probability forecasts , 1981 .

[40]  Arnold Kaufmann,et al.  Fundamental theoretical elements , 1975 .

[41]  J. Hackman,et al.  Group tasks, group interaction process, and group performance effectiveness: A review and proposed integration , 1975 .

[42]  Robert L. Winkler,et al.  Probabilistic Prediction: Some Experimental Results , 1971 .

[43]  John Rohrbaugh,et al.  Improving the quality of group judgment: Social judgment analysis and the Delphi technique. , 1979 .

[44]  Howard Wainer,et al.  Estimating Coefficients in Linear Models: It Don't Make No Nevermind , 1976 .

[45]  Peter G. Goldschmidt,et al.  Scientific inquiry or political critique?: Remarks on Delphi assessment, expert opinion, forecasting, and group process by H. Sackman , 1975 .

[46]  Barbara C. Goodman,et al.  Action selection and likelihood ratio estimation by individuals and groups , 1972 .

[47]  Lotfi A. Zadeh,et al.  The concept of a linguistic variable and its application to approximate reasoning-III , 1975, Inf. Sci..

[48]  R. Bordley.  A Multiplicative Formula for Aggregating Probability Assessments , 1982 .

[49]  Lewis A. Dexter.  Impressions About Utility and Wastefulness in Applied Social Science Studies , 1966 .

[50]  David G. Myers,et al.  Group-Induced Polarization of Attitudes and Behavior , 1978 .

[51]  Peter A. Morris,et al.  An Axiomatic Approach to Expert Resolution , 1983 .

[52]  E. H. Mamdani,et al.  A GENERAL APPROACH TO LINGUISTIC APPROXIMATION , 1979 .

[53]  M. Bacharach.  Group Decisions in the Face of Differences of Opinion , 1975 .

[54]  I. Steiner.  Group process and productivity , 1972 .