Consensus and accuracy in accounting studies of decision-making: A note on a new measure of consensus

Abstract Despite the common use of consensus amongst experts as a “surrogate for truth” in experimental studies of decision-making, there has been little discussion as to its appropriateness. A new measure of consensus for dichotomous predictions, which takes into account expert' confidence in the correctness of their decisions, is proposed and its relationship with accuracy is experimentally evaluated. The experiment also extends Ashton's ( The Accounting Review , April 1985, pp. 173–185) empirical work on the conventional consensus measure by checking the robustness of Ashton's results to a different experimental setting (U.K. trade credit specialists) and to increases in the cues made available to experimental participants. The paper concludes that the new measure of consensus complements the conventional measure, in that it provides information over and above that provided by the conventional consensus measure. The paper also concludes that the results noted by Ashton for the U.S. also hold for the U.K. and are robust to increases in the cue set made available to participants.

[1]  Barry L. Lewis,et al.  Human information processing research in accounting: The state of the art in 1982 , 1977 .

[2]  A. H. Murphy,et al.  “Good” Probability Assessors , 1968 .

[3]  B. Fischhoff,et al.  Calibration of probabilities: the state of the art to 1980 , 1982 .

[4]  P. A. Adams,et al.  Training in confidence-judgments. , 1958, The American journal of psychology.

[5]  A. H. Murphy,et al.  forecasters and probability forecasts: some current problems1,2 , 1971 .

[6]  S. Oskamp.  OVERCONFIDENCE IN CASE-STUDY JUDGMENTS. , 1965, Journal of consulting psychology.

[7]  M. Bar-Hillel.  The base-rate fallacy in probability judgments. , 1980 .

[8]  Cameron R. Peterson,et al.  Odds Versus Probabilities for Categorical Events. , 1973 .

[9]  Thomas Kida.  AN INVESTIGATION INTO AUDITORS CONTINUITY AND RELATED QUALIFICATION JUDGMENTS , 1980 .

[10]  J. Dowie,et al.  On the Efficiency and Equity of Betting Markets , 1976 .

[11]  B. Brown,et al.  THE DELPHI METHOD, IV: EFFECT OF PERCENTILE FEEDBACK AND FEED-IN OF RELEVANT FACTS, , 1970 .

[12]  L. Nystedt.  Consensus Among Judges as a Function of Amount of Information , 1974 .

[13]  Baruch Fischhoff,et al.  How Well do Probability Experts Assess Probabilities , 1980 .

[14]  Kevin Keasey,et al.  The Prediction of Small Company Failure: Some Behavioural Evidence for the UK , 1986 .

[15]  Frank R. Clarke.  Confidence Ratings, Second‐Choice Responses, and Confusion Matrices in Intelligibility Tests , 1960 .

[16]  Jay J.J. Christensen-Szalanski,et al.  Physicians' use of probabilistic information in a real clinical setting. , 1981 .

[17]  A. H. Ashton,et al.  Does consensus imply accuracy in accounting studies of decision making?: Alison Hubbard Ashton, Accounting Review 60 (185) 173–85 , 1986 .

[18]  G. Glass,et al.  Statistical methods in education and psychology , 1970 .

[19]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[20]  D. Magnusson,et al.  PREDICTIVE EFFICIENCY AS A FUNCTION OF AMOUNT OF INFORMATION. , 1972, Multivariate behavioral research.

[21]  F. W. Huff,et al.  A study of the effects on agreement among clinicians of redundant and new information, confidence, and time available for assessments. , 1967, The Journal of general psychology.

[22]  Arnold Well,et al.  The influence of irrelevant information on speeded classification tasks , 1971 .

[23]  A. H. Murphy,et al.  subjective probability forecasting experiments in meteorology: some preliminary results , 1974 .

[24]  Joan E. Sieber,et al.  Effects of decision importance on ability to generate warranted subjective uncertainty. , 1974 .

[25]  Hillel J. Einhorn,et al.  Judgment under uncertainty: Learning from experience and suboptimal rules in decision making , 1982 .

[26]  Norman Crolee Dalkey,et al.  An experimental study of group opinion , 1969 .

[27]  Elizabeth C. Hirschman,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[28]  H. J. Einhorn.  Use of nonlinear, noncompensatory models as a function of task and amount of information , 1971 .

[29]  Raymond S. Nickerson,et al.  Confidence Ratings and Level of Performance on a Judgmental Task , 1965, Perceptual and motor skills.

[30]  B. Fischhoff,et al.  Training for calibration. , 1980 .

[31]  P. A. Adams,et al.  Realism of confidence judgments. , 1961, Psychological review.

[32]  S. Lichtenstein,et al.  Do those who know more also know more about how much they know?*1 , 1977 .