Effects of justification and a mechanical aid on judgment performance

Abstract This study examines the effect on judgment performance of an explicit justification requirement and of having available the recommendations of a mechanical judgment aid. The subjects are professional auditors, and the task is the classification of industrial bond issues into rating categories that reflect differential levels of financial quality. It is found that both the justification requirement and the availability of the mechanical aid are associated with significantly greater judgment accuracy which can be traced to significantly greater consistency. Both factors are also associated with significantly greater consensus, or agreement among judges, which can be important when accuracy cannot be assessed. Several directions for further research on justification and judgment aiding in applied contexts are discussed.

[1]  Marshall B. Romney,et al.  Calibration of auditors' probabilistic judgments: Some empirical evidence , 1982 .

[2]  Robert H. Ashton,et al.  Evidence-responsiveness in professional judgment: Effects of positive versus negative evidence and presentation mode , 1990 .

[3]  Henry Montgomery,et al.  Attractiveness of decision rules , 1980 .

[4]  J. I. Kim,et al.  Accountability and judgment processes in a personality prediction task. , 1987, Journal of personality and social psychology.

[5]  Michael Gibbins,et al.  Good judgment in public accounting: Quality and justification* , 1987 .

[6]  H. Arkes,et al.  Two methods of reducing overconfidence , 1987 .

[7]  John W. Payne,et al.  Contingent decision behavior. , 1982 .

[8]  I. Simonson,et al.  Choice Based on Reasons: The Case of Attraction and Compromise Effects , 1989 .

[9]  Philip E. Tetlock,et al.  Accountability and complexity of thought. , 1983 .

[10]  James C. Baxter,et al.  Influence of role pressures on the perceiver: Judgments of videotaped interviews varying judge accountability and responsibility. , 1981 .

[11]  J. Pearl,et al.  Causal and diagnostic inferences: A comparison of validity , 1981 .

[12]  C. N. Uhl.  Effects of multiple stimulus validity and criterion dispersion on learning of interval concepts. , 1966, Journal of experimental psychology.

[13]  P. Slovic.  Choice Between Equally Valued Alternatives. , 1975 .

[14]  N. C. Silver,et al.  Averaging Correlation Coefficients: Should Fishers z Transformation Be Used? , 1987 .

[15]  R. Ashton.  A Descriptive Study Of Information Evaluation , 1981 .

[16]  S. Chaiken.  Heuristic versus systematic information processing and the use of source versus message cues in persuasion. , 1980 .

[17]  P. Tetlock,et al.  Accountability: a social magnifier of the dilution effect. , 1989, Journal of personality and social psychology.

[18]  H. J. Einhorn,et al.  Accepting error to make less error. , 1986, Journal of personality assessment.

[19]  R. Ashton.  Research in audit decision making : rationale, evidence, and implications , 1983 .

[20]  Kevin Keasey,et al.  Consensus and accuracy in accounting studies of decision-making: A note on a new measure of consensus , 1989 .

[21]  Berndt Brehmer,et al.  Response consistency in probabilistic inference tasks , 1978 .

[22]  Berndt Brehmer,et al.  Does having to justify one's judgments change the nature of the judgment process? , 1983 .

[23]  Stephen A. Butler.  Application Of A Decision Aid In The Judgmental Evaluation Of Substantive Test Of Details Samples , 1985 .

[24]  P. Tetlock.  Accountability: The neglected social context of judgment and choice. , 1985 .

[25]  Leonard Adelman.  Involving Users in the Development of Decision-Analytic Aids: The Principal Factor in Successful Implementation , 1982 .

[26]  Paul Slovic,et al.  Structuring Knowledge Retrieval: An Analysis of Decomposed Quantitative Judgments , 1988 .

[27]  A. H. Ashton,et al.  Does consensus imply accuracy in accounting studies of decision making?: Alison Hubbard Ashton, Accounting Review 60 (185) 173–85 , 1986 .

[28]  A. Tversky,et al.  Intuitive Prediction: Biases and Corrective Procedures , 1982 .

[29]  George Cvetkovich,et al.  Cognitive accommodation, language, and social responsibility. , 1978 .

[30]  H. Arkes,et al.  Factors influencing the use of a decision rule in a probabilistic task , 1986 .

[31]  R. Ashton,et al.  Consistency among alternative performance measures in an applied judgement setting , 1987 .

[32]  Dane K. Peterson,et al.  Effect of input from a mechanical model on clinical judgment. , 1986 .

[33]  Robert H. Ashton,et al.  Using and evaluating audit decision aids , 1988 .

[34]  B. Fischhoff,et al.  KNOWING WHAT YOU WANT: MEASURING LABILE VALUES , 1988 .

[35]  Robert H. Ashton,et al.  Pressure And Performance In Accounting Decision Settings - Paradoxical Effects Of Incentives, Feedback, And Justification , 1990 .

[36]  P. Tetlock.  Accountability and the perseverance of first impressions. , 1983 .

[37]  A. Tversky.  Elimination by aspects: A theory of choice. , 1972 .

[38]  I. Janis,et al.  Decision Making: A Psychological Analysis of Conflict, Choice, and Commitment , 1977 .

[39]  C. Dunnett.  A Multiple Comparison Procedure for Comparing Several Treatments with a Control , 1955 .

[40]  Daniel W. McAllister,et al.  The Contingency Model for the Selection of Decision Strategies: An Empirical Test. , 1978 .

[41]  A. Wisudha.  Design of Decision-Aiding System , 1985 .

[42]  P. Tetlock,et al.  Social and cognitive strategies for coping with accountability: conformity, complexity, and bolstering. , 1989, Journal of personality and social psychology.

[43]  Charles W. Dunnett,et al.  New tables for multiple comparisons with a control. , 1964 .

[44]  L. Tucker.  A SUGGESTED ALTERNATIVE FORMULATION IN THE DEVELOPMENTS BY HURSCH, HAMMOND, AND HURSCH, AND BY HAMMOND, HURSCH, AND TODD. , 1964, Psychological review.