Effects of rater goals on rating patterns: evidence from an experimental field study.

The goal-based perspective of performance appraisals suggests that raters who pursue different goals give different performance ratings. Yet previous studies have not provided strong empirical evidence that there are different impacts of different goals on mean rating and discriminability, nor have they provided evidence of a goal-rating causality. The authors extend the literature by manipulating rater goals in the context of peer evaluations of graded group projects with a sample of 104 undergraduate students. They find that (a) pursuing a harmony goal increased mean rating and decreased discriminability, and (b) pursuing a fairness goal increased mean rating and decreased discriminability when the group projects had not ended and increased mean rating but did not change discriminability when the group projects had ended.

[1]  William K. Balzer,et al.  Rater errors and rating accuracy. , 1989 .

[2]  Kin Fai Ellick Wong,et al.  Between-individual comparisons in performance evaluation: a perspective from prospect theory. , 2005, The Journal of applied psychology.

[3]  J. W. Atkinson.  Motives In Fantasy, Action, And Society , 1958 .

[4]  Karen S. Cook,et al.  Distributive Justice: A Social-Psychological Perspective. , 1986 .

[5]  Gerardo Marí.  PERCEIVING JUSTICE ACROSS CULTURES: EQUITY VS. EQUALITY IN COLOMBIA AND IN THE UNITED STATES , 1981 .

[6]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[7]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[8]  G. Leventhal,et al.  Inequity and interpersonal conflict: reward allocation and secrecy about reward as methods of preventing conflict. , 1972, Journal of personality and social psychology.

[9]  J. S. Adams,et al.  Inequity In Social Exchange , 1965 .

[10]  K. N. Wexley,et al.  Training interviewers to eliminate contrast effects in employment interviews. , 1973 .

[11]  Kevin R. Murphy,et al.  Understanding Performance Appraisal: Social, Organizational, and Goal-Based Perspectives , 1995 .

[12]  Kevin R. Murphy,et al.  Multiple uses of performance appraisal: Prevalence and correlates. , 1989 .

[13]  V. Vroom.  Work and motivation , 1964 .

[14]  Shirley F. Harper,et al.  Performance Appraisal: Research and Practice. , 1963 .

[15]  H. P. Sims,et al.  Behind the Mask: The Politics of Employee Appraisal , 1987 .

[16]  S. Gilliland.  The Perceived Fairness of Selection Systems: An Organizational Justice Perspective , 1993 .

[17]  Clinton O. Longenecker,et al.  Delving into the dark side: The politics of executive appraisal , 1994 .

[18]  S. Ng.  Intergroup Allocation Bias Before and after Group Discussion , 1984 .

[19]  Angelo S. DeNisi,et al.  A cognitive approach to performance appraisal : a program of research , 1996 .

[20]  Kevin R. Murphy,et al.  Performance appraisal: An organizational perspective. , 1991 .

[21]  Dan J. Putka,et al.  Two studies examining the negative effect of self-efficacy on performance. , 2002, The Journal of applied psychology.

[22]  B. Bass.  Reducing Leniency in Merit Ratings , 1956 .

[23]  A. Morrison,et al.  Performance Appraisal on the Line , 1981 .

[24]  W. T. Powers.  Behavior, the control of perception , 1973 .

[25]  Neil M. A. Hauenstein.  An information-processing approach to leniency in performance judgments , 1992 .

[26]  C. Carver,et al.  On the Self-Regulation of Behavior , 1998 .

[27]  Michael J. Burke,et al.  Estimating Interrater Agreement with the Average Deviation Index: A User’s Guide , 2002 .

[28]  Kevin R. Murphy,et al.  INTERRATER CORRELATIONS DO NOT ESTIMATE THE RELIABILITY OF JOB PERFORMANCE RATINGS , 2000 .

[29]  Peter A. Heslin,et al.  The effect of implicit person theory on performance appraisals. , 2005, The Journal of applied psychology.

[30]  Jack M. Feldman,et al.  Beyond Attribution Theory: Cognitive Processes in Performance Appraisal , 1981 .

[31]  William K. Balzer,et al.  Halo and performance appraisal research: A critical examination. , 1992 .

[32]  L. Cronbach.  Processes affecting scores on understanding of others and assumed similarity. , 1955, Psychological bulletin.

[33]  M. Deutsch.  Equity, Equality, and Need: What Determines Which Value Will Be Used as the Basis of Distributive Justice? , 1975 .

[34]  Jeanette N Cleveland,et al.  Raters who pursue different goals give different ratings. , 2004, The Journal of applied psychology.

[35]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[36]  J. Greenberg,et al.  Justice in social relations , 1986 .

[37]  David E. Smith,et al.  Accuracy of performance ratings as affected by rater training and perceived purpose of rating. , 1984 .

[38]  Michael J. Burke,et al.  On Average Deviation Indices for Estimating Interrater Agreement , 1999 .

[39]  Chockalingam Viswesvaran,et al.  Comparative analysis of the reliability of job performance ratings , 1996 .