Raters who pursue different goals give different ratings.

J. N. Cleveland and K. R. Murphy (1992) suggested that phenomena such as rater errors and interrater disagreements could be understood in terms of differences in the goals pursued by various raters. We measured 19 rating goals of students at the beginning of a semester, grouped them into scales, and correlated these with teacher evaluations collected at the end of the semester. We found significant multiple correlations, both within classes and in an analysis of the pooled sample (adjusting for instructor mean differences, incremental R2 =.08). Measures of rating goals obtained after raters had observed a significant proportion of ratee performance accounted for variance (incremental R2 =.07) not accounted for by measures of goals obtained at the beginning of the semester.

[1]  J. Harter.  'Appraisal from Aristotle on ...' , 1986 .

[2]  Hylton Boothroyd.  Image Theory: Decision Making in Personal and Organizational Contexts. , 1990 .

[3]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[4]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[5]  H. P. Sims,et al.  Behind the Mask: The Politics of Employee Appraisal , 1987 .

[6]  Clinton O. Longenecker,et al.  Delving into the dark side: The politics of executive appraisal , 1994 .

[7]  T. T. Herbert.  Management education and development , 1978 .

[8]  Kevin R. Murphy,et al.  Toward Narrowing the Research-Practice Gap in Performance Appraisal. , 1985 .

[9]  R. Downey,et al.  Rating the ratings: Assessing the psychometric quality of rating data , 1980 .

[10]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[11]  G. Milkovich,et al.  The Current State of Performance Appraisal Research and Practice: Concerns, Directions, and Implications , 1992 .

[12]  Chockalingam Viswesvaran,et al.  Comparative analysis of the reliability of job performance ratings , 1996 .

[13]  Lee Roy Beach,et al.  Image theory: Principles, goals, and plans in decision making , 1987 .

[14]  John Schaubroeck,et al.  A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings. , 1988 .

[15]  L. A. Pervin.  The stasis and flow of behavior: toward a theory of goals. , 1983, Nebraska Symposium on Motivation. Nebraska Symposium on Motivation.

[16]  Kevin R. Murphy,et al.  Understanding Performance Appraisal: Social, Organizational, and Goal-Based Perspectives , 1995 .

[17]  K. Murphy,et al.  Statistical Power Analysis: A Simple and General Model for Traditional and Modern Hypothesis Tests, Second Ediction , 1998 .

[18]  Lawrence A. Pervin,et al.  Personality : current theory and research , 1983 .

[19]  C. Dweck,et al.  A social-cognitive approach to motivation and personality , 1988 .

[20]  Kevin R. Murphy,et al.  Performance appraisal: An organizational perspective. , 1991 .

[21]  Allen I. Huffcutt,et al.  Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer, and self-ratings. , 1997 .

[22]  David G Bjerke,et al.  Officer Fitness Report Evaluation Study , 1987 .

[23]  A. Bandura,et al.  Social learning and personality development , 1964 .

[24]  J Noak,et al.  Performance rating. , 1999, Nursing standard (Royal College of Nursing (Great Britain) : 1987).

[25]  Kevin R. Murphy,et al.  INTERRATER CORRELATIONS DO NOT ESTIMATE THE RELIABILITY OF JOB PERFORMANCE RATINGS , 2000 .