Using frame-of-reference training to understand the implications of rater idiosyncrasy for rating accuracy.

Frame-of-reference (FOR) rater training is one technique used to impart a theory of work performance to raters. In this study, the authors explored how raters' implicit performance theories may differ from a normative performance theory taught during training. The authors examined how raters' level and type of idiosyncrasy predicts their rating accuracy and found that rater idiosyncrasy negatively predicts rating accuracy. Moreover, although FOR training may improve rating accuracy even for trainees with lower performance theory idiosyncrasy, it may be more effective in improving errors of omission than commission. The discussion focuses on the roles of idiosyncrasy in FOR training and the implications of this research for future FOR research and practice.

[1]  Elaine D. Pulakos,et al.  The development of training programs to increase accuracy with different rating tasks , 1986 .

[2]  Karen S. Loup,et al.  Looking backwards with the personnel evaluation standards: An analysis of the development and implementation of a statewide teacher assessment program , 1996 .

[3]  L. Sulsky,et al.  The Effects of Task Information Training and Frame-of-Reference Training With Situational Constraints on Rating Accuracy , 2001 .

[4]  Barry Markovsky.  CURRENT RESEARCH IN SOCIAL PSYCHOLOGY , 1996 .

[5]  Lorne M. Sulsky,et al.  Performance appraisal in the changing world of work: Implications for the meaning and measurement of work performance. , 1998 .

[6]  S. Chaiken,et al.  Attitude structure and function. , 1998 .

[7]  S. Fiske,et al.  The Handbook of Social Psychology , 1935 .

[8]  Patricia G. Devine,et al.  Attitude importance and resistance to persuasion : It's not just the thought that counts , 1996 .

[9]  A. Tesser,et al.  Thought-induced attitude change: The effects of schema structure and commitment. , 1986 .

[10]  R. Tharp,et al.  Standards Performance Continuum: Development and Validation of a Measure of Effective Pedagogy , 2002 .

[11]  Elaine D. Pulakos,et al.  A comparison of rater training programs: Error training and accuracy training. , 1984 .

[12]  H. John Bernardin,et al.  Strategies in Rater Training , 1981 .

[13]  Walter C. Borman,et al.  Individual Differences Correlates of Accuracy in Evaluating Others' Performance Effectiveness , 1979 .

[14]  Frame-of-reference training and cognitive categorization: an empirical investigation of rater memory issues. , 1992 .

[15]  Yining Chen,et al.  Student Evaluation of Teaching Effectiveness: An assessment of student perception and motivation , 2003 .

[16]  James A. Kulik,et al.  Schematic effects of social attitudes on information processing and recall. , 1980 .

[17]  J. Landman.  Regret and Elation Following Action and Inaction , 1987 .

[18]  E. A. Locke,et al.  Generalizing From Laboratory to Field Settings. , 1987 .

[19]  Neil M. A. Hauenstein,et al.  Rater training and rating accuracy: training for dimensional accuracy versus training for ratee differentiation , 1993 .

[20]  Neil M. A. Hauenstein,et al.  FROM LABORATORY TO PRACTICE: NEGLECTED ISSUES IN IMPLEMENTING FRAME‐OF‐REFERENCE RATER TRAINING , 1989 .

[21]  J. Baron,et al.  Omission and commission in judgment and choice , 1991 .

[22]  Bert P. M. Creemers,et al.  School Effectiveness and School Improvement: Sustaining Links , 1997 .

[23]  L. Kyriakides,et al.  Generating Criteria for Measuring Teacher Effectiveness Through a Self-Evaluation Approach: A Complementary Way of Measuring Teacher Effectiveness , 2002 .

[24]  David V. Day,et al.  Effects of frame-of-reference training and information configuration on memory organization and rating accuracy. , 1995 .

[25]  Deidra J. Schleicher,et al.  A Cognitive Evaluation of Frame-of-Reference Rater Training: Content and Process Issues. , 1998, Organizational behavior and human decision processes.

[26]  William K. Balzer,et al.  Meaning and measurement of performance rating accuracy: Some methodological and theoretical concerns. , 1988 .

[27]  Richard E Petty,et al.  Resistance to Persuasion and Attitude Certainty: The Moderating Role of Elaboration , 2004, Personality & social psychology bulletin.

[28]  S. Fiske,et al.  Structure and development of social schemata: Evidence from positive and negative transfer effects. , 1985 .

[29]  Peter Villanova,et al.  Effective performance management: A focus on precision, customers, and situational constraints. , 1998 .

[30]  David Nicol,et al.  Teaching Tips: Strategies, Research, and Theory for College and University Teachers , 1994 .

[31]  R. Petty,et al.  What doesn't kill me makes me stronger: the effects of resisting persuasion on attitude certainty. , 2002, Journal of personality and social psychology.