Frame of Reference Rater Training Issues: Recall, Time and Behavior Observation Training

This study investigated three controversial issues regarding frame of reference training (FOR), a type of rater training: (1) Does it improve behavioral recall, (2) Can it be improved by incorporating behavior observation training (BOT), and (3) Can its effects persist over time? Results suggested that even though FOR training increases the number of behaviors recalled, it does not necessarily improve the quality of the recalled behaviors, but a combined FOR BOT program does improve recall quality. Lastly, results suggested that FOR training can improve rating accuracy, even after a two-week delay between rater training and the rating task.

[1]  T. K. Srull,et al.  Category accessibility: Some theoretical and empirical issues concerning the processing of social stimulus information , 1981 .

[2]  L. Sulsky,et al.  Impact of Frame-of-Reference and Behavioral Observation Training on Alternative Training Effectiveness Criteria in a Canadian Military Sample , 2001 .

[3]  David J. Woehr,et al.  Understanding frame-of-reference training: the impact of training on the recall of performance information , 1994 .

[4]  David J. Woehr,et al.  Processing objective and question order effects on the causal relation between memory and judgment in performance appraisal: the tip of the iceberg , 1993 .

[5]  Gary P. Latham,et al.  Increasing productivity through performance appraisal , 1981 .

[6]  David E. Smith,et al.  Accuracy of performance ratings as affected by rater training and perceived purpose of rating. , 1984 .

[7]  Steven L. Neuberg,et al.  A Continuum of Impression Formation, from Category-Based to Individuating Processes: Influences of Information and Motivation on Attention and Interpretation , 1990 .

[8]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[9]  Jack M. Feldman,et al.  Performance appraisal: A process focus. , 1983 .

[10]  W. Mischel,et al.  Traits as prototypes: Effects on recognition memory. , 1977 .

[11]  H. John Bernardin,et al.  Effects of rater training: Creating new response sets and decreasing accuracy. , 1980 .

[12]  D. Day,et al.  Frame-of-reference training and cognitive categorization: an empirical investigation of rater memory issues. , 1992, The Journal of applied psychology.

[13]  Robert Rosenthal,et al.  Contemporary Issues in the Analysis of Data: A Survey of 551 Psychologists , 1993 .

[14]  David V. Day,et al.  Effects of frame-of-reference training and information configuration on memory organization and rating accuracy. , 1995 .

[15]  Kevin R. Murphy,et al.  Understanding Performance Appraisal: Social, Organizational, and Goal-Based Perspectives , 1995 .

[16]  John M. Ivancevich,et al.  Longitudinal study of the effects of rater training on psychometric error in ratings. , 1979 .

[17]  R. H. Moorman,et al.  JUSTICE AS A MEDIATOR OF THE RELATIONSHIP BETWEEN METHODS OF MONITORING AND ORGANIZATIONAL CITIZENSHIP BEHAVIOR , 1993 .

[18]  George C. Thornton,et al.  Training to improve observer accuracy. , 1980 .

[19]  Robert L. Heneman.  Traits, Behaviors, and Rater Training: Some Unexpected Results , 1988 .

[20]  Deidra J. Schleicher,et al.  A new frame for frame-of-reference training: enhancing the construct validity of assessment centers. , 2002, The Journal of applied psychology.

[21]  Eleanor Rosch,et al.  Principles of Categorization , 1978 .

[22]  Robert G. Lord,et al.  Cognitive categorization and dimensional schemata: A process approach to the study of halo in performance ratings. , 1983 .

[23]  T. L. Tang,et al.  Distributive and Procedural Justice as Related to Satisfaction and Commitment. , 1996 .

[24]  Michael J. Kavanagh,et al.  Improving the accuracy of performance evaluations: comparison of three methods of performance appraiser training , 1988 .

[25]  L. Cronbach.  Processes affecting scores on understanding of others and assumed similarity. , 1955, Psychological bulletin.

[26]  William K. Balzer,et al.  Meaning and measurement of performance rating accuracy: Some methodological and theoretical concerns. , 1988 .

[27]  R. Hastie,et al.  Person memory: Personality traits as organizing principles in memory for behaviors. , 1979 .

[28]  Neil M. A. Hauenstein,et al.  Rater training and rating accuracy: training for dimensional accuracy versus training for ratee differentiation , 1993 .

[29]  P. Sweeney,et al.  Distributive and procedural justice as predictors of satisfaction with personal and organizational outcomes. , 1992 .

[30]  Elaine D. Pulakos,et al.  The development of training programs to increase accuracy with different rating tasks , 1986 .

[31]  Miguel A. Quiñones,et al.  A survey of assessment center practices in organizations in the United States. , 1997 .

[32]  Jack M. Feldman,et al.  Beyond Attribution Theory: Cognitive Processes in Performance Appraisal , 1981 .

[33]  T. K. Srull,et al.  A Dual process model of impression formation , 1988 .

[34]  H. P. Sims,et al.  Behind the Mask: The Politics of Employee Appraisal , 1987 .

[35]  Juan I. Sanchez,et al.  A second look at the relationship between rating and behavioral accuracy in performance appraisal. , 1996 .

[36]  Robert L. Heneman,et al.  The Effects of Time Delay in Rating and Amount of Information Observed on Performance Rating Accuracy , 1983 .

[37]  Mark D. Spool,et al.  TRAINING PROGRAMS FOR OBSERVERS OF BEHAVIOR: A REVIEW , 1978 .

[38]  David J. Woehr,et al.  Rater training for performance appraisal: A quantitative review , 1994 .

[39]  Catherine C. H. Chiu,et al.  Distributive and procedural justice as predictors of employee outcomes in Hong Kong , 2000 .

[40]  Robert H. Moorman,et al.  Relationship between organizational justice and organizational citizenship behaviors : do fairness perceptions influence employee citizenship ? , 1991 .

[41]  R. Folger,et al.  Effects of Procedural and Distributive Justice on Reactions to Pay Raise Decisions , 1989 .

[42]  P. Sweeney,et al.  Research Notes. Distributive and Procedural Justice as Predictors of Satisfaction with Personal and Organizational Outcomes , 1992 .

[43]  Robert G. Lord,et al.  An information processing approach to social perceptions, leadership and behavioral measurement in organizations. , 1985 .

[44]  Kevin R. Murphy,et al.  Multiple uses of performance appraisal: Prevalence and correlates. , 1989 .

[45]  D. Hamilton,et al.  Stereotypes and stereotyping: An overview of the cognitive approach. , 1986 .

[46]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[47]  Robert M. McIntyre,et al.  Effect of rater training on rater accuracy: Levels-of-processing theory and social facilitation theory perspectives. , 1987 .

[48]  Neil M. A. Hauenstein,et al.  FROM LABORATORY TO PRACTICE: NEGLECTED ISSUES IN IMPLEMENTING FRAME‐OF‐REFERENCE RATER TRAINING , 1989 .

[49]  Kevin R. Murphy,et al.  Criterion issues in performance appraisal research: Behavioral accuracy versus classification accuracy. , 1991 .

[50]  David V. Day,et al.  Effects of frame-of-reference training on rater accuracy under alternative time delays , 1994 .

[51]  H. John Bernardin,et al.  Strategies in Rater Training , 1981 .

[52]  Deidra J. Schleicher,et al.  A Cognitive Evaluation of Frame-of-Reference Rater Training: Content and Process Issues. , 1998, Organizational behavior and human decision processes.

[53]  James W. Smither,et al.  True intercorrelation among job components, time delay in rating, and rater intelligence as determinants of accuracy in performance ratings , 1987 .

[54]  K. Murphy,et al.  Effect of purpose of observation on accuracy of immediate and delayed performance ratings , 1989 .

[55]  Jon M. Werner,et al.  EXPLAINING U.S. COURTS OF APPEALS DECISIONS INVOLVING PERFORMANCE APPRAISAL: ACCURACY, FAIRNESS, AND VALIDATION , 1997 .

[56]  Task Force on Assessment Center Guidelines.  Guidelines and ethical considerations for Assessment Center operations , 1989 .