A Comparison of Rating Formats After Corrections for Attenuation

It was noted that format comparisons for performance rating scales rarely report the most critical statistic in terms of assessing score comparability; i.e., the disattenuated correlation of scores between formats. This study examined the degree to which scores on different formats are correlated after disattenuation. Corrected correlations ranged from .7 to above 1.00. While it appears the scales are measuring essentially the same thing, several cautionary statements are made regarding the interpretation of the results. The failure to consider the purpose for data collection in these studies was one major problem cited.

[1]  H. Weiss,et al.  Effects of Standard Extremity on Mixed Standard Scale Performance Ratings. , 1984 .

[2]  Nambury S. Raju,et al.  A Comparison of Five Methods for Combining Multiple Criteria into a Single Composite , 1982 .

[3]  Gary P. Latham,et al.  EFFECTS OF TRAINING AND RATING SCALES ON RATING ERRORS , 1982 .

[4]  W. Sauser,et al.  Shortcut Methods for Deriving Behaviorally Anchored Rating Scales , 1981 .

[5]  H. John Bernardin,et al.  A clarification of some issues regarding the development and use of behaviorally anchored ratings scales (BARS). , 1981 .

[6]  A. R. Bass,et al.  A CRITICAL ANALYSIS OF STUDIES COMPARING BEHAVIORALLY ANCHORED RATING SCALES (BARS) AND OTHER RATING FORMATS , 1981 .

[7]  John E. Hunter,et al.  Halo in a multidimensional forced-choice performance evaluation scale. , 1980 .

[8]  Sheldon Zedeck,et al.  EXPECTATIONS OF BEHAVIORALLY ANCHORED RATING SCALES , 1980 .

[9]  H. J. Bernardin,et al.  A Critical Assessment of Mixed Standard Rating Scales. , 1980 .

[10]  T. Dickinson,et al.  A comparison of the behaviorally anchored rating and mixed standard scale formats. , 1980 .

[11]  W. Borman,et al.  Format and training effects on rating accuracy and rater errors , 1979 .

[12]  H. G. Osburn,et al.  Behaviorally based rating scales: Effects of specific anchors and disguised scale continua. , 1977 .

[13]  H. John Bernardin,et al.  Behavioral expectation scales versus summated scales: A fairer comparison. , 1977 .

[14]  Gary P. Latham,et al.  BEHAVIORAL OBSERVATION SCALES FOR PERFORMANCE APPRAISAL PURPOSES , 1977 .

[15]  Robert F. Burnaska,et al.  An empirical comparison of the relative effects of rater response biases on three rating scale formats. , 1974 .

[16]  Richard D. Arvey,et al.  A Guttman approach to the development of behaviorally based rating scales for systems analysts and programmer/analysts. , 1974 .

[17]  Marvin D. Dunnette,et al.  The development and evaluation of behaviorally based rating scales , 1973 .

[18]  Edwin E. Ghiselli,et al.  THE MIXED STANDARD SCALE: A NEW RATING SYSTEM , 1972 .

[19]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[20]  C. Bartlett,et al.  Effect of instructional conditions in producing leniency on two types of rating scales. , 1969 .

[21]  J. Lepkowski,et al.  Development of a forced-choice rating scale for engineer evaluation. , 1963 .

[22]  James R. Berkshire,et al.  Forced-Choice Performance Rating—A Methodological Study* , 1953 .

[23]  A. W. Bendig The reliability of self-ratings as a function of the amount of verbal anchoring and of the number of categories on the scale. , 1953 .

[24]  L. Mcquitty,et al.  A New Application of Forced‐Choice Ratings* , 1950 .

[25]  H. John Bernardin,et al.  Performance appraisal : assessing human behavior at work , 1984 .

[26]  C. Bartlett,et al.  What's the difference between valid and invalid halo? Forced-choice measurement without forcing a choice. , 1983 .

[27]  Raymond Lee,et al.  Multitrait-multimethod-multirater analysis of performance ratings for law enforcement personnel. , 1981 .

[28]  Zur Shapira,et al.  New issues in the use of behaviorally anchored rating scales: Level of analysis, the effects of incident frequency, and external validation. , 1980 .

[29]  D. W. Zimmerman,et al.  Is classical test theory "robust" under violation of the assumption of uncorrelated errors? , 1980 .

[30]  J. Cotton,et al.  The general applicability of a scale for rating research productivity. , 1960 .