Measurement Error in Psychological Research: Lessons From 26 Research Scenarios

technical principles to concrete research problems that appear in a wide variety of forms and with a myriad of obscuring features, details, and idiosyncracies. This article is based on a different approach: examination of a series of concrete research scenarios that we have encountered in our work as researchers, advisors to researchers, or reviewers. This article examines 26 real-world "case studies" and explicates them on the basis of the principles of reliability theory found in Cronbach (1947, 1951) and Cronbach et al. (1972). Although we cite and rely on these sources, other sources (e.g., Thorndike, 1951) would yield identical resolutions; only the terminology would be sometimes (slightly) different. In classical measurement theory, the fundamental general formula for the observed correlation between any two measures, x and y, is yields the dissattenuation formula:

[1]  J. Hedge,et al.  Personnel selection. , 1997, Annual review of psychology.

[2]  M Borenstein,et al.  The case for confidence intervals in controlled clinical trials. , 1994, Controlled clinical trials.

[3]  Steven D. Maurer,et al.  The validity of employment interviews: A comprehensive review and meta-analysis. , 1994 .

[4]  Mark W. Lipsey,et al.  The efficacy of psychological, educational, and behavioral treatment. Confirmation from meta-analysis. , 1993, The American psychologist.

[5]  K. Murphy.  Personnel Selection in Organizations , 1992 .

[6]  Frank L. Schmidt,et al.  What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. , 1992 .

[7]  John E. Hunter,et al.  Development of a Causal Model of Processes Determining Job Performance , 1992 .

[8]  John E. Hunter,et al.  Methods of Meta-Analysis: Correcting Error and Bias in Research Findings , 1991 .

[9]  Hannah R. Rothstein,et al.  Interrater reliability of job performance ratings: Growth to asymptote level with increasing opportunity to observe. , 1990 .

[10]  John P. Campbell,et al.  Modeling the performance prediction problem in industrial and organizational psychology. , 1990 .

[11]  R. Linn.  Educational measurement, 3rd ed. , 1989 .

[12]  Phillip L. Ackerman,et al.  Within-task intercorrelations of skilled performance: Implications for predicting individual differences? A comment on Henry & Hulin, 1987. , 1989 .

[13]  John B. Willett,et al.  Satisfying a Simplex Structure is Simpler than it should be , 1985 .

[14]  Joseph M. Hillery,et al.  FURTHER WITHIN-SETTING EMPIRICAL TESTS OF THE SITUATIONAL SPECIFICITY HYPOTHESIS IN PERSONNEL SELECTION , 1984 .

[15]  Gregg B. Jackson,et al.  Meta-Analysis: Cumulating Research Findings Across Studies , 1982 .

[16]  John E. Hunter,et al.  Validity generalization results for two job groups in the petroleum industry. , 1981 .

[17]  John E. Hunter,et al.  Halo in a multidimensional forced-choice performance evaluation scale. , 1980 .

[18]  John E. Hunter,et al.  Validity generalization results for tests used to predict job proficiency and training success in clerical occupations. , 1980 .

[19]  Eva Nick,et al.  The dependability of behavioral measurements: theory of generalizability for scores and profiles , 1973 .

[20]  Donald B. Rubin,et al.  The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. , 1974 .

[21]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[22]  P. C. Smith,et al.  Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. , 1963 .

[23]  Angus Campbell,et al.  The American voter , 1960 .

[24]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[25]  L. Cronbach.  Test “reliability”: Its meaning and determination , 1947, Psychometrika.