Teacher-Based Judgments of Academic Achievement: A Review of Literature

The focus of this paper is on data reflecting the match between teacher-based assessments of students’ achievement levels and an objective measure of student learning. These data are treated as relevant to the validity or accuracy of the judgmental measures. The paper begins with a discussion of two contexts in which such judgments are relevant: the teacher decision-making and assessment contexts. The second section presents a review of studies in which data are presented on the match between judgments and test scores. Two types of studies are reviewed. The first represents an indirect test of validity in the sense that there is a discrepancy between the judgmental measure (usually a rating of achievement) and the criterion measure (a score on a standardized achievement test). The second provides a more direct test of validity in that teachers are directly asked to estimate the achievement test performance of their students. On the whole, the results revealed high levels of validity for the teacher-judgment measures. The studies revealed, however, some variability across teachers in accuracy levels and suggested the operation of certain other moderator variables. The paper concludes with a set of recommendations for future research on the judgments and a set of recommendations for improvements in the teacher-assessment process.

[1]  Martin J. Wiese,et al.  Teacher Judgment in Student Evaluation: A Comparison of Grading Methods , 1988 .

[2]  T. Achenbach,et al.  Child/adolescent behavioral and emotional problems: implications of cross-informant correlations for situational specificity. , 1987, Psychological bulletin.

[3]  O. Egan,et al.  The Accuracy of Teachers’ Ratings of Ability: A Regression Model , 1985 .

[4]  Gaea Leinhardt,et al.  Novice and expert knowledge of individual student's achievement , 1983 .

[5]  R. Hoge,et al.  The use of teacher-judgment measures in the identification of gifted pupils , 1986 .

[6]  Penelope L. Peterson,et al.  Teachers' and Students' Cognitional Knowledge for Classroom Teaching and Learning , 1988 .

[7]  Andreas Helmke,et al.  Interactional effects of instructional quality and teacher judgement accuracy on achievement , 1987 .

[8]  R Wood,et al.  Aptitude and achievement , 1982 .

[9]  Richard J. Shavelson,et al.  Research on Teachers’ Pedagogical Thoughts, Judgments, Decisions, and Behavior , 1981 .

[10]  Henry Beaumont.  The Measurement of Teaching , 1938 .

[11]  T. Coladarci.  Accuracy of teacher judgments of student responses to standardized test items. , 1986 .

[12]  J. Sattler.  Assessment of children, 3rd ed. , 1988 .

[13]  Michael M. Gerber,et al.  Teacher as imperfect test: Reconceptualizing the referral process , 1984 .

[14]  R. Hoge.  Psychometric Properties of Teacher-Judgment Measures of Pupil Aptitudes, Classroom Behaviors, and Achievement Levels , 1983 .

[15]  R. Farr,et al.  MEASURING SUBSKILLS OF READING: INTERCORRELATIONS BETWEEN STANDARDIZED READING TESTS, TEACHERS’ RATINGS, AND READING SPECIALISTS’ RATINGS , 1971 .

[16]  J. M. Cattell.  Aptitude and Achievement , 1926, Nature.

[17]  David Tom,et al.  Influences of student background and teacher authoritarianism on teacher expectations. , 1984 .

[18]  C. Sharpley,et al.  Teachers' Ratings vs. Standardized Tests: An Empirical Investigation of Agreement between Two Indices of Achievement. , 1986 .

[19]  Robert D. Hoge,et al.  Analysis of teacher judgments of pupil achievement levels. , 1984 .

[20]  R. D. Hoge,et al.  Relations Among Teacher Rankings, Pupil-Teacher Interactions, and Academic Achievement: A Test of the Teacher Expectancy Hypothesis , 1978 .

[21]  T. P. Carpenter,et al.  Using Knowledge of Children’s Mathematics Thinking in Classroom Teaching: An Experimental Study , 1989 .

[22]  L. Cronbach,et al.  Conceptual and methodological problems in interpersonal perception. , 1955, Psychological review.

[23]  T. P. Carpenter,et al.  Teachers' pedagogical content knowledge of students' problem solving in elementary arithmetic , 1988 .

[24]  H. Borko,et al.  Research on Teachers' Decisions in Planning Instruction. , 1979 .

[25]  T. P. Carpenter,et al.  Teachers' knowledge of students' knowledge in mathematics problem solving: correlational and case analyses , 1989 .

[26]  D. A. Kenny,et al.  Accuracy in interpersonal perception: a social relations analysis. , 1987, Psychological bulletin.

[27]  Craig Edebrock.  Problems and Issues in Using Rating Scales to Assess Child Personality and Psychopathology. , 1983 .

[28]  Jere Brophy,et al.  Research on the Self-Fulfilling Prophecy and Teacher Expectations. , 1983 .

[29]  Peter W. Airasian,et al.  Do Teacher Ratings and Standardized Test Results of Students Yield the Same Information? , 1980 .

[30]  F. Gresham,et al.  Teachers as "Tests": Classification Accuracy and Concurrent Validation in the Identification of Learning Disabled Children. , 1987 .

[31]  Jere Brophy,et al.  Teacher-student relationships: causes and consequences , 1974 .

[32]  R. Hoge.  The Definition and Measurement of Teacher Expectations: Problems and Prospects. , 1984 .

[33]  Frank M. Gresham,et al.  Social Skills Training with Handicapped Children: A Review , 1981 .

[34]  Michael Conolly,et al.  How accurately can primary school teachers predict the scores of their pupils in standardised tests of attainment? A study of some non-cognitive factors that influence specific judgements. , 1985 .

[35]  J. Dusek,et al.  The Bases of Teacher Expectancies: A Meta-Analysis. , 1983 .

[36]  H. Borko,et al.  Individual Differences in Teachers' Decision Strategies: An Investigation of Classroom Organization and Management Decisions. , 1982 .

[37]  Kathleen McNair.  Capturing Inflight Decisions: Thoughts while Teaching. , 1978 .

[38]  R. Linn.  Educational Testing and Assessment: Research Needs and Policy Issues. , 1986 .

[39]  Peter W. Airasian,et al.  Proportion and Direction of Teacher Rating Changes of Pupils' Progress Attributable to Standardized Test Information. , 1977 .

[40]  A. Silverstein,et al.  Convergent and Discriminant Validation of Two Methods of Assessing Three Academic Traits , 1983 .

[41]  F. Gresham,et al.  Teacher and Observer Ratings of Children's Social Skills: Validation of the Social Skills Rating Scales , 1988 .

[42]  Teachers' Interactive Thoughts About Pupil Cognition , 1982 .

[43]  K. Hopkins,et al.  THE CONCURRENT VALIDITY OF STANDARDIZED ACHIEVEMENT TESTS BY CONTENT AREA USING TEACHERS' RATINGS AS CRITERIA , 1985 .

[44]  R. Rosenthal,et al.  Pygmalion, Galatea, and the Golem: Investigations of biased and unbiased teachers. , 1982 .