Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility.

This article reviews research indicating that, under appropriate conditions, students' evaluations of teaching (SETs) are (a) multidimensional; (b) reliable and stable; (c) primarily a function of the instructor who teaches a course rather than the course that is taught; (d) relatively valid against a variety of indicators of effective teaching; (e) relatively unaffected by a variety of variables hypothesized as potential biases (e.g., grading leniency, class size, workload, prior subject interest); and (f) useful in improving teaching effectiveness when SETS are coupled with appropriate consultation. The authors recommend rejecting a narrow criterion-related approach to validity and adopting a broad construct-validation approach, recognizing that effective teaching and SETs that reflect teaching effectiveness are multidimensional; no single criterion of effective teaching is sufficient; and tentative interpretations of relations with validity criteria and potential biases should be evaluated critically in different contexts, in relation to multiple criteria of effective teaching, theory, and existing knowledge.

[1]  R. E. Redding,et al.  Students' Evaluations of Teaching Fuel Grade Inflation , 1998 .

[2]  W. McKeachie.  Student ratings: The validity of use. , 1997 .

[3]  P. Abrami,et al.  Navigating student ratings of instruction. , 1997 .

[4]  A. Greenwald,et al.  Grading leniency is a removable contaminant of student ratings. , 1997, The American psychologist.

[5]  Robert E. Haskell,et al.  Academic Freedom, Tenure, and Student Evaluation of Faculty: Galloping Polls in the 21st Century. , 1997 .

[6]  Raymond P. Perry,et al.  Effective teaching in higher education : research and practice , 1997 .

[7]  A. Greenwald.  Validity concerns and usefulness of student ratings of instruction. , 1997, The American psychologist.

[8]  John Hattie,et al.  The Relationship Between Research and Teaching: A Meta-Analysis , 1996 .

[9]  H. Marsh.  Still weighting for the right criteria to validate student evaluations of teaching in the IDEA system , 1995 .

[10]  H. G. Murray,et al.  Using Multiple Outcomes to Validate Student Ratings of Overall Teacher Effectiveness. , 1995 .

[11]  Herbert W. Marsh,et al.  Weighting for the right criteria in the Instructional Development and Effectiveness Assessment (IDEA) system: Global and specific ratings of teaching effectiveness and their relation to course objectives. , 1994 .

[12]  John A. Centra,et al.  Reflective Faculty Evaluation: Enhancing Teaching and Determining Faculty Effectiveness. The Jossey-Bass Higher and Adult Education Series. , 1993 .

[13]  Herbert W. Marsh,et al.  The Use of Students’ Evaluations and an Individually Structured Intervention to Enhance University Teaching Effectiveness , 1993 .

[14]  H. Marsh,et al.  Multidimensional Students' Evaluations of Teaching Effectiveness: A Profile Analysis. , 1993 .

[15]  William E. Cashin,et al.  Using global student rating items for summative evaluation. , 1992 .

[16]  H. Marsh.  A Multidimensional Perspective on Students' Evaluations of Teaching Effectiveness: Reply to Abrami and d'Apollonia (1991). , 1991 .

[17]  Philip C. Abrami,et al.  Multidimensional students' evaluations of teaching effectiveness: Generalizability of "N = 1" research: Comment on Marsh (1991). , 1991 .

[18]  Adrian Blunt,et al.  The Effects of Anonymity and Manipulated Grades on Student Ratings of Instructors , 1991 .

[19]  Herbert W. Marsh,et al.  Students' evaluations of teaching effectiveness: The stability of mean ratings of the same teachers over a 13-year period , 1991 .

[20]  Herbert W. Marsh,et al.  The multidimensionality of students' evaluations of teaching effectiveness: The generality of factor structures across academic discipline, instructor level, and course level , 1991 .

[21]  Kenneth A. Feldman,et al.  The association between student ratings of specific instructional dimensions and student achievement: Refining and extending the synthesis of data from multisection validity studies , 1989 .

[22]  Kenneth A. Feldman,et al.  Instructional effectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators, and external (neutral) observers , 1989 .

[23]  P. Abrami,et al.  Students' Evaluations of University Teaching: Research Findings, Methodological Issues, and Directions for Future Research , 1987 .

[24]  R. C. Wilson.  Improving Faculty Teaching: Effective Use of Student Evaluations and Consultants. , 1986 .

[25]  S. Maxwell,et al.  Construct validity of measures of college teaching effectiveness. , 1985 .

[26]  H. Marsh.  Students’ Evaluations of University Teaching: Dimensionality, Reliability, Validity, Potential Biases and Usefulness , 1984 .

[27]  Kenneth A. Feldman,et al.  Seniority and experience of college teachers as related to evaluations they receive from students , 1983 .

[28]  H. Marsh.  Multidimensional ratings of teaching effectiveness by students from different academic settings and their relation to student/course/instructor characteristics. , 1983 .

[29]  H. G. Murray,et al.  Low-inference classroom teaching behaviors and student ratings of college teaching effectiveness. , 1983 .

[30]  T. Chacko,et al.  Student ratings of instruction: A function of grading standards. , 1983 .

[31]  Philip C. Abrami,et al.  Educational Seduction , 1982 .

[32]  Scott E. Maxwell,et al.  Do grades contaminate student evaluations of instruction? , 1982 .

[33]  J. Ware,et al.  Effects of expressiveness, content coverage, and incentive on multidimensional student rating scales: new interpretations of the Dr. Fox effect , 1982 .

[34]  P. Cranton,et al.  The Relationships Between Student Ratings and Instructor Behavior: Implications for Improving Teaching , 1981 .

[35]  Scott E. Maxwell,et al.  Correlation between student satisfaction and grades: A case of mistaken causation? , 1980 .

[36]  Peter A. Cohen,et al.  Effectiveness of student-rating feedback for improving college instruction: A meta-analysis of findings , 1980 .

[37]  Herbert W. Marsh,et al.  Students' Evaluations of Instruction: A Longitudinal Study of Their Stability. , 1980 .

[38]  W. J. Dickens,et al.  Do Teacher Standards for Assigning Grades Affect Student Evaluations of Instruction , 1980 .

[39]  John A. Centra,et al.  Determining faculty effectiveness , 1980 .

[40]  Paul T. P. Wong,et al.  Effects of earned and assigned grades on student evaluations of an instructor. , 1979 .

[41]  Herbert W. Marsh,et al.  Midterm feedback from students: Its relationship to instructional improvement and students' cognitive and affective outcomes. , 1979 .

[42]  Wilbert J. McKeachie,et al.  Student Ratings of Faculty: A Reprise. , 1979 .

[43]  R. F. Sarmiento,et al.  Liberal Grading Improves Evaluations But Not Performance. , 1979 .

[44]  Michael T. Kane,et al.  The Generalizability of Student Ratings of Instruction: Estimation of the Teacher and Course Components. , 1978 .

[45]  P. Frey.  A two-dimensional analysis of student ratings of instruction , 1978 .

[46]  Robert W. Powell,et al.  Grades, learning, and student evaluation of instruction , 1977 .

[47]  Kenneth A. Feldman,et al.  Grades and college students' evaluations of their courses and teachers , 1976 .

[48]  D. S. Holmes,et al.  Effects of grades and disconfirmed grade expectancies on students' evaluations of their instructor. , 1972 .