Counseling University Instructors Based on Student Evaluations of Their Teaching Effectiveness: A Multilevel Test of its Effectiveness Under Consideration of Bias and Unfairness Variables

Counseling instructors using evaluations made by their students has shown to be a fruitful approach to enhancing teaching quality. However, prior experimental studies are questionable in terms of external validity. Therefore, we conducted a non-experimental intervention study in which all of the courses offered by a specific department at a German university were evaluated twice with a standardized student evaluation questionnaire (HILVE-II; overall 44 instructors, 140 courses, and 2,546 student evaluations). Additionally, twelve full time instructors received counseling after the first measurement point. Long-term effects over a period of 2 years and transfer effects to other courses were analyzed using multi-level analyses with three levels. Possible influences by bias and unfairness variables were controlled for. Our results indicate a moderate to large effect of counseling on teaching quality. In conclusion, if students’ evaluations are accompanied by counseling based on the evaluation results, they present a useful method to assure and increase teaching quality in higher education.

[1]  Glenn R. Erickson,et al.  Improving College Teaching: An Evaluation of a Teaching Consultation Procedure , 1979 .

[2]  G. Meisenberg,et al.  Quality of Instruction Improved by Evaluation and Consultation of Instructors , 2007 .

[3]  R. C. Wilson.  Improving Faculty Teaching: Effective Use of Student Evaluations and Consultants. , 1986 .

[4]  A. Rotem,et al.  The effects of feedback from students to university instructors: An experimental study , 1978 .

[5]  George S. Howard,et al.  Methodological considerations in the evaluation of a teacher-training program. , 1980 .

[6]  Raymond P. Perry,et al.  The Scholarship of Teaching and Learning in Higher Education: An Evidence-Based Perspective , 2007 .

[7]  Kathleen T. Brinko,et al.  Effects of Student Evaluation Feedback: A Meta-Analysis of Higher Education Research. , 1986 .

[8]  Christopher Knapper,et al.  Using consultants to improve teaching , 1999 .

[9]  Carolina Cristi,et al.  The impact of individual consultation on student ratings of teaching , 1999 .

[10]  P. A. Cohen.  Effectiveness of student ratings feedback and consultation for improving instruction in dental school. , 1991, Journal of dental education.

[11]  H. Marsh.  Students’ Evaluations of University Teaching: Dimensionality, Reliability, Validity, Potential Biases and Usefulness , 1984 .

[12]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[13]  K. Ting.  Cross-level effects of class characteristics on students' perceptions of teaching quality , 2000 .

[14]  Kathleen T. Brinko,et al.  Methodological explanations for the modest effects of feedback from student ratings. , 1990 .

[15]  P. Cranton,et al.  Reconsidering the Unit of Analysis: A Model of Student Ratings of Instruction. , 1990 .

[16]  John A. Centra,et al.  Will Teachers Receive Higher Student Evaluations by Giving Higher Grades and Less Course Work? , 2003 .

[17]  John T. E. Richardson,et al.  Instruments for obtaining student feedback: a review of the literature , 2005 .

[18]  H. Rindermann,et al.  Generalizability of Multidimensional Student Ratings of University Instruction Across Courses and Teachers , 2001 .

[19]  Raymond P. Perry,et al.  Effective teaching in higher education : research and practice , 1997 .

[20]  S. Alexander,et al.  The Influence of Individual- and Class-Level Fairness-Related Perceptions on Student Satisfaction. , 2005 .

[21]  Herbert W. Marsh,et al.  Students' evaluations of teaching effectiveness: The stability of mean ratings of the same teachers over a 13-year period , 1991 .

[22]  A. Greenwald.  Validity concerns and usefulness of student ratings of instruction. , 1997, The American psychologist.

[23]  R. Coe,et al.  Effectiveness of Consultation on Student Ratings Feedback: A Meta-Analysis , 2004 .

[24]  F. Försterling,et al.  Attributional retraining: a review. , 1985, Psychological bulletin.

[25]  K. Heller,et al.  The Benefit of Gifted Classes and Talent Schools for Developing Students' Competences and Enhancing Academic Self-Concept , 2005 .

[26]  Herbert W. Marsh,et al.  Effects of Grading Leniency and Low Workload on Students' Evaluations of Teaching: Popular Myth, Bias, Validity, or Innocent Bystanders? , 2000 .

[27]  Wilbert J. McKeachie,et al.  Good Teaching Makes a Difference—And we Know What It Is , 2007 .

[28]  H. Marsh,et al.  Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. , 1997 .

[29]  H. G. Murray,et al.  Speech and drama training for lectures as a means of improving university teaching , 1980 .

[30]  K. Feldman.  Identifying Exemplary Teachers and Teaching: Evidence from Student Ratings 1 , 2007 .

[31]  Herbert W. Marsh,et al.  Do university teachers become more effective with experience? A multilevel growth model of students' evaluations of teaching over 13 years. , 2007 .

[32]  Robert A. Reiser,et al.  Effects of a Theory-Based Feedback and Consultation Process on Instruction and Learning in College Classrooms , 2004 .

[33]  Kathleen T. Brinko.  Instructional Consultation with Feedback in Higher Education. , 1990 .

[34]  Herbert W. Marsh,et al.  SEEQ: A Reliable, Valid, and Useful Instrument for Collecting Students' Evaluations of University Teaching. , 1982 .

[35]  Philip C. Abrami,et al.  The Dimensionality of Student Ratings of Instruction: What We Know and What We Do Not* , 2007 .

[36]  W. McKeachie,et al.  Using Student Ratings and Consultation to Improve Instruction. , 1980 .