Effects of Course and Instructor Characteristics on Student Evaluation of Teaching across a College of Engineering

Background Student evaluations of teaching (SETs) are a widely used metric to evaluate instructor effectiveness and are used to make promotion, tenure, and retention decisions for faculty. There is also growing interest by those outside the university community to use these metrics to evaluate faculty and broader academic performance. Purpose (Hypothesis) This study seeks to understand if and how course and instructor characteristics affect SETs and thereby to improve the usefulness of these metrics. This article aims to statistically examine the relationship between course and instructor characteristics and SETs. Design/Method SETs from a large engineering college at a major public university were evaluated over a seven-semester period that covered 3938 courses taught by 549 unique engineering instructors. Course and instructor demographic data were statistically evaluated for their effects on SETs. Results Course characteristics such as class size, course level, and whether a course was an elective or required had statistically significant effects on SETs. Instructor characteristics of gender and academic rank affected SETs and average course grades, respectively. Average course grades were positively correlated with SETs. Conclusions Data analysis showed that course characteristics, faculty demographics, and average course grades had statistically significant effects on SETs; however, in some cases the effect sizes of these variables were small. Administrators and senior faculty members should be cognizant of these relevant factors and their effects when assigning faculty to certain courses and evaluating their teaching effectiveness using SETs.

[1]  T. Joiner,et al.  Factors influencing teaching evaluations in higher education. , 2006 .

[2]  Markus Dresel,et al.  Counseling University Instructors Based on Student Evaluations of Their Teaching Effectiveness: A Multilevel Test of its Effectiveness Under Consideration of Bias and Unfairness Variables , 2011 .

[3]  Scott P. Hippensteel,et al.  Increasing the Significance of Course Evaluations in Large-Enrollment Geoscience Classes , 2005 .

[4]  S. Bennett,et al.  Student perceptions of and expectations for male and female instructors: Evidence relating to the question of gender bias in teaching evaluation. , 1982 .

[5]  Francisco Zabaleta,et al.  The use and misuse of student evaluations of teaching , 2007 .

[6]  Jane Kenney,et al.  Do grading standards affect student evaluations of teaching? Some new evidence on an old question. , 1982 .

[7]  Larry H. Ludlow,et al.  Understanding Student Evaluations of Teaching Quality: The Contributions of Class Attendance , 2005 .

[8]  James J. Divoky,et al.  Student Perceptions of the Relative Importances of Dimensions of Teaching Performance across Type of Class. , 1988 .

[9]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[10]  Scott E. Maxwell,et al.  Correlation between student satisfaction and grades: A case of mistaken causation? , 1980 .

[11]  Lawrence M. Aleamoni,et al.  Differential Relationships of Student, Instructor, and Course Characteristics to General and Specific Items on a Course Evaluation Questionnaire , 1980 .

[12]  George D. Lovell,et al.  Forced-Choice Applied to College Faculty Rating , 1955 .

[13]  Lawrence M. Aleamoni,et al.  A review of the research on student evaluation and a report on the effect of different sets of instructions on student course and instructor evaluation , 1973 .

[14]  R. Oskrochi,et al.  What matters most? Predictors of student satisfaction in public health educational courses. , 2006, Public health.

[15]  Lee Ellis,et al.  Student Grades and Average Ratings of Instructional Quality: The Need for Adjustment , 2003 .

[16]  D. S. Holmes,et al.  Effects of grades and disconfirmed grade expectancies on students' evaluations of their instructor. , 1972 .

[17]  P. A. Cohen,et al.  Do grades influence students' evaluations of clinical courses? , 1989, Journal of dental education.

[18]  David Watkins,et al.  STUDENT RATINGS OF TERTIARY COURSES FOR ‘ALTERNATIVE CALENDAR’ PURPOSES , 1990 .

[19]  L. Langbein.  Management by Results: Student Evaluation of Faculty Teaching and the Mis-Measurement of Performance. , 2008 .

[20]  Dennis E. Clayson,et al.  Student Evaluations of Teaching: Are They Related to What Students Learn? , 2009 .

[21]  Eugene F. Stone,et al.  Effects of grading practices and time of rating on student ratings of faculty performance and student learning , 1991 .

[22]  N. Gage,et al.  The Appraisal of College Teaching: An Analysis of Ends and Means , 1961 .

[23]  Ram B. Misra,et al.  An Evaluation of Factors Regarding Students’ Assessment of Faculty in a Business School* , 2008 .

[24]  Murray A. Straus,et al.  Class Size and Student Evaluations of Faculty , 1974 .

[25]  J. Chow,et al.  Sources of Variation in Students' Evaluations of Instruction in a Graduate Social Work Program , 1988 .

[26]  H. Kaiser.  The Application of Electronic Computers to Factor Analysis , 1960 .

[27]  P. A. Cohen.  Student Ratings of Instruction and Student Achievement: A Meta-analysis of Multisection Validity Studies , 1981 .

[28]  Herbert W. Marsh,et al.  Class Size, Students’ Evaluations, and Instructional Effectiveness , 1979 .

[29]  Thomas K. Hazlet,et al.  "Fix the Law" project , 2004 .

[30]  N. Gage,et al.  The Appraisal of College Teaching , 1961 .

[31]  John A. Centra,et al.  Research productivity and teaching effectiveness , 1981 .

[32]  L. R. Trick,et al.  Do Grades Affect Faculty Teaching Evaluations , 1993 .

[33]  Harinder Singh,et al.  Do Higher Grades Lead to Favorable Student Evaluations? , 2005 .

[34]  Charles D. Bailey,et al.  Do students’ judgment models of instructor effectiveness differ by course level, course content, or individual instructor? , 2000 .

[35]  Jeffrey B. Adams.  What Makes the Grade? Faculty and Student Perceptions , 2005 .

[36]  Jennifer Franklin.  Grade Inflation and Student Ratings: A Closer Look. , 1991 .

[37]  H. C. Ratz,et al.  Factors in the Evaluation of Instructors by Students , 1975, IEEE Transactions on Education.

[38]  Guy A. Boysen,et al.  Revenge and Student Evaluations of Teaching , 2008 .

[39]  R. Cattell.  The Scree Test For The Number Of Factors. , 1966, Multivariate behavioral research.

[40]  Herbert W. Marsh,et al.  Do university teachers become more effective with experience? A multilevel growth model of students' evaluations of teaching over 13 years. , 2007 .

[41]  James F. Ragan,et al.  Differences in Student Evaluations of Principles and Other Economics Courses and the Allocation of Faculty across Courses , 2010 .

[42]  Kenneth A. Feldman,et al.  College students' views of male and female college teachers: Part II—Evidence from students' evaluations of their classroom teachers , 1993 .

[43]  David E. Kapel,et al.  Assessment of a conceptually based instructor evaluation form , 1974 .

[44]  K. Feldman.  Identifying Exemplary Teachers and Teaching: Evidence from Student Ratings 1 , 2007 .

[45]  H. Marsh.  Students’ Evaluations of University Teaching: Dimensionality, Reliability, Validity, Potential Biases and Usefulness , 1984 .

[46]  Dennis E. Clayson,et al.  Grades and the Student Evaluation of Instruction: A Test of the Reciprocity Effect , 2006 .

[47]  David A. Kravitz,et al.  Effects of Procedural Fairness on Student Judgments of Professors. , 1994 .

[48]  Herbert W. Marsh,et al.  Effects of Grading Leniency and Low Workload on Students' Evaluations of Teaching: Popular Myth, Bias, Validity, or Innocent Bystanders? , 2000 .

[49]  Harvey J. Brightman.  Mentoring Faculty to Improve Teaching and Student Learning , 2005 .

[50]  Richard Remedios,et al.  I Liked Your Course because You Taught Me Well: The Influence of Grades, Workload, Expectations and Goals on Students' Evaluations of Teaching. , 2008 .

[51]  H. Schuckman,et al.  Students' Perceptions of Faculty and Graduate Students as Classroom Teachers , 1990 .

[52]  Joakim Westerlund.  Class Size and Student Evaluations in Sweden , 2006 .

[53]  Adrian Blunt,et al.  The Effects of Anonymity and Manipulated Grades on Student Ratings of Instructors , 1991 .

[54]  Blaine R. Worthen,et al.  Class Level as a Possible Mediator of the Relationship between Grades and Student Ratings of Teaching , 1997 .

[55]  J. Centra,et al.  Is There Gender Bias in Student Evaluations of Teaching? , 2000 .

[56]  K. Feldman.  Research productivity and scholarly accomplishment of college teachers as related to their instructional effectiveness: A review and exploration , 1987 .

[57]  Samer Kherfi,et al.  Whose Opinion Is It Anyway? Determinants of Participation in Student Evaluation of Teaching , 2009 .

[58]  Myungsup Kim,et al.  What Determines Student Evaluation Scores? A Random Effects Analysis of Undergraduate Economics Classes , 2009 .

[59]  Gerald F. Goldberg,et al.  Objectivity of Student Evaluations of Instructors , 1991 .

[60]  Carol L. Patrick.  Student evaluations of teaching: effects of the Big Five personality traits, grades and the validity hypothesis , 2011 .

[61]  Scott E. Maxwell,et al.  Do grades contaminate student evaluations of instruction? , 1982 .

[62]  Wen-ying Lin,et al.  Is Class Size a Bias to Student Evaluations of Faculty? A Review. , 1992 .

[63]  Pieter Spooren,et al.  On the credibility of the judge: A cross-classified multilevel analysis on students’ evaluation of teaching , 2010 .

[64]  Patricia B. Elmore,et al.  Effect of teacher sex, student sex, and teacher warmth on the evaluation of college instructors. , 1975 .

[65]  Richard A. Johnson,et al.  A new family of power transformations to improve normality or symmetry , 2000 .

[66]  Robert S. Kidd,et al.  Student Evaluations: Are They Valid Measures of Course Effectiveness? , 2004 .

[67]  Michael A. McPherson.  Determinants of How Students Evaluate Teachers , 2006 .

[68]  R. Todd Jewell,et al.  Leveling the Playing Field: Should Student Evaluation Scores be Adjusted?* , 2007 .

[69]  Miguel A. Mateo,et al.  Incidence of Class Size on the Evaluation of University Teaching Quality , 1996 .

[70]  Larry H. Ludlow,et al.  Can Downsizing College Class Sizes Augment Student Outcomes?: An Investigation of the Effects of Class Size on Student Learning , 2010 .

[71]  J. Pohlmann,et al.  A Multivariate Analysis of Selected Class Characteristic and Student Rating of Instruction. , 1975, Multivariate behavioral research.

[72]  H. Marsh,et al.  Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. , 1997 .