Systematic Influences on Teaching Evaluations: The Case for Caution

In this paper, we examine eight years of Quality of Teaching (QOT) responses from an Economics Department in an Australian University. This is done to determine what factors, besides the instructor, have an impact on the raw average student evaluation scores. Most of the previous research on student ratings has been conducted in the US. One significant difference between US and Australian tertiary education is that, on average, the number of foreign undergraduate students in Australia is ten times the number in US institutions. We find that cultural background significantly affects student evaluations. Other factors that have an influence on the average QOT score include: year level; enrolment size; the quantitative nature of the subject; the gender of the student; fee-paying status by gender; course of study; the differences between the course mark and previous marks; the quality of workbooks; the quality of textbooks; and the QOT score relative to those in other subjects taught at the same time. In addition, average QOT scores for instructors who have taught in a mix of subjects are similar to those based on scores adjusted to account for subject and student characteristics.

[1]  Omer Gokcekus.  How do university students value economics courses? A hedonic approach , 2000 .

[2]  Kwok-fai Ting,et al.  A Multilevel Perspective on Student Ratings of Instruction: Lessons from the Chinese Experience , 2000 .

[3]  K. Arrow.  Higher education as a filter , 1973 .

[4]  H. White.  A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity , 1980 .

[5]  W. McKeachie,et al.  Teaching tips: A guidebook for the beginning college teacher. , 1986 .

[6]  Li-fang Zhang,et al.  Do university students’ thinking styles matter in their preferred teaching approaches? , 2004 .

[7]  P. Abrami,et al.  Students' Evaluations of University Teaching: Research Findings, Methodological Issues, and Directions for Future Research , 1987 .

[8]  Philip Banyard,et al.  The Validity of Student Evaluation of Teaching in Higher Education: Love me, love my lectures? , 2000 .

[9]  Youth,et al.  Guide to good practice , 1993 .

[10]  A. Worthington.  The Impact of Student Perceptions and Characteristics on Teaching Evaluations: A case study in finance education , 2002 .

[11]  Ruth Williams,et al.  Collecting and using student feedback Date: A guide to good practice , 2004 .

[12]  Harinder Singh,et al.  Do Higher Grades Lead to Favorable Student Evaluations? , 2005 .

[13]  S. Bigger,et al.  Teaching appraisal in higher education: an Australian perspective , 1997 .

[14]  Kenneth A. Feldman,et al.  Course characteristics and college students' ratings of their teachers: What we know and what we don't , 1978 .

[15]  Judy A. Siguaw,et al.  Student Evaluations of Teaching: An Exploratory Study of the Faculty Response , 2000 .

[16]  S. Basow.  Best and Worst Professors: Gender Patterns in Students' Choices , 2000 .

[17]  Anthony G. Greenwald,et al.  No Pain , No Gain ? The Importance of Measuring Course Workload in Student Ratings of Instruction , 1997 .

[18]  Jeffrey W. Steagall,et al.  Student evaluations of faculty: A new procedure for using aggregate measures of performance , 1995 .

[19]  Terri A. Scandura,et al.  Grade Inflation and Student Individual Differences as Systematic Bias in Faculty Evaluations. , 2005 .

[20]  Peter Kuhn,et al.  Making Nutritional Information Digestible : Effects of a Receipt-Based Intervention on Fast Food Purchases , 2013 .

[21]  J. P. Rushton,et al.  Teacher personality traits and student instructional ratings in six types of university courses , 1990 .

[22]  J. C. Ory,et al.  Assessing Faculty Work: Enhancing Individual and Institutional Performance , 1994 .

[23]  J. Ware,et al.  Effects of expressiveness, content coverage, and incentive on multidimensional student rating scales: new interpretations of the Dr. Fox effect , 1982 .

[24]  Susan A. Basow,et al.  Student evaluations of college professors: Are female and male professors rated differently? , 1987 .

[25]  William D. Bosshardt,et al.  How Instructors Make a Difference: Panel Data Estimates from Principles of Economic Courses , 1991 .

[26]  W. McKeachie.  Teaching tips: A guidebook for the beginning college teacher , 1963 .

[27]  J. P. Rushton,et al.  Teacher personality traits and student instructional ratings in six types of university courses , 1990 .

[28]  S. Simmons.  Reflective faculty evaluation: Enhancing teaching and determining faculty effectiveness , 1996 .

[29]  Jon P. Nelson,et al.  Grade Inflation, Real Income, Simultaneity, and Teaching Evaluations , 1984 .

[30]  D. Kember,et al.  Does the Use of Student Feedback Questionnaires Improve the Overall Quality of Teaching? , 2002 .

[31]  Ruth Neumann,et al.  Communicating Student Evaluation of Teaching Results: Rating Interpretation Guides (RIGs) , 2000 .

[32]  Dennis J. Aigner,et al.  On Student Evaluation of Teaching Ability , 1986 .

[33]  Jeffrey Chin,et al.  Faculty Assessment and the Scholarship of Teaching and Learning: Knowledge Available/Knowledge Needed. , 2003 .

[34]  L. F. Jameson Boex,et al.  Attributes of Effective Economics Instructors: An Analysis of Student Evaluations , 2000 .

[35]  William E. Cashin.  Student Ratings of Teaching: The Research Revisited. IDEA Paper No. 32. , 1995 .

[36]  G. Palmer.  The Ideal Teacher , 2008 .

[37]  Stephen J. DeCanio,et al.  Student Evaluations of Teaching—A Multinominal Logit Approach , 1986 .

[38]  Michael A. McPherson.  Determinants of How Students Evaluate Teachers , 2006 .

[39]  Philip C. Abrami,et al.  Educational Seduction , 1982 .

[40]  K. Goh,et al.  BIASES IN STUDENT EVALUATION OF TEACHING: THE CASE OF FACULTY OF ECONOMICS & ADMINISTRATION, UNIVERSITY OF MALAYA , 2002 .

[41]  Frank Costin,et al.  Student Ratings of College Teaching: Reliability, Validity, and Usefulness , 1973 .

[42]  William D. Bosshardt,et al.  Comparing Student and Instructor Evaluations of Teaching , 2001 .