Can you increase teacher engagement with evaluation simply by improving the evaluation system?

We know various factors can influence how teaching staff engage with student evaluation, such as institutional policies or staff beliefs. However, little research has investigated the influence of the technical processes of an evaluation system. In this article, we present a case study of the effects of changing the technical system for administering student evaluations at one New Zealand university. We develop a socio-technical model of the institutional evaluation system, and use this model to examine whether introducing an online system for ordering student feedback questionnaires and reducing processing time influenced academic staff engagement with evaluation. Survey responses, interview comments and data about ordering trends suggest the change did increase staff engagement by: (1) improving staff perceptions of evaluation and (2) increasing engaged behaviour, such as voluntarily ordering more evaluations. The outcomes of this study imply that the ‘practical implementation’ of an evaluation system is an important factor in influencing engagement with evaluation. We conclude that we can increase teacher engagement with evaluation simply by improving the ‘practical implementation’ of the evaluation system.

[1]  M. Reeves It can be done. , 1969, Nursing mirror and midwives journal.

[2]  J. Newton,et al.  Feeding the Beast or Improving Quality?: Academics' perceptions of quality assurance and quality monitoring , 2000 .

[3]  K. Ryan,et al.  How Do Student Ratings Measure Up to a New Validity Framework , 2001 .

[4]  D. Kember,et al.  Does the Use of Student Feedback Questionnaires Improve the Overall Quality of Teaching? , 2002 .

[5]  F. Geels From sectoral systems of innovation to socio-technical systems: Insights about dynamics and change from sociology and institutional theory , 2004 .

[6]  R. Coe,et al.  Effectiveness of Consultation on Student Ratings Feedback: A Meta-Analysis , 2004 .

[7]  Robert W. Hanna,et al.  Gathering faculty teaching evaluations by in‐class and online surveys: their effects on response rates and evaluations , 2004 .

[8]  Enid Mumford,et al.  The story of socio‐technical design: reflections on its successes, failures and potential , 2006, Inf. Syst. J..

[9]  G. Hendry,et al.  Teachers' approaches to teaching and responses to student evaluation in a problem‐based medical program , 2007 .

[10]  Duncan David Nulty,et al.  The adequacy of response rates to online and paper surveys: what can be done? , 2008 .

[11]  Calvin Smith Building effectiveness in teaching through targeted evaluation and response: connecting evaluation to teaching improvement in higher education , 2008 .

[12]  Kristina Edström,et al.  Doing course evaluation as if learning matters most , 2008 .

[13]  Tanya N. Beran,et al.  Instructors’ perspectives on the utility of student ratings of instruction , 2009 .

[14]  Tena B. Crews,et al.  Online Course Evaluations: Faculty Perspective and Strategies for Improved Response Rates , 2011 .

[15]  Nathan E. Gonyea,et al.  Reining in student comments: a model for categorising and studying online student comments , 2012 .

[16]  Sangmi Chai,et al.  A socio-technical approach to knowledge contribution behavior: An empirical investigation of social networking sites users , 2012, Int. J. Inf. Manag..

[17]  V. Bamber,et al.  Evaluating learning and teaching: institutional needs and individual practices , 2012 .

[18]  William E. Cashin,et al.  Student Ratings of Teaching : A Summary of Research and Literature , 2012 .

[19]  Sarah J. Stein,et al.  Tertiary teachers and student evaluations: never the twain shall meet? , 2013 .

[20]  Bart Rienties,et al.  Understanding academics’ resistance towards (online) student evaluation , 2014 .

[21]  Steve Sawyer,et al.  Sociotechnical Approaches to the Study of Information Systems , 2014, Computing Handbook, 3rd ed..