An Evaluation of Semi-Automated, Collaborative Marking and Feedback Systems: Academic Staff Perspectives.

Online marking and feedback systems are critical for providing timely and accurate feedback to students and maintaining the integrity of results in large class teaching. Previous investigations have involved much in-house development and more consideration is needed for deploying or customising off the shelf solutions. Furthermore, keeping up to date with the state of the art from both academia and industry is essential. This paper is motivated by a project aiming to identify a marking and feedback system for deployment at the authors' university. A detailed investigation is described which is open minded towards adopting or modifying an existing product, or the implementation of a new solution, with key features and shortcomings described in detail. Moodle Workshops, Turnitin GradeMark, Waypoint and WebMark were shortlisted and carried forward for user analysis testing. The outcomes have not only provided key conclusions concerning the suitability of existing solutions, but resulted in a comprehensive collection of functional requirements that leaders of new projects should consider. This paper should be of interest for anyone considering the adoption or upgrade of any marking and feedback system at their home institution.

[1]  David Sharp,et al.  Technical Review of Plagiarism Detection Software Report , 2001 .

[2]  Mark R. Shortis,et al.  A review of the status of online, semi-automated marking and feedback systems , 2009 .

[3]  Peter Henderson.  Electronic Grading and Marking: A Note on Turnitin’s Grademark Function , 2008 .

[4]  Helen J. Parkin,et al.  Using technology to encourage student engagement with feedback: a literature review , 2011 .

[5]  Denise Wood,et al.  CAFAS: an automated computer aided assessment tool for providing student feedback and managing assessment information , 2006 .

[6]  Alistair Campbell,et al.  Application of ICT and rubrics to the assessment process where professional judgement is involved: the features of an e‐marking tool , 2005 .

[7]  David Jackson,et al.  Grading student programs using ASSYST , 1997, SIGCSE '97.

[8]  Kenneth J. Stevens,et al.  The Introduction and Assessment of Three Teaching Tools (WebCT, Mindtrail, EVE) into a Post Graduate Course , 2002, J. Inf. Technol. Educ..

[9]  Pete Thomas,et al.  Reducing the distance in distance education , 2000 .

[10]  David Nicol,et al.  From monologue to dialogue: improving written feedback processes in mass higher education , 2010 .

[11]  Fintan Culwin,et al.  A REVIEW OF ELECTRONIC SERVICES FOR PLAGIARISM DETECTION IN STUDENT SUBMISSIONS , 2000 .

[12]  Chun Zhang,et al.  Storing and querying ordered XML using a relational database system , 2002, SIGMOD '02.

[13]  Shelly Savage.  Staff and Student Responses to a Trial of Turnitin Plagiarism Detection Software , 2004 .

[14]  R. Higgins,et al.  The Conscientious Consumer: Reconsidering the role of assessment feedback in student learning , 2002 .

[15]  Darrall Thompson,et al.  Integrating Graduate Attributes with Student Self-Assessment , 2007 .

[16]  Michael Luck,et al.  Effective electronic marking for on-line assessment , 1998, ITiCSE '98.

[17]  Salvatore Valenti,et al.  An Overview of Current Research on Automated Essay Grading , 2003, J. Inf. Technol. Educ..

[18]  Eva Heinrich,et al.  Literature review on the use of e-learning tools for formative essay-type assessment , 2006 .

[19]  Daryl J. D'Souza.  Management of Teaching in a Complex Setting Steven Burrows , 2022 .

[20]  Shane McKenzie.  Assessing quality of feedback in online marking databases : An opportunity for academic professional development or just Big Brother ? , 2004 .

[21]  Carl Reidsema,et al.  Educational design and online support for an innovative project-based course in engineering design , 2006 .

[22]  Daniel Shawcross Wilkerson,et al.  Winnowing: local algorithms for document fingerprinting , 2003, SIGMOD '03.

[23]  George Fernandez,et al.  Weblearn : a common gateway interface ( CGI)-based enviroment for interactive learning , 2001 .

[24]  Ruth Grady,et al.  Staff and student perceptions of computer-assisted assessment for physiology practical classes. , 2006, Advances in physiology education.

[25]  Bruce Armstrong,et al.  Electronic assessment software for distance education , 2000, Proceedings International Workshop on Advanced Learning Technologies. IWALT 2000. Advanced Learning Technology: Design and Development Issues.

[26]  Anne Gardner,et al.  Assessment for learning: using minor assessment to promote major learning , 2009 .

[27]  M. Price,et al.  Feedback : all that effort, but what is the effect? , 2010 .

[28]  P. J. Radcliffe,et al.  Plagiarism : a Survival Strategy or Recipe for Disaster? , 2007 .

[29]  Michael Philippsen,et al.  Finding Plagiarisms among a Set of Programs with JPlag , 2002, J. Univers. Comput. Sci..

[30]  Rosemary Skeele,et al.  Innovation in e-Assessment: Exploring a Multidimensional Tool , 2007 .