A Systematic Analysis of Peer Assessment in the MOOC Era and Future Perspectives

Massive Online Open Courses (MOOCs) have become a cost and time effective choice for learners all across the globe. This has led to new challenges for teachers such as providing valuable and quality assessment and feedback on such a large scale. Recent studies have found peer assessment where learners assess the work of their peers to be a viable and cost effective alternative to teacher/staff evaluation. This study systematically analyzes the current research on peer assessment published in the context of MOOCs and the online tools that are being used in MOOCs for peer assessment. 48 peer reviewed papers and 17 peer assessment tools were selected for the comparison in this study and were assessed on three main dimensions, namely, system design, efficiency and effectiveness. In the light of the comparison and discussion of current research in terms of these categories, we present future visions and research dimensions to improve the peer assessment process in MOOCs. Keywords-Open Assessment; Peer Assessment; MOOC; Blended Learning; Peer Reviews, Online Assessment.

[1]  George Siemens.  Connectivism: A Learning Theory for the Digital Age , 2004 .

[2]  Andrii Vozniuk,et al.  Peer assessment based on ratings in a social media course , 2014, LAK '14.

[3]  Ulrik Schroeder,et al.  The Effect of Peer Assessment Rubrics on Learners' Satisfaction and Performance Within a Blended MOOC Environment , 2015, CSEDU.

[4]  Michael S. Bernstein,et al.  PeerStudio: Rapid Peer Feedback Emphasizes Revision and Improves Performance , 2015, L@S.

[5]  Claire Harris,et al.  Cognitive Mapping in Organizational Research , 2004 .

[6]  Amanda M. Clifford,et al.  ‘Just enough to make you take it seriously’: exploring students’ attitudes towards peer assessment , 2013 .

[7]  Daniel M. Russell,et al.  Self-evaluation in advanced power searching and mapping with google MOOCs , 2014, L@S.

[8]  Christian D. Schunn,et al.  Students’ perceptions about peer assessment for writing: their origin and impact on revision work , 2011 .

[9]  P. A. Carlson,et al.  Calibrated peer review/sup TM/ and assessing learning outcomes , 2003, 33rd Annual Frontiers in Education, 2003. FIE 2003..

[10]  Michael S. Bernstein,et al.  Scaling short-answer grading by combining peer assessment with algorithmic scoring , 2014, L@S.

[11]  Jan Marco Leimeister,et al.  Theory-driven Design of an IT-based Peer Assessment to Assess High Cognitive Levels of Educational Objectives in Large-scale Learning Services , 2015, ECIS.

[12]  Anthony C. Robinson,et al.  Peer Grading in a MOOC: Reliability, Validity, and Perceived Effects. , 2014 .

[13]  Lino Montoro Moreno,et al.  Student perceptions of peer assessment: an interdisciplinary study , 2014 .

[14]  Arlene Fink,et al.  Conducting research literature reviews : from the internet to paper , 2014 .

[15]  Yao-Ting Sung,et al.  The design and application of a web-based self- and peer-assessment system , 2005, Comput. Educ..

[16]  Tim Vogelsang,et al.  On the validity of peer grading and a cloud teaching assistant system , 2015, LAK.

[17]  Steve Joordens,et al.  The Pedagogical Anatomy of Peer-Assessment : Dissecting a peerScholar Assignment , 2013 .

[18]  Fiona Spence,et al.  Peer assessment using aropä , 2007 .

[19]  Krzysztof Z. Gajos,et al.  Organic Peer Assessment , 2014 .

[20]  Ulrik Schroeder,et al.  MOOCs - A Review of the State-of-the-Art , 2014, CSEDU.

[21]  Andrew Luxton-Reilly,et al.  A systematic review of tools that support peer assessment , 2009, Comput. Sci. Educ..

[22]  Patris van Boxel,et al.  Summative peer assessment using ‘Turnitin’ and a large cohort of students: a case study , 2006 .

[23]  Thorsten Joachims,et al.  Bayesian Ordinal Peer Grading , 2015, L@S.

[24]  K. Topping.  Peer Assessment Between Students in Colleges and Universities , 1998 .

[25]  R. B. O'Toole,et al.  Pedagogical strategies and technologies for peer assessment in Massively Open Online Courses (MOOCs) , 2013 .

[26]  Ying Liu,et al.  Toward Motivating Participants to Assess Peers’ Work More Fairly , 2014, ArXiv.

[27]  Peter Willmot,et al.  MULTI-DISCIPLINARY PEER-MARK MODERATION OF GROUP WORK: THE CASE OF WEBPA , 2012 .

[28]  Ben He,et al.  Automated Essay Scoring by Maximizing Human-Machine Agreement , 2013, EMNLP.

[29]  Edward F. Gehringer,et al.  Electronic peer review and peer grading in computer-science courses , 2001, SIGCSE '01.

[30]  Damith C. Rajapakse,et al.  Teammates: A cloud-based peer evaluation tool for student team projects , 2011, CSEE&T.

[31]  Hugh Somervell,et al.  Issues in Assessment, Enterprise and Higher Education: the case for self‐peer and collaborative assessment , 1993 .