Peer Grading in a MOOC: Reliability, Validity, and Perceived Effects.

Peer grading affords a scalable and sustainable way of providing assessment and feedback to a massive student population, and has been used in massive open online courses (MOOCs) on the Coursera platform. However, currently there is little empirical evidence to support the credentials of peer grading as a learning assessment method in the MOOC context. To address this research need, this study examined 1825 peer grading assignments collected from a Coursera MOOC with the purpose of investigating the reliability and validity of peer grading as well as its perceived effects on students’ MOOC learning experience. The empirical findings proved that the aggregate ratings of student graders can provide peer grading scores that were fairly consistent and highly similar to the instructor grading scores. Student responses to a survey also show that the peer grading activity was well received as the majority of MOOC students believed it was fair, useful, beneficial, and would recommend it to be included in future MOOC offerings. Based on the empirical results, this study concludes with a set of principles for designing and implementing peer grading activities in the MOOC context.

[1]  Zhenghao Chen,et al.  Tuned Models of Peer Assessment in MOOCs , 2013, EDM.

[2]  Mark G. Simkin,et al.  The Consistency of Peer Review in Student Writing Projects , 1995 .

[3]  Douglas Magin,et al.  Reciprocity as a Source of Bias in Multiple Peer Assessment of Group Work , 2001 .

[4]  Linda Bol,et al.  A Comparison of Anonymous versus Identifiable e-Peer Review on College Student Writing Performance and the Extent of Critical Feedback. , 2007 .

[5]  K. Oldfield,et al.  Peer and Self‐assessment at Tertiary Level ‐‐ an experiential report , 1995 .

[6]  Jan-Willem Strijbos,et al.  Peer feedback content and sender's competence level in academic writing revision tasks: Are they critical for feedback perceptions and efficiency? , 2010 .

[7]  Gloria Dall'Alba,et al.  Students’ experience of peer assessment in a professional course , 2007 .

[8]  N. Law,et al.  Online peer assessment: effects of cognitive and affective feedback , 2011, Instructional Science.

[9]  K. Topping,et al.  Formative Peer Assessment of Academic Writing Between Postgraduate Students , 2000 .

[10]  Book Reviews: Teaching Comprehensive Medical Care. A psychological study of a change in medical education , 1959 .

[11]  Jan-Willem Strijbos,et al.  Unravelling peer assessment: Methodological, functional, and conceptual developments , 2010 .

[12]  N. Falchikov,et al.  Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks , 2000 .

[13]  Alain Jaillet,et al.  Can Online Peer Assessment be Trusted? , 2009, J. Educ. Technol. Soc..

[14]  Bo Zhang,et al.  Assessing the reliability of self‐ and peer rating in student group work , 2008 .

[15]  Christian D. Schunn,et al.  Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives , 2006 .

[16]  Winnie Cheng,et al.  Peer and Teacher Assessment of the Oral and Written Tasks of a Group Project , 1999 .

[17]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[18]  Amanda M. Clifford,et al.  ‘Just enough to make you take it seriously’: exploring students’ attitudes towards peer assessment , 2013 .

[19]  P. Sadler,et al.  The Impact of Self- and Peer-Grading on Student Learning , 2006 .

[20]  Richard A. Kaimann.  The Coincidence of Student Evaluation by Professor and Peer Group Using Rank Correlation. , 1974 .

[21]  J. Mok.  A case study of students' perceptions of peer assessment in Hong Kong , 2011 .

[22]  David A. F. Haaga,et al.  Peer Review of Term Papers in Graduate Psychology Courses , 1993 .

[23]  Stuart A. Fry.  IMPLEMENTATION AND EVALUATION OF PEER MARKING IN HIGHER EDUCATION , 1990 .

[24]  Mark Freeman,et al.  Peer Assessment by Groups of Group Work , 1995 .

[25]  Graham Mowl,et al.  Using Self and Peer Assessment to Improve Students’ Essay Writing: a Case Study from Geography , 1995 .

[26]  L. Stefani.  Peer, self and tutor assessment: Relative reliabilities , 1994 .

[27]  P. Miller,et al.  The Effect of Scoring Criteria Specificity on Peer and Self-assessment , 2003 .

[28]  K. Topping.  Peer Assessment Between Students in Colleges and Universities , 1998 .

[29]  Gavin T. L. Brown.  Student peer assessment , 2015 .

[30]  Geoffrey E. Mills,et al.  Educational Research: Competencies for Analysis and Application , 1995 .

[31]  Peter Knight.  Using and Experiencing Assessment , 2012 .

[32]  B. Bloom.  Taxonomy of educational objectives , 1956 .

[33]  M. Korman,et al.  Medical shcool evaluation and internship performance. , 1971, Journal of medical education.

[34]  H. L. Billington.  Poster presentations and peer assessment: novel forms of evaluation and assessment , 1997 .

[35]  Christopher Orpen,et al.  Student versus lecturer assessment of learning: A research note , 1982 .

[36]  A. C. Butcher,et al.  Analysis of Peer‐, Self‐ and Staff‐assessment in Group Project Work , 1995 .