Quantitative analysis of approaches to group marking

Abstract Group work, where students work on projects to overcome challenges together, has numerous advantages, including learning of important transferable skills, better learning experience and increased motivation. However, in many academic systems the advantages of group projects clash with the need to assign individualised marks to students. A number of different schemes have been proposed to individualise group project marks, these include marking of individual reflexive accounts of the group work and peer assessment. Here, we explore a number of these schemes in computational experiments with an artificial student population. Our analysis highlights the advantages and disadvantages of each scheme and particularly reveals the power of a new scheme proposed here that we call pseudoinverse marking. Abbreviations SOPP: Self organised peer assessment; RA: Reflexive accounts; MRA: Mark-adjusted reflexive accounts; NPA: Normalised peer assessment; PR: Peer ranking; PiM: Pseudoinverse marking

[1]  Naoki Masuda,et al.  A network-based dynamical ranking system for competitive sports , 2012, Scientific Reports.

[2]  Gerard Arbat,et al.  Peer and self-assessment applied to oral presentations from a multidisciplinary perspective , 2016 .

[3]  M. Newman,et al.  A network-based ranking system for US college football , 2005, physics/0505169.

[4]  J. Willison,et al.  Implementation and outcomes of online self and peer assessment on group based honours research projects , 2014 .

[5]  Martin R. Fellenz.  Toward Fairness in Assessing Student Groupwork: A Protocol for Peer Evaluation of Individual Contributions , 2006 .

[6]  Martin Davies.  Groupwork as a form of assessment: common problems and recommended solutions , 2009 .

[7]  Q. Bai,et al.  Exploiting Vagueness for Multi-Agent Consensus , 2016 .

[8]  Branko Bognar,et al.  Evaluation in Higher Education. , 2014 .

[9]  Jo McKenzie,et al.  SPARK, a confidential web-based template for self and peer assessment of student teamwork: benefits of evaluating across different subjects , 2002, Br. J. Educ. Technol..

[10]  C. Hmelo‐Silver.  Problem-Based Learning: What and How Do Students Learn? , 2004 .

[11]  Mark Lejk,et al.  Peer Assessment of Contributions to a Group Project: A comparison of holistic and category-based approaches , 2001 .

[12]  T. Lawson,et al.  Spark , 2011 .

[13]  Jacob Pearce,et al.  Assessment and Feedback in Higher Education: A Review of Literature for the Higher Education Academy , 2017 .

[14]  J. Dijkstra,et al.  Assessing the “I” in group work assessment: State of the art and recommendations for practice , 2016, Medical teacher.

[15]  Timothy S. O'Connell,et al.  Assessing the quality of reflection in student journals: a review of the research , 2011 .

[16]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[17]  Judy Goldfinch,et al.  Further Developments in Peer Assessment of Group Projects , 1994 .

[18]  Stephen J. Sharp.  Deriving individual student marks from a tutor’s assessment of group work , 2006 .

[19]  Michael A. Abelson,et al.  Peer Evaluation Within Group Projects: a Suggested Mechanism and Process , 1986 .

[20]  J. C. A. Barata,et al.  The Moore–Penrose Pseudoinverse: A Tutorial Review of the Theory , 2011, 1110.6882.

[21]  Sung-Seok Ko,et al.  Peer assessment in group projects accounting for assessor reliability by an iterative method , 2014 .

[22]  V. Kutija,et al.  A robust approach for mapping group marks to individual marks using peer assessment , 2015 .