Performance measures for peer assessment

Peer review is an evaluation strategy increasingly used in educational environments. It has now a central role in the field of teaching, but there is an extensive experience in the field of research. This evaluation method implies an author reviewing, usually anonymously, the work of his or her colleagues who, in turn, can become reviewers of the author's work. This peer evaluation system enriches the evaluation itself, and it can even become one of the few ways to assess the works in absence of a hierarchical authority on the subject. In massive access learning platforms it has even become the main assessment method. All in all, it is necessary to assure the quality of the assessment process. In this research we consider the process of peer review as a classification process, in which we have several classifiers (reviewers) that, before an input (work to revise) should assign a particular class (work grading). In this analysis two performance measures are proposed to assess the degree of goodness of the reviewers, in order to help improve the quality of peer review processes providing a more objective assessment of the reviewers task.

[1]  M. Andrés,et al.  Una experiencia de autoevaluación y evaluación por compañeros , 2013 .

[2]  Hendrik Drachsler,et al.  Translating Learning into Numbers: A Generic Framework for Learning Analytics , 2012, J. Educ. Technol. Soc..

[4]  K. Topping Peer Assessment Between Students in Colleges and Universities , 1998 .

[5]  Rebecca Ferguson,et al.  Social learning analytics: five approaches , 2012, LAK.

[6]  Judy Kay,et al.  MOOCs: So Many Learners, So Much Potential ... , 2013, IEEE Intelligent Systems.

[7]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[8]  Winnie Cheng,et al.  Peer and Teacher Assessment of the Oral and Written Tasks of a Group Project , 1999 .

[9]  P. Sadler,et al.  The Impact of Self- and Peer-Grading on Student Learning , 2006 .

[10]  N. Falchikov,et al.  Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks , 2000 .

[11]  María Bordons,et al.  Evaluación de la actividad científica a través de indicadores bibliométricos , 1999 .

[12]  D. J. Magin,et al.  A Novel Technique for Comparing the Reliability of Multiple Peer Assessments with that of Single Teacher Assessments of Group Process Work , 2001 .

[13]  Mark Freeman,et al.  Peer Assessment by Groups of Group Work , 1995 .

[14]  W. Rod Cullen,et al.  Peer assessment of oral presentations: effects of student gender, university affiliation and participation in the development of assessment criteria , 2005 .

[16]  Pei-Lin Hsu,et al.  Evaluating Online Peer Assessment as an Educational Tool for Promoting Self-Regulated Learning , 2015, MISNC.

[17]  Yiming Yang,et al.  An Evaluation of Statistical Approaches to Text Categorization , 1999, Information Retrieval.

[18]  Ibrahim S. Al-fallay The Role of Some Selected Psychological and Personality Traits of the Rater in the Accuracy of Self- and Peer-Assessment. , 2004 .

[19]  Douglas Magin,et al.  Reciprocity as a Source of Bias in Multiple Peer Assessment of Group Work , 2001 .

[20]  Gregorio Rodríguez Gómez,et al.  La evaluación entre iguales : beneficios y estrategias para su práctica en la universidad , 2012 .

[21]  Richard L. Weaver,et al.  Peer evaluation: A case study , 1986 .

[22]  EL SISTEMA DE REVISIÓN POR EXPERTOS (PEER REVIEW): MUCHOS PROBLEMAS Y POCAS SOLUCIONES. , 2002 .

[23]  John A. Dutton Peer Grading in a MOOC : Reliability , Validity , and Perceived Effects , 2014 .