A comparative analysis of the consistency and difference among online self-, peer-, external- and instructor-assessments: The competitive effect

In the last few years, self- and peer-assessment have been increasingly employed not only as an evaluation method, but also as a learning procedure. The consistency and difference between self- and peer-assessments as compared to instructor-assessments have been previously studied, and a friendship bias was discovered. In this study, we introduce external-assessment (products are assessed by students from a different university that are enrolled in a similar course), and compare self-, peer-, external- and instructor-assessments. The experience was conducted at two different universities separated by a significant distance, during two consecutive years, including a total of 97 students. At both universities, students developed websites and online tools were employed to organise the different types of assessments. The obtained results indicate that there is a high-level of consistency across the different kinds of assessments. Moreover, a competitive effect was discovered: students tended to award higher grades to students from their same university while they were harsher with the products from a distant university. From the learning perspective, and according to the students' final grade, the assessment experience correlated with learning gains. Self- and peer-assessment have been increasingly employed in the last few years.We introduce external-assessment from students from a different university.A high-level consistency among the different kinds of assessments is obtained.A competitive effect is found: students are stricter with products from a distant university.The assessment experience was correlated with learning gains.

[1]  Jan-Willem Strijbos,et al.  Unravelling peer assessment: Methodological, functional, and conceptual developments , 2010 .

[2]  Lan Li,et al.  Assessor or assessee: How student learning improves by giving and receiving peer feedback , 2010, Br. J. Educ. Technol..

[3]  J. Kruger,et al.  Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. , 1999, Journal of personality and social psychology.

[4]  Paul D. Ellis,et al.  The essential guide to effect sizes : statistical power, meta-analysis, and the interpretation of research results , 2010 .

[5]  N. Falchikov Involving Students in Assessment , 2004 .

[6]  Eric Zhi-Feng Liu,et al.  Web-based peer assessment: feedback for students with various thinking-styles , 2001, J. Comput. Assist. Learn..

[7]  Chi-Cheng Chang,et al.  A comparative analysis of the consistency and difference among teacher-assessment, student self-assessment and peer-assessment in a Web-based portfolio assessment environment for high school students , 2012, Comput. Educ..

[8]  Tero Vartiainen,et al.  Educating IT Project Managers through Project-Based Learning: A Working-Life Perspective , 2009, Commun. Assoc. Inf. Syst..

[9]  Hongli Li,et al.  Peer assessment in the digital age: a meta-analysis comparing peer and teacher ratings , 2016 .

[10]  Nian-Shing Chen,et al.  Effects of high level prompts and peer assessment on online learners' reflection levels , 2009, Comput. Educ..

[11]  Andrew Luxton-Reilly,et al.  A comparison of peer and tutor feedback , 2015 .

[12]  Gwo-Jen Hwang,et al.  An interactive peer-assessment criteria development approach to improving students' art design performance using handheld devices , 2015, Comput. Educ..

[13]  P. Sadler,et al.  The Impact of Self- and Peer-Grading on Student Learning , 2006 .

[14]  Christian D. Schunn,et al.  Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives , 2006 .

[15]  N. Falchikov,et al.  Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks , 2000 .

[16]  Catherine E. Amiot,et al.  The Role of Autonomy in Intergroup Processes: Toward an Integration of Self-Determination Theory and Intergroup Approaches , 2014 .

[17]  Keith J. Topping,et al.  Methodological quandaries in studying process and outcomes in peer assessment , 2010 .

[18]  Richard T. Snodgrass,et al.  Editorial: Single- versus double-blind reviewing , 2007, TODS.

[19]  César Domínguez,et al.  Database design learning: A project-based approach organized through a course management system , 2010, Comput. Educ..

[20]  K. Topping Self and Peer Assessment in School and University: Reliability, Validity and Utility , 2003 .

[21]  Chris Piech,et al.  Deconstructing disengagement: analyzing learner subpopulations in massive open online courses , 2013, LAK '13.

[22]  Zacharias C. Zacharia,et al.  Peer versus expert feedback: An investigation of the quality of peer feedback among secondary school students , 2014, Comput. Educ..

[23]  Harald Søndergaard,et al.  Collaborative learning through formative peer review: pedagogy, programs and potential , 2012, Comput. Sci. Educ..

[24]  Silvia Sanz-Santamaría,et al.  Student-generated online videos to develop cross-curricular and curricular competencies in Nursing Studies , 2014, Comput. Hum. Behav..

[25]  Geoffrey J. Kennedy,et al.  Peer-assessment in Group Projects: Is It Worth It? , 2005, ACE.

[26]  Kyparisia A. Papanikolaou,et al.  Alternative assessment methods in technology enhanced project-based learning , 2013, Int. J. Learn. Technol..

[27]  J. Hair Multivariate data analysis , 1972 .

[28]  K. Topping Peer Assessment Between Students in Colleges and Universities , 1998 .

[29]  David Dunning,et al.  How unaware are the unskilled? Empirical tests of the “signal extraction” counterexplanation for the Dunning–Kruger effect in self-evaluation of performance , 2013 .

[30]  Amanda M. Clifford,et al.  ‘Just enough to make you take it seriously’: exploring students’ attitudes towards peer assessment , 2013 .

[31]  Chin-Chung Tsai,et al.  On-line peer assessment and the role of the peer feedback: A study of high school computer course , 2007, Comput. Educ..

[32]  Anthony C. Robinson,et al.  Peer Grading in a MOOC: Reliability, Validity, and Perceived Effects. , 2014 .

[33]  Lino Montoro Moreno,et al.  Student perceptions of peer assessment: an interdisciplinary study , 2014 .

[34]  César Domínguez,et al.  Interuniversity telecollaboration to improve academic results and identify preferred communication tools , 2013, Comput. Educ..

[35]  Maryam Azarnoosh,et al.  Peer assessment in an EFL context: attitudes and friendship bias , 2013, Language Testing in Asia.

[36]  Jan-Willem Strijbos,et al.  The impact of a rubric and friendship on peer assessment: Effects on construct validity, performance, and perceptions of fairness and comfort , 2013 .

[37]  Chao-hsiu Chen,et al.  The implementation and evaluation of a mobile self- and peer-assessment system , 2010, Comput. Educ..

[38]  Yao-Ting Sung,et al.  The design and application of a web-based self- and peer-assessment system , 2005, Comput. Educ..

[39]  Patrick Onghena,et al.  An inventory of peer assessment diversity , 2011 .

[40]  E. Pronin,et al.  Perception and misperception of bias in human judgment , 2007, Trends in Cognitive Sciences.

[41]  Justin Cheng,et al.  Peer and self assessment in massive online classes , 2013, ACM Trans. Comput. Hum. Interact..