Examining the Effects of Automatic Comment Classification on Comment Types in Peer Review for Graduate Students' Academic Writing

Although previous studies have confirmed the effect of peer review on academic writing, graduate students are usually unable to provide appropriate comments in peer review. As a result, their comments cannot effectively help authors revise academic papers. For this reason, in order to improve the quality of students' comments, this study aims to provide peer reviewers with feedback of the automatic classification of their comments in an existing peer review system, so that reviewers can improve their comments for writers. In order to examine its effect, this study designs an experiment in an academic writing course for first-year graduate students in two semesters. In the experimental group, peer reviewers are provided with the feedback, while students in the control group did not receive any feedback. The results showed that the graduate students in the experimental group consciously improved their comments after receiving the feedback, especially on the cognitive dimensions (e.g., increasing guidance) and the metacognitive dimensions (e.g., increasing evaluating comments). The findings may provide a direction for students to write high-quality comments and improve writing proficiency.

[1]  Christian D. Schunn,et al.  Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system , 2007, Comput. Educ..

[2]  Young Hoan Cho,et al.  Peer reviewers learn from giving comments , 2011 .

[3]  Niall Seery,et al.  Promoting deep learning in a teacher education programme through self- and peer-assessment and feedback , 2012 .

[4]  Eric Zhi-Feng Liu,et al.  Web-based peer assessment: feedback for students with various thinking-styles , 2001, J. Comput. Assist. Learn..

[5]  M. K. A. Singh.  International Graduate Students' Academic Writing Practices in Malaysia: Challenges and Solutions. , 2015 .

[6]  Yuqin Yang,et al.  Improving Learning in MOOCs Through Peer Feedback: How Is Learning Improved by Providing and Receiving Feedback? , 2017 .

[7]  N. Law,et al.  Online peer assessment: effects of cognitive and affective feedback , 2011, Instructional Science.

[8]  Patrick Onghena,et al.  Improving the effectiveness of peer feedback for learning , 2009 .

[9]  Lino Montoro Moreno,et al.  Student perceptions of peer assessment: an interdisciplinary study , 2014 .

[10]  Guan-Yu Lin,et al.  Anonymous versus identified peer assessment via a Facebook-based learning application: Effects on quality of peer feedback, perceived learning, perceived fairness, and attitude toward the system , 2018, Comput. Educ..

[11]  D. Boud,et al.  Rethinking models of feedback for learning: the challenge of design , 2013 .

[12]  Daniel Marcu,et al.  Finding the WRITE Stuff: Automatic Identification of Discourse Structure in Student Essays , 2003, IEEE Intell. Syst..

[13]  Edward F. Gehringer,et al.  Automated Assessment of the Quality of Peer Reviews using Natural Language Processing Techniques , 2017, International Journal of Artificial Intelligence in Education.

[14]  K. Topping.  Peer Assessment Between Students in Colleges and Universities , 1998 .

[15]  Hussein Meihami,et al.  An emic perspective toward challenges and solutions of self- and peer-assessment in writing courses , 2016 .

[16]  Jyh-Chong Liang,et al.  Examining the role of feedback messages in undergraduate students' writing performance during an online peer assessment activity , 2015, Internet High. Educ..

[17]  Chin-Chung Tsai,et al.  An educational research course facilitated by online peer assessment , 2009 .

[18]  Rob Martens,et al.  Exploring the Value of Peer Feedback in Online Learning for the Provider. , 2017 .

[19]  Diane J. Litman,et al.  Iterative Design and Classroom Evaluation of Automated Formative Feedback for Improving Peer Feedback Localization , 2016, International Journal of Artificial Intelligence in Education.

[20]  C. Vleuten,et al.  Quality of feedback following performance assessments: does assessor expertise matter? , 2013 .

[21]  Larry Ambrose,et al.  The power of feedback. , 2002, Healthcare executive.

[22]  Christian D. Schunn,et al.  Students’ perceptions about peer assessment for writing: their origin and impact on revision work , 2011 .

[23]  D. Carless,et al.  Peer feedback: the learning element of peer assessment , 2006 .