Are You Being Rhetorical? An Open Corpus of Machine Annotated Rhetorical Moves

Writing analytics has emerged as a sub-field of learning analytics, with applications including the provision of formative feedback to students in developing their writing capacities. Rhetorical markers in writing have become a key feature in this feedback, with a number of tools being developed across research and teaching contexts. However, there is no shared corpus of texts annotated by these tools, nor is it clear how the tool annotations compare. Thus, resources are scarce for comparing tools for both tool development and pedagogic purposes. In this paper, we conduct such a comparison and introduce a sample corpus of texts representative of the particular genres, a subset of which has been annotated using three rhetorical analysis tools (one of which has two versions). This paper aims to provide both a description of the tools and a shared dataset in order to support extensions of existing analyses and tool design in support of writing skill development. We intend the description of these tools, which share a focus on rhetorical structures, alongside the corpus, to be a preliminary step to enable further research, with regard to both tool development and tool interaction.

[1]  Maria Kuteeva,et al.  Graduate students’ genre knowledge and perceived disciplinary practices: Creating a research space across disciplines , 2016 .

[2]  Elena Cotos,et al.  Studying disciplinary corpora to teach the craft of discussion , 2016 .

[3]  Jill Burstein,et al.  Handbook of Automated Essay Evaluation Current Applications and New Directions , 2018 .

[4]  Elena Cotos,et al.  Potential of Automated Writing Evaluation Feedback. , 2011 .

[5]  Laurence Anthony,et al.  Mover: a machine learning tool to assist in the reading and writing of technical papers , 2003 .

[6]  Elena Cotos,et al.  Furthering and applying move/step constructs: Technology-driven marshalling of Swalesian genre theory for EAP pedagogy , 2015 .

[7]  Simon Buckingham Shum,et al.  Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation Study , 2012, Computer Supported Cooperative Work (CSCW).

[8]  Atsushi Mizumoto,et al.  Applying the Bundle–Move Connection Approach to the Development of an Online Writing Support Tool for Research Articles , 2017 .

[9]  Antonette Shibani,et al.  Turning the TAP on writing analytics , 2018 .

[10]  Simon Knight,et al.  Design and implementation of a pedagogic intervention using writing analytics , 2017 .

[11]  Adon Christian Michael Moskal,et al.  Text analytic tools to illuminate student learning , 2018, Learning Analytics in the Classroom.

[12]  Simon Buckingham Shum,et al.  Contextualizable Learning Analytics Design: A Generic Model and Writing Analytics Evaluations , 2019, LAK.

[13]  Dietrich Rebholz-Schuhmann,et al.  Automatic recognition of conceptualization zones in scientific articles and two life science applications , 2012, Bioinform..

[14]  Aaron N. Kaplan,et al.  Discovering Paradigm Shift Patterns in Biomedical Abstracts: Application to Neurodegenerative Diseases , 2005 .

[15]  Duanli Yan,et al.  Handbook of Automated Scoring , 2020 .

[16]  Jean-Pierre Chanod,et al.  Robustness beyond shallowness: incremental deep parsing , 2002, Natural Language Engineering.

[17]  Elena Cotos,et al.  Discourse classification into rhetorical functions for AWE feedback , 2016 .

[18]  Linda Flower,et al.  Problem-Solving Strategies and the Writing Process , 1977 .

[19]  Laurence Anthony,et al.  Writing research article introductions in software engineering: how accurate is a standard model? , 1999 .

[20]  Laura K. Allen,et al.  Critical perspectives on writing analytics , 2016, LAK.

[21]  Elena Cotos,et al.  Computer-Assisted Research Writing in the Disciplines , 2016 .

[22]  Simon Buckingham Shum,et al.  AcaWriter: A Learning Analytics Tool for Formative Feedback on Academic Writing , 2020, Journal of Writing Research.

[23]  Ágnes Sándor,et al.  Detecting key sentences for automatic assistance in peer reviewing research articles in educational sciences , 2009 .

[24]  Chih-Hua Kuo,et al.  A Corpus-Based Approach to Online Materials Development for Writing Research Articles , 2011 .

[25]  M. Scardamalia.  Knowledge telling and knowledge transforming in written composition , 1987 .

[26]  Stephanie Link,et al.  Understanding Graduate Writers’ Interaction with and Impact of the Research Writing Tutor during Revision , 2020, Journal of Writing Research.

[27]  Elena Cotos,et al.  Enhancing writing pedagogy with learner corpus data , 2014, ReCALL.

[28]  John M. Swales,et al.  Genre Analysis: English in Academic and Research Settings , 1993 .

[29]  M. Warschauer,et al.  Automated Writing Assessment in the Classroom , 2008 .

[30]  Ken Hyland,et al.  Genre pedagogy: Language, literacy and L2 writing instruction , 2007 .

[31]  Elena Cotos,et al.  Effects of DDL technology on genre learning , 2017 .

[32]  Xiaolong Wang,et al.  Reflecting on reflective writing analytics: assessment challenges and iterative evaluation of a prototype tool , 2016, LAK.

[33]  Laura K. Allen,et al.  Computer-Based Writing Instruction. , 2016 .

[34]  Jing Cai.  An exploratory study on an integrated genre-based approach for the instruction of academic lexical phrases , 2016 .

[35]  Elena Cotos,et al.  A move/step model for methods sections: Demonstrating Rigour and Credibility , 2017 .

[36]  Marc Moens,et al.  Articles Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status , 2002, CL.

[37]  William C. Mann,et al.  Rhetorical Structure Theory: Description and Construction of Text Structures , 1987 .

[38]  Simon Buckingham Shum,et al.  Educator perspectives on learning analytics in classroom practice , 2020, Internet High. Educ..