Reflective writing analytics for actionable feedback

Reflective writing can provide a powerful way for students to integrate professional experience and academic learning. However, writing reflectively requires high quality actionable feedback, which is time-consuming to provide at scale. This paper reports progress on the design, implementation, and validation of a Reflective Writing Analytics platform to provide actionable feedback within a tertiary authentic assessment context. The contributions are: (1) a new conceptual framework for reflective writing; (2) a computational approach to modelling reflective writing, deriving analytics, and providing feedback; (3) the pedagogical and user experience rationale for platform design decisions; and (4) a pilot in a student learning context, with preliminary data on educator and student acceptance, and the extent to which we can evidence that the software provided actionable feedback for reflective writing.

[1]  Bo Chang.  Reflection in Learning , 2019, Online Learning.

[2]  Jean-Pierre Chanod,et al.  Robustness beyond shallowness: incremental deep parsing , 2002, Natural Language Engineering.

[3]  Xiaolong Wang,et al.  Designing Academic Writing Analytics for Civil Law Student Self-Assessment , 2018, International Journal of Artificial Intelligence in Education.

[4]  D. Boud,et al.  Aligning assessment with long‐term learning , 2006 .

[5]  Ana Caetano,et al.  Defining personal reflexivity , 2015 .

[6]  Jasmine Luk,et al.  Assessing teaching practicum reflections: Distinguishing discourse features of the “high” and “low” grade reports , 2008 .

[7]  Sinthia Bosnic-Anticevich,et al.  A Model for Assessing Reflective Practices in Pharmacy Education , 2015, American Journal of Pharmaceutical Education.

[8]  N. Hatton,et al.  Reflection in teacher education: Towards definition and implementation , 1995 .

[9]  Peter Bruza,et al.  Towards the Discovery of Learner Metacognition From Reflective Writing , 2016, J. Learn. Anal..

[10]  Rosanne Birney,et al.  Reflective Writing: Quantitative Assessment and Identification of Linguistic Features , 2012 .

[11]  Peter A. C. Smith.  Action Learning and Reflective Practice in Project Environments that are Related to Leadership Development , 2001 .

[12]  William B. Borgers.  DEMOCRACY AND EDUCATION. , 1919 .

[13]  Mary E. Ryan,et al.  Reflexive writers: Re-thinking writing development and assessment in schools , 2014 .

[14]  Caroline Brun.  Detecting Opinions Using Deep Syntactic Analysis , 2011, RANLP.

[15]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[16]  Gregory F. Lorenz,et al.  A study of outcomes-oriented student reflection during internship: The integrated, coordinated, and reflection based model of learning and experiential education , 2010 .

[17]  D. Schoen,et al.  The Reflective Practitioner: How Professionals Think in Action , 1985 .

[18]  Linda R. Valli.  Listening to Other Voices: A Description of Teacher Reflection in the United States , 1997 .

[19]  John Flowerdew,et al.  John Swales's approach to pedagogy in Genre Analysis: A perspective from 25 years on ☆ , 2015 .

[20]  David Rose,et al.  Working with Discourse: Meaning Beyond the Clause , 2003 .

[21]  D. Kember,et al.  Assessing the level of student reflection from reflective journals. , 1995, Journal of advanced nursing.

[22]  John Sandars,et al.  The use of reflection in medical education: AMEE Guide No. 44 , 2009, Medical teacher.

[23]  Frederick Steier.  Reflexivity and Methodology: An Ecological Constructionism , 1991 .

[24]  Rebecca Ferguson,et al.  XIP Dashboard: visual analytics from automated rhetorical parsing of scientific metadiscourse , 2013 .

[25]  M. Manen.  On the Epistemology of Reflective Practice , 1995 .

[26]  Jack Mezirow,et al.  Fostering Critical Reflection in Adulthood: A Guide to Transformative and Emancipatory Learning , 1990 .

[27]  Sinthia Bosnic-Anticevich,et al.  A Retrospective Study on Students’ and Teachers’ Perceptions of the Reflective Ability Clinical Assessment , 2016, American Journal of Pharmaceutical Education.

[28]  R. J. Bogumil,et al.  The reflective practitioner: How professionals think in action , 1985, Proceedings of the IEEE.

[29]  Carl Reidsema,et al.  Enabling the Reflective Practitioner in Engineering Design Courses , 2010 .

[30]  Jill Willis,et al.  Turning points in narratives of research design: Research innovation stimulating unique responses to existing challenges for beginning rural teachers , 2014 .

[31]  Victoria Hoban,et al.  The Reflective Practitioner , 2013 .

[32]  Joseph M. Paxton,et al.  Reflection and Reasoning in Moral Judgment , 2022 .

[33]  Rudi Dallos,et al.  Reflective Practice In Psychotherapy And Counselling , 2009 .

[34]  Xiaolong Wang,et al.  Reflecting on reflective writing analytics: assessment challenges and iterative evaluation of a prototype tool , 2016, LAK.

[35]  D. Nicol.  Assessment for learner self‐regulation: enhancing achievement in the first year using learning technologies , 2009 .

[36]  David Boud,et al.  Introduction: What is Reflection in Learning? , 2013 .

[37]  D. Sadler.  Formative assessment and the design of instructional systems , 1989 .

[38]  John M. Swales,et al.  Genre Analysis: English in Academic and Research Settings , 1993 .

[39]  Larry Ambrose,et al.  The power of feedback. , 2002, Healthcare executive.

[40]  Mary E. Ryan,et al.  Theorising a model for teaching and assessing reflective learning in higher education , 2013 .

[41]  Jae-Won Moon.  Reflection in Learning and Professional Development: Theory and Practice , 2005 .

[42]  Ágnes Sándor,et al.  Modeling metadiscourse conveying the author's rhetorical strategy in biomedical research abstracts , 2007 .

[43]  Amy Beth Warriner,et al.  Norms of valence, arousal, and dominance for 13,915 English lemmas , 2013, Behavior Research Methods.

[44]  J. Mezirow,et al.  How Critical Reflection Triggers Transformative Learning , 2002 .

[45]  Mary Ryan.  The Teachable Moment: The Washington Center Internship Program. , 1988 .

[46]  Andrew Dillon,et al.  Information architecture in JASIST: Just where did we come from? , 2002, J. Assoc. Inf. Sci. Technol..

[47]  Simon Buckingham Shum,et al.  Learning Analytics for 21st Century Competencies , 2016, J. Learn. Anal..