Writing motivation and ability profiles and transition during a technology-based writing intervention

Students exhibit heterogeneity in writing motivation and ability. Profiles based on measures of motivation and ability might help to describe this heterogeneity and better understand the effects of interventions aimed at improving students’ writing outcomes. We aimed to identify writing motivation and ability profiles in U.S. middle-school students participating in an automated writing evaluation (AWE) intervention using MI Write, and to identify transition paths between profiles as a result of the intervention. We identified profiles and transition paths of 2,487 students using latent profile and latent transition analysis. Four motivation and ability profiles emerged from a latent transition analysis with self-reported writing self-efficacy, attitudes toward writing, and a measure of writing writing: Low, Low/Mid, Mid/High, and High. Most students started the school year in the Low/Mid (38%) and Mid/High (30%) profiles. Only 11% of students started the school year in the High profile. Between 50 and 70% of students maintained the same profile in the Spring. Approximately 30% of students were likely to move one profile higher in the Spring. Fewer than 1% of students exhibited steeper transitions (e.g., from High to Low profile). Random assignment to treatment did not significantly influence transition paths. Likewise, gender, being a member of a priority population, or receiving special education services did not significantly influence transition paths. Results provide a promising profiling strategy focused on students’ attitudes, motivations, and ability and show students’ likeliness to belong to each profile based on their demographic characteristics. Finally, despite previous research indicating positive effects of AWE on writing motivation, results indicate that simply providing access to AWE in schools serving priority populations is insufficient to produce meaningful changes in students’ writing motivation profiles or writing outcomes. Therefore, interventions targeting writing motivation, in conjunction with AWE, could improve results.

[1]  Rui-ling Li Still a fallible tool? Revisiting effects of automated writing evaluation from activity theory perspective , 2022, Br. J. Educ. Technol..

[2]  Bram De Wever,et al.  Do cognitive processes and motives for argumentative writing converge in writer profiles? , 2022, The Journal of Educational Research.

[3]  G. A. Troia,et al.  Latent Profiles of Writing-related Skills, Knowledge, and Motivation for Elementary Students and Their Relations to Writing Performance across Multiple Genres , 2022, Contemporary Educational Psychology.

[4]  H. Van den Bergh,et al.  How students perform synthesis tasks: An empirical study into dynamic process configurations. , 2022, Journal of Educational Psychology.

[5]  S. Graham,et al.  A teacher technology tango shows strong results on 5th graders persuasive writing , 2022, Educational technology research and development.

[6]  Andrew Potter,et al.  Integrating goal-setting and automated feedback to improve writing outcomes: a pilot study , 2022, Innovation in Language Learning and Teaching.

[7]  K. Harris,et al.  Writing motivation questionnaire: validation and application as a formative assessment , 2022, Assessment in Education: Principles, Policy & Practice.

[8]  Haoran Xie,et al.  A review of AWE feedback: types, learning outcomes, and implications , 2022, Computer Assisted Language Learning.

[9]  Johannes Bauer A primer to latent profile and latent class analysis , 2021 .

[10]  Clarence Ng,et al.  Relationships between writing motives, writing self-efficacy and time on writing among Chinese students: path models and cluster analyses , 2021, Reading and Writing.

[11]  H. Van Keer,et al.  Relations among motivation, behaviour, and performance in writing: A multiple-group structural equation modeling study. , 2021, The British journal of educational psychology.

[12]  Joshua Wilson,et al.  Elementary teachers' perceptions of automated feedback and automated scoring: Transforming the teaching and learning of writing using automated writing evaluation , 2021, Comput. Educ..

[13]  Jochen De Weerdt,et al.  A review of automated feedback systems for learners: Classification framework, challenges and opportunities , 2021, Comput. Educ..

[14]  Sara K. Johnson Latent profile transition analyses and growth mixture models: A very non-technical guide for researchers in child and adolescent development. , 2021, New directions for child and adolescent development.

[15]  Joshua Wilson,et al.  Automated Feedback and Automated Scoring in the Elementary Grades: Usage, Attitudes, and Associations with Writing Outcomes in a Districtwide Implementation of MI Write , 2021, International Journal of Artificial Intelligence in Education.

[16]  Anna Elizabeth Talley,et al.  Misspecification of longitudinal measurement invariance within the latent transition analysis framework , 2021 .

[17]  Zsuzsa Bakk,et al.  Relating latent class membership to external variables: An overview , 2020, The British journal of mathematical and statistical psychology.

[18]  P. Boscolo,et al.  Writing Motivation in School: a Systematic Review of Empirical Research in the Early Twenty-First Century , 2020 .

[19]  Joshua Wilson,et al.  Implementing Automated Writing Evaluation in Different Instructional Contexts: A Mixed-Methods Study , 2020, Journal of Writing Research.

[20]  Stephanie Link,et al.  Impact of automated writing evaluation on teacher feedback, student revision, and writing improvement , 2020, Computer Assisted Language Learning.

[21]  Rod D. Roscoe,et al.  Automated Writing Evaluation and Feedback: Multiple Metrics of Efficacy , 2020, Journal of Educational Computing Research.

[22]  Katherine Landau Wright,et al.  Writing Motivation and Middle School: An Examination of Changes in Students’ Motivation for Writing , 2020 .

[23]  Muhammad M. M. Abdel Latif Unresolved issues in defining and assessing writing motivational constructs: A review of conceptualization and measurement perspectives , 2019, Assessing Writing.

[24]  K. Harris,et al.  The roles of writing knowledge, motivation, strategic behaviors, and skills in predicting elementary students’ persuasive writing from source material , 2019 .

[25]  Glenda Andrews,et al.  Gender Differences in Reading and Writing Achievement: Evidence From the National Assessment of Educational Progress (NAEP) , 2019, The American psychologist.

[26]  Michael Hebert,et al.  Generalizability of Automated Scores of Writing Quality in Grades 3–5 , 2019, Journal of Educational Psychology.

[27]  Antje Proske,et al.  Digital support for academic writing: A review of technologies and pedagogies , 2019, Comput. Educ..

[28]  Evan J. Fishman,et al.  Writing Skills, Knowledge, Motivation, and Strategic Behavior Predict Students’ Persuasive Writing Performance in the Context of Robust Writing Instruction , 2019, The Elementary School Journal.

[29]  N. Hockly Automated writing evaluation , 2018, ELT Journal.

[30]  Samantha G. Daley,et al.  Do writing motivational beliefs predict middle school students' writing performance? , 2018, Journal of Research in Reading.

[31]  Steve Graham,et al.  A Revised Writer(s)-Within-Community Model of Writing , 2018, Educational Psychologist.

[32]  P. Deane The Challenges of Writing in School: Conceptualizing Writing Development Within a Sociocognitive Framework , 2018, Educational Psychologist.

[33]  S. Zumbrunn,et al.  Clarifying an Elusive Construct: a Systematic Review of Writing Attitudes , 2018 .

[34]  S. Graham,et al.  Effects of SRSD college entrance essay exam instruction for high school students with disabilities or at-risk for writing difficulties , 2018, Reading and Writing.

[35]  Kristen D. Ritchey,et al.  An Analysis of First-Grade Writing Profiles and Their Relationship to Compositional Quality , 2018, Journal of learning disabilities.

[36]  C. Palermo,et al.  Teacher implementation of Self-Regulated Strategy Development with an automated writing evaluation system: Effects on the argumentative writing performance of middle school students , 2018 .

[37]  Cynthia S. Puranik,et al.  Profiles of Emergent Writing Skills Among Preschool Children , 2018 .

[38]  Barry Rosenfeld,et al.  Full Information Maximum Likelihood Estimation for Latent Variable Interactions With Incomplete Indicators , 2017, Multivariate behavioral research.

[39]  Joshua Wilson,et al.  Automated essay evaluation software in English Language Arts classrooms: Effects on teacher feedback, student motivation, and writing quality , 2016, Comput. Educ..

[40]  N. S Moore,et al.  Student Use of Automated Essay Evaluation Technology during Revision , 2016 .

[41]  John P. Meyer,et al.  Multiple-Group Analysis of Similarity in Latent Profile Solutions , 2016 .

[42]  C. MacArthur,et al.  A Multicomponent Measure of Writing Motivation With Basic College Writers , 2016 .

[43]  Steve Graham,et al.  Formative Assessment and Writing , 2015, The Elementary School Journal.

[44]  Paige Ware,et al.  Feedback for Adolescent Writers in the English Classroom: Exploring Pen-and-Paper, Electronic, and Automated Options , 2014 .

[45]  Mark D. Shermis,et al.  State-of-the-art automated essay scoring: Competition, results, and future directions from a United States demonstration , 2014 .

[46]  Douglas F. Kauffman,et al.  Examining Dimensions of Self-Efficacy for Writing. , 2013 .

[47]  Stephen Graham,et al.  Word processing programs and weaker writers/readers: a meta-analysis of research findings , 2012 .

[48]  Kimberly A Wolbers,et al.  Motivation Research in Writing: Theoretical and Empirical Considerations , 2012 .

[49]  P. Boscolo,et al.  Teaching Elementary School Students to Play With Meanings and Genre , 2012 .

[50]  Christophe Maïano,et al.  General Growth Mixture Analysis of Adolescents' Developmental Trajectories of Anxiety: The Impact of Untested Invariance Assumptions on Substantive Interpretations , 2011 .

[51]  Mark Warschauer,et al.  Utility in a Fallible Tool: A Multi-Site Case Study of Automated Writing Evaluation. , 2010 .

[52]  Thomas Quinlan,et al.  Does Automated Feedback Help Students Learn to Write? , 2010 .

[53]  Stephanie T. Lanza,et al.  Latent Class and Latent Transition Analysis: With Applications in the Social, Behavioral, and Health Sciences , 2009 .

[54]  Ulrich Trautwein,et al.  Please Scroll down for Article Structural Equation Modeling: a Multidisciplinary Journal Classical Latent Profile Analysis of Academic Self-concept Dimensions: Synergy of Person-and Variable-centered Approaches to Theoretical Models of Self-concept , 2022 .

[55]  M. Warschauer,et al.  Automated Writing Assessment in the Classroom , 2008 .

[56]  B. Muthén,et al.  Deciding on the Number of Classes in Latent Class Analysis and Growth Mixture Modeling: A Monte Carlo Simulation Study , 2007 .

[57]  Virginia W. Berninger,et al.  The structural relationship between writing attitude and writing achievement in first and third grade students , 2007 .

[58]  Carl W. Swartz,et al.  Subtypes of Written Expression in Elementary School Children: A Linguistic-Based Model , 2006, Developmental neuropsychology.

[59]  Carl W. Swartz,et al.  Aptitude–Treatment Interactions Revisited: Effect of Metacognitive Intervention on Subtypes of Written Expression in Elementary School Students , 2006, Developmental neuropsychology.

[60]  M. Daane,et al.  The Nation's Report Card: Writing, 2002. , 2003 .

[61]  Gale H. Roid,et al.  Patterns of Writing Skills Derived From Cluster Analysis of Direct-Writing Assessments , 1994 .

[62]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[63]  Aek Phakiti,et al.  The effects of computer-generated feedback on the quality of writing , 2014 .

[64]  Charles A. MacArthur,et al.  Self-Regulated Strategy Instruction in College Developmental Writing. , 2014 .

[65]  A. Bandura Self-efficacy mechanism in human agency. , 1982 .