Analyzing Linguistic Complexity and Accuracy in Academic Language Development of German across Elementary and Secondary School

We track the development of writing complexity and accuracy in German students’ early academic language development from first to eighth grade. Combining an empirically broad approach to linguistic complexity with the high-quality error annotation included in the Karlsruhe Children’s Text corpus (Lavalley et al. 2015) used, we construct models of German academic language development that successfully identify the student’s grade level. We show that classifiers for the early years rely more on accuracy development, whereas development in secondary school is better characterized by increasingly complex language in all domains: linguistic system, language use, and human sentence processing characteristics. We demonstrate the generalizability and robustness of models using such a broad complexity feature set across writing topics.

[1]  Danielle S. McNamara,et al.  Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners , 2014 .

[2]  P. Robinson.  Task complexity, task difficulty, and task production: exploring interactions in a componential framework , 2001 .

[3]  Shunji Inagaki,et al.  Second Language Development in Writing: Measures of Fluency, Accuracy, and Complexity , 1998 .

[4]  Kristopher Kyle,et al.  Measuring Syntactic Development in L2 Writing: Fine Grained Indices of Syntactic Complexity and Usage-Based Indices of Syntactic Sophistication , 2016 .

[5]  Elena Volodina,et al.  Classification of Language Proficiency Levels in Swedish Learners’ Texts , 2016 .

[6]  Heidi Byrnes,et al.  Realizing Advanced Foreign Language Writing Development in Collegiate Education: Curricular Design, Pedagogy, Assessment. , 2010 .

[7]  E. Gibson.  The dependency locality theory: A distance-based theory of linguistic complexity. , 2000 .

[8]  Ute Knoch,et al.  Does the writing of undergraduate ESL students develop after one year of study in an English-medium university? , 2014 .

[9]  Monika S. Schmid,et al.  A dynamic usage based perspective on L2 writing , 2012 .

[10]  Sebastian Stüker,et al.  Preparing children's writing database for automated processing , 2015, LTLT@SLaTE.

[11]  Walt Detmar Meurers,et al.  Modeling the Readability of German Targeting Adults and Children: An empirically broad analysis and its cross-corpus validation , 2018, COLING.

[12]  Hyung-Jo Yoon,et al.  The Linguistic Development of Students of English as a Second Language in Two Written Genres , 2017 .

[13]  Walt Detmar Meurers,et al.  Broad linguistic modeling is beneficial for German L2 proficiency assessment , 2017 .

[14]  Folkert Kuiken,et al.  Complexity, Accuracy and Fluency: Definitions, Measurement and Research. , 2012 .

[15]  Robert Reynolds,et al.  Russian natural language processing for computer-assisted language learning: Capturing the benefits of deep morphological analysis in real-life applications , 2016 .

[16]  Nina Vyatkina,et al.  The Development of Second Language Writing Complexity in Groups and Individuals: A Longitudinal Learner Corpus Study , 2012 .

[17]  Ute Knoch,et al.  What happens to ESL students’ writing after three years of study at an English medium university? , 2015 .

[18]  Rod Ellis,et al.  Analysing Learner Language , 2005 .

[19]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[20]  Xiaofei Lu,et al.  Different topics, different discourse: Relationships among writing topic, measures of syntactic complexity, and judgments of writing quality , 2015 .

[21]  P. Skehan.  A FRAMEWORK FOR THE IMPLEMENTATION OF TASK-BASED INSTRUCTION , 1996 .

[22]  D. Larsen-Freeman.  The Emergence of Complexity, Fluency, and Accuracy in the Oral and Written Production of Five Chinese Learners of English , 2006 .

[23]  Helen Yannakoudakis,et al.  Developing an automated writing placement system for ESL learners , 2018 .

[24]  Walt Detmar Meurers,et al.  Task Effects on Linguistic Complexity and Accuracy: A Large-Scale Learner Corpus Analysis Employing Natural Language Processing Techniques , 2017 .

[25]  Walt Detmar Meurers,et al.  Readability Classification for German using Lexical, Syntactic, and Morphological Features , 2012, COLING.

[26]  A. Jacobs,et al.  The word frequency effect: a review of recent developments and implications for the choice of frequency estimates in German. , 2011, Experimental psychology.

[27]  Mary Lou Vercellotti,et al.  The Development of Complexity, Accuracy, and Fluency in Second Language Performance: A Longitudinal Study , 2017 .

[28]  William Schuler,et al.  Memory access during incremental sentence processing causes reading time latency , 2016, CL4LC@COLING 2016.

[29]  Alex Housen,et al.  Conceptualizing and measuring short-term changes in L2 writing complexity , 2014 .

[30]  Zarah Weiss,et al.  Computationally Modeling the Impact of Task-Appropriate Language Complexity and Accuracy on Human Grading of German Essays , 2019, BEA@ACL.

[31]  Hyung-Jo Yoon,et al.  Linguistic complexity in L2 writing revisited: Issues of topic, proficiency, and construct multidimensionality , 2017 .

[32]  Simonetta Montemagni,et al.  Assessing document and sentence readability in less resourced languages and across textual genres , 2014 .

[33]  Danielle S. McNamara,et al.  The Development of Writing Proficiency as a Function of Grade Level: A Linguistic Analysis , 2011 .

[34]  Thea van der Geest,et al.  Online Test Tool to Determine the CEFR Reading Comprehension Level of Text , 2013, DSAI.

[35]  Heidi Byrnes,et al.  Emergent L2 German writing ability in a curricular context: A longitudinal study of grammatical metaphor , 2009 .