Conceptualizing and measuring short-term changes in L2 writing complexity

Abstract This study aims to ascertain the nature and extent of the development of English L2 writing proficiency of 45 adult ESL learners over the time of an intensive short-term academic English language programme by means of quantitative measures targeting different components of the lexical and syntactic complexity of the learners’ writing performance, and to compare the scores on these measures with subjective ratings of learners’ overall writing quality. Results reveal several linguistic complexity measures that can adequately and validly capture changes in L2 writing in short-term ESL courses, though these do not include “popular” measures such as subordination ratios and lexical richness/diversity measures. Results also suggest that different subcomponents of syntactic and lexical complexity in L2 writing develop at a different pace, underlining the importance of calculating a sufficiently wide range of judiciously selected complexity measures in order to get a comprehensive picture of L2 writing development. Interestingly, the set of progress-sensitive complexity measures identified in this study does not coincide with the set of complexity measures that best predict subjective perceptions of writing quality.

[1]  Peter Skehan.  Lexical Performance by Native and Non-Native Speakers on Language-Learning Tasks , 2009 .

[2]  Robert L. Politzer,et al.  An Exploratory Study of Learning Behaviors and Their Relationship to Gains in Linguistic and Communicative Competence , 1985 .

[3]  Marjolijn Verspoor,et al.  Variability in Second Language Development From a Dynamic Systems Perspective , 2008 .

[4]  E. White.  Teaching and assessing writing , 1996 .

[5]  Alex Housen,et al.  Complexity, accuracy and fluency in second language acquisition , 2009 .

[6]  Carmen Muñoz,et al.  Age-related differences in foreign language learning. Revisiting the empirical evidence , 2008 .

[7]  Helmut Daller,et al.  Modelling and Assessing Vocabulary Knowledge: What is in a teacher's mind? Teacher ratings of EFL essays and different aspects of lexical richness , 2007 .

[8]  Diane Larsen-Freeman,et al.  An ESL Index of Development , 1978 .

[9]  Neomy Storch,et al.  The impact of studying in a second language (L2) medium university on the development of L2 writing , 2009 .

[10]  Peter Skehan,et al.  Cognition and Second Language Instruction: Cognition and tasks , 2001 .

[11]  Maria Mos,et al.  Cognitive task complexity and second language writing performance , 2005 .

[12]  Freek Van de Velde.  Review of Givón (2009): The genesis of syntactic complexity, Diachrony, ontogeny, neurocognition, evolution , 2012 .

[13]  Philip M. McCarthy,et al.  MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment , 2010, Behavior research methods.

[14]  Thomas Givon,et al.  The Genesis of Syntactic Complexity: Diachrony, Ontogeny, Neuro-Cognition, Evolution , 2009 .

[15]  Rod Ellis,et al.  The Study of Second Language Acquisition , 1994 .

[16]  L. Given,et al.  The SAGE encyclopedia of qualitative research methods , 2011 .

[17]  Yoko Kozaki,et al.  Using GENOVA and FACETS to set multiple standards on performance assessment for certification in medical translation from Japanese into English , 2004 .

[18]  Patrick Juola.  Assessing linguistic complexity , 2008 .

[19]  G. Pullum,et al.  The Cambridge Grammar of the English Language , 2002 .

[20]  P. Guiraud.  Problèmes et méthodes de la statistique linguistique , 1960 .

[21]  I. Nation.  How Large a Vocabulary Is Needed for Reading and Listening? , 2006 .

[22]  J. Norris,et al.  Towards an Organic Approach to Investigating CAF in Instructed SLA: The Case of Complexity , 2009 .

[23]  David Crystal,et al.  The Cambridge Encyclopedia of Language , 2012, Modern Language Review.

[24]  Rosemary Wette,et al.  Evaluating student learning in a university-level EAP unit on writing using sources , 2010 .

[25]  Ute Knoch,et al.  Re-Training Writing Raters Online: How Does It Compare with Face-to-Face Training?. , 2007 .

[26]  Alan Tonkyn,et al.  Measuring and Perceiving Changes in Oral Complexity, Accuracy and Fluency: Examining Instructed Learners' Short-Term Gains. , 2012 .

[27]  L. Ortega.  Syntactic Complexity Measures and Their Relationship to L2 Proficiency: A Research Synthesis of College-Level L2 Writing. , 2003 .

[28]  Margaret Thomas,et al.  Assessment of L2 Proficiency in Second Language Acquisition Research , 1994 .

[29]  Folkert Kuiken,et al.  Task complexity and linguistic performance in L2 writing and speaking: the effect of mode , 2011 .

[30]  Eric Friginal,et al.  Exploring multiple profiles of L2 writing using multi-dimensional analysis , 2014 .

[31]  Elsa Tragant,et al.  A Longitudinal Analysis of the Effects of One Year Abroad , 2012 .

[32]  Folkert Kuiken,et al.  Complexity, Accuracy and Fluency: Definitions, Measurement and Research. , 2012 .

[33]  Betty Lou Leaver,et al.  Developing Professional-Level Language Proficiency: Index , 2002 .

[34]  Monika S. Schmid,et al.  A dynamic usage based perspective on L2 writing , 2012 .

[35]  Alex Housen,et al.  Defining and operationalising L2 complexity , 2012 .

[36]  D. Larsen-Freeman,et al.  THE CONSTRUCTION OF A SECOND LANGUAGE ACQUISITION INDEX OF DEVELOPMENT1 , 1977 .

[37]  D. Larsen-Freeman,et al.  Complex Systems and Applied Linguistics , 2008 .

[38]  Ingo Wegener,et al.  Complexity Theory , 2005 .

[39]  Averil Coxhead.  A New Academic Word List , 2000 .

[40]  Alex Housen,et al.  Investigating lexical proficiency development over time – the case of Dutch-speaking learners of French in Brussels , 2008, Journal of French Language Studies.

[41]  George Engelhard,et al.  Examining Rater Errors in the Assessment of Written Composition With a Many-Faceted Rasch Model , 1994 .

[42]  Heidi Byrnes,et al.  Realizing Advanced Foreign Language Writing Development in Collegiate Education: Curricular Design, Pedagogy, Assessment. , 2010 .

[43]  Danielle S. McNamara,et al.  Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners , 2014 .

[44]  Folkert Kuiken,et al.  Communicative adequacy and linguistic complexity in L2 writing , 2010 .

[45]  D. Larsen-Freeman.  The Emergence of Complexity, Fluency, and Accuracy in the Oral and Written Production of Five Chinese Learners of English , 2006 .

[46]  Östen Dahl,et al.  The growth and maintenance of linguistic complexity , 2004 .

[47]  P. Robinson.  Task complexity, task difficulty, and task production: exploring interactions in a componential framework , 2001 .

[48]  Wang Chuming.  The length approach to ELT , 2005 .

[49]  W. Levelt,et al.  Speaking: From Intention to Articulation , 1990 .

[50]  Benjamin A. Rifkin,et al.  A Ceiling Effect in Traditional Classroom Foreign Language Instruction: Data from Russian , 2005 .

[51]  Cheryl A. Engber.  The relationship of lexical proficiency to the quality of ESL compositions , 1995 .

[52]  G. Pallotti.  CAF: Defining, Refining and Differentiating Constructs , 2009 .

[53]  Paul Lennon,et al.  Assessing short-term change in advanced oral proficiency: problems of reliability & validity in four case studies , 1995 .

[54]  Geoffrey Sampson,et al.  Language complexity as an evolving variable , 2009 .

[55]  Norman W. Evans,et al.  Effects of Dynamic Corrective Feedback on ESL Writing Accuracy , 2010 .

[56]  Jeff Connor-Linton,et al.  Comparing perspectives on L2 writing: Multiple analyses of a common corpus , 2014 .

[57]  Scott Jarvis,et al.  vocd: A theoretical and empirical evaluation , 2007 .

[58]  B. MacWhinney.  The CHILDES project: tools for analyzing talk , 1992 .

[59]  R. Hout,et al.  Lexical richness in the spontaneous speech of bilinguals , 2003 .

[60]  R. Ellis.  The Differential Effects of Three Types of Task Planning on the Fluency, Complexity, and Accuracy in L2 Oral Production , 2009 .

[61]  J. McWhorter,et al.  The worlds simplest grammars are creole grammars , 2001 .

[62]  Pauline Foster,et al.  Native Speakers and Task Performance: Comparing Effects on Complexity, Fluency, and Lexical Diversity. , 2009 .

[63]  P. Nation,et al.  Vocabulary size and use: Lexical richness in L2 written production , 1995 .

[64]  Benedikt Szmrecsanyi,et al.  Introduction: Linguistic complexity Second Language Acquisition, indigenization, contact , 2012 .

[65]  P. Skehan.  Task-based instruction , 2003, Language Teaching.

[66]  Folkert Kuiken,et al.  Dimensions of L2 Performance and Proficiency: Complexity, Accuracy and Fluency in SLA. Language Learning & Language Teaching. Volume 32. , 2012 .

[67]  William C. Ritchie,et al.  Handbook of second language acquisition , 1999 .

[68]  J. Charles Alderson.  Testing in EAP: progress? Achievement? Proficiency? , 2000 .

[69]  Charlene Polio,et al.  An investigation into current measures of linguistic accuracy in second language writing research , 2014 .

[70]  P. Skehan.  语言学习认知法 = A cognitive approach to language learning , 1998 .

[71]  Lourdes Ortega,et al.  Interlanguage complexity A construct in search of theoretical renewal , 2012 .

[72]  Wayne Rimmer,et al.  Measuring grammatical complexity: the Gordian knot , 2006 .

[73]  Marianne Spoelman,et al.  Dynamic Patterns in Development of Accuracy and Complexity: A Longitudinal Case Study in the Acquisition of Finnish , 2010 .

[74]  Melanie Mitchell,et al.  Complexity - A Guided Tour , 2009 .

[75]  Peter Skehan,et al.  Complexity, accuracy, fluency and lexis in task-based performance: a meta-analysis of the Ealing research , 2012 .

[76]  Betty Lou Leaver,et al.  Developing Professional-Level Language Proficiency: Principles and practices in teaching Superior-level language skills: not just more of the same , 2002 .

[77]  Scott Jarvis,et al.  Capturing the Diversity in Lexical Diversity. , 2013 .

[78]  Matti Miestamo,et al.  Grammatical complexity in cross-linguistic perspective , 2008 .

[79]  Pauline Foster,et al.  Task type and task processing conditions as influences on foreign language performance , 1997 .

[80]  Nina Vyatkina,et al.  The Development of Second Language Writing Complexity in Groups and Individuals: A Longitudinal Learner Corpus Study , 2012 .

[81]  Peter Skehan,et al.  Modelling Second Language Performance: Integrating Complexity, Accuracy, Fluency, and Lexis , 2009 .

[82]  Barbara F. Freed.  An Overview of Issues and Research in Language Learning in a Study Abroad Setting. , 1998 .

[83]  Kim Sauter,et al.  English Sentence Analysis: An introductory course , 2000 .

[84]  John W. Oller,et al.  ORAL PROFICIENCY SCALES: CONSTRUCT VALIDITY AND THE HALO EFFECT , 1980 .

[85]  Shunji Inagaki,et al.  Second Language Development in Writing: Measures of Fluency, Accuracy, and Complexity , 1998 .

[86]  David Malvern,et al.  Lexical Diversity and Language Development: Quantification and Assessment , 2004 .

[87]  John McWhorter,et al.  Linguistic Simplicity and Complexity: Why Do Languages Undress? , 2011 .

[88]  Rod Ellis,et al.  Analysing Learner Language , 2005 .