Using global complexity measures to assess second language proficiency: Comparing CLIL and non-CLIL learners of English and Dutch in French-speaking Belgium

This study falls within the framework of an interdisciplinary project on Content and Language Integrated Learning (CLIL) in French-speaking Belgium. One of the project’s aims is to compare the L2 language proficiency of CLIL and non-CLIL French-speaking learners of English and Dutch. In the present paper we focus on learners’ global proficiency and use of different types of metrics to assess syntactic and lexical complexity in the learners’ written productions. Using various computational tools, we extracted lexical and syntactic complexity scores for texts written by CLIL and non-CLIL pupils in their L2 (English or Dutch) and their L1 (French). These scores were then compared to investigate the potential influence of CLIL education on the pupils’ language proficiency as CLIL programs provide more target language input than non-CLIL programs. We therefore hypothesized that CLIL pupils would display a more nativelike competence in the target language, i.e. a more native-like ...

[1]  Rie Koizumi,et al.  Relationships between text length and lexical diversity measures: Can we use short texts of less than 100 tokens? , 2012 .

[2]  Pauline Foster,et al.  Task Design and Second Language Performance: The Effect of Narrative Type on Learner Output , 2008 .

[3]  Lourdes Ortega,et al.  Interlanguage complexity A construct in search of theoretical renewal , 2012 .

[4]  M. Juan-Garau,et al.  Writing Development Under CLIL Provision , 2015 .

[5]  Xiaofei Lu,et al.  Automatic analysis of syntactic complexity in second language writing , 2010 .

[6]  Thomas François,et al.  Les apports du traitement automatique des langues à la lisibilité du français langue étrangère , 2011 .

[7]  Patrick Parslow,et al.  Back to Basics: How Measures of Lexical Diversity Can Help Discriminate between CEFR Levels , 2016 .

[8]  Teresa Navés,et al.  CLIL in Catalonia: An Overview of Research Studies , 2010 .

[9]  Danielle S. McNamara,et al.  Computational assessment of lexical differences in L1 and L2 writing , 2009 .

[10]  Lawrence T. Frase,et al.  COMPUTER ANALYSIS OF THE TOEFL TEST OF WRITTEN ENGLISH , 1998 .

[11]  A. E. Pierce,et al.  Language Acquisition and Syntactic Theory: A Comparative Analysis of French and English Child Grammars , 1992 .

[12]  Elisabet Pladevall-Ballester.  Exploring primary school CLIL perceptions in Catalonia: students', teachers' and parents' opinions and expectations , 2015 .

[13]  Alex Housen,et al.  Evaluating short-term changes in L2 complexity development , 2015 .

[14]  C. Pérez-Vidal,et al.  CLIL in Context: Profiling Language Abilities , 2015 .

[15]  L. Ortega.  Syntactic Complexity Measures and Their Relationship to L2 Proficiency: A Research Synthesis of College-Level L2 Writing. , 2003 .

[16]  Guoxing Yu,et al.  Lexical Diversity in Writing and Speaking Task Performances , 2010 .

[17]  Nina Vyatkina,et al.  The Development of Second Language Writing Complexity in Groups and Individuals: A Longitudinal Learner Corpus Study , 2012 .

[18]  Heidi Fox,et al.  Phrasal Cohesion and Statistical Machine Translation , 2002, EMNLP.

[20]  Batia Laufer,et al.  The Relationship Between Passive and Active Vocabularies: Effects of LanguageLearning Context , 1998 .

[21]  Folkert Kuiken,et al.  Complexity, Accuracy and Fluency: Definitions, Measurement and Research. , 2012 .

[22]  Ritualised routines and L2 acquisition: Acquisition strategies in an immersion program , 1991 .

[23]  T. McNamara,et al.  Assessed Levels of Second Language Speaking Proficiency: How Distinct? , 2007 .

[24]  David Marsh,et al.  CLIL/EMILE - the European dimension : actions, trends and foresight potential , 2002 .

[25]  K. W. Hunt.  Grammatical structures written at three grade levels , 1965 .

[26]  Fred Genesee,et al.  Second Language Learning in School Settings: Lessons from Immersion , 2014 .

[28]  J. Norris,et al.  Towards an Organic Approach to Investigating CAF in Instructed SLA: The Case of Complexity , 2009 .

[29]  Xi Chen,et al.  Cross-Language Transfer of Morphological Awareness in Chinese-English Bilinguals. , 2011 .

[30]  Christiane Dalton-Puffer,et al.  The CLIL differential: Comparing the writing of CLIL and non-CLIL students in higher colleges of technology , 2010 .

[31]  Leslie Grant,et al.  Using Computer-Tagged Linguistic Features to Describe L2 Writing Differences , 2000 .

[32]  Philip M. McCarthy,et al.  MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment , 2010, Behavior research methods.

[33]  Danielle S. McNamara,et al.  Chapter 4. Validating lexical measures using human scores of lexical proficiency , 2013 .

[34]  Danielle S. McNamara,et al.  Predicting the proficiency level of language learners using lexical indices , 2012 .

[35]  R. Hout,et al.  Lexical richness in the spontaneous speech of bilinguals , 2003 .

[36]  Alex Housen,et al.  Defining and operationalising L2 complexity , 2012 .

[37]  Laura K. Allen,et al.  Linguistic Microfeatures to Predict L2 Writing Proficiency: A Case Study in Automated Writing Evaluation. , 2014 .

[38]  Manfred Görlach,et al.  English in Europe , 2004 .

[39]  Dana R. Ferris,et al.  Lexical and Syntactic Features of ESL Writing by Students at Different Levels of L2 Proficiency , 1994 .

[40]  Jeanine Treffers-Daller.  Chapter 3. Measuring lexical diversity among L2 learners of French , 2013 .

[41]  Arthur C. Graesser,et al.  Coh-Metrix: Analysis of text on cohesion and language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.