Scaling Up Intervention Studies to Investigate Real-Life Foreign Language Learning in School

Abstract Intervention studies typically target a focused aspect of language learning that is studied over a relatively short time frame for a relatively small number of participants in a controlled setting. While for many research questions, this is effective, it can also limit the ecological validity and relevance of the results for real-life language learning. In educational science, large-scale randomized controlled field trials (RCTs) are seen as the gold standard method for addressing this challenge—yet they require intervention to scale to hundreds of learners in their varied, authentic contexts. We discuss the use of technology in support of large-scale interventions that are fully integrated in regular classes in secondary school. As an experimentation platform, we developed a web-based workbook to replace a printed workbook widely used in German schools. The web-based FeedBook provides immediate scaffolded feedback to students on form and meaning for various exercise types, covering the full range of constructions in the seventh-grade English curriculum. Following the conceptual discussion, we report on the first results of an ongoing, yearlong RCT. The results confirm the effectiveness of the scaffolded feedback, and the approach makes students and learning process variables accessible for the analysis of learning in a real-world context.

[1]  Manfred Pienemann.  Processability Theory and Teachability , 2012 .

[2]  Masatoshi Sato,et al.  The Routledge handbook of instructed second language acquisition , 2017 .

[3]  Walt Detmar Meurers,et al.  On using intelligent computer-assisted language learning in real-life foreign language teaching and learning , 2011, ReCALL.

[4]  Walt Detmar Meurers,et al.  Evidence and Interpretation in Language Learning Research: Opportunities for Collaboration With Computational Linguistics: Evidence and Interpretation , 2017 .

[5]  Venessa Keesler,et al.  Scaling-Up Exemplary Interventions , 2006 .

[6]  D. Ferris.  SECOND LANGUAGE WRITING RESEARCH AND WRITTEN CORRECTIVE FEEDBACK IN SLA , 2010, Studies in Second Language Acquisition.

[7]  G. Pallotti,et al.  An operational definition of the emergence criterion , 2007 .

[8]  L. Hedges,et al.  How Large Are Teacher Effects? , 2004 .

[9]  Jara Rodríguez Montesinos.  English as a first foreign language , 2019 .

[10]  R. Lyster,et al.  ORAL FEEDBACK IN CLASSROOM SLA , 2010, Studies in Second Language Acquisition.

[11]  Walt Detmar Meurers,et al.  Feedback Strategies for Form and Meaning in a Real-life Language Tutoring System , 2018 .

[12]  Shawn Loewen.  Instructed Second Language Acquisition , 2019 .

[13]  Detmar Meurers,et al.  Integrating parallel analysis modules to evaluate the meaning of answers to reading comprehension questions , 2011 .

[14]  John Bitchener,et al.  Written Corrective Feedback for L2 Development , 2016 .

[15]  Rod Ellis,et al.  Task-based Language Learning and Teaching , 2003 .

[16]  Hossein Nassaji,et al.  Corrective Feedback in Second Language Teaching and Learning : Research, Theory, Applications, Implications , 2017 .

[17]  Chris Dede,et al.  The Cambridge Handbook of the Learning Sciences: Scaling Up , 2005 .

[18]  Shaofeng Li,et al.  Cognitive differences and ISLA , 2017 .

[19]  Albert T. Corbett,et al.  Chapter 37 – Intelligent Tutoring Systems , 1997 .

[20]  Vincent Aleven,et al.  Defining "Ill-Defined Domains"; A literature survey. , 2006 .

[21]  Younghee Sheen,et al.  Corrective Feedback, Individual Differences and Second Language Learning , 2011 .

[22]  Noriko Nagata,et al.  Some Design Issues for an Online Japanese Textbook , 2010 .

[23]  Z. Dörnyei,et al.  Validation of the C-test amongst Hungarian EFL learners , 1992 .

[24]  Arthur C. Graesser,et al.  AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring , 2014, International Journal of Artificial Intelligence in Education.

[25]  S. Vaughn,et al.  Improving Content Knowledge and Comprehension for English Language Learners: Findings From a Randomized Control Trial , 2017 .

[26]  Tom Routen,et al.  Intelligent Tutoring Systems , 1996, Lecture Notes in Computer Science.

[27]  Alison Mackey,et al.  Interactional feedback in synchronous computer-mediated communication : A Review of the State of the Art , 2017 .

[28]  Johannes Sörgel.  Englisch als erste Fremdsprache , 1908 .

[29]  Aline Godfroid,et al.  SLA for all? Reproducing SLA research in non-academic samples , 2018 .

[30]  Ruiz Hernández,et al.  Individual Differences and Instructed Second Language Acquisition: Insights from Intelligent Computer Assisted Language Learning , 2019 .

[31]  B. MacWhinney.  A Shared Platform for Studying Second Language Acquisition. , 2017 .

[32]  Walt Detmar Meurers,et al.  Generating Feedback for English Foreign Language Exercises , 2018, BEA@NAACL-HLT.

[33]  Trude Heift,et al.  Developing an Intelligent Language Tutor , 2010 .

[34]  Walt Detmar Meurers,et al.  Automatic Focus Annotation: Bringing Formal Pragmatics Alive in Analyzing the Information Structure of Authentic Data , 2018, NAACL.

[35]  Chris Dede,et al.  Scaling Up: Evolving Innovations beyond Ideal Settings to Challenging Contexts of Practice , 2005 .

[36]  Nina Spada,et al.  4. The effectiveness of corrective feedback for the acquisition of L2 grammar: A meta-analysis of the research , 2006 .

[37]  Jeffrey L. Foster,et al.  Measuring Working Memory Capacity on the Web with the Online Working Memory Lab (the OWL) , 2016 .

[38]  J. Hattie,et al.  The Power of Feedback , 2007 .

[39]  T. Heift,et al.  Errors and Intelligence in Computer-Assisted Language Learning: Parsers and Pedagogues. Routledge Studies in Computer Assisted Language Learning. , 2007 .

[40]  R. Ellis.  Focus on form: A critical review , 2016 .

[41]  John Truscott,et al.  Review Article The Case Against Grammar Correction in L2 Writing Classes , 1996 .

[42]  Walt Detmar Meurers,et al.  Developing a web-based workbook for English supporting the interaction of students and teachers , 2017 .

[43]  Walt Detmar Meurers,et al.  Question Generation for Language Learning: From ensuring texts are read to supporting learning , 2017, BEA@EMNLP.

[44]  J. Metcalfe,et al.  Scaffolding feedback to maximize long-term error correction , 2010 .

[45]  J. Lantolf,et al.  Negative Feedback as Regulation and Second Language Learning in the Zone of Proximal Development , 1994 .

[46]  Ulrich Trautwein,et al.  Assessing task values in five subjects during secondary school: Measurement structure and mean level differences across grade level, gender, and academic subject , 2017 .

[47]  Alison Mackey,et al.  Input, Interaction, and Corrective Feedback in L2 Learning , 2012 .

[48]  Anita Lämmerer.  Classroom-based research , 2015 .

[49]  Volker Hegelheimer,et al.  Computer-assisted corrective feedback and language learning , 2017 .

[50]  Noriko Nagata,et al.  Robo-Sensei's NLP-Based Error Detection and Feedback Generation , 2013 .

[51]  L. Hedges,et al.  Randomised trials in education in the USA , 2018, Educational Research.

[52]  Walt Detmar Meurers,et al.  Linguistically Aware Information Retrieval: Providing Input Enrichment for Second Language Learners , 2016, BEA@NAACL-HLT.

[53]  Masatoshi Sato,et al.  Chapter 3. Methodological strengths, challenges, and joys of classroom-based quasi-experimental research , 2019, Language Learning & Language Teaching.

[54]  Detmar Meurers,et al.  How Can Writing Tasks Be Characterized in a Way Serving Pedagogical Goals and Automatic Analysis Needs , 2015 .

[55]  D. Ferris.  The ‘‘Grammar Correction’ ’ Debate in L2 Writing: , 2022 .

[56]  Luke Plonsky,et al.  The CALL-SLA Interface: Insights from a Second-Order Synthesis. , 2016 .

[57]  Dragan Gasevic,et al.  Handbook of Learning Analytics , 2017 .

[58]  James A. Kulik,et al.  Effectiveness of Intelligent Tutoring Systems , 2016 .

[59]  Walt Detmar Meurers,et al.  Interdisciplinary Research at the Intersection of CALL, NLP, and SLA: Methodological Implications from an Input Enhancement Project. , 2017 .