Using Trialogues to Measure English Language Skills

We explored the use of technology-assisted, trialogue-based tasks to measure the English language proficiency of students learning English as a second or foreign language. A presumed benefit of the system for language assessment is its suitability for use in scenario-based tasks that integrate multiple language skills. This integration allows test developers to simulate real-life language use situations by administering more authentic test tasks to strengthen the link between test performance and its interpretation as an indicator of expected success in performing real-life language use tasks. We conducted a usability study with 20 English language learners (ELLs) in grades 3 to 5, representing ten different first languages. Results suggested that the trialoguebased tasks engaged students in an authentic experience and provided evidence of their English proficiency. Perception data indicated that students perceived the trialogue-based tasks positively and understood how to respond to the virtual characters. Implications are discussed for future research and the potential of using trialogue-based assessment tasks for measuring the proficiency of young ELLs.

[1]  D. Borsboom.  Educational Measurement (4th ed.) , 2009 .

[2]  Lyle F. Bachman,et al.  Language assessment in practice : developing language assessments and justifying their use in the real world , 2010 .

[3]  Aanshi Varshney,et al.  Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments , 2014 .

[4]  H. Widdowson,et al.  Teaching Language as Communication , 1979 .

[5]  Kristen N. Moreno,et al.  Development and evaluation of a lesson authoring tool for AutoTutor , 2003 .

[6]  Claude Frasson,et al.  Intelligent Tutoring Systems: At the Crossroads of Artificial Intelligence and Education , 1990 .

[7]  Arthur C. Graesser,et al.  ARIES: An Intelligent Tutoring System Assisted by Conversational Agents , 2009, AIED.

[8]  Nancy H. Hornberger,et al.  Encyclopedia of language and education , 2008 .

[9]  V. Rideout,et al.  Generation M2: Media in the Lives of 8- to 18-Year-Olds , 2010 .

[10]  Anne Strauss,et al.  Assessing Young Language Learners , 2016 .

[11]  Arthur C. Graesser,et al.  Teaching Tactics and Dialog in AutoTutor , 2001 .

[12]  Angela Hasselgreen,et al.  Assessing the language of young learners , 2005 .

[13]  Arthur Baskin,et al.  Learning companion systems , 1989 .

[14]  Keith A. Johnson.  Communicative Syllabus Design and Methodology , 1982 .

[15]  A. Hasselgren,et al.  The assessment of the English ability of young learners in Norwegian schools: an innovative approach , 2000 .

[16]  James C. Lester,et al.  Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments , 2000 .

[17]  Lynne Cameron,et al.  Challenges for ELT from the expansion in teaching children , 2003 .

[18]  Arthur C. Graesser,et al.  Critiquing Media Reports with Flawed Scientific Findings: Operation ARIES! A Game with Animated Agents and Natural Language Trialogues , 2010, Intelligent Tutoring Systems.

[19]  Penelope A. McKay,et al.  RESEARCH INTO THE ASSESSMENT OF SCHOOL-AGE LANGUAGE LEARNERS , 2005, Annual Review of Applied Linguistics.