Developing and evaluating SynctoLearn, a fully automatic video and transcript synchronization tool for EFL learners

Authentic videos are always motivational for foreign language learners. According to the findings of many empirical studies, subtitled L2 videos are particularly useful for foreign language learning. Although there are many authentic English videos available on the Internet, most of these videos do not have subtitles. If subtitles can be added to videos, EFL students might better understand the video content and pick up more vocabulary items. To help EFL students make better use of authentic videos, an automatic video and transcript synchronization system called SynctoLearn was developed. The SynctoLearn system uses speech recognition technologies to automatically synchronize voices in audios and videos with their respective transcripts. Two studies were conducted in order to assess the effectiveness of this listening tool and investigate EFL learners' perception toward this device. In the first study, two college classes at similar proficiency levels were divided in two groups. One group viewed the synchronized video and another viewed the same video without any synchronized subtitles. The group that watched the synchronized video outperformed the other group in summarizing the key points of the video. In the second study, two college EFL classes used the SynctoLearn website for a whole semester and were invited to fill in a questionnaire. Based on the survey results, students indicated that they enjoyed watching synchronized videos. They regarded the design of the SynctoLearn system positively and particularly lauded the options that allowed them to click and replay any video segments and show/hide the subtitles at any time. Students, however, suggested that the video quality and subtitle presentation can be further improved. Moreover, additional learning support such as online dictionaries can be included in the system. The results of the experiment and the survey indicate that the SynctoLearn tool, which provides synchronized subtitles, can be used as a supporting tool to help EFL students reduce their cognitive loads and anxiety levels when tackling authentic videos. The findings and the technologies used in this article should be useful for teachers and researchers who want to use or develop similar tools for foreign language learning.

[1]  Michael H. Long.  The Role of the Linguistic Environment in Second Language Acquisition , 1996 .

[2]  David E. Eskey,et al.  The Effects of Closed-Captioned Television on the Listening Comprehension of Intermediate English as a Second Language (ESL) Students , 1999 .

[3]  Cees M. Koolstra,et al.  Children's vocabulary acquisition in a foreign language through watching subtitled television programs at home , 1999 .

[4]  Patricia S. Koskinen,et al.  Captioned Television and the Vocabulary Acquisition of Adult Second Language Correlational Facility Residents , 1996 .

[5]  Robert Vanderplank,et al.  The value of teletext sub-titles in language learning , 1988 .

[6]  Michael Levy,et al.  Computer applications in second language acquisition : Foundations for teaching , testing and research , 2009 .

[7]  Rita M. Bean,et al.  Using closed captioned television to teach reading to adults , 1989 .

[8]  Joan-Tomás Pujolă,et al.  CALLing for help: researching language learning strategies using help facilities in a web-based multimedia program , 2002, ReCALL.

[9]  Isabel Borrás,et al.  Effects of Multimedia Courseware Subtitling on the Speaking Performance of College Students of French , 1994 .

[10]  Helen Gant Guillory,et al.  The Effects of Keyword Captions to Authentic French Video on Learner Comprehension , 2013 .

[11]  Carol A. Chapelle,et al.  Multimedia CALL: Lessons to be Learned from Research on Instructed SLA , 1998 .

[12]  Volker Hegelheimer,et al.  Help Options and Multimedia Listening: Students' Use of Subtitles and the Transcript , 2007 .

[13]  Susan M. Gass,et al.  The effects of captioning videos used for foreign language listening activities , 2010 .

[14]  Tetyana Sydorenko,et al.  Modality of Input and Vocabulary Acquisition , 2010 .

[15]  Patricia S. Koskinen,et al.  Captioned Television as Comprehensible Input: Effects of Incidental Word Learning from Context for Language Minority Students. , 1992 .

[16]  T. Pica.  Research on Negotiation: What Does It Reveal about Second-Language Learning Conditions, Processes, and Outcomes?. , 1994 .

[17]  Paul Markham,et al.  The Influence of Culture-Specific Background Knowledge and Captions on Second Language Comprehension , 2001 .

[18]  Linda C. Jones,et al.  Supporting Listening Comprehension and Vocabulary Acquisition with Multimedia Annotations: The Students' Voice , 2013 .

[19]  T. Murugavel,et al.  Using Authentic Video in the Language Classroom , 2005 .

[20]  J. Chung,et al.  The Effects of Using Video Texts Supported with Advance Organizers and Captions on Chinese College Students' Listening Comprehension: An Empirical Study , 1999 .

[21]  Thomas J. Garza,et al.  Evaluating the Use of Captioned Video Materials in Advanced Foreign Language Learning , 1991 .