Subjective Evaluation of Comprehensibility in Movie Interactions

Various research works have dealt with the comprehensibility of textual, audio, or audiovisual documents, and showed that factors related to text (e.g. linguistic complexity), sound (e.g. speech intelligibility), image (e.g. presence of visual context), or even to cognition and emotion can play a major role in the ability of humans to understand the semantic and pragmatic contents of a given document. However, to date, no reference human data is available that could help investigating the role of the linguistic and extralinguistic information present at these different levels (i.e., linguistic, audio/phonetic, and visual) in multimodal documents (e.g., movies). The present work aimed at building a corpus of human annotations that would help to study further how much and in which way the human perception of comprehensibility (i.e., of the difficulty of comprehension, referred in this paper as overall difficulty) of audiovisual documents is affected (1) by lexical complexity, grammatical complexity, and speech intelligibility, and (2) by the modality/ies (text, audio, video) available to the human recipient.

[1]  Beth M. Phillips,et al.  Cognitive Correlates of Listening Comprehension. , 2014 .

[2]  Kevyn Collins-Thompson,et al.  Predicting reading difficulty with statistical language models , 2005, J. Assoc. Inf. Sci. Technol..

[3]  E. Thorndike.  The Teacher's Word Book , 2007 .

[4]  M. Alibali,et al.  Gesture's role in speaking, learning, and creating language. , 2013, Annual review of psychology.

[5]  M. C. Levorato,et al.  The relationship between listening comprehension of text and sentences in preschoolers: Specific or mediated by lower and higher level components? , 2011, Applied Psycholinguistics.

[6]  Tomoko Kurita,et al.  Issues in Second Language Listening Comprehension and the Pedagogical Implications , 2012 .

[8]  H. Magen.  The perception of foreign-accented speech , 1998 .

[9]  Jeremy Harmer,et al.  The Practice of English Language Teaching , 1983 .

[10]  The Roles of Verbal Short-Term Memory and Working Memory in the Acquisition of Grammar by Children With Williams Syndrome , 2003, Developmental neuropsychology.

[11]  G. Gougenheim,et al.  L'élaboration du français fondamental (1er degré) : étude sur l'établissement d'un vocabulaire et d'une grammaire de base , 1964 .

[12]  R. Flesch.  A new readability yardstick. , 1948, The Journal of applied psychology.

[13]  Anne Cutler,et al.  Non-native speech perception in adverse conditions: A review , 2010, Speech Commun..

[14]  The Effect of Speech Rate on Listening Comprehension of EFL learners , 2010 .

[15]  William E. Tunmer,et al.  The role of language-related factors in reading disability. , 1989 .

[16]  Thomas François.  Modèles statistiques pour l’estimation automatique de la difficulté de textes de FLE , 2009, JEPTALNRECITAL.

[17]  Eileen Kay Blau,et al.  More on Comprehensible Input: The Effect of Pauses and Hesitation Markers on Listening Comprehension. , 1991 .

[18]  S. Stokes,et al.  Factors that influence vocabulary development in two-year-old children. , 2009, Journal of child psychology and psychiatry, and allied disciplines.

[19]  John O. Willis,et al.  Comprehensive Assessment of Spoken Language , 2014 .

[20]  Slava Kalyuga,et al.  Managing split-attention and redundancy in multimedia instruction , 1999 .

[21]  Faisal Mustafa,et al.  How Much Videos Win over Audios in Listening Instruction for EFL Learners. , 2017 .

[22]  J. Read,et al.  Reducing Listening Test Anxiety Through Various Forms of Listening Support , 2008 .

[23]  S. Scott,et al.  Comprehension of familiar and unfamiliar native accents under adverse listening conditions. , 2009, Journal of experimental psychology. Human perception and performance.

[24]  Abdolmajid Hayati,et al.  The effect of films with and without subtitles on listening comprehension of EFL learners , 2011, Br. J. Educ. Technol..

[25]  T. J. McCarthy,et al.  The Effects of Native Language vs. Target Language Captions on Foreign Language Students' DVD Video Comprehension , 2001 .

[26]  John H. L. Hansen,et al.  Perceptual Recognition Cues in Native English Accent Variation: "Listener Accent, Perceived Accent, and Comprehension" , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[27]  Christine C. M. Goh,et al.  How Much Do Learners Know about the Factors That Influence Their Listening Comprehension , 1999 .

[28]  Véronique Traverso,et al.  L'analyse des conversations , 1999 .

[29]  Jan-Louis Kruger,et al.  Measuring the impact of subtitles on cognitive load: eye tracking and dynamic audiovisual texts , 2013, ETSA '13.

[30]  Wim Van Den Noortgate,et al.  Captioned video for L2 listening and vocabulary learning: A meta-analysis , 2013 .

[31]  Hussein M. Elkhafaifi.  Listening Comprehension and Anxiety in the Arabic Language Classroom , 2005 .

[32]  Charles A. Perfetti,et al.  Word Knowledge in a Theory of Reading Comprehension , 2014 .

[33]  Shawn A. Weil,et al.  The impact of phonetic dissimilarity on the perception of foreign accented speech , 2003 .

[34]  J. McQueen,et al.  Foreign Subtitles Help but Native-Language Subtitles Harm Foreign Speech Perception , 2009, PloS one.

[35]  Yong Zhao.  The Effects of Listeners' Control of Speech Rate on Second Language Comprehension , 1997 .

[36]  I. Nation.  How Large a Vocabulary Is Needed for Reading and Listening? , 2006 .

[37]  Susan Nissan,et al.  AN ANALYSIS OF FACTORS AFFECTING THE DIFFICULTY OF DIALOGUE ITEMS IN TOEFL LISTENING COMPREHENSION , 1995 .

[38]  G. Buck.  Assessing Listening , 2001 .

[39]  B Voss.  Hesitation Phenomena as Sources of Perceptual Errors for Non-Native Speakers , 1979, Language and speech.

[40]  D. Hardison,et al.  The Role of Gestures and Facial Cues in Second Language Listening Comprehension , 2005 .

[41]  Lucius Adelno Sherman,et al.  Analytics of Literature: A Manual for the Objective Study of English Prose and Poetry , 2009 .

[42]  John R. Anderson.  Cognitive Psychology and Its Implications , 1980 .

[43]  王英兰.  Factors Affecting Listening Comprehension , 2004 .

[44]  Susan Kellerman,et al.  ‘I See What You Mean’: The Role of Kinesic Behaviour in Listening and Implications for Foreign and Second Language Learning , 1992 .

[45]  Cédrick Fairon,et al.  An “AI readability” Formula for French as a Foreign Language , 2012, EMNLP.

[46]  Birgitta Larsby,et al.  Cognitive performance and perceived effort in speech processing tasks: effects of different noise backgrounds in normal-hearing and hearing-impaired subjects. , 2005, International journal of audiology.

[47]  D. Tannen.  Spoken and written language : exploring orality and literacy , 1984 .

[48]  T. Dahl,et al.  How I See What You're Saying: The Role of Gestures in Native and Foreign Language Listening Comprehension , 2014 .

[49]  M. Daneman,et al.  Working memory and language comprehension: A meta-analysis , 1996, Psychonomic bulletin & review.