The effect of bimodal input on implicit and explicit memory: An investigation into the benefits of within-language subtitling

Two experiments examined the effect of single-modality (sound or text) and bimodal (sound and text) presentation on word learning, as measured by both improvements in spoken word recognition efficiency (long lag repetition priming) and recognition memory. Native and advanced nonnative speakers of English were tested. In Experiment 1 auditory lexical decisions on familiar words were equally primed by prior bimodal and sound-only presentation, whereas there were no priming effects for nonwords. Experiment 2 employed a rhyme judgment task using nonwords. Repetition priming of auditory rhyme judgment decisions was now obtained, and this was greater in the bimodal than the sound-only condition. In both experiments prior bimodal presentation improved recognition memory for spoken words and nonwords compared to single modality presentation. We conclude that simultaneous text presentation can aid novel word learning under certain conditions, as assessed by both explicit and implicit memory tests.

[1]  Mark S. Seidenberg,et al.  Multiple code activation in word recognition: evidence from rhyme monitoring. , 1981, Journal of experimental psychology. Human learning and memory.

[2]  Anne Cutler,et al.  The syllable's differing role in the segmentation of French and English. , 1986 .

[3]  Randall S. Davis.  Captioned Video: Making It Work for You , 1998 .

[4]  L. Jacoby.  Remembering the data: analyzing interactive processes in reading , 1983 .

[5]  Colin M. Macleod,et al.  Repetition Priming in Speeded Word Reading: Contributions of Perceptual and Conceptual Processing Episodes , 2000 .

[6]  Jing-Mei Chung,et al.  The Effects of Using Advance Organizers and Captions To Introduce Video in the Foreign Language Classroom. , 1996 .

[7]  R Frost,et al.  Toward a strong phonological theory of visual word recognition: true issues and false trails. , 1998, Psychological bulletin.

[8]  B. Repp,et al.  Can speech perception be influenced by simultaneous presentation of print , 1988 .

[9]  M. Masson.  A distributed memory model of semantic priming. , 1995 .

[10]  Andrew W. Ellis,et al.  Modality-specific repetition priming of auditory word recognition , 1982 .

[11]  R. Vanderplank.  Paying attention to the words: Practical and theoretical problems in watching television programmes with uni-lingual (CEEFAX) sub-titles , 1990 .

[12]  F. Craik,et al.  Depth of processing and the retention of words , 1975 .

[13]  Jonathan Grainger,et al.  Masked Orthographic and Phonological Priming in Visual Word Recognition and Naming: Cross-Task Comparisons , 1996 .

[14]  Thomas J. Garza,et al.  Evaluating the Use of Captioned Video Materials in Advanced Foreign Language Learning , 1991 .

[15]  M. Coltheart.  Lexical access in simple reading tasks , 1978 .

[16]  U. Frauenfelder,et al.  Grapheme Context Effects on Phonemic Processing , 1989 .

[17]  Martine Danan,et al.  Reversed subtitling and dual coding theory: New directions for foreign language instruction , 1992 .

[18]  Jordan Grafman,et al.  Handbook of Neuropsychology , 1991 .

[19]  William D. Marslen-Wilson,et al.  Access and integration: projecting sound onto meaning , 1989 .

[20]  D. Allport,et al.  Components of the Mental Lexicon , 1981 .

[21]  Mark S. Seidenberg,et al.  Orthographic effects on rhyme monitoring. , 1979 .

[22]  F. Craik,et al.  Levels of Pro-cessing: A Framework for Memory Research , 1975 .

[23]  John D. Bransford,et al.  Levels of processing versus transfer appropriate processing , 1977 .

[24]  L. Squire,et al.  Intact priming of words and nonwords in amnesia , 1991, Psychobiology.

[25]  Isabel Borrás,et al.  Effects of Multimedia Courseware Subtitling on the Speaking Performance of College Students of French , 1994 .

[26]  John Morton,et al.  Facilitation in Word Recognition: Experiments Causing Change in the Logogen Model , 1979 .

[27]  Don L. Scarborough,et al.  Accessing lexical memory: The transfer of word repetition effects across task and modality , 1979 .

[28]  Ronald A. Cole,et al.  Sound and spelling in spoken word recognition. , 1985 .

[29]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: part 1.: an account of basic findings , 1988 .

[30]  A. Healy,et al.  Detecting phonemes and letters in text: Interactions between different types and levels of processes , 1993, Memory & cognition.

[31]  N. Ellis,et al.  Implicit and explicit learning of languages , 1997 .

[32]  L. Jacoby,et al.  On the relationship between autobiographical memory and perceptual learning. , 1981, Journal of experimental psychology. General.

[33]  J. Morton,et al.  Facilitation of auditory word recognition , 1984, Memory & cognition.

[34]  G. Stone,et al.  Word identification in reading and the promise of subsymbolic psycholinguistics. , 1990, Psychological review.

[35]  H. Roediger.  Implicit memory in normal human subjects , 1993 .

[36]  D Milech,et al.  Common and modality-specific processes in the mental lexicon , 1983, Memory & cognition.

[37]  Wallace E. Lambert,et al.  PAIRING SCRIPT AND DIALOGUE: COMBINATIONS THAT SHOW PROMISE FOR SECOND OR FOREIGN LANGUAGE LEARNING , 1984 .

[38]  J. B. Wyman,et al.  What is reading ability , 1921 .

[39]  Kim Kirsner,et al.  Modality effects in word identification , 1974, Memory & cognition.

[40]  Daniel L. Schacter,et al.  Priming and Multiple Memory Systems: Perceptual Mechanisms of Implicit Memory , 1992, Journal of Cognitive Neuroscience.

[41]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[42]  Alison Moore,et al.  Learning to listen. , 2002, Nursing older people.

[43]  R M Shiffrin,et al.  Episodic and lexical contributions to the repetition effect in word identification. , 1983, Journal of experimental psychology. General.

[44]  P. Robinson.  Attention, Memory, and the “Noticing” Hypothesis , 1995 .

[45]  D. Balota,et al.  Are lexical decisions a good measure of lexical access? The role of word frequency in the neglected decision stage. , 1984, Journal of experimental psychology. Human perception and performance.

[46]  J. Ziegler,et al.  The Feedback Consistency Effect in Lexical Decision and Naming , 1997 .

[47]  Udo O. H. Jung.  THE CHALLENGE OF BROADCAST VIDEOTEX TO APPLIED LINGUISTICS , 1990 .

[48]  Wallace E. Lambert,et al.  Choosing the languages of subtitles and spoken dialogues for media presentations: Implications for second language education , 1981, Applied Psycholinguistics.

[49]  N. Ellis.  Vocabulary acquisition: The implicit ins and outs of explicit cognitive mediation. , 1994 .

[50]  Bruce F. Pennington,et al.  Word identification in reading and the promise of subsymbolic psycholinguistics. , 1990, Psychological review.