RESEARCH ON TEXT COMPREHENSION IN MULTIMEDIA ENVIRONMENTS

Based on underlying theories of L2 reading comprehension and text comprehension with multimedia, and in light of the new tools and modes of information presentation that are now available, researchers have a variety of avenues for studying how people comprehend text in a second language with the help of multimodal instructional materials. In this paper we discuss how L2 reading research is focusing increasingly on the cognitive processes involved in reading, that is, the interaction of lower-level, bottom-up processes such as vocabulary acquisition with higherlevel, top-down processes such as activating prior knowledge. We combine this knowledge with current research on learning with multimedia, focusing on how learners integrate verbal and visual information, particularly with respect to the individual differences among learners that moderate, if not determine, learning processes. INTRODUCTION The process of reading and comprehending in one's native language (L1) is very complex, due to the myriad of factors that interact with each other in a non-linear and non-sequential manner. The issues and their accompanying complexity are further compounded when describing and understanding reading comprehension in a second or foreign language (L2). In facilitating L2 reading comprehension, the use of sound, pictures, and animated pictures or video in addition to text have played an important role in vocabulary acquisition and in overall text comprehension, and are unquestioned components of instructional materials for language learning (Chun & Plass, 1996a, 1996b; Cohen, 1987; Hanley, Herron, & Cole, 1995; Leow, 1995; Oller, 1996; Omaggio, 1979; Secules, Herron, & Tomasello, 1992). The possibility of an instructional use for these different modes of information on a computer raises questions concerning learning from media (Clark, 1994; Clark & Salomon, 1986; Kozma, 1991; Ross, 1994), and concerning the specifics of language learning with multimedia. Of special interest are questions regarding the difference in cognitive processes in learning from different sources, and regarding the effect of individual learner differences on learning from media. How does the process of comprehension of text differ from comprehension of pictures, and how does the combination of pictures and text affect comprehension? What effect do students' learning preferences, abilities, and cognitive styles have on the answers to these questions? This paper is concerned with the research regarding text comprehension in multimedia environments. We first summarize research results in L2 reading comprehension and text comprehension with multimedia, and synthesize these findings into a model of L2 reading comprehension with multimedia. We then review the research questions currently asked by scholars working in the field and, based on the model proposed, derive new and more specific research questions. Volker Hegelheimer & Carol Chapelle Methodological Issues in Research on Learner Computer ... Language Learning & Technology 56 L2 READING COMPREHENSION Historical Perspective From a historical perspective, our understanding of reading in a second or foreign language (L2) has changed considerably in the last several decades. In the midto late 1960s, reading was considered a skill for learners to acquire, mainly to reinforce the grammar and vocabulary being taught orally (e.g., under audiolingualism). In the 1970s researchers argued that greater importance should be placed on reading and advocated a psycholinguistic model or theory of reading (Goodman, 1967; Smith, 1971, 1979). During the 1980s the perspectives of Goodman and Smith were extended, and L2 researchers began to examine L1 reading research more closely (e.g., Bernhardt, 1991; Goodman, 1985; Smith, 1982). Currently, there are two overlapping approaches to viewing and describing reading that reflect, in part, the development of reading research. The first might be termed a "reading components" perspective in that reading is subdivided into skills and knowledge areas. This approach goes well beyond viewing reading as a simple collection of skills or knowledge in that it also focuses on the cognitive processes involved in reading in general. The second approach to describing reading is in terms of metaphors: The most prevalent metaphors in the literature are the bottom-up approaches, the top-down approaches, and the interactive approaches. Due to the fact that many of the current views of L2 reading have been shaped by research on L1 readers, this section first summarizes the main characterizations of L1 reading comprehension. The additional factors that come into play when dealing with L2 reading comprehension will be discussed next. Finally, a comparison of L1 and L2 readers in terms of cognitive comprehension processes will lead into the following section on text comprehension with multimedia. Reading Components Approach Briefly, the view of reading from a so-called "reading components" perspective proposes to subdivide reading into six general component skills and knowledge areas (as summarized by Grabe, 1991): (a) automatic recognition skills; (b) vocabulary and structural knowledge; (c) formal discourse structure knowledge; (d) content/world background knowledge; (e) synthesis and evaluation skills/strategies; and (f) metacognitive knowledge and skills monitoring. Some researchers rank these factors in terms of importance for the comprehension process. For example, Laufer & Sim (1985) posit the following factors in order of decreasing importance: knowledge of vocabulary, subject matter, discourse markers, syntactic structure. In essence, they find that vocabulary is most important, syntax least important. It is important to note that the above-mentioned skills are increasingly being described by cognitive and educational psychologists in terms of the processes that learners go through in developing such skills or knowledge. For example, the so-called "automatic recognition skills" that typically involve recognizing letters, characters, and words have recently begun to be recognized as important in L2 reading. Automatic lexical access involves lower-level processing and, in a fluent reader, requires little processing capacity (cf. Koda, 1992). With the shift in emphasis from a pure skills approach to a process approach, researchers turned to the comprehension research that formed a major domain of the field of cognitive psychology. Thus, most of the current views of L2 reading have been shaped by the solid and extensive body of research by cognitive psychologists on L1 reading comprehension (Grabe, 1991). There are a number of excellent reviews of the literature on L1 reading comprehension (e.g., Bernhardt, 1991; Grabe, 1991; Horiba, 1996; Swaffar, Arens, & Byrnes, 1991). As an overall description of the process of reading comprehension, Swaffar et al. state that "fluent readers synthesize textual subsystems (e.g., content, context, intent, language) into a larger metasystem of meaning" (p. 21) and that "in short, readers comprehend a text Volker Hegelheimer & Carol Chapelle Methodological Issues in Research on Learner Computer ... Language Learning & Technology 57 when they construct a mental representation for incoming pieces of verbal information" (p. 22), see section "Text Comprehension with Multimedia." De Beaugrande (1982), among others, concludes that "what is in fact comprehended is not sentences, but conceptual content" (p. 180). One cognitive view of reading suggests that it is a process or set of ordered stages, consisting of a beginning state, an end state, and intervening transformations (e.g., Just & Carpenter, 1987). Some researchers believe that reading comprehension involves multiple cognitive processes that are hierarchically related to one another (e.g., Horiba, 1996). These processes (from recognizing letters, characters, and words, to analyzing the syntactic and semantic structure of clauses and sentences, to generating inferences) must take place in an orchestrated manner (Rayner & Pollatsek, 1989). However, neither the "hierarchy of processes" view nor the proposition that they take place in an "orchestrated manner" insists that the process be linear. Just and Carpenter (1980) define the process as a clearly nonlinear one. Readers are active in the selection of portions of the text for processing, and former portions of the text may inform latter ones, just as latter portions of the text may inform former ones through feedback. Similarly, other researchers believe that reading comprehension results from interactive variables that operate simultaneously rather than sequentially (Samuels & Kamil, 1984; Swaffar et al., 1991). A second major approach to understanding the reading process is to describe it using metaphors. The three most common metaphors for describing the reading process are bottom-up processing, top-down processing, and interactive processing (cf. Samuels and Kamil, 1984; Silberstein, 1987; Swaffar et al., 1991). These terms and metaphors are described below. Metaphoric Approaches Bottom-up processing models place primary emphasis on textual decoding. They can be seen as datadriven and emphasize the priority of the text as input and, hence, lower-level processes such as letter and word recognition. In contrast, top-down models place primary emphasis on reader interpretation and prior knowledge. They are seen as concept-driven, in the sense that the text is "sampled" and predictions are made on the basis of the reader's prior syntactic and semantic knowledge (Goodman, 1967). Other models are more balanced in suggesting that linguistic knowledge from several sources (orthographic, lexical, syntactic and semantic) interacts in the L1 reading process (Rumelhart, 1977). Readers may also try to compensate for deficiencies at one level (e.g., word recognition) by relying more on a source at a lower or higher level (e.g., contextual knowledge) (Stanovich, 1980). T