Modeling semantic memory: Effects of presenting semantic information in different modalities

Abstract How is semantic information from different modalities integrated and stored? If related ideas are encountered in French and English, or in pictures and sentences, is the result a single representation in memory, or two modality-dependent ones? Subjects were presented with items in different modalities, then were asked whether or not subsequently presented items were identical with the former ones. Subjects frequently accepted translations and items semantically consistent with those presented earlier as identical, although not as often as they accepted items actually seen previously. The same pattern of results was found when the items were French and English sentences, and when they were pictures and sentences. The results can be explained by the hypothesis that subjects integrate information across modalities into a single underlying semantic representation. A computer model, embodying this hypothesis, made predictions in close agreement with the data.

[1]  Walter Kintsch,et al.  Storage of Complex Information in Memory: Some Implications of the Speed with which Inferences Can be Made. , 1972 .

[2]  John D. Bransford,et al.  The abstraction of linguistic ideas , 1971 .

[3]  John D. Bransford,et al.  Considerations of some problems of comprehension. , 1973 .

[4]  Murray Singer,et al.  The Role of Grammatical Relations in the Abstraction of Linguistic Ideas. , 1973 .

[5]  Herbert H. Clark,et al.  ON THE MEETING OF SEMANTICS AND PERCEPTION , 1973 .

[6]  David R. Olson,et al.  On the comprehension of active and passive sentences , 1972 .

[7]  S Katz,et al.  The Bransford-Franks linear effect: Integration or artifact? , 1974, Memory & cognition.

[8]  R. Shepard,et al.  CHRONOMETRIC STUDIES OF THE ROTATION OF MENTAL IMAGES , 1973 .

[9]  J. Bransford,et al.  A brief note on linguistic integration , 1974 .

[10]  Roger C. Schank,et al.  Conceptual dependency: A theory of natural language understanding , 1972 .

[11]  H. H. Clark,et al.  Mental operations in the comparison of sentences and pictures. , 1972 .

[12]  Paul Gruenewald,et al.  The abstraction of linguistic ideas in “meaningless” sentences , 1974, Memory & cognition.

[13]  G. Bower,et al.  Storage and later recognition of exemplars of concepts , 1973 .

[14]  P Wright,et al.  Transformations and the Understanding of Sentences , 1969, Language and speech.

[15]  Stuart Katz.  Role of instructions in abstraction of linguistic ideas. , 1973 .

[16]  Charles J. Fillmore,et al.  THE CASE FOR CASE. , 1967 .

[17]  Wallace E. Lambert,et al.  Bilingual processing of mixed-language associative networks , 1969 .