On the process of comparing sentences against pictures

Abstract The present study outlines a theory of how people compare sentences against pictures. This theory was tested in four experiments in which S s were timed as they judged whether a sentence (e.g., Star isn't above plus ) was true or false of a picture (e.g., + ∗ ). The latencies in these tasks were consistent with the thesis that: (1) sentences are represented in terms of elementary propositions; (2) pictures are encoded in the same interpretive format; (3) these two codes are compared in an algorithmic series of mental operations, each of which contributes additively to the response latency; and (4) sentence encoding, picture encoding, comparing, and responding are four serially ordered stages, and their component latencies are additive. From these results, it was also possible to rule out certain explanations based on visual imagery, conversion (e.g., converting isn't above into is below ), reading time, normative word frequencies, and other factors. Finally, it was shown that this theory is consistent with previous studies on sentence comprehension, sentence verification, concept verification, and other related phenomena.

[1]  Roderick A. Jacobs,et al.  Readings in English transformational grammar , 1970 .

[2]  J. Fodor,et al.  Some syntactic determinants of sentential complexity , 1967 .

[3]  Rivka R. Eifermann,et al.  Negation: A linguistic variable , 1961 .

[4]  J. Fodor,et al.  Some syntactic determinants of sentential complexity, II : Verb structure , 1968 .

[5]  D. Hakes,et al.  Sentence comprehension and relative pronouns , 1970 .

[6]  R. L. Dominowski.  Anagram solving as a function of letter moves , 1966 .

[7]  D. J. Foss.  Decision processes during sentence comprehension: Effects of lexical item difficulty and position upon decision times , 1969 .

[8]  W. Watt.  Morphology of the Nevada cattle brands and their blazons , 1966 .

[9]  T. Trabasso,et al.  Storage and verification stages in processing concepts. , 1971 .

[10]  Joseph H. Greenberg,et al.  Current trends in linguistics. , 1959, Science.

[11]  H. H. Clark,et al.  Mental operations in the comparison of sentences and pictures. , 1972 .

[12]  R. Grieve,et al.  What Is so difficult about negation? , 1969 .

[13]  L. Brooks.  Spatial and verbal components of the act of recall. , 1968 .

[14]  R. C. Grimsdale.  Automatic Interpretation and Classification of Images, A. Grasselli (Ed.). Academic Press, New York (1969), 436 pp. $14.00. , 1972 .

[15]  J. Hayes.  Cognition and the development of language , 1970 .

[16]  Sheila J. Jones.  DECODING A DECEPTIVE INSTRUCTION , 1966 .

[17]  W G Chase,et al.  Semantics in the perception of verticality. , 1971, British journal of psychology.

[18]  G. Bower,et al.  Cognition in learning and memory , 1974 .

[19]  R. Leeper.  A Study of a Neglected Portion of the Field of Learning—the Development of Sensory Organization , 1935 .

[20]  D. Slobin.  Grammatical transformations and sentence comprehension in childhood and adulthood , 1966 .

[21]  P. Wason,et al.  Response to affirmative and negative binary statements. , 1961, British journal of psychology.

[22]  Sheila J. Jones,et al.  Instructions, Self-Instructions and Performance , 1968, The Quarterly journal of experimental psychology.

[23]  J M Greene,et al.  Syntactic Form and Semantic Function , 1970, The Quarterly journal of experimental psychology.

[24]  Philip B. Gough,et al.  Grammatical transformations and speed of understanding , 1965 .

[25]  J. Greene,et al.  THE SEMANTIC FUNCTION OF NEGATIVES AND PASSIVES , 1970 .

[26]  S Fillenbaum.  Recall for Answers to "Conducive" Questions , 1968, Language and speech.

[27]  Frieda Goldman-Eisler,et al.  Is N, P, and PN difficulty a valid criterion of transformational operations? , 1970 .

[28]  P H Seymour,et al.  Response latencies in judgements of spatial location. , 1969, British journal of psychology.

[29]  Herbert H. Clark,et al.  Linguistic processes in deductive reasoning. , 1969 .

[30]  Philip B. Gough,et al.  The verification of sentences: The effects of delay of evidence and sentence length , 1966 .

[31]  Donald J. Foss,et al.  Decision processes during sentence comprehension: Effects of surf ace structure on decision times , 1969 .

[32]  Barbara Tversky,et al.  Pictorial and verbal encoding in a short-term memory task , 1969 .

[33]  P. Wason,et al.  NEGATIVES: DENOTATION AND CONNOTATION. , 1963, British journal of psychology.

[34]  Sher ry Folsom-Meek,et al.  Human Performance , 2020, Nature.

[35]  Saul Sternberg,et al.  The discovery of processing stages: Extensions of Donders' method , 1969 .

[36]  Michael I. Posner,et al.  Abstraction and The Process of Recognition , 1970 .

[37]  Sheila J. Jones.  The effect of a negative qualifier in an instruction , 1966 .

[38]  J. Brožek.  Attention and Performance II. , 1971 .

[39]  A. Paivio.  Mental imagery in associative learning and memory , 1969 .

[40]  E. Smith,et al.  Choice reaction time: an analysis of the major theoretical positions. , 1968, Psychological bulletin.

[41]  Marcel Adam Just,et al.  Comprehension of negation with quantification , 1971 .

[42]  Talmy Givón,et al.  Notes on the Semantic Structure of English Adjectives , 1970 .

[43]  H. H. Clark.  SPACE, TIME, SEMANTICS, AND THE CHILD , 1973 .

[44]  Herbert H. Clark,et al.  THE USE OF SYNTAX IN UNDERSTANDING SENTENCES , 1968 .