Perceptual and Linguistic Interactions in Speeded Classification: Tests of the Semantic Coding Hypothesis

We tested the semantic coding hypothesis, which states that cross-modal interactions observed in speeded classification tasks arise after perceptual information is recoded into an abstract format common to perceptual and linguistic systems. Using a speeded classification task, we first confirmed the presence of congruence interactions between auditory pitch and visual lightness and observed Garner-type interference with nonlinguistic (perceptual) stimuli (low-frequency and high-frequency tones, black and white squares). Subsequently, we found that modifying the visual stimuli by (a) making them lexical (related words) or (b) reducing their compactness or figural ‘goodness’ altered congruence effects and Garner interference. The results are consistent with the semantic coding hypothesis, but only in part, and suggest the need for additional assumptions regarding the role of perceptual organization in cross-modal dimensional interactions.

[1]  L. Marks.  The Unity of the Senses , 1978 .

[2]  D. Vickers,et al.  Evidence for an accumulator model of psychophysical discrimination. , 1970, Ergonomics.

[3]  W. Petrusic,et al.  Semantic congruity effects and theories of the comparison process. , 1992, Journal of experimental psychology. Human perception and performance.

[4]  R. Melara.  Dimensional interaction between color and pitch , 1989 .

[5]  Lawrence E. Marks,et al.  Visual-auditory interaction in speeded classification: Role of stimulus difference , 1995, Perception & psychophysics.

[6]  L E Marks,et al.  Dimensional interactions in language processing: investigating directions and levels of crosstalk. , 1990, Journal of experimental psychology. Learning, memory, and cognition.

[7]  L. Marks.  Bright sneezes and dark coughs, loud sunlight and soft moonlight. , 1982, Journal of experimental psychology. Human perception and performance.

[8]  Lawrence E. Marks,et al.  On Perceptual Metaphors , 1996 .

[9]  G. von Békésy,et al.  Similarities between hearing and skin sensations. , 1959, Psychological review.

[10]  K. Kirsner,et al.  Language processing in partial English-Japanese bilinguals: evidence from Stroop-test results. , 1983, The International journal of neuroscience.

[11]  James R. Pomerantz,et al.  CHAPTER 1 – Visual Form Perception: An Overview* , 1986 .

[12]  P. Barber,et al.  An auditory Stroop effect with judgments of speaker gender , 1981, Perception & psychophysics.

[13]  R. Melara,et al.  Dimensional interaction between color and pitch. , 1989, Journal of experimental psychology. Human perception and performance.

[14]  L E Marks,et al.  Processing linguistic and perceptual dimensions of speech: interactions in speeded classification. , 1999, Journal of experimental psychology. Human perception and performance.

[15]  L E Marks,et al.  On cross-modal similarity: auditory-visual interactions in speeded discrimination. , 1987, Journal of experimental psychology. Human perception and performance.

[16]  D. LaBerge.  A recruitment theory of simple behavior , 1962 .

[17]  J W Grau,et al.  The distinction between integral and separable dimensions: evidence for the integrality of pitch and loudness. , 1988, Journal of experimental psychology. General.

[18]  Shor Re.  An auditory analog of the Stroop Test. , 1975 .

[19]  I H Bernstein,et al.  Effects of some variations in auditory input upon visual choice reaction time. , 1971, Journal of experimental psychology.

[20]  G. Logan.  Attention and automaticity in Stroop and priming tasks: Theory and data , 1980, Cognitive Psychology.

[21]  R. Shor.  An auditory analog of the Stroop Test. , 1975, The Journal of general psychology.

[22]  Herbert H. Clark,et al.  Position, direction, and their perceptual integrality , 1976 .

[23]  S Smith,et al.  Stroop Interference Based on the Synaesthetic Qualities of Auditory Pitch , 1984, Perception.

[24]  D M Green,et al.  A neural timing theory for response times and the psychophysics of intensity. , 1972, Psychological review.

[25]  L. Marks,et al.  Cross-modality matching of brightness and loudness. , 1965, Proceedings of the National Academy of Sciences of the United States of America.

[26]  Georg v. Békésy,et al.  Similarities between hearing and skin sensations. , 1959 .

[27]  L. Marks,et al.  Processes underlying dimensional interactions: Correspondences between linguistic and nonlinguistic dimensions , 1990, Memory & cognition.

[28]  James R. Pomerantz,et al.  Stimules configuration in selective attention tasks , 1973 .

[29]  David J. Lewkowicz,et al.  Cross-modal equivalence in early infancy: Auditory–visual intensity matching. , 1980 .

[30]  Ellen Winner,et al.  "Metaphorical" Mapping in Human Infants , 1981 .

[31]  H. Egeth,et al.  Toward a translational model of Stroop interference , 1985, Memory & cognition.

[32]  Robert D. Melara,et al.  Interaction between synesthetically corresponding dimensions. , 1987 .

[33]  L E Marks,et al.  Interaction among auditory dimensions: Timbre, pitch, and loudness , 1990, Perception & psychophysics.