When audition dominates vision: evidence from cross-modal statistical learning.

Presenting information to multiple sensory modalities sometimes facilitates and sometimes interferes with processing of this information. Research examining interference effects shows that auditory input often interferes with processing of visual input in young children (i.e., auditory dominance effect), whereas visual input often interferes with auditory processing in adults (i.e., visual dominance effect). The current study used a cross-modal statistical learning task to examine modality dominance in adults. Participants ably learned auditory and visual statistics when auditory and visual sequences were presented unimodally and when auditory and visual sequences were correlated during training. However, increasing task demands resulted in an important asymmetry: Increased task demands attenuated visual statistical learning, while having no effect on auditory statistical learning. These findings are consistent with auditory dominance effects reported in young children and have important implications for our understanding of how sensory modalities interact while learning the structure of cross-modal information.

[1]  S. Shimojo,et al.  Sensory modalities are not separate modalities: plasticity and interactions , 2001, Current Opinion in Neurobiology.

[2]  M. Goldsmith,et al.  Statistical Learning by 8-Month-Old Infants , 1996 .

[3]  John J. Foxe,et al.  Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. , 2004, Cerebral cortex.

[4]  E. Newport,et al.  PSYCHOLOGICAL SCIENCE Research Article INCIDENTAL LANGUAGE LEARNING: Ustening (and Learning) out of the Comer of Your Ear , 2022 .

[5]  Vladimir M. Sloutsky,et al.  The Role of Words and Sounds in Infants' Visual Processing: From Overshadowing to Attentional Tuning , 2008, Cogn. Sci..

[6]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[7]  Morten H. Christiansen,et al.  Modality-constrained statistical learning of tactile, visual, and auditory sequences. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[8]  Aaron R. Seitz,et al.  Simultaneous and Independent Acquisition of Multisensory and Unisensory Associations , 2007, Perception.

[9]  C. Spence,et al.  The co-occurrence of multisensory competition and facilitation. , 2008, Acta psychologica.

[10]  D. H. Warren,et al.  Immediate perceptual response to intersensory discrepancy. , 1980, Psychological bulletin.

[11]  J. Pernier,et al.  Dynamics of cortico-subcortical cross-modal operations involved in audio-visual object detection in humans. , 2002, Cerebral cortex.

[12]  V. Sloutsky,et al.  Is a picture worth a thousand words? Preference for auditory modality in young children. , 2003, Child development.

[13]  Scott P. Johnson,et al.  Visual statistical learning in infancy , 2010 .

[14]  Christopher W. Robinson,et al.  Evidence for auditory dominance in a passive oddball task , 2010 .

[15]  R. Gómez,et al.  Infant artificial language learning and language acquisition , 2000, Trends in Cognitive Sciences.

[16]  Richard N Aslin,et al.  Statistical learning of new visual feature combinations by infants , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Christopher W. Robinson,et al.  Effects of multimodal presentation and stimulus familiarity on auditory and visual processing. , 2010, Journal of experimental child psychology.

[18]  Vladimir M. Sloutsky,et al.  The Role of Words in Cognitive Tasks: What, When, and How? , 2012, Front. Psychology.

[19]  Brenda Rapp,et al.  Principles of cross-modal competition: Evidence from deficits of attention , 2003, Psychonomic bulletin & review.

[20]  Elizabeth K. Johnson,et al.  Statistical learning of tone sequences by human infants and adults , 1999, Cognition.

[21]  Scott Sinnett,et al.  Speech segmentation by statistical learning depends on attention , 2005, Cognition.

[22]  Christopher W. Robinson,et al.  Development of cross-modal processing. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[23]  M. Posner,et al.  Visual dominance: an information-processing account of its origins and significance. , 1976, Psychological review.

[24]  Scott Sinnett,et al.  Coexistence of Multiple Modal Dominances , 2011, CogSci.

[25]  Christopher W. Robinson,et al.  Auditory dominance and its change in the course of development. , 2004, Child development.

[26]  Morten H. Christiansen,et al.  PSYCHOLOGICAL SCIENCE Research Article Statistical Learning Within and Between Modalities Pitting Abstract Against Stimulus-Specific Representations , 2022 .

[27]  Scott P. Johnson,et al.  Location, location, location: development of spatiotemporal sequence learning in infancy. , 2007, Child development.

[28]  C. Spence,et al.  Visual dominance and attention: The Colavita effect revisited , 2007, Perception & psychophysics.

[29]  C. Spence,et al.  Repetition blindness and the Colavita effect , 2010, Neuroscience Letters.

[30]  F. Colavita Human sensory dominance , 1974 .

[31]  C. Spence Explaining the Colavita visual dominance effect. , 2009, Progress in brain research.

[32]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[33]  R. Aslin,et al.  Statistical learning of higher-order temporal structure from visual shape sequences. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[34]  Jeff Miller,et al.  Divided attention: Evidence for coactivation with redundant signals , 1982, Cognitive Psychology.

[35]  Scott P. Johnson,et al.  Visual statistical learning in infancy: evidence for a domain general learning mechanism , 2002, Cognition.

[36]  A. Diederich,et al.  The race model inequality: interpreting a geometric measure of the amount of violation. , 2006, Psychological review.

[37]  B. Scholl,et al.  The Automaticity of Visual Statistical Learning Statistical Learning , 2005 .