The role of cross-modal associations in statistical learning

Our environment is richly structured, with objects producing correlated information within and across sensory modalities. A prominent challenge faced by our perceptual system is to learn such regularities. Here, we examined statistical learning and addressed learners’ ability to track transitional probabilities between elements in the auditory and visual modalities. Specifically, we investigated whether cross-modal information affects statistical learning within a single modality. Participants were familiarized with a statistically structured modality (e.g., either audition or vision) accompanied by different types of cues in a second modality (e.g., vision or audition). The results revealed that statistical learning within either modality is affected by cross-modal information, with learning being enhanced or reduced according to the type of cue provided in the second modality.

[1]  H. Pashler.  The Psychology of Attention , 1997 .

[2]  R. Lickliter,et al.  Intersensory redundancy guides attentional selectivity and perceptual learning in infancy. , 2000, Developmental psychology.

[3]  Aaron R. Seitz,et al.  Influences of Multisensory Experience on Subsequent Unisensory Processing , 2011, Front. Psychology.

[4]  E. Newport,et al.  Learning at a distance I. Statistical learning of non-adjacent dependencies , 2004, Cognitive Psychology.

[5]  Tamra J. Bireta,et al.  Irrelevant speech effects and statistical learning. , 2009, Quarterly journal of experimental psychology.

[6]  Joanne Arciuli,et al.  Statistical Learning Is Related to Reading Ability in Children and Adults , 2012, Cogn. Sci..

[7]  Noam Siegelman,et al.  What Predicts Successful Literacy Acquisition in a Second Language? , 2013, Psychological science.

[8]  Aaron R. Seitz,et al.  Benefits of Stimulus Congruency for Multisensory Facilitation of Visual Learning , 2008, PloS one.

[9]  Morten H. Christiansen,et al.  PSYCHOLOGICAL SCIENCE Research Article Statistical Learning Within and Between Modalities Pitting Abstract Against Stimulus-Specific Representations , 2022 .

[10]  S. Lynn,et al.  Varieties of Anomalous Experience: Examining the Scientific Evidence , 2000 .

[11]  Erik D. Thiessen.  Effects of Visual Information on Adults' and Infants' Auditory Statistical Learning , 2010, Cogn. Sci..

[12]  James S. Magnuson,et al.  The link between statistical segmentation and word learning in adults , 2008, Cognition.

[13]  Ram Frost,et al.  Towards a universal model of reading , 2012, Behavioral and Brain Sciences.

[14]  Antoni Rodríguez-Fornells,et al.  Speech segmentation is facilitated by visual cues , 2010 .

[15]  Antoni Rodríguez-Fornells,et al.  Bridging the gap between speech segmentation and word-to-world mappings: Evidence from an audiovisual statistical learning task , 2010 .

[16]  Scott Sinnett,et al.  Speech segmentation by statistical learning depends on attention , 2005, Cognition.

[17]  L. Marks,et al.  Cross-modality matching of brightness and loudness. , 1965, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Robert M. Gonyea,et al.  Learning at a Distance : , 2009 .

[19]  Marvin M. Chun,et al.  Neural Evidence of Statistical Learning: Efficient Detection of Visual Regularities Without Awareness , 2009, Journal of Cognitive Neuroscience.

[20]  Richard I. Ivry,et al.  Attention and structure in sequence learning. , 1990 .

[21]  M. Goldsmith,et al.  Statistical Learning by 8-Month-Old Infants , 1996 .

[22]  M. Murray,et al.  The role of multisensory memories in unisensory object discrimination. , 2005, Brain research. Cognitive brain research.

[23]  E. Gibson,et al.  Principles of Perceptual Learning and Development , 1973 .

[24]  R. Aslin,et al.  Statistical learning of higher-order temporal structure from visual shape sequences. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[25]  A. Reber.  Implicit learning of artificial grammars , 1967 .

[26]  Scott P. Johnson,et al.  Visual statistical learning in infancy: evidence for a domain general learning mechanism , 2002, Cognition.

[27]  C. Spence.  Crossmodal correspondences: A tutorial review , 2011, Attention, perception & psychophysics.

[28]  B. Scholl,et al.  Multidimensional Visual Statistical Learning Visual Statistical Learning , 2005 .

[29]  B. Scholl,et al.  The Automaticity of Visual Statistical Learning Statistical Learning , 2005 .

[30]  Michael C. Frank,et al.  Information from multiple modalities helps 5-month-olds learn abstract rules. , 2009, Developmental science.

[31]  Daphne Maurer,et al.  Do small white balls squeak? Pitch-object correspondences in young children , 2004, Cognitive, affective & behavioral neuroscience.

[32]  M. Ernst.  Learning to integrate arbitrary signals from vision and touch. , 2007, Journal of vision.

[33]  Aaron R. Seitz,et al.  Simultaneous and Independent Acquisition of Multisensory and Unisensory Associations , 2007, Perception.

[34]  Richard N Aslin,et al.  Statistical learning of new visual feature combinations by infants , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[35]  P. Reuter-Lorenz,et al.  Visual-auditory interactions in sensorimotor processing: saccades versus manual responses. , 1994, Journal of experimental psychology. Human perception and performance.

[36]  S. S. Stevens,et al.  Cross-modality matching of brightness to loudness by 5-year-olds , 1969 .

[37]  E. Newport,et al.  Science Current Directions in Psychological Statistical Learning : from Acquiring Specific Items to Forming General Rules on Behalf Of: Association for Psychological Science , 2022 .

[38]  Evan Kidd,et al.  Implicit statistical learning is directly associated with the acquisition of syntax. , 2012, Developmental psychology.

[39]  J. Fiser.  Perceptual learning and representational learning in humans and animals , 2009, Learning & behavior.