Synaesthetic Correspondence between Auditory Clips and Colors: An Empirical Study

This study investigated cross modal matching between some common auditory clips and colors, with two groups of participants: normal control group and the expertise group. Participants were required and encouraged to select a specific color to match a given auditory clip (normal onomatopoeia sounds and music clips) and give appraisal of their fondness to the clip, using a 7-Likert point scale. The results showed that two groups showed similar fondness of the auditory clips. However, the expertise group showed higher variance in the clip-color matching and were more sensitive to the luminance of the colors. The expertise group tended to match the low luminance color to the clips, although both groups did not differ in the average score of hue selection in this mapping.

[1]  C. Spence.  Crossmodal correspondences: A tutorial review , 2011, Attention, perception & psychophysics.

[2]  FURTHER VALIDATION OF SUBJECTIVE SCALES FOR LOUDNESS AND BRIGHTNESS BY MEANS OF CROSS-MODALITY MATCHING. , 1965, The American journal of psychology.

[3]  J. Enns,et al.  Spatial selection and target identification are separable processes in visual search. , 2010, Journal of vision.

[4]  William M. Stern,et al.  Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex , 2007, Nature Neuroscience.

[5]  F. W. Wicker,et al.  Mapping the intersensory regions of perceptual space. , 1968, The American journal of psychology.

[6]  I. Bernstein,et al.  Hue-Tone Sensory Interaction: A Negative Result , 1971, Perceptual and motor skills.

[7]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[8]  S. S. Stevens,et al.  Cross-modality matching of brightness to loudness by 5-year-olds , 1969 .

[9]  L. Marks,et al.  Cross-modality matching of brightness and loudness. , 1965, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Peter B. L. Meijer,et al.  An experimental system for auditory image representations , 1992, IEEE Transactions on Biomedical Engineering.

[11]  L E Marks,et al.  On cross-modal similarity: auditory-visual interactions in speeded discrimination. , 1987, Journal of experimental psychology. Human perception and performance.

[12]  John T. Cowles,et al.  An experimental study of the pairing of certain auditory and visual stimuli. , 1935 .

[13]  C. E. Osgood,et al.  Studies in Synesthetic Thinking: II. The Rôle Of Form In Visual Responses To Music , 1942 .

[14]  Robert D. Melara,et al.  Interaction between synesthetically corresponding dimensions. , 1987 .

[15]  A. Meltzoff,et al.  Intermodal matching by human neonates , 1979, Nature.

[16]  L E Marks,et al.  On associations of light and sound: the mediation of brightness, pitch, and loudness. , 1974, The American journal of psychology.

[17]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[18]  Anne Treisman,et al.  Natural cross-modal mappings between visual and auditory features. , 2011, Journal of vision.