Sound-Colour Synaesthesia: to What Extent Does it Use Cross-Modal Mechanisms Common to us All?

This study examines a group of synaesthetes who report colour sensations in response to music and other sounds. Experiment 1 shows that synaesthetes choose more precise colours and are more internally consistent in their choice of colours given a set of sounds of varying pitch, timbre and composition (single notes or dyads) relative to a group of controls. In spite of this difference, both controls and synaesthetes appear to use the same heuristics for matching between auditory and visual domains (e.g., pitch to lightness). We take this as evidence that synaesthesia may recruit some of the mechanisms used in normal cross-modal perception. Experiment 2 establishes that synaesthetic colours are automatically elicited insofar as they give rise to cross-modal Stroop interference. Experiment 3 uses a variant of the cross-modal Posner paradigm in which detection of a lateralised target is enhanced when combined with a non-informative but synaesthetically congruent sound-colour pairing. This suggests that synaesthesia uses the same (or an analogous) mechanism of exogenous cross-modal orienting as normal perception. Overall, the results support the conclusion that this form of synaesthesia recruits some of the same mechanisms used in normal cross-modal perception rather than using direct, privileged pathways between unimodal auditory and unimodal visual areas that are absent in most other adults.

[1]  L E Marks,et al.  On cross-modal similarity: auditory-visual interactions in speeded discrimination. , 1987, Journal of experimental psychology. Human perception and performance.

[2]  R. Plomp Pitch of complex tones. , 1966, The Journal of the Acoustical Society of America.

[3]  Richard E. Cytowic Synesthesia: A union of the senses, 2nd ed. , 2002 .

[4]  L. Marks On colored-hearing synesthesia: cross-modal translations of sensory dimensions. , 1975, Psychological bulletin.

[5]  Rudolf E. Radocy,et al.  A Case Study of a Chromesthetic , 1981 .

[6]  C. Frith,et al.  The neural correlates of conscious experience: an experimental framework , 1999, Trends in Cognitive Sciences.

[7]  [Color hearing]. , 1985, Ceskoslovenska otolaryngologie.

[8]  James Ward,et al.  The British Journal of Psychology , 1904 .

[9]  C. Frith,et al.  Modulation of human visual cortex by crossmodal spatial attention. , 2000, Science.

[10]  I. Knopf Chemical Psychoses: LSD and Related Drugs. , 1968 .

[11]  C. Spence,et al.  Attracting attention to the illusory location of a sound: reflexive crossmodal orienting and ventriloquism , 2000, Neuroreport.

[12]  Carl E. Seashore,et al.  Color Music , 1938 .

[13]  J. Driver Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading , 1996, Nature.

[14]  J. Carroll,et al.  Two Cases of Synesthesia for Color and Musical Tonality Associated with Absolute Pitch Ability , 1961 .

[15]  Paola Campadelli,et al.  System for the automatic selection of conspicuous color sets for qualitative data display and visual interface design , 2000, IS&T/SPIE Electronic Imaging.

[16]  S. Baron-Cohen,et al.  Hearing Words and Seeing Colours: An Experimental Investigation of a Case of Synaesthesia , 1987, Perception.

[17]  L E Marks,et al.  On associations of light and sound: the mediation of brightness, pitch, and loudness. , 1974, The American journal of psychology.

[18]  J. W. Bernard,et al.  Messiaen's Synaesthesia: The Correspondence between Color and Sound Structure in His Music , 1986 .

[19]  Jamie Ward,et al.  Lexical-gustatory synaesthesia: linguistic and conceptual factors , 2003, Cognition.

[20]  D. Levitin Absolute memory for musical pitch: Evidence from the production of learned melodies , 1994, Perception & psychophysics.

[21]  Christopher T. Lovelace,et al.  Mechanisms of synesthesia: cognitive and physiological constraints , 2001, Trends in Cognitive Sciences.

[22]  Bruno Nettl Music in primitive culture , 1956 .

[23]  M. Zigler Tone Shapes: A Novel Type of Synaesthesia , 1930 .

[24]  Vilayanur S. Ramachandran,et al.  Acquired synesthesia in retinitis pigmentosa , 1999 .

[25]  N. Freimer,et al.  Familial aggregation of absolute pitch. , 2000, American journal of human genetics.

[26]  D. Deutsch,et al.  The Psychology of Music , 1983 .

[27]  David J. Lewkowicz,et al.  Cross-modal equivalence in early infancy: Auditory–visual intensity matching. , 1980 .

[28]  Charles E. Osgood,et al.  The cross-cultural generality of visual-verbal synesthetic tendencies. , 2007 .

[29]  H. Kennedy,et al.  Transient projections from the fronto-parietal and temporal cortex to areas 17, 18 and 19 in the kitten , 2004, Experimental Brain Research.

[30]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[31]  Synesthesia and Sound Spectra , 1952 .

[32]  J. Driver,et al.  Audiovisual links in exogenous covert spatial orienting , 1997, Perception & psychophysics.

[33]  L. Hollister,et al.  Effect of mescaline, lysergic acid diethylamide and psilocybin on color perception , 1963, Psychopharmacologia.

[34]  Kenneth J. Peacock,et al.  Synesthetic Perception: Alexander Scriabin's Color Hearing , 1985 .

[35]  J. E. Harrison,et al.  Synaesthesia: The Strangest Thing , 2001 .

[36]  M. Posner,et al.  Components of visual orienting , 1984 .

[37]  C. Maitland,et al.  Auditory-visual synesthesia. Report of a case with intact visual pathways. , 1984, Archives of neurology.

[38]  D. Maurer Neonatal Synesthesia: Implications for the Processing of Speech And Faces , 1993 .

[39]  Charles S. Myers,et al.  A CASE OF SYNAESTHESIA , 1911 .

[40]  T L Hubbard,et al.  Synesthesia-like mappings of lightness, pitch, and melodic interval. , 1996, The American journal of psychology.

[41]  L. Marks Bright sneezes and dark coughs, loud sunlight and soft moonlight. , 1982, Journal of experimental psychology. Human perception and performance.

[42]  R. Shepard Circularity in Judgments of Relative Pitch , 1964 .

[43]  E. Bullmore,et al.  Response amplification in sensory-specific cortices during crossmodal binding. , 1999, Neuroreport.

[44]  Riitta Hari,et al.  Audiovisual Integration of Letters in the Human Brain , 2000, Neuron.

[45]  Herbert Sidney Langfeld Discussion: Note on a Case of Chrom?sthesia. , 1914 .

[46]  R. Melara,et al.  Dimensional interaction between color and pitch. , 1989, Journal of experimental psychology. Human perception and performance.

[47]  Alan Cowey,et al.  The 30th Sir Frederick Bartlett Lecture: Fact, Artefact, and Myth about Blindsight , 2004, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[48]  Lawrence E. Marks,et al.  On colored-hearing synesthesia: cross-modal translations of sensory dimensions. , 1975 .

[49]  Simon Baron-Cohen,et al.  Synaesthesia : classic and contemporary readings , 1997 .

[50]  S. Iversen,et al.  Detection of Audio-Visual Integration Sites in Humans by Application of Electrophysiological Criteria to the BOLD Effect , 2001, NeuroImage.

[51]  C. Spence,et al.  Multisensory perception: Beyond modularity and convergence , 2000, Current Biology.

[52]  G. Schlaug,et al.  Absolute Pitch and Planum Temporale , 2001, NeuroImage.

[53]  P F OSTWALD,et al.  Color Hearing: A Missing Link Between Normal Perception and the Hallucination , 1964 .

[54]  Simon Baron-Cohen,et al.  Is There a Normal Phase of Synaesthesia in Development , 1996 .

[55]  C. Mills,et al.  DIGIT SYNAESTHESIA: A CASE STUDY USING A STROOP-TYPE TEST , 1999 .

[56]  T. D. Cutsforth The Role of Emotion in a Synaesthetic Subject , 1925 .

[57]  G. Calvert Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[58]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[59]  George L. Rogers Four Cases of Pitch-Specific Chromesthesia in Trained Musicians with Absolute Pitch , 1987 .

[60]  Simon Baron-Cohen,et al.  Synaesthesia: Prevalence and Familiality , 1996, Perception.

[61]  G. Whipple Two Cases of Synaesthesia , 1900 .

[62]  S. Baron-Cohen,et al.  Coloured Speech Perception: Is Synaesthesia what Happens when Modularity Breaks Down? , 1993, Perception.

[63]  E. L. Mudge The Common Synaesthesia of Music. , 1920 .

[64]  L E Marks,et al.  Perceiving similarity and comprehending metaphor. , 1988, Monographs of the Society for Research in Child Development.

[65]  J. H. Flowers,et al.  An Investigation of the Cognitive and Perceptual Dynamics of a Colour–Digit Synaesthete , 1999, Perception.

[66]  Peter G. Grossenbacher,et al.  Perception and sensory information in synaesthetic experience. , 1997 .