Pre-attentive modulation of brain responses to tones in coloured-hearing synesthetes

BackgroundColoured-hearing (CH) synesthesia is a perceptual phenomenon in which an acoustic stimulus (the inducer) initiates a concurrent colour perception (the concurrent). Individuals with CH synesthesia "see" colours when hearing tones, words, or music; this specific phenomenon suggesting a close relationship between auditory and visual representations. To date, it is still unknown whether the perception of colours is associated with a modulation of brain functions in the inducing brain area, namely in the auditory-related cortex and associated brain areas. In addition, there is an on-going debate as to whether attention to the inducer is necessarily required for eliciting a visual concurrent, or whether the latter can emerge in a pre-attentive fashion.ResultsBy using the EEG technique in the context of a pre-attentive mismatch negativity (MMN) paradigm, we show that the binding of tones and colours in CH synesthetes is associated with increased MMN amplitudes in response to deviant tones supposed to induce novel concurrent colour perceptions. Most notably, the increased MMN amplitudes we revealed in the CH synesthetes were associated with stronger intracerebral current densities originating from the auditory cortex, parietal cortex, and ventral visual areas.ConclusionsThe automatic binding of tones and colours in CH synesthetes is accompanied by an early pre-attentive process recruiting the auditory cortex, inferior and superior parietal lobules, as well as ventral occipital areas.

[1]  André Brechmann,et al.  Sound level dependence of auditory evoked potentials: simultaneous EEG recording and low-noise fMRI. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[2]  John J. Foxe,et al.  Seeing voices: High-density electrical mapping and source-analysis of the multisensory mismatch negativity evoked during the McGurk illusion , 2007, Neuropsychologia.

[3]  Joost X. Maier,et al.  Multisensory Integration of Dynamic Faces and Voices in Rhesus Monkey Auditory Cortex , 2005 .

[4]  Lutz Jäncke,et al.  Electrical brain imaging reveals spatio-temporal dynamics of timbre perception in humans , 2006, NeuroImage.

[5]  G. Bargary,et al.  Synaesthesia and cortical connectivity , 2008, Trends in Neurosciences.

[6]  K. Hugdahl,et al.  The neural correlate of colour distances revealed with competing synaesthetic and real colours , 2011, Cortex.

[7]  Karsten Specht,et al.  An independent component analysis of fMRI data of grapheme-colour synaesthesia. , 2011, Journal of neuropsychology.

[8]  Kenneth Hugdahl,et al.  Asymmetry of evoked potential latency to speech sounds predicts the ear advantage in dichotic listening. , 2005, Brain research. Cognitive brain research.

[9]  T Wüstenberg,et al.  Short-term functional plasticity in the human auditory cortex: an fMRI study. , 2001, Brain research. Cognitive brain research.

[10]  Jamie Ward,et al.  Seeing Sounds and Hearing Colors: An Event-related Potential Study of Auditory–Visual Synesthesia , 2009, Journal of Cognitive Neuroscience.

[11]  Daniel Smilek,et al.  Synaesthetic Photisms Influence Visual Perception , 2001, Journal of Cognitive Neuroscience.

[12]  Juha Virtanen,et al.  Hemispheric lateralization in preattentive processing of speech sounds , 1998, Neuroscience Letters.

[13]  N. Rothen,et al.  When conditioned responses “fire back”: Bidirectional cross-activation creates learning opportunities in synesthesia , 2007, Neuroscience.

[14]  G. Calvert Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[15]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[16]  Gereon R Fink,et al.  Grapheme-colour synaesthetes show increased grey matter volumes of parietal and fusiform cortex. , 2009, Brain : a journal of neurology.

[17]  Lucas Spierer,et al.  Auditory spatio-temporal brain dynamics and their consequences for multisensory interactions in humans , 2009, Hearing Research.

[18]  R. Goebel,et al.  Integration of Letters and Speech Sounds in the Human Brain , 2004, Neuron.

[19]  A. Friederici,et al.  Noise affects auditory and linguistic processing differently: an MEG study , 2000, Neuroreport.

[20]  S. Posse,et al.  Intensity coding of auditory stimuli: an fMRI study , 1998, Neuropsychologia.

[21]  S A Hillyard,et al.  An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. , 2002, Brain research. Cognitive brain research.

[22]  Mathias S. Oechslin,et al.  Absolute Pitch—Functional Evidence of Speech-Relevant Auditory Acuity , 2009, Cerebral cortex.

[23]  Lutz Jäncke,et al.  The multiple synaesthete E.S. — Neuroanatomical basis of interval-taste and tone-colour synaesthesia , 2008, NeuroImage.

[24]  P. Deltenre,et al.  Mismatch negativity evoked by the McGurk–MacDonald effect: a phonetic representation within short-term memory , 2002, Clinical Neurophysiology.

[25]  E. Halgren,et al.  Intracerebral potentials to rare target and distractor auditory and visual stimuli. III. Frontal cortex. , 1995, Electroencephalography and clinical neurophysiology.

[26]  R. Kakigi,et al.  Serial and parallel processing in the human auditory cortex: a magnetoencephalographic study. , 2006, Cerebral cortex.

[27]  Henning Scheich,et al.  Anomalous Auditory Cortex Activations in Colored Hearing Synaesthetes: An fMRI-Study. , 2011, Seeing and perceiving.

[28]  John J. Foxe,et al.  Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. , 2002, Brain research. Cognitive brain research.

[29]  A. King,et al.  Visual–auditory spatial processing in auditory cortical neurons , 2008, Brain Research.

[30]  T. Sejnowski,et al.  Removing electroencephalographic artifacts by blind source separation. , 2000, Psychophysiology.

[31]  Lutz Jäncke,et al.  Globally Altered Structural Brain Network Topology in Grapheme-Color Synesthesia , 2011, The Journal of Neuroscience.

[32]  Lutz Jäncke,et al.  The Plasticity of the Superior Longitudinal Fasciculus as a Function of Musical Expertise: A Diffusion Tensor Imaging Study , 2009, Front. Hum. Neurosci..

[33]  Lutz Jäncke,et al.  Time course of neural activity correlated with colored-hearing synesthesia. , 2008, Cerebral cortex.

[34]  Hans-Jochen Heinze,et al.  Scanning silence: Mental imagery of complex sounds , 2005, NeuroImage.

[35]  Seana Coulson,et al.  Magnetoencephalography reveals early activation of V4 in grapheme-color synesthesia , 2010, NeuroImage.

[36]  Lutz Jäncke,et al.  ‘Hearing’ syllables by ‘seeing’ visual stimuli , 2004, The European journal of neuroscience.

[37]  Gereon R Fink,et al.  Intrinsic Network Connectivity Reflects Consistency of Synesthetic Experiences , 2012, The Journal of Neuroscience.

[38]  Mikko Sams,et al.  Visual Processing Affects the Neural Basis of Auditory Discrimination , 2008, Journal of Cognitive Neuroscience.

[39]  V. Ramachandran,et al.  Psychophysical investigations into the neural basis of synaesthesia , 2001, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[40]  Dries J. W. Froyen,et al.  Evidence for a specific cross-modal association deficit in dyslexia: an electrophysiological study of letter-speech sound processing. , 2011, Developmental science.

[41]  Lutz Jäncke,et al.  The spatiotemporal characteristics of elementary audiovisual speech and music processing in musically untrained subjects. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[42]  P Ullsperger,et al.  The P300 to novel and target events: a spatio–temporal dipole model analysis , 1995, Neuroreport.

[43]  John J. Foxe,et al.  Audio-visual multisensory integration in superior parietal lobule revealed by human intracranial recordings. , 2006, Journal of neurophysiology.

[44]  Dominic W. Massaro,et al.  Speechreading: illusion or window into pattern recognition , 1999, Trends in Cognitive Sciences.

[45]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[46]  BMC Neuroscience , 2003 .

[47]  Phineas de Thornley Head,et al.  Synaesthesia: Pitch-Colour Isomorphism in RGB-Space? , 2006, Cortex.

[48]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[49]  Tony Ro,et al.  Neural Substrates of Sound–Touch Synesthesia after a Thalamic Lesion , 2008, The Journal of Neuroscience.

[50]  R. Näätänen Mismatch negativity (MMN) as an index of central auditory system plasticity , 2008, International journal of audiology.

[51]  C. Sinke,et al.  Disinhibited feedback as a cause of synesthesia: Evidence from a functional connectivity study on auditory-visual synesthetes , 2012, Neuropsychologia.

[52]  D. Eagleman,et al.  A standardized test battery for the study of synesthesia , 2007, Journal of Neuroscience Methods.

[53]  H. Scheich,et al.  Phonetic Perception and the Temporal Cortex , 2002, NeuroImage.

[54]  Christoph Kayser,et al.  Behavioral/systems/cognitive Functional Imaging Reveals Visual Modulation of Specific Fields in Auditory Cortex , 2022 .

[55]  S Baron-Cohen,et al.  The physiology of coloured hearing. A PET activation study of colour-word synaesthesia. , 1995, Brain : a journal of neurology.

[56]  Romke Rouw,et al.  Neural Basis of Individual Differences in Synesthetic Experiences , 2010, The Journal of Neuroscience.

[57]  L. Jäncke,et al.  A strong parietal hub in the small-world network of coloured-hearing synaesthetes during resting state EEG. , 2011, Journal of neuropsychology.

[58]  G. Triebig,et al.  Multiple choice vocabulary test MWT as a valid and short test to estimate premorbid intelligence , 1995, Acta neurologica Scandinavica.

[59]  John J. Foxe,et al.  Multisensory processing of naturalistic objects in motion: A high-density electrical mapping and source estimation study , 2007, NeuroImage.

[60]  Danko Nikolić,et al.  Swimming-style synesthesia , 2011, Cortex.

[61]  Christopher T. Lovelace,et al.  Mechanisms of synesthesia: cognitive and physiological constraints , 2001, Trends in Cognitive Sciences.

[62]  C. Sinke,et al.  The neural correlates of coloured music: A functional MRI investigation of auditory–visual synaesthesia , 2012, Neuropsychologia.

[63]  Vilayanur S. Ramachandran,et al.  Acquired synesthesia in retinitis pigmentosa , 1999 .

[64]  Phineas de Thornley Head Synaesthesia: Pitch-Colour Isomorphism in RGB-Space? , 2006, Cortex.

[65]  L. Jäncke,et al.  Influence of acoustic masking noise in fMRI of the auditory cortex during phonetic discrimination , 1999, Journal of magnetic resonance imaging : JMRI.

[66]  E. T. Bullmore,et al.  Functional magnetic resonance imaging of synesthesia: activation of V4/V8 by spoken words , 2002, Nature Neuroscience.

[67]  T. Nijboer,et al.  Multiple dimensions in bi‐directional synesthesia , 2009, The European journal of neuroscience.

[68]  C. Schroeder,et al.  Neuronal Oscillations and Multisensory Interaction in Primary Auditory Cortex , 2007, Neuron.

[69]  H. Scheich,et al.  Nonauditory Events of a Behavioral Procedure Activate Auditory Cortex of Highly Trained Monkeys , 2005, The Journal of Neuroscience.

[70]  K. Alho Cerebral Generators of Mismatch Negativity (MMN) and Its Magnetic Counterpart (MMNm) Elicited by Sound Changes , 1995, Ear and hearing.

[71]  Edward M. Hubbard,et al.  Neurophysiology of synesthesia , 2007, Current psychiatry reports.

[72]  Olivier Bertrand,et al.  Electrophysiological (EEG, sEEG, MEG) evidence for multiple audiovisual interactions in the human auditory cortex , 2009, Hearing Research.

[73]  E. Halgren,et al.  Intracerebral potentials to rare target and distractor auditory and visual stimuli. I. Superior temporal plane and parietal lobe. , 1995, Electroencephalography and clinical neurophysiology.

[74]  E. Halgren,et al.  Intracerebral potentials to rare target and distractor auditory and visual stimuli. II. Medial, lateral and posterior temporal lobe. , 1995, Electroencephalography and clinical neurophysiology.

[75]  R. Näätänen,et al.  Automatic auditory intelligence: An expression of the sensory–cognitive core of cognitive processes , 2010, Brain Research Reviews.

[76]  E. Bullmore,et al.  Response amplification in sensory-specific cortices during crossmodal binding. , 1999, Neuroreport.

[77]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[78]  J Pernier,et al.  Neurophysiological mechanisms of auditory selective attention in humans. , 2000, Frontiers in bioscience : a journal and virtual library.

[79]  J. Pernier,et al.  Early auditory-visual interactions in human cortex during nonredundant target identification. , 2002, Brain research. Cognitive brain research.

[80]  Josep Marco-Pallarés,et al.  Combined ICA-LORETA analysis of mismatch negativity , 2005, NeuroImage.

[81]  R. Zatorre,et al.  Voice-selective areas in human auditory cortex , 2000, Nature.

[82]  R. Knight,et al.  Neural Mechanisms of Involuntary Attention to Acoustic Novelty and Change , 1998, Journal of Cognitive Neuroscience.

[83]  Olivier David,et al.  Waves of consciousness: ongoing cortical patterns during binocular rivalry , 2004, NeuroImage.

[84]  P. Eslinger,et al.  Colored hearing synesthesia , 1989, Neurology.

[85]  John J. Foxe,et al.  The case for feedforward multisensory convergence during early cortical processing , 2005, Neuroreport.

[86]  F. Wood,et al.  Synesthesia II. Psychophysical relations in the synesthesia of geometrically shaped taste and colored hearing , 1982, Brain and Cognition.

[87]  John J. Foxe,et al.  The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. , 2002, Brain research. Cognitive brain research.