Modality-specific spectral dynamics in response to visual and tactile sequential shape information processing tasks: An MEG study using multivariate pattern classification analysis

Brain regions that respond to more than one sensory modality are characterized as multisensory regions. Studies on the processing of shape or object information have revealed recruitment of the lateral occipital cortex, posterior parietal cortex, and other regions regardless of input sensory modalities. However, it remains unknown whether such regions show similar (modality-invariant) or different (modality-specific) neural oscillatory dynamics, as recorded using magnetoencephalography (MEG), in response to identical shape information processing tasks delivered to different sensory modalities. Modality-invariant or modality-specific neural oscillatory dynamics indirectly suggest modality-independent or modality-dependent participation of particular brain regions, respectively. Therefore, this study investigated the modality-specificity of neural oscillatory dynamics in the form of spectral power modulation patterns in response to visual and tactile sequential shape-processing tasks that are well-matched in terms of speed and content between the sensory modalities. Task-related changes in spectral power modulation and differences in spectral power modulation between sensory modalities were investigated at source-space (voxel) level, using a multivariate pattern classification (MVPC) approach. Additionally, whole analyses were extended from the voxel level to the independent-component level to take account of signal leakage effects caused by inverse solution. The modality-specific spectral dynamics in multisensory and higher-order brain regions, such as the lateral occipital cortex, posterior parietal cortex, inferior temporal cortex, and other brain regions, showed task-related modulation in response to both sensory modalities. This suggests modality-dependency of such brain regions on the input sensory modality for sequential shape-information processing.

[1]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[2]  S. Kosslyn,et al.  Representations in mental imagery and working memory: Evidence from different types of visual masks , 2012, Memory & cognition.

[3]  Hyukchan Kwon,et al.  A whole-head magnetoencephalography system with compact axial gradiometer structure , 2009 .

[4]  Anders M. Dale,et al.  An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest , 2006, NeuroImage.

[5]  O. Jensen,et al.  Shaping Functional Architecture by Oscillatory Alpha Activity: Gating by Inhibition , 2010, Front. Hum. Neurosci..

[6]  Jean-Lon Chen,et al.  Dynamic changes of ICA‐derived EEG functional connectivity in the resting state , 2013, Human brain mapping.

[7]  V Menon,et al.  Modality effects in verbal working memory: differential prefrontal and parietal responses to auditory and visual stimuli , 2004, NeuroImage.

[8]  Bevil R. Conway,et al.  Parallel, multi-stage processing of colors, faces and shapes in macaque inferior temporal cortex , 2013, Nature Neuroscience.

[9]  N. N. Rüther,et al.  Observed manipulation of novel tools leads to mu rhythm suppression over sensory-motor cortices , 2014, Behavioural Brain Research.

[10]  Toshio Inui,et al.  The role of the posterior parietal cortex in human object recognition: a functional magnetic resonance imaging study , 1999, Neuroscience Letters.

[11]  David L. Sheinberg,et al.  Neural Dynamics in Inferior Temporal Cortex during a Visual Working Memory Task , 2009, The Journal of Neuroscience.

[12]  T. Hendler,et al.  Visuo-haptic object-related activation in the ventral visual pathway , 2001, Nature Neuroscience.

[13]  S. Hanslmayr,et al.  Oscillatory power decreases and long-term memory: the information via desynchronization hypothesis , 2012, Front. Hum. Neurosci..

[14]  W. Klimesch.  Alpha-band oscillations, attention, and controlled access to stored information , 2012, Trends in Cognitive Sciences.

[15]  S. Makeig.  Auditory event-related dynamics of the EEG spectrum and effects of exposure to tones. , 1993, Electroencephalography and clinical neurophysiology.

[16]  T. Hendler,et al.  Convergence of visual and tactile shape processing in the human lateral occipital complex. , 2002, Cerebral cortex.

[17]  E. Macaluso,et al.  Multisensory spatial interactions: a window onto functional integration in the human brain , 2005, Trends in Neurosciences.

[18]  William M. Stern,et al.  Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex , 2007, Nature Neuroscience.

[19]  J. Rauschecker,et al.  Modality-specific frontal and parietal areas for auditory and visual spatial localization in humans , 1999, Nature Neuroscience.

[20]  Steven L. Bressler,et al.  The Role of Neural Context in Large-Scale Neurocognitive Network Operations , 2007 .

[21]  C. Forn,et al.  Differential dorsolateral prefrontal cortex activation during a verbal n-back task according to sensory modality , 2009, Behavioural Brain Research.

[22]  John J. Foxe,et al.  Crossmodal binding through neural coherence: implications for multisensory processing , 2008, Trends in Neurosciences.

[23]  David G. Pearson,et al.  Body image, visual working memory and visual mental imagery , 2015, PeerJ.

[24]  Ursula Bellugi,et al.  Neural Organization of Linguistic Short-term Memory is Sensory Modalitydependent: Evidence from Signed and Spoken Language , 2008, Journal of Cognitive Neuroscience.

[25]  P. Mamassian,et al.  Multisensory processing in review: from physiology to behaviour. , 2010, Seeing and perceiving.

[26]  P. Pietrini,et al.  Spatial imagery relies on a sensory independent, though sensory sensitive, functional organization within the parietal cortex: A fMRI study of angle discrimination in sighted and congenitally blind individuals , 2015, Neuropsychologia.

[27]  I. Fujita.  The inferior temporal cortex: Architecture, computation, and representation , 2002, Journal of neurocytology.

[28]  N. Crone,et al.  Estimation of short-time cross-correlation between frequency bands of event related EEG , 2006, Journal of Neuroscience Methods.

[29]  Y. Jeong,et al.  Sensory modality-specific spatio-temporal dynamics in response to counting tasks , 2014, Neuroscience Letters.

[30]  G. Calvert,et al.  Multisensory integration: methodological approaches and emerging principles in the human brain , 2004, Journal of Physiology-Paris.

[31]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[32]  Dwight J. Kravitz,et al.  A new neural framework for visuospatial processing , 2011, Nature Reviews Neuroscience.

[33]  S. Makeig,et al.  Mining event-related brain dynamics , 2004, Trends in Cognitive Sciences.

[34]  Amir Amedi,et al.  A Putative Model of Multisensory Object Representation , 2009, Brain Topography.

[35]  Diana K. Sarko,et al.  Convergent approaches toward the study of multisensory perception , 2013, Front. Syst. Neurosci..

[36]  G. Calvert.  Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[37]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[38]  Katherine Roe,et al.  Selective involvement of superior frontal cortex during working memory for shapes. , 2010, Journal of neurophysiology.

[39]  Á. Pascual-Leone,et al.  α-Band Electroencephalographic Activity over Occipital Cortex Indexes Visuospatial Attention Bias and Predicts Visual Target Detection , 2006, The Journal of Neuroscience.

[40]  Chris I. Baker,et al.  Disentangling visual imagery and perception of real-world objects , 2012, NeuroImage.

[41]  M. Sereno,et al.  Multisensory maps in parietal cortex☆ , 2014, Current Opinion in Neurobiology.

[42]  Randall Stilla,et al.  Selective visuo‐haptic processing of shape and texture , 2008, Human brain mapping.

[43]  I. Toni,et al.  Shared Representations for Working Memory and Mental Imagery in Early Visual Cortex , 2013, Current Biology.

[44]  Sayan Mukherjee,et al.  Permutation Tests for Classification , 2005, COLT.

[45]  Matthew J. Brookes,et al.  Measuring functional connectivity using MEG: Methodology and comparison with fcMRI , 2011, NeuroImage.

[46]  Richard M. Leahy,et al.  Brainstorm: A User-Friendly Application for MEG/EEG Analysis , 2011, Comput. Intell. Neurosci..

[47]  R. Ilmoniemi,et al.  Magnetoencephalography-theory, instrumentation, and applications to noninvasive studies of the working human brain , 1993 .

[48]  W. Klimesch,et al.  EEG alpha oscillations: The inhibition–timing hypothesis , 2007, Brain Research Reviews.

[49]  Christopher L Asplund,et al.  Amodal Processing in Human Prefrontal Cortex , 2013, The Journal of Neuroscience.

[50]  T. James,et al.  The neural basis of haptic object processing. , 2007, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[51]  Annette M. Molinaro,et al.  Prediction error estimation: a comparison of resampling methods , 2005, Bioinform..

[52]  Kaspar Meyer,et al.  Cross-Modal Multivariate Pattern Analysis , 2011, Journal of visualized experiments : JoVE.

[53]  C. Gross,et al.  Representation of visual stimuli in inferior temporal cortex. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[54]  Scott Makeig,et al.  Information-based modeling of event-related brain dynamics. , 2006, Progress in brain research.

[55]  Fernando Henrique Lopes da Silva,et al.  Interactions between different EEG frequency bands and their effect on alpha–fMRI correlations , 2009, NeuroImage.

[56]  Carolyn Jeane Perry,et al.  Feature integration and object representations along the dorsal stream visual hierarchy , 2014, Front. Comput. Neurosci..

[57]  J. Palva,et al.  New vistas for α-frequency band oscillations , 2007, Trends in Neurosciences.

[58]  Michael S. Beauchamp,et al.  Statistical criteria in fMRI studies of multisensory integration , 2005, Neuroinformatics.

[59]  D. F. Marks,et al.  Topographical distribution of EEG activity accompanying visual and motor imagery in vivid and non-vivid imagers. , 1995, British journal of psychology.

[60]  Andrea Brovelli,et al.  Multivoxel Pattern Analysis for fMRI Data: A Review , 2012, Comput. Math. Methods Medicine.

[61]  Erkki Oja,et al.  Independent component approach to the analysis of EEG and MEG recordings , 2000, IEEE Transactions on Biomedical Engineering.

[62]  J. S. Guntupalli,et al.  Decoding neural representational spaces using multivariate pattern analysis. , 2014, Annual review of neuroscience.

[63]  Olaf Hauk,et al.  Comparison of noise-normalized minimum norm estimates for MEG analysis using multiple resolution metrics , 2011, NeuroImage.

[64]  A. McIntosh,et al.  Modulation of Ventral Prefrontal Cortex Functional Connections Reflects the Interplay of Cognitive Processes and Stimulus Characteristics , 2008, Cerebral cortex.

[65]  M. Corbetta,et al.  Large-scale cortical correlation structure of spontaneous oscillatory activity , 2012, Nature Neuroscience.

[66]  Robert Oostenveld,et al.  Modality-independent decoding of semantic information from the human brain. , 2014, Cerebral cortex.

[67]  Kingson Man,et al.  Sight and Sound Converge to Form Modality-Invariant Representations in Temporoparietal Cortex , 2012, The Journal of Neuroscience.

[68]  R. Oostenveld,et al.  Independent EEG Sources Are Dipolar , 2012, PloS one.

[69]  Sydney S. Cash,et al.  Decoding word and category-specific spatiotemporal representations from MEG and EEG , 2011, NeuroImage.

[70]  Jason B. Mattingley,et al.  Modality-Specific Control of Strategic Spatial Attention in Parietal Cortex , 2004, Neuron.

[71]  Jasna Martinovic,et al.  Time Course of Information Processing in Visual and Haptic Object Classification , 2012, Front. Hum. Neurosci..

[72]  W. Penny,et al.  Decoding oscillatory representations and mechanisms in memory , 2013, Neuropsychologia.

[73]  Arnaud Delorme,et al.  Single-Trial Normalization for Event-Related Spectral Decomposition Reduces Sensitivity to Noisy Trials , 2011, Front. Psychology.