Accurate decoding of sub-TR timing differences in stimulations of sub-voxel regions from multi-voxel response patterns

We investigated the decoding of ocular dominance stimulations with millisecond-order timing difference from the blood oxygen level dependent (BOLD) signal in human functional magnetic resonance imaging (fMRI). In our experiment, ocular dominance columns were activated by monocular visual stimulation with 500- or 100- ms onset differences. We observed that the event-related hemodynamic response (HDR) in the human visual cortex was sensitive to the subtle onset difference. The HDR shapes were related to the stimulus timings in various manners: the timing difference was represented in either the amplitude of positive peak, amplitude of negative peak, delay of peak time, or response duration of HDR. These complex relationships were different across voxels and subjects. To find an informative feature of HDR for discriminating the subtle timing difference of ocular dominance stimulations, we examined various characteristics of HDR including response amplitude, time to peak, full width at half-maximum response, as inputs for decoding analysis. Using a canonical HDR function for estimating the voxel's response did not yield good decoding scores, suggesting that information may reside in the variability of HDR shapes. Using all the values from the deconvolved HDR also showed low performance, which could be due to an over-fitting problem with the large data dimensionality. When using either positive or negative peak amplitude of the deconvolved HDR, high decoding performance could be achieved for both the 500ms and the 100ms onset differences. The high accuracy even for the 100ms difference, given that the signal was sampled at a TR of 250ms and 2×2×3-mm voxels, implies a possibility of spatiotemporally hyper-resolution decoding. Furthermore, both down-sampling and smoothing did not affect the decoding accuracies very much. These results suggest a complex spatiotemporal relationship between the multi-voxel pattern of the BOLD response and the population activation of neuronal columns. The demonstrated possibility of decoding stimulations for columnar-level organization with 100-ms onset difference using lower resolution imaging data may broaden the scope of application of the BOLD fMRI.

[1]  Nikolaus Kriegeskorte,et al.  Analyzing for information, not activation, to exploit high-resolution fMRI , 2007, NeuroImage.

[2]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[3]  S. Yamane,et al.  Population dynamics of face-responsive neurons in the inferior temporal cortex. , 2005, Cerebral cortex.

[4]  Justin L. Gardner,et al.  Is cortical vasculature functionally organized? , 2010, NeuroImage.

[5]  Hans P. Op de Beeck,et al.  Against hyperacuity in brain reading: Spatial smoothing does not hurt multivariate fMRI analyses? , 2010, NeuroImage.

[6]  M. Diamond,et al.  Deciphering the Spike Train of a Sensory Neuron: Counts and Temporal Patterns in the Rat Whisker Pathway , 2006, The Journal of Neuroscience.

[7]  Adrian T. Lee,et al.  Discrimination of Large Venous Vessels in Time‐Course Spiral Blood‐Oxygen‐Level‐Dependent Magnetic‐Resonance Functional Neuroimaging , 1995, Magnetic resonance in medicine.

[8]  Karl J. Friston,et al.  Nonlinear event‐related responses in fMRI , 1998, Magnetic resonance in medicine.

[9]  G. Rees,et al.  Predicting the orientation of invisible stimuli from activity in human primary visual cortex , 2005, Nature Neuroscience.

[10]  J. Gore,et al.  Measurements of the Temporal fMRI Response of the Human Auditory Cortex to Trains of Tones , 1998, NeuroImage.

[11]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[12]  F. Mechler,et al.  Temporal coding of contrast in primary visual cortex: when, what, and why. , 2001, Journal of neurophysiology.

[13]  R. Turner,et al.  Detecting Latency Differences in Event-Related BOLD Responses: Application to Words versus Nonwords and Initial versus Repeated Face Presentations , 2002, NeuroImage.

[14]  S. Ogawa,et al.  An approach to probe some neural systems interaction by functional MRI at neural time scale down to milliseconds. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Joshua E. Motelow,et al.  Negative BOLD with large increases in neuronal activity. , 2008, Cerebral cortex.

[16]  Essa Yacoub,et al.  Mechanisms underlying decoding at 7 T: Ocular dominance columns, broad structures, and macroscopic blood vessels in V1 convey information on the stimulated eye , 2010, NeuroImage.

[17]  G. Rees,et al.  Neuroimaging: Decoding mental states from brain activity in humans , 2006, Nature Reviews Neuroscience.

[18]  R. Christopher deCharms,et al.  Primary cortical representation of sounds by the coordination of action-potential timing , 1996, Nature.

[19]  Ravi S. Menon,et al.  Mental chronometry using latency-resolved functional MRI. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[21]  N. Logothetis,et al.  Phase-of-Firing Coding of Natural Visual Stimuli in Primary Visual Cortex , 2008, Current Biology.

[22]  Afonso C. Silva,et al.  Spatiotemporal Evolution of the Functional Magnetic Resonance Imaging Response to Ultrashort Stimuli , 2011, The Journal of Neuroscience.

[23]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[24]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[25]  Peter A. Bandettini,et al.  Does feature selection improve classification accuracy? Impact of sample size and feature selection on classification using anatomical magnetic resonance images , 2012, NeuroImage.

[26]  Brian N. Pasley,et al.  Analysis of oxygen metabolism implies a neural origin for the negative BOLD response in human visual cortex , 2007, NeuroImage.

[27]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[28]  Jascha D. Swisher,et al.  Multiscale Pattern Analysis of Orientation-Selective Activity in the Primary Visual Cortex , 2010, The Journal of Neuroscience.

[29]  Rainer Goebel,et al.  Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns , 2008, NeuroImage.

[30]  R. Menon,et al.  Spatial and temporal resolution of functional magnetic resonance imaging. , 1998, Biochemistry and cell biology = Biochimie et biologie cellulaire.

[31]  Satoru Miyauchi,et al.  Circulatory basis of fMRI signals: relationship between changes in the hemodynamic parameters and BOLD signal intensity , 2004, NeuroImage.

[32]  Yasuhito Sawahata,et al.  Spatial smoothing hurts localization but not information: Pitfalls for brain mappers , 2010, NeuroImage.

[33]  Jeremy Freeman,et al.  Orientation Decoding Depends on Maps, Not Columns , 2011, The Journal of Neuroscience.

[34]  Martin A. Lindquist,et al.  Detection of time-varying signals in event-related fMRI designs , 2008, NeuroImage.

[35]  Kenji Kawano,et al.  Global and fine information coded by single neurons in the temporal visual cortex , 1999, Nature.

[36]  W. Bair Spike timing in the mammalian visual system , 1999, Current Opinion in Neurobiology.

[37]  L. Optican,et al.  Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. III. Information theoretic analysis. , 1987, Journal of neurophysiology.

[38]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[39]  Nikolaus Kriegeskorte,et al.  How does an fMRI voxel sample the neuronal activity pattern: Compact-kernel or complex spatiotemporal filter? , 2010, NeuroImage.

[40]  Jeff H. Duyn,et al.  Temporal dynamics of the BOLD fMRI impulse response , 2005, NeuroImage.

[41]  Wolf Singer,et al.  Time as coding space? , 1999, Current Opinion in Neurobiology.

[42]  Nikolaus Kriegeskorte,et al.  Comparison of multivariate classifiers and response normalizations for pattern-information fMRI , 2010, NeuroImage.

[43]  Essa Yacoub,et al.  Modeling and analysis of mechanisms underlying fMRI-based decoding of information conveyed in cortical columns , 2011, NeuroImage.

[44]  G. Glover Deconvolution of Impulse Response in Event-Related BOLD fMRI1 , 1999, NeuroImage.

[45]  Simon B. Eickhoff,et al.  A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data , 2005, NeuroImage.

[46]  S. Ogawa,et al.  Temporal feature of BOLD responses varies with temporal patterns of movement , 2008, Neuroscience Research.

[47]  Peter König,et al.  Invariant representations of visual patterns in a temporal population code , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[48]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[49]  A. Aertsen,et al.  Dynamics of neuronal interactions in monkey cortex in relation to behavioural events , 1995, Nature.

[50]  Ravi S. Menon,et al.  Spatial and temporal limits in cognitive neuroimaging with fMRI , 1999, Trends in Cognitive Sciences.

[51]  Lawrence C. Sincich,et al.  Complete Pattern of Ocular Dominance Columns in Human Primary Visual Cortex , 2007, The Journal of Neuroscience.

[52]  David Badre,et al.  Temporal Sensitivity of Event-Related fMRI , 2002, NeuroImage.