Computational advances towards linking BOLD and behavior

Traditionally, fMRI studies have focused on analyzing the mean response amplitude within a cortical area. However, the mean response is blind to many important patterns of cortical modulation, which severely limits the formulation and evaluation of linking hypotheses between neural activity, BOLD responses, and behavior. More recently, multivariate pattern classification analysis (MVPA) has been applied to fMRI data to evaluate the information content of spatially distributed activation patterns. This approach has been remarkably successful at detecting the presence of specific information in targeted brain regions, and provides an extremely flexible means of extracting that information without a precise generative model for the underlying neural activity. However, this flexibility comes at a cost: since MVPA relies on pooling information across voxels that are selective for many different stimulus attributes, it is difficult to infer how specific sub-sets of tuned neurons are modulated by an experimental manipulation. In contrast, recently developed encoding models can produce more precise estimates of feature-selective tuning functions, and can support the creation of explicit linking hypotheses between neural activity and behavior. Although these encoding models depend on strong - and often untested - assumptions about the response properties of underlying neural generators, they also provide a unique opportunity to evaluate population-level computational theories of perception and cognition that have previously been difficult to assess using either single-unit recording or conventional neuroimaging techniques.

[1]  N. Logothetis,et al.  Negative functional MRI response correlates with decreases in neuronal activity in monkey visual area V1 , 2006, Nature Neuroscience.

[2]  L. Pessoa,et al.  Decoding near-threshold perception of fear from distributed single-trial brain activation. , 2006, Cerebral cortex.

[3]  Andreas Kleinschmidt,et al.  The Blind, the Lame, and the Poor Signals of Brain Function—a Comment on Sirotin and Das (2009) , 2022 .

[4]  J. Gallant,et al.  Predicting neuronal responses during natural vision , 2005, Network.

[5]  Mitsuo Kawato,et al.  Sparse linear regression for reconstructing muscle activity from human cortical fMRI , 2008, NeuroImage.

[6]  D. Teller Linking propositions , 1984, Vision Research.

[7]  A. Pouget,et al.  Tuning curve sharpening for orientation selectivity: coding efficiency and the impact of correlations , 2004, Nature Neuroscience.

[8]  Nikolaus Kriegeskorte,et al.  Analyzing for information, not activation, to exploit high-resolution fMRI , 2007, NeuroImage.

[9]  Rajesh P. N. Rao,et al.  Bayesian brain : probabilistic approaches to neural coding , 2006 .

[10]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Jeremy Freeman,et al.  Orientation Decoding Depends on Maps, Not Columns , 2011, The Journal of Neuroscience.

[12]  Anthony J. Movshon,et al.  Optimal representation of sensory information by neural populations , 2006, Nature Neuroscience.

[13]  Nikos K Logothetis,et al.  Interpreting the BOLD signal. , 2004, Annual review of physiology.

[14]  Brian A. Wandell,et al.  Population receptive field estimates in human visual cortex , 2008, NeuroImage.

[15]  A. Zador,et al.  Neural representation and the cortical code. , 2000, Annual review of neuroscience.

[16]  R. Andersen,et al.  Cognitive neural prosthetics. , 2010, Annual review of psychology.

[17]  J. Gallant,et al.  Identifying natural images from human brain activity , 2008, Nature.

[18]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[19]  Kendrick N Kay,et al.  I can see what you see , 2009, Nature Neuroscience.

[20]  S. Treue,et al.  Feature-Based Attention Increases the Selectivity of Population Responses in Primate Visual Cortex , 2004, Current Biology.

[21]  T. Sanger,et al.  Probability density estimation for the interpretation of neural population codes. , 1996, Journal of neurophysiology.

[22]  J. Jonides,et al.  Overlapping mechanisms of attention and spatial working memory , 2001, Trends in Cognitive Sciences.

[23]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[24]  Leslie G. Ungerleider,et al.  Mechanisms of visual attention in the human cortex. , 2000, Annual review of neuroscience.

[25]  B. Postle Working memory as an emergent property of the mind and brain , 2006, Neuroscience.

[26]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[27]  Edward F. Ester,et al.  PSYCHOLOGICAL SCIENCE Research Article Stimulus-Specific Delay Activity in Human Primary Visual Cortex , 2022 .

[28]  M. Corbetta,et al.  The Reorienting System of the Human Brain: From Environment to Theory of Mind , 2008, Neuron.

[29]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[30]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[31]  R. Zemel,et al.  Inference and computation with population codes. , 2003, Annual review of neuroscience.

[32]  M. Paradiso,et al.  A theory for the use of visual orientation information which exploits the columnar structure of striate cortex , 2004, Biological Cybernetics.

[33]  D. Heeger,et al.  In this issue , 2002, Nature Reviews Drug Discovery.

[34]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[35]  J. Serences,et al.  Spatial attention improves the quality of population codes in human visual cortex. , 2010, Journal of neurophysiology.

[36]  Denis Schluppeck,et al.  The role of early visual cortex in visual short-term memory and visual attention , 2009, Vision Research.

[37]  Daniel A. Handwerker,et al.  Hemodynamic signals not predicted? Not so: A comment on Sirotin and Das (2009) , 2011, NeuroImage.

[38]  Nikolaus Kriegeskorte,et al.  Combining the tools: Activation- and information-based fMRI analysis , 2007, NeuroImage.

[39]  Stefan Treue,et al.  Feature-based attention influences motion processing gain in macaque visual cortex , 1999, Nature.

[40]  J. Maunsell,et al.  Effects of Attention on the Processing of Motion in Macaque Middle Temporal and Medial Superior Temporal Visual Cortical Areas , 1999, The Journal of Neuroscience.

[41]  Leslie G. Ungerleider,et al.  The neural systems that mediate human perceptual decision making , 2008, Nature Reviews Neuroscience.

[42]  D. G. Albrecht,et al.  Spikes versus BOLD: what does neuroimaging tell us about neuronal activity? , 2000, Nature Neuroscience.

[43]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[44]  Fei-Fei Li,et al.  Exploring Functional Connectivities of the Human Brain using Multivariate Information Analysis , 2009, NIPS.

[45]  Tom Michael Mitchell,et al.  Predicting Human Brain Activity Associated with the Meanings of Nouns , 2008, Science.

[46]  J. Maunsell,et al.  Using Neuronal Populations to Study the Mechanisms Underlying Spatial and Feature Attention , 2011, Neuron.

[47]  J. Maunsell,et al.  A Neuronal Population Measure of Attention Predicts Behavioral Performance on Individual Trials , 2010, The Journal of Neuroscience.

[48]  D. Heeger,et al.  Neuronal basis of contrast discrimination , 1999, Vision Research.

[49]  J. Haynes Brain Reading: Decoding Mental States From Brain Activity In Humans , 2011 .

[50]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[51]  Nicole C. Rust,et al.  Do We Know What the Early Visual System Does? , 2005, The Journal of Neuroscience.

[52]  N. Logothetis,et al.  Neurophysiological investigation of the basis of the fMRI signal , 2001, Nature.

[53]  D. Heeger,et al.  Cross-orientation suppression in human visual cortex. , 2011, Journal of neurophysiology.

[54]  David S. Brée,et al.  Linking Propositions , 1986, COLING.

[55]  D. Heeger,et al.  Decoding and Reconstructing Color from Responses in Human Visual Cortex , 2009, The Journal of Neuroscience.

[56]  Jack L. Gallant,et al.  Encoding and decoding in fMRI , 2011, NeuroImage.

[57]  Alex R. Wade,et al.  Early Suppressive Mechanisms and the Negative Blood Oxygenation Level-Dependent Response in Human Visual Cortex , 2010, The Journal of Neuroscience.

[58]  John H. R. Maunsell,et al.  Attentional modulation of visual motion processing in cortical areas MT and MST , 1996, Nature.

[59]  T. Pasternak,et al.  Working memory in primate sensory systems , 2005, Nature Reviews Neuroscience.

[60]  Geoffrey M Boynton,et al.  Imaging orientation selectivity: decoding conscious perception in V1 , 2005, Nature Neuroscience.

[61]  J. Gallant,et al.  Complete functional characterization of sensory neurons by system identification. , 2006, Annual review of neuroscience.

[62]  S. Rossitti Introduction to Functional Magnetic Resonance Imaging, Principles and Techniques , 2002 .

[63]  B. Wandell Computational neuroimaging of human visual cortex. , 1999, Annual review of neuroscience.

[64]  David D. Cox,et al.  Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex , 2003, NeuroImage.

[65]  G. Brindley Physiology of the Retina and the Visual Pathway , 1960 .

[66]  Jascha D. Swisher,et al.  Multiscale Pattern Analysis of Orientation-Selective Activity in the Primary Visual Cortex , 2010, The Journal of Neuroscience.

[67]  D. Heeger,et al.  The Normalization Model of Attention , 2009, Neuron.

[68]  Ryan J. Prenger,et al.  Bayesian Reconstruction of Natural Images from Human Brain Activity , 2009, Neuron.

[69]  Kâmil Uğurbil,et al.  Mental maze solving: directional fMRI tuning and population coding in the superior parietal lobule , 2005, Experimental Brain Research.

[70]  D. Heeger,et al.  Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1 , 1996, The Journal of Neuroscience.

[71]  Tom M. Mitchell,et al.  Machine learning classifiers and fMRI: A tutorial overview , 2009, NeuroImage.

[72]  Yevgeniy B. Sirotin,et al.  What could underlie the trial-related signal? A response to the commentaries by Drs. Kleinschmidt and Muller, and Drs. Handwerker and Bandettini , 2011, NeuroImage.

[73]  Yevgeniy B. Sirotin,et al.  Anticipatory haemodynamic signals in sensory cortex not predicted by local neuronal activity. , 2009, Nature.

[74]  Benjamin J. Tamber-Rosenau,et al.  Decoding cognitive control in human parietal cortex , 2009, Proceedings of the National Academy of Sciences.

[75]  G. Boynton Attention and visual perception , 2005, Current Opinion in Neurobiology.

[76]  John-Dylan Haynes,et al.  Decoding visual consciousness from human brain signals , 2009, Trends in Cognitive Sciences.

[77]  K. Sen,et al.  Spectral-temporal Receptive Fields of Nonlinear Auditory Neurons Obtained Using Natural Sounds , 2022 .

[78]  Rainer Goebel,et al.  Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns , 2008, NeuroImage.

[79]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[80]  M. Schönwiesner,et al.  Spectro-temporal modulation transfer function of single voxels in the human auditory cortex measured with high-resolution fMRI , 2009, Proceedings of the National Academy of Sciences.

[81]  Kevin L. Briggman,et al.  Multifunctional pattern-generating circuits. , 2008, Annual review of neuroscience.

[82]  Hans P. Op de Beeck,et al.  Against hyperacuity in brain reading: Spatial smoothing does not hurt multivariate fMRI analyses? , 2010, NeuroImage.

[83]  Timothy A. Machado,et al.  Functional connectivity in the retina at the resolution of photoreceptors , 2010, Nature.

[84]  N. Logothetis What we can do and what we cannot do with fMRI , 2008, Nature.

[85]  M. Brass,et al.  Unconscious determinants of free decisions in the human brain , 2008, Nature Neuroscience.

[86]  G. Boynton,et al.  Feature-Based Attentional Modulations in the Absence of Direct Visual Stimulation , 2007, Neuron.

[87]  J. Maunsell,et al.  Attention improves performance primarily by reducing interneuronal correlations , 2009, Nature Neuroscience.

[88]  G. Brindley The physiology of the retina and visual pathways , 1962 .

[89]  Alice J. O'Toole,et al.  Theoretical, Statistical, and Practical Perspectives on Pattern-based Classification Approaches to the Analysis of Functional Neuroimaging Data , 2007, Journal of Cognitive Neuroscience.

[90]  Peter E. Latham,et al.  Narrow Versus Wide Tuning Curves: What's Best for a Population Code? , 1999, Neural Computation.

[91]  F. Tong,et al.  Decoding Seen and Attended Motion Directions from Activity in the Human Visual Cortex , 2006, Current Biology.

[92]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[93]  R. Douglas,et al.  Recurrent neuronal circuits in the neocortex , 2007, Current Biology.

[94]  Jude F. Mitchell,et al.  Spatial Attention Decorrelates Intrinsic Activity Fluctuations in Macaque Area V4 , 2009, Neuron.

[95]  S. Yantis,et al.  Selective visual attention and perceptual coherence , 2006, Trends in Cognitive Sciences.

[96]  Carrie J. McAdams,et al.  Effects of Attention on Orientation-Tuning Functions of Single Neurons in Macaque Cortical Area V4 , 1999, The Journal of Neuroscience.

[97]  Justin L. Gardner,et al.  Is cortical vasculature functionally organized? , 2010, NeuroImage.

[98]  Edward F. Ester,et al.  Spatially Global Representations in Human Primary Visual Cortex during Working Memory Maintenance , 2009, The Journal of Neuroscience.

[99]  R. Shapley,et al.  Information Tuning of Populations of Neurons in Primary Visual Cortex , 2004, The Journal of Neuroscience.

[100]  Geraint Rees,et al.  Decoding the neural correlates of consciousness. , 2010, Current opinion in neurology.

[101]  Masa-aki Sato,et al.  Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders , 2008, Neuron.

[102]  Daniel A. Handwerker,et al.  Simple explanations before complex theories: Alternative interpretations of Sirotin and Das' observations , 2011, NeuroImage.

[103]  E. Seidemann,et al.  Effect of spatial attention on the responses of area MT neurons. , 1999, Journal of neurophysiology.

[104]  Jean-Baptiste Poline,et al.  Inverse retinotopy: Inferring the visual content of images from brain activation patterns , 2006, NeuroImage.

[105]  G. Rees,et al.  Predicting the orientation of invisible stimuli from activity in human primary visual cortex , 2005, Nature Neuroscience.

[106]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[107]  Mark D'Esposito,et al.  From cognitive to neural models of working memory , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[108]  I. Fried,et al.  Coupling Between Neuronal Firing, Field Potentials, and fMRI in Human Auditory Cortex , 2005, Science.

[109]  Giancarlo Valente,et al.  Multivariate analysis of fMRI time series: classification and regression of brain responses using machine learning. , 2008, Magnetic resonance imaging.

[110]  R. Desimone,et al.  Attention Increases Sensitivity of V4 Neurons , 2000, Neuron.