Imaging decision-related neural cascades in the human brain

Perceptual decisions depend on coordinated patterns of neural activity cascading across the brain, running in time from stimulus to response and in space from primary sensory regions to the frontal lobe. Measuring this cascade and how it flows through the brain is key to developing an understanding of how our brains function. However observing, let alone understanding, this cascade, particularly in humans, is challenging. Here, we report a significant methodological advance allowing this observation in humans at unprecedented spatiotemporal resolution. We use a novel encoding model to link simultaneously measured electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) signals to infer the high-resolution spatiotemporal brain dynamics taking place during rapid visual perceptual decision-making. After demonstrating the methodology replicates past results, we show that it uncovers a previously unobserved sequential reactivation of a substantial fraction of the pre-response network whose magnitude correlates with decision confidence. Our results illustrate that a temporally coordinated and spatially distributed neural cascade underlies perceptual decision-making, with our methodology illuminating complex brain dynamics that would otherwise be unobservable using conventional fMRI or EEG separately. We expect this methodology to be useful in observing brain dynamics in a wide range of other mental processes.

[1]  Thomas E. Nichols,et al.  Controlling the familywise error rate in functional neuroimaging: a comparative review , 2003, Statistical methods in medical research.

[2]  J. Gallant,et al.  Identifying natural images from human brain activity , 2008, Nature.

[3]  Y Kamitani,et al.  Neural Decoding of Visual Imagery During Sleep , 2013, Science.

[4]  Thomas Dierks,et al.  BOLD correlates of EEG alpha phase-locking and the fMRI default mode network , 2009, NeuroImage.

[5]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[6]  Éva M. Bankó,et al.  Dissociating the Effect of Noise on Sensory Processing and Overall Decision Difficulty , 2011, The Journal of Neuroscience.

[7]  A. Hayes,et al.  Combining independent p values: extensions of the Stouffer and binomial methods. , 2000, Psychological methods.

[8]  Takashi Hanakawa,et al.  Spontaneous Slow Fluctuation of EEG Alpha Rhythm Reflects Activity in Deep-Brain Structures: A Simultaneous EEG-fMRI Study , 2013, PloS one.

[9]  Stephen D. Mayhew,et al.  Spontaneous EEG alpha oscillation interacts with positive and negative BOLD responses in the visual–auditory cortices and default-mode network , 2013, NeuroImage.

[10]  J. Gallant,et al.  Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies , 2011, Current Biology.

[11]  Christopher Summerfield,et al.  Metacognition in human decision-making: confidence and error monitoring , 2012, Philosophical Transactions of the Royal Society B: Biological Sciences.

[12]  Karl J. Friston,et al.  Ten simple rules for dynamic causal modeling , 2010, NeuroImage.

[13]  Nadim Joni Shah,et al.  Attention to Detail: Why Considering Task Demands Is Essential for Single-Trial Analysis of BOLD Correlates of the Visual P1 and N1 , 2014, Journal of Cognitive Neuroscience.

[14]  Michael M. Plichta,et al.  Sequential inhibitory control processes assessed through simultaneous EEG–fMRI , 2014, NeuroImage.

[15]  Paul Sajda,et al.  Knowing when not to swing: EEG evidence that enhanced perception–action coupling underlies baseball batter expertise , 2015, NeuroImage.

[16]  P. Sajda,et al.  EEG-Informed fMRI Reveals Spatiotemporal Characteristics of Perceptual Decision Making , 2007, The Journal of Neuroscience.

[17]  Bin He,et al.  Negative covariation between task-related responses in alpha/beta-band activity and BOLD in human sensorimotor cortex: An EEG and fMRI study of motor imagery and movements , 2010, NeuroImage.

[18]  Rainer Goebel,et al.  Multimodal imaging: an evaluation of univariate and multivariate methods for simultaneous EEG/fMRI. , 2010, Magnetic resonance imaging.

[19]  Gregor Leicht,et al.  Single-trial EEG–fMRI coupling of the emotional auditory early posterior negativity , 2012, NeuroImage.

[20]  Nadim Joni Shah,et al.  Single-trial P3 amplitude and latency informed event-related fMRI models yield different BOLD response patterns to a target detection task , 2009, NeuroImage.

[21]  M. Philiastides,et al.  TITLE : Two spatiotemporally distinct value systems shape reward-based learning in the human brain , 2015 .

[22]  D. Poeppel,et al.  Coupled neural systems underlie the production and comprehension of naturalistic narrative speech , 2014, Proceedings of the National Academy of Sciences.

[23]  Kathleen A. Hansen,et al.  Topographic Organization in and near Human Visual Area V4 , 2007, The Journal of Neuroscience.

[24]  N. Logothetis What we can do and what we cannot do with fMRI , 2008, Nature.

[25]  Juliana Yordanova,et al.  Simultaneous EEG and fMRI Reveals a Causally Connected Subcortical-Cortical Network during Reward Anticipation , 2013, The Journal of Neuroscience.

[26]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[27]  Stephen M. Smith,et al.  Threshold-free cluster enhancement: Addressing problems of smoothing, threshold dependence and localisation in cluster inference , 2009, NeuroImage.

[28]  Jack L. Gallant,et al.  Natural Scene Statistics Account for the Representation of Scene Categories in Human Visual Cortex , 2013, Neuron.

[29]  Vinh Thai Nguyen,et al.  The superior temporal sulcus and the N170 during face processing: Single trial analysis of concurrent EEG–fMRI , 2014, NeuroImage.

[30]  Paul Sajda,et al.  You Can’t Think and Hit at the Same Time: Neural Correlates of Baseball Pitch Classification , 2012, Front. Neurosci..

[31]  Markus Siegel,et al.  Cortical information flow during flexible sensorimotor decisions , 2015, Science.

[32]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[33]  Thomas V. Wiecki,et al.  HDDM: Hierarchical Bayesian estimation of the Drift-Diffusion Model in Python , 2013, Front. Neuroinform..

[34]  N. Yeung,et al.  Decision Processes in Human Performance Monitoring , 2010, The Journal of Neuroscience.

[35]  R. Hess,et al.  What causes non-monotonic tuning of fMRI response to noisy images? , 2002, Current Biology.

[36]  R. Ratcliff,et al.  Neural Representation of Task Difficulty and Decision Making during Perceptual Categorization: A Timing Diagram , 2006, The Journal of Neuroscience.

[37]  Andrew S. Kayser,et al.  The neural representation of sensorimotor transformations in a human perceptual decision making network , 2013, NeuroImage.

[38]  Jason Sherwin,et al.  Musical experts recruit action-related neural structures in harmonic anomaly detection: Evidence for embodied cognition in expertise , 2013, Brain and Cognition.

[39]  L. Parra,et al.  Single-Trial Analysis of Neuroimaging Data: Inferring Neural Networks Underlying Perceptual Decision-Making in the Human Brain , 2009, IEEE Reviews in Biomedical Engineering.

[40]  Lorena Deuker,et al.  Memory Consolidation by Replay of Stimulus-Specific Neural Activity , 2013, The Journal of Neuroscience.

[41]  Paul Sajda,et al.  Simultaneous EEG–fMRI reveals a temporal cascade of task-related and default-mode activations during a simple target detection task , 2014, NeuroImage.

[42]  Alexander G. Huth,et al.  Attention During Natural Vision Warps Semantic Representation Across the Human Brain , 2013, Nature Neuroscience.

[43]  David Friedman,et al.  Single-trial discrimination for integrating simultaneous EEG and fMRI: Identifying cortical areas contributing to trial-to-trial variability in the auditory oddball task , 2009, NeuroImage.

[44]  Leslie G. Ungerleider,et al.  A general mechanism for perceptual decision-making in the human brain , 2004, Nature.

[45]  Jack L. Gallant,et al.  Encoding and decoding in fMRI , 2011, NeuroImage.

[46]  M. Botvinick,et al.  Conflict monitoring and cognitive control. , 2001, Psychological review.

[47]  Marios G. Philiastides,et al.  Neural representations of confidence emerge from the process of decision formation during perceptual choices , 2015, NeuroImage.

[48]  R. Dolan,et al.  Prefrontal Contributions to Metacognition in Perceptual Decision Making , 2012, The Journal of Neuroscience.

[49]  Paul Sajda,et al.  Fast Bootstrapping and Permutation Testing for Assessing Reproducibility and Interpretability of Multivariate fMRI Decoding Models , 2013, PloS one.

[50]  Karl J. Friston,et al.  Comparing Families of Dynamic Causal Models , 2010, PLoS Comput. Biol..

[51]  Bart Vanrumste,et al.  EEG/MEG Source Imaging: Methods, Challenges, and Open Issues , 2009, Comput. Intell. Neurosci..

[52]  Lucas C. Parra,et al.  Recipes for the linear analysis of EEG , 2005, NeuroImage.

[53]  João Jorge,et al.  EEG–fMRI integration for the study of human brain function , 2014, NeuroImage.

[54]  P. Sajda,et al.  Simultaneous EEG-fMRI Reveals Temporal Evolution of Coupling between Supramodal Cortical Attention Networks and the Brainstem , 2013, The Journal of Neuroscience.

[55]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[56]  René J. Huster,et al.  Methods for Simultaneous EEG-fMRI: An Introductory Review , 2012, The Journal of Neuroscience.

[57]  Paul Sajda,et al.  Signal Processing and Machine Learning for Single-trial Analysis of Simultaneously Acquired EEG and fMRI , 2010 .

[58]  Cees van Leeuwen,et al.  Donders is dead: cortical traveling waves and the limits of mental chronometry in cognitive neuroscience , 2015, Cognitive Processing.

[59]  Sabine Van Huffel,et al.  The BOLD correlates of the visual P1 and N1 in single-trial analysis of simultaneous EEG-fMRI recordings during a spatial detection task , 2011, NeuroImage.

[60]  Russell A. Poldrack,et al.  Deconvolving BOLD activation in event-related designs for multivoxel pattern classification analyses , 2012, NeuroImage.

[61]  G. Strang Introduction to Linear Algebra , 1993 .

[62]  P. Sajda,et al.  A System for Single-trial Analysis of Simultaneously Acquired EEG and fMRI , 2007, 2007 3rd International IEEE/EMBS Conference on Neural Engineering.

[63]  P. Sajda,et al.  Human Scalp Potentials Reflect a Mixture of Decision-Related Signals during Perceptual Choices , 2014, The Journal of Neuroscience.

[64]  Hauke R. Heekeren,et al.  A task-independent neural representation of subjective certainty in visual perception , 2015, Front. Hum. Neurosci..

[65]  M. Shadlen,et al.  Choice Certainty Is Informed by Both Evidence and Decision Time , 2014, Neuron.

[66]  Bruce Fischl,et al.  Accurate and robust brain image alignment using boundary-based registration , 2009, NeuroImage.

[67]  P. Sajda,et al.  Temporal characterization of the neural correlates of perceptual decision making in the human brain. , 2006, Cerebral cortex.

[68]  Stephen M. Smith,et al.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm , 2001, IEEE Transactions on Medical Imaging.

[69]  Karl J. Friston,et al.  Dynamic causal modelling , 2003, NeuroImage.