Neural responses to naturalistic clips of behaving animals in two different task contexts

Neuroimaging studies of object and action representation often use controlled stimuli and implicitly assume that the relevant neural representational spaces are fixed and context-invariant. Here we present functional MRI data measured while participants freely viewed brief naturalistic video clips of behaving animals in two different task contexts. Participants performed a 1-back category repetition detection task requiring them to attend to either animal taxonomy or animal behavior. The data and analysis code are freely available, and have been curated according to the Brain Imaging Data Structure (BIDS) standard. We thoroughly describe the data, provide quality control metrics, and perform a searchlight classification analysis to demonstrate the potential utility of the data. These data are intended to provide a test bed for investigating how task demands alter the neural representation of complex stimuli and their semantic qualities.

[1]  John-Dylan Haynes,et al.  Valid population inference for information-based imaging: From the second-level t-test to prevalence inference , 2015, NeuroImage.

[2]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[3]  A. Dale,et al.  High‐resolution intersubject averaging and a coordinate system for the cortical surface , 1999, Human brain mapping.

[4]  William W. Graves,et al.  Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. , 2009, Cerebral cortex.

[5]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[6]  Satrajit S. Ghosh,et al.  The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments , 2016, Scientific Data.

[7]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[8]  Yi Chen,et al.  Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): Random permutations and cluster size control , 2011, NeuroImage.

[9]  Alexander Borst,et al.  How does Nature Program Neuron Types? , 2008, Front. Neurosci..

[10]  James L. McClelland,et al.  The parallel distributed processing approach to semantic cognition , 2003, Nature Reviews Neuroscience.

[11]  J. Gallant,et al.  Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies , 2011, Current Biology.

[12]  E. Rosch.  Cognitive Representations of Semantic Categories. , 1975 .

[13]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[14]  Jack L. Gallant,et al.  A Continuous Semantic Space Describes the Representation of Thousands of Object and Action Categories across the Human Brain , 2012, Neuron.

[15]  Abraham Z. Snyder,et al.  Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion , 2012, NeuroImage.

[16]  Bruce Fischl,et al.  Accurate and robust brain image alignment using boundary-based registration , 2009, NeuroImage.

[17]  Peter Gärdenfors,et al.  Using Conceptual Spaces to Model Actions and Events , 2012, J. Semant..

[18]  R. Shepard.  Stimulus and response generalization: tests of a model relating generalization to distance in psychological space. , 1958, Journal of experimental psychology.

[19]  Yaroslav O. Halchenko,et al.  Neuroscience Runs on GNU/Linux , 2011, Front. Neuroinform..

[20]  Thomas T. Liu,et al.  A component based noise correction method (CompCor) for BOLD and perfusion based fMRI , 2007, NeuroImage.

[21]  Alfonso Caramazza,et al.  Action categories in lateral occipitotemporal cortex are organized along sociality and transitivity. , 2016, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[22]  Samuel A. Nastase,et al.  Attention Selectively Reshapes the Geometry of Distributed Semantic Representation , 2016, bioRxiv.

[23]  W. R. Garner,et al.  Integrality of stimulus dimensions in various types of information processing , 1970 .

[24]  J. Kruschke,et al.  ALCOVE: an exemplar-based connectionist model of category learning. , 1992, Psychological review.

[25]  Thomas E. Nichols,et al.  Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate , 2002, NeuroImage.

[26]  Michael Hanke,et al.  A studyforrest extension, retinotopic mapping and localization of higher visual areas , 2016, Scientific Data.

[27]  Alexander G. Huth,et al.  Attention During Natural Vision Warps Semantic Representation Across the Human Brain , 2013, Nature Neuroscience.

[28]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[29]  P. Boesiger,et al.  SENSE: Sensitivity encoding for fast MRI , 1999, Magnetic resonance in medicine.

[30]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[31]  T. Poggio,et al.  Cognitive neuroscience: Neural mechanisms for the recognition of biological movements , 2003, Nature Reviews Neuroscience.

[32]  Kingson Man,et al.  Multivariate cross-classification: applying machine learning techniques to characterize abstraction in neural representations , 2015, Front. Hum. Neurosci..

[33]  Satrajit S. Ghosh,et al.  Nipype: A Flexible, Lightweight and Extensible Neuroimaging Data Processing Framework in Python , 2011, Front. Neuroinform..

[34]  N. Kriegeskorte,et al.  Author ' s personal copy Representational geometry : integrating cognition , computation , and the brain , 2013 .

[35]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[36]  N. Logothetis,et al.  Neurophysiological investigation of the basis of the fMRI signal , 2001, Nature.

[37]  R. Shepard.  Attention and the metric structure of the stimulus space. , 1964 .

[38]  Christiane Fellbaum,et al.  English Verbs as a Semantic Net , 1990 .

[39]  S Edelman,et al.  Representation is representation of similarities , 1996, Behavioral and Brain Sciences.

[40]  Paul E. Downing,et al.  Crossmodal and action-specific: neuroimaging the human mirror neuron system , 2013, Trends in Cognitive Sciences.

[41]  J. I.  The Design of Experiments , 1936, Nature.

[42]  Satrajit S. Ghosh,et al.  BIDS apps: Improving ease of use, accessibility, and reproducibility of neuroimaging data analysis methods , 2016, bioRxiv.

[43]  Jonathan W. Peirce,et al.  PsychoPy—Psychophysics software in Python , 2007, Journal of Neuroscience Methods.

[44]  Brian E. Granger,et al.  IPython: A System for Interactive Scientific Computing , 2007, Computing in Science & Engineering.

[45]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[46]  Russell A. Poldrack,et al.  OpenfMRI: Open sharing of task fMRI data , 2017, NeuroImage.

[47]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[48]  Joset A. Etzel.  MVPA significance testing when just above chance, and related properties of permutation tests , 2017, 2017 International Workshop on Pattern Recognition in Neuroimaging (PRNI).

[49]  Paul E. Downing,et al.  A comparison of volume-based and surface-based multi-voxel pattern analysis , 2011, NeuroImage.

[50]  Geoffrey Karl Aguirre,et al.  Continuous carry-over designs for fMRI , 2007, NeuroImage.

[51]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[52]  D. Heeger,et al.  Reliability of cortical activity during natural stimulation , 2010, Trends in Cognitive Sciences.

[53]  R. Nosofsky.  Attention, similarity, and the identification-categorization relationship. , 1986, Journal of experimental psychology. General.

[54]  Stephen M. Smith,et al.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm , 2001, IEEE Transactions on Medical Imaging.

[55]  Lawrence L. Wald,et al.  Comparison of physiological noise at 1.5 T, 3 T and 7 T and optimization of fMRI acquisition parameters , 2005, NeuroImage.

[56]  J. S. Guntupalli,et al.  The Representation of Biological Classes in the Human Brain , 2012, The Journal of Neuroscience.

[57]  Krzysztof J. Gorgolewski,et al.  MRIQC: Advancing the automatic prediction of image quality in MRI from unseen sites , 2016, bioRxiv.

[58]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[59]  Stefan Pollmann,et al.  PyMVPA: a Python Toolbox for Multivariate Pattern Analysis of fMRI Data , 2009, Neuroinformatics.

[60]  Gidon Felsen,et al.  A natural approach to studying vision , 2005, Nature Neuroscience.

[61]  A. Tversky.  Features of Similarity , 1977 .

[62]  F ATTNEAVE,et al.  Dimensions of similarity. , 1950, The American journal of psychology.

[63]  S. Zeki,et al.  Functional brain mapping during free viewing of natural scenes , 2004, Human brain mapping.

[64]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[65]  Yaroslav O. Halchenko,et al.  Cross-modal searchlight classification: methodological challenges and recommended solutions , 2016, 2016 International Workshop on Pattern Recognition in Neuroimaging (PRNI).

[66]  J. S. Guntupalli,et al.  Decoding neural representational spaces using multivariate pattern analysis. , 2014, Annual review of neuroscience.

[67]  Thomas E. Nichols,et al.  Fixing the stimulus-as-fixed-effect fallacy in task fMRI , 2016, bioRxiv.

[68]  Brenna Argall,et al.  SUMA: an interface for surface-based intra- and inter-subject analysis with AFNI , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[69]  Yaroslav O. Halchenko,et al.  The Animacy Continuum in the Human Ventral Vision Pathway , 2015, Journal of Cognitive Neuroscience.

[70]  Mark S. Cohen,et al.  Parametric Analysis of fMRI Data Using Linear Systems Methods , 1997, NeuroImage.

[71]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[72]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[73]  Bryan R. Conroy,et al.  A Common, High-Dimensional Model of the Representational Space in Human Ventral Temporal Cortex , 2011, Neuron.

[74]  Thomas E. Nichols,et al.  Fixing the stimulus-as-fixed-effect fallacy in task fMRI , 2016 .

[75]  Oliver Speck,et al.  A high-resolution 7-Tesla fMRI dataset from complex natural stimulation with an audio movie , 2014, Scientific Data.