Mental encoding and neural decoding of abstract cognitive categories: A commentary and simulation

The premise of Multi-Voxel Pattern Analysis (MVPA) of functional Magnetic Resonance Image (fMRI) data is that mental encodings or states give rise to patterns of neural activation, which in turn, give rise to patterns of blood-oxygen level dependent (BOLD) responses distributed across sets of voxels. Statistical learning algorithms can then be used to detect relationships between mental encodings and BOLD responses, typically through pattern classification. Amongst many other applications, this technique has been used to evidence abstract category representation in an assortment of brain areas and across a range of cognitive domains. In this commentary, we address a critical domain-general caveat to inferring abstract category representation from MVPA that has been partly overlooked in the recent literature: specifically, the distinction between representing specific exemplars within categories, and representing the abstract categories themselves. Using a simulation, we demonstrate that certain forms of MVPA training and testing do not constitute sufficient evidence of category representation, and illustrate prospective and novel retrospective resolutions for this issue.

[1]  N. Kanwisher,et al.  How Distributed Is Visual Category Information in Human Occipito-Temporal Cortex? An fMRI Study , 2002, Neuron.

[2]  Michael Esterman,et al.  Decoding Task-based Attentional Modulation during Face Categorization , 2011, Journal of Cognitive Neuroscience.

[3]  Thomas Serre,et al.  Reading the mind's eye: Decoding category information during mental imagery , 2010, NeuroImage.

[4]  Noël Staeren,et al.  Sound Categories Are Represented as Distributed Patterns in the Human Auditory Cortex , 2009, Current Biology.

[5]  Mark D'Esposito,et al.  The Prefrontal Cortex Modulates Category Selectivity in Human Extrastriate Cortex , 2011, Journal of Cognitive Neuroscience.

[6]  David D. Cox,et al.  Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex , 2003, NeuroImage.

[7]  Dirk B. Walther,et al.  Natural Scene Categories Revealed in Distributed Patterns of Activity in the Human Brain , 2009, The Journal of Neuroscience.

[8]  T. Ethofer,et al.  Decoding of emotional information in voice-sensitive cortices , 2009, NeuroImage.

[9]  Tom Michael Mitchell,et al.  From the SelectedWorks of Marcel Adam Just 2008 Using fMRI brain activation to identify cognitive states associated with perception of tools and dwellings , 2016 .

[10]  J. Y. Goulermas,et al.  Multivoxel fMRI analysis of color tuning in human primary visual cortex. , 2009, Journal of vision.

[11]  N. Kriegeskorte,et al.  Revealing representational content with pattern-information fMRI--an introductory guide. , 2009, Social cognitive and affective neuroscience.

[12]  Bertrand Thirion,et al.  Deciphering Cortical Number Coding from Human Brain Activity Patterns , 2009, Current Biology.

[13]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[14]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.