The impact of certain methodological choices on multivariate analysis of fMRI data with support vector machines

Multivoxel pattern analysis of functional magnetic resonance imaging (fMRI) data is continuing to increase in popularity. Like all fMRI analyses, these analyses require extensive data processing and methodological choices, but the impact of these decisions on the final results is not always known. This study explores the impact of four methodological choices on analysis outcomes and introduces the technique of partitioning on random runs for characterizing temporal dependencies and evaluating partitioning methods. The analyses were performed on two fMRI data sets, which were repeatedly analyzed with support vector machines, varying the method of temporal compression, smoothing, voxel-wise detrending, and partitioning into training and testing sets. Smoothing sometimes slightly increased classification accuracy. Partitioning other than on the runs increased classification accuracy, and the random runs technique allowed us to attribute this improvement to the increased amount of training data, rather than to bias. The impact of the temporal compression and detrending methods varied so strongly with data set that general recommendations could not be drawn. These interactions suggest that, rather than searching for a universally superior set of methodological choices, researchers must carefully consider each choice in the context of each experiment.

[1]  Hans P. Op de Beeck,et al.  Against hyperacuity in brain reading: Spatial smoothing does not hurt multivariate fMRI analyses? , 2010, NeuroImage.

[2]  Johan Wagemans,et al.  Perceived Shape Similarity among Unfamiliar Objects and the Organization of the Human Object Vision Pathway , 2008, The Journal of Neuroscience.

[3]  Jody Tanabe,et al.  See Blockindiscussions, Blockinstats, Blockinand Blockinauthor Blockinprofiles Blockinfor Blockinthis Blockinpublication Comparison Blockinof Blockindetrending Blockinmethods Blockinfor Optimal Blockinfmri Blockinpreprocessing , 2022 .

[4]  S.C. Strother,et al.  Evaluating fMRI preprocessing pipelines , 2006, IEEE Engineering in Medicine and Biology Magazine.

[5]  Polina Golland,et al.  Permutation Tests for Classification: Towards Statistical Significance in Image-Based Studies , 2003, IPMI.

[6]  Sayan Mukherjee,et al.  Permutation Tests for Classification , 2005, COLT.

[7]  Tom M. Mitchell,et al.  Learning to Decode Cognitive States from Brain Images , 2004, Machine Learning.

[8]  Tom M. Mitchell,et al.  Beyond brain blobs: machine learning classifiers as instruments for analyzing functional magnetic resonance imaging data , 2007 .

[9]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[10]  Geoffrey E. Hinton,et al.  A Comparison of Classification Methods for Longitudinal fMRI Studies , 2009, NeuroImage.

[11]  Masa-aki Sato,et al.  Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders , 2008, Neuron.

[12]  N. Kriegeskorte,et al.  Revealing representational content with pattern-information fMRI--an introductory guide. , 2009, Social cognitive and affective neuroscience.

[13]  Tom Michael Mitchell,et al.  From the SelectedWorks of Marcel Adam Just 2008 Using fMRI brain activation to identify cognitive states associated with perception of tools and dwellings , 2016 .

[14]  Essa Yacoub,et al.  The Evaluation of Preprocessing Choices in Single-Subject BOLD fMRI Using NPAIRS Performance Metrics , 2003, NeuroImage.

[15]  C. Keysers,et al.  An introduction to anatomical ROI-based fMRI classification analysis , 2009, Brain Research.

[16]  Janaina Mourão Miranda,et al.  Classifying brain states and determining the discriminating activation patterns: Support Vector Machine on functional MRI data , 2005, NeuroImage.

[17]  Janaina Mourão Miranda,et al.  The impact of temporal compression and space selection on SVM analysis of single-subject and multi-subject fMRI data , 2006, NeuroImage.

[18]  G. Rees,et al.  Neuroimaging: Decoding mental states from brain activity in humans , 2006, Nature Reviews Neuroscience.

[19]  Tom M. Mitchell,et al.  Machine learning classifiers and fMRI: A tutorial overview , 2009, NeuroImage.

[20]  D. Bates,et al.  Mixed-Effects Models in S and S-PLUS , 2001 .

[21]  Francisco Herrera,et al.  Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power , 2010, Inf. Sci..

[22]  Polina Golland,et al.  AI Memo AIM-2003-019 Permutation Tests for Classication , 2003 .

[23]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[24]  Nikolaus Kriegeskorte,et al.  How does an fMRI voxel sample the neuronal activity pattern: Compact-kernel or complex spatiotemporal filter? , 2010, NeuroImage.

[25]  C.-Y.C. Chu,et al.  Pattern recognition and machine learning for magnetic resonance images with kernel methods , 2009 .

[26]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[27]  Arthur Gretton,et al.  Comparison of Pattern Recognition Methods in Classifying High-resolution Bold Signals Obtained at High Magnetic Field in Monkeys , 2008 .

[28]  Yasuhito Sawahata,et al.  Spatial smoothing hurts localization but not information: Pitfalls for brain mappers , 2010, NeuroImage.

[29]  Stephen C. Strother,et al.  Support vector machines for temporal classification of block design fMRI data , 2005, NeuroImage.

[30]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[31]  Cameron S. Carter,et al.  Multivariate Pattern Analysis of Functional Magnetic Resonance Imaging Data Reveals Deficits in Distributed Representations in Schizophrenia , 2008, Biological Psychiatry.

[32]  Steven Salzberg,et al.  On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach , 1997, Data Mining and Knowledge Discovery.

[33]  Simon B. Eickhoff,et al.  Testing anatomically specified hypotheses in functional imaging using cytoarchitectonic maps , 2006, NeuroImage.

[34]  Yoshua Bengio,et al.  Inference for the Generalization Error , 1999, Machine Learning.

[35]  Hans P. Op de Beeck,et al.  Probing the mysterious underpinnings of multi-voxel fMRI analyses , 2010, NeuroImage.

[36]  Rainer Goebel,et al.  Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns , 2008, NeuroImage.

[37]  Simon B. Eickhoff,et al.  A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data , 2005, NeuroImage.

[38]  Alice J. O'Toole,et al.  Theoretical, Statistical, and Practical Perspectives on Pattern-based Classification Approaches to the Analysis of Functional Neuroimaging Data , 2007, Journal of Cognitive Neuroscience.