Decoding of visual activity patterns from fMRI responses using multivariate pattern analyses and convolutional neural network.

Decoding of human brain activity has always been a primary goal in neuroscience especially with functional magnetic resonance imaging (fMRI) data. In recent years, Convolutional neural network (CNN) has become a popular method for the extraction of features due to its higher accuracy, however it needs a lot of computation and training data. In this study, an algorithm is developed using Multivariate pattern analysis (MVPA) and modified CNN to decode the behavior of brain for different images with limited data set. Selection of significant features is an important part of fMRI data analysis, since it reduces the computational burden and improves the prediction performance; significant features are selected using t-test. MVPA uses machine learning algorithms to classify different brain states and helps in prediction during the task. General linear model (GLM) is used to find the unknown parameters of every individual voxel and the classification is done using multi-class support vector machine (SVM). MVPA-CNN based proposed algorithm is compared with region of interest (ROI) based method and MVPA based estimated values. The proposed method showed better overall accuracy (68.6%) compared to ROI (61.88%) and estimation values (64.17%).

[1]  Russell A. Poldrack,et al.  Deconvolving BOLD activation in event-related designs for multivoxel pattern classification analyses , 2012, NeuroImage.

[2]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[3]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[4]  Wei Xie,et al.  Accurate Cancer Classification Using Expressions of Very Few Genes , 2007, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[5]  Ryan J. Prenger,et al.  Bayesian Reconstruction of Natural Images from Human Brain Activity , 2009, Neuron.

[6]  Janaina Mourão Miranda,et al.  Classifying brain states and determining the discriminating activation patterns: Support Vector Machine on functional MRI data , 2005, NeuroImage.

[7]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[8]  Russell A. Poldrack,et al.  The impact of study design on pattern estimation for single-trial multivariate pattern analysis , 2014, NeuroImage.

[9]  Tom Heskes,et al.  Linear reconstruction of perceived images from human brain activity , 2013, NeuroImage.

[10]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[11]  Jean-Baptiste Poline,et al.  Inverse retinotopy: Inferring the visual content of images from brain activation patterns , 2006, NeuroImage.

[12]  Sebastian Stober,et al.  Using Convolutional Neural Networks to Recognize Rhythm Stimuli from Electroencephalography Recordings , 2014, NIPS.

[13]  Ulrich H.-G. Kreßel,et al.  Pairwise classification and support vector machines , 1999 .

[14]  J. Gallant,et al.  Identifying natural images from human brain activity , 2008, Nature.

[15]  Luca Maria Gambardella,et al.  Convolutional Neural Support Vector Machines: Hybrid Visual Pattern Classifiers for Multi-robot Systems , 2012, 2012 11th International Conference on Machine Learning and Applications.

[16]  Nikolaus Kriegeskorte,et al.  Deep neural networks: a new framework for modelling biological vision and brain information processing , 2015, bioRxiv.

[17]  Michael S. Pratte,et al.  Decoding patterns of human brain activity. , 2012, Annual review of psychology.

[18]  G. Rees,et al.  Neuroimaging: Decoding mental states from brain activity in humans , 2006, Nature Reviews Neuroscience.

[19]  Aamir Saeed Malik,et al.  Decoding of visual information from human brain activity: A review of fMRI and EEG studies. , 2015, Journal of integrative neuroscience.

[20]  Tzyy-Ping Jung,et al.  Feature extraction with deep belief networks for driver's cognitive states prediction from EEG data , 2015, 2015 IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP).

[21]  W. Art Chaovalitwongse,et al.  Voxel Selection Framework in Multi-Voxel Pattern Analysis of fMRI Data for Prediction of Neural Response to Visual Stimuli , 2014, IEEE Transactions on Medical Imaging.

[22]  D E McMillan,et al.  Increased levels of acute-phase serum proteins in diabetes. , 1989, Metabolism: clinical and experimental.

[23]  Panqu Wang,et al.  Encoding Voxels with Deep Learning , 2015, The Journal of Neuroscience.

[24]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[25]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[26]  Hubert Cecotti,et al.  Convolutional Neural Networks for P300 Detection with Application to Brain-Computer Interfaces , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Yann LeCun,et al.  Comparing SVM and convolutional networks for epileptic seizure prediction from intracranial EEG , 2008, 2008 IEEE Workshop on Machine Learning for Signal Processing.

[28]  Lipo Wang,et al.  Data Mining With Computational Intelligence , 2006, IEEE Transactions on Neural Networks.