Real‐time fMRI using brain‐state classification

We have implemented a real‐time functional magnetic resonance imaging system based on multivariate classification. This approach is distinctly different from spatially localized real‐time implementations, since it does not require prior assumptions about functional localization and individual performance strategies, and has the ability to provide feedback based on intuitive translations of brain state rather than localized fluctuations. Thus this approach provides the capability for a new class of experimental designs in which real‐time feedback control of the stimulus is possible—rather than using a fixed paradigm, experiments can adaptively evolve as subjects receive brain‐state feedback. In this report, we describe our implementation and characterize its performance capabilities. We observed ∼80% classification accuracy using whole brain, block‐design, motor data. Within both left and right motor task conditions, important differences exist between the initial transient period produced by task switching (changing between rapid left or right index finger button presses) and the subsequent stable period during sustained activity. Further analysis revealed that very high accuracy is achievable during stable task periods, and that the responsiveness of the classifier to changes in task condition can be much faster than signal time‐to‐peak rates. Finally, we demonstrate the versatility of this implementation with respect to behavioral task, suggesting that our results are applicable across a spectrum of cognitive domains. Beyond basic research, this technology can complement electroencephalography‐based brain computer interface research, and has potential applications in the areas of biofeedback rehabilitation, lie detection, learning studies, virtual reality‐based training, and enhanced conscious awareness. Hum Brain Mapp 2006. © 2006 Wiley‐Liss, Inc.

[1]  Stephen C. Strother,et al.  Support vector machines for temporal classification of block design fMRI data , 2005, NeuroImage.

[2]  S. Ogawa,et al.  Oxygenation‐sensitive contrast in magnetic resonance image of rodent brain at high magnetic fields , 1990, Magnetic resonance in medicine.

[3]  D. Tank,et al.  Brain magnetic resonance imaging with contrast dependent on blood oxygenation. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[4]  Gary H. Glover,et al.  Learned regulation of spatially localized brain activation using real-time fMRI , 2004, NeuroImage.

[5]  Seung-Schik Yoo,et al.  Functional MRI for neurofeedback: feasibility studyon a hand motor task , 2002, Neuroreport.

[6]  Luiz Pessoa,et al.  Quantitative prediction of perceptual decisions during near-threshold fear detection. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[7]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[8]  Jason Weston,et al.  Trading convexity for scalability , 2006, ICML.

[9]  P. Boesiger,et al.  Dynamic characteristics of oxygenation-sensitive MRI signal in different temporal protocols for imaging human brain activity , 2000, Neuroradiology.

[10]  Alice J. O'Toole,et al.  Partially Distributed Representations of Objects and Faces in Ventral Temporal Cortex , 2005, Journal of Cognitive Neuroscience.

[11]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[12]  G. Rees,et al.  Predicting the Stream of Consciousness from Activity in Human Visual Cortex , 2005, Current Biology.

[13]  Stephen C. Strother,et al.  Penalized Discriminant Analysis of [15O]-water PET Brain Images with Prediction Error Selection of Smoothness and Regularization , 2001, IEEE Trans. Medical Imaging.

[14]  D.M. Taylor,et al.  Information conveyed through brain-control: cursor versus robot , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  David D. Cox,et al.  Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex , 2003, NeuroImage.

[16]  Stephen José Hanson,et al.  Combinatorial codes in ventral temporal lobe for object recognition: Haxby (2001) revisited: is there a “face” area? , 2004, NeuroImage.

[17]  Dinggang Shen,et al.  Classifying spatial patterns of brain activity with machine learning methods: Application to lie detection , 2005, NeuroImage.

[18]  John D E Gabrieli,et al.  Control over brain activation and pain learned by using real-time functional MRI. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[19]  K. Uğurbil,et al.  Spatial and temporal differentiation of fMRI BOLD response in primary visual cortex of human brain during sustained visual simulation , 1998, Magnetic resonance in medicine.

[20]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[21]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[22]  Gary F. Egan,et al.  Evaluating subject specific preprocessing choices in multisubject fMRI data sets using data-driven performance metrics , 2003, NeuroImage.

[23]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[24]  S. Ogawa.  Brain magnetic resonance imaging with contrast-dependent oxygenation , 1990 .

[25]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[26]  E Yacoub,et al.  Detection of the early negative response in fMRI at 1.5 Tesla , 1999, Magnetic resonance in medicine.

[27]  S. C. Strother,et al.  The Quantitative Evaluation of Functional Neuroimaging Experiments: Mutual Information Learning Curves , 2002, NeuroImage.

[28]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[29]  Tom M. Mitchell,et al.  Learning to Decode Cognitive States from Brain Images , 2004, Machine Learning.

[30]  Lars Kai Hansen,et al.  Optimizing the fMRI data-processing pipeline using prediction and reproducibility performance metrics: I. A preliminary group analysis , 2004, NeuroImage.

[31]  Leslie G. Ungerleider,et al.  Distributed representation of objects in the human ventral visual pathway. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[32]  Manel Martínez-Ramón,et al.  fMRI pattern classification using neuroanatomically constrained boosting , 2006, NeuroImage.

[33]  Robert A. Lordo,et al.  Learning from Data: Concepts, Theory, and Methods , 2001, Technometrics.

[34]  S. Strother,et al.  Penalized discriminant analysis of [/sup 15/O]-water PET brain images with prediction error selection of smoothness and regularization hyperparameters , 2001, IEEE Transactions on Medical Imaging.

[35]  E Donchin,et al.  Brain-computer interface technology: a review of the first international meeting. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[36]  Vaidehi S. Natu,et al.  Category-Specific Cortical Activity Precedes Retrieval During Memory Search , 2005, Science.

[37]  L. K. Hansen,et al.  The Quantitative Evaluation of Functional Neuroimaging Experiments: The NPAIRS Data Analysis Framework , 2000, NeuroImage.

[38]  Essa Yacoub,et al.  The Evaluation of Preprocessing Choices in Single-Subject BOLD fMRI Using NPAIRS Performance Metrics , 2003, NeuroImage.

[39]  S. Ogawa,et al.  An approach to probe some neural systems interaction by functional MRI at neural time scale down to milliseconds. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[40]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[41]  Janaina Mourão Miranda,et al.  Classifying brain states and determining the discriminating activation patterns: Support Vector Machine on functional MRI data , 2005, NeuroImage.

[42]  Soo-Young Lee,et al.  Brain–computer interface using fMRI: spatial navigation by thoughts , 2004, Neuroreport.

[43]  Frank Schneider,et al.  Real-time fMRI of temporolimbic regions detects amygdala activation during single-trial self-induced sadness , 2003, NeuroImage.

[44]  N. Kanwisher,et al.  Testing cognitive models of visual attention with fMRI and MEG , 2001, Neuropsychologia.

[45]  Michael Erb,et al.  Physiological self-regulation of regional brain activity using real-time functional magnetic resonance imaging (fMRI): methodology and exemplary data , 2003, NeuroImage.

[46]  L. K. Hansen,et al.  Plurality and Resemblance in fMRI Data Analysis , 1999, NeuroImage.