Brain-Voyant: A General Purpose Machine-Learning Tool for Real-Time fMRI Whole-Brain Pattern Classification

We have developed Brain-Voyant, an efficient general-purpose machine learning tool for real-time functional magnetic resonance imaging classification using whole-brain data, which can be used to explore novel brain-computer interface paradigms or advanced neurofeedback protocols. We have created a convenient and configurable front-end tool that receives fMRI-based multi-voxel raw brain data as input. Our tool processes, analyses, classifies and transfers the classification to an external object such as a virtual avatar or a humanoid robot in real-time. Our tool is focused on minimizing delay time, and to that end, it employs a method that is based on examining in advance the voxels that have been found to be task-relevant in the machine learning model training phase.The tool's code base was designed to be easily extended to support additional feature reduction, normalization and classification algorithms. This tool was used in several published studies using motor execution, motor imagery, and visual category classification in cue-based and free-choice brain-computer interface experiments, with both healthy and amputated subjects. This tool is not limited by number of classes, is not limited to predefined regions of interest, and classifier instances can run in parallel to combine multiple classification tasks in real time. Finally, our tool is able use the slow peaking blood-oxygen-level dependent signal to classify our subjects' intention during the two-second window TR. We release this tool as open-source for non-commercial usage.

[1]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[2]  J. Platt Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[3]  P. Boesiger,et al.  Dynamic characteristics of oxygenation-sensitive MRI signal in different temporal protocols for imaging human brain activity , 2000, Neuroradiology.

[4]  S Thesen,et al.  Prospective acquisition correction for head motion with image‐based tracking for real‐time fMRI , 2000, Magnetic resonance in medicine.

[5]  R. Veit,et al.  Self-regulation of local brain activity using real-time functional magnetic resonance imaging (fMRI) , 2004, Journal of Physiology-Paris.

[6]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[7]  G. Rees,et al.  Neuroimaging: Decoding mental states from brain activity in humans , 2006, Nature Reviews Neuroscience.

[8]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[9]  Horst Bischof,et al.  Toward Self-Paced Brain–Computer Communication: Navigation Through Virtual Worlds , 2008, IEEE Transactions on Biomedical Engineering.

[10]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[11]  F. Jolesz,et al.  Brain–machine interface via real-time fMRI: Preliminary study on thought-controlled robotic arm , 2009, Neuroscience Letters.

[12]  Niels Birbaumer,et al.  Hemodynamic brain-computer interfaces for communication and rehabilitation , 2009, Neural Networks.

[13]  Niels Birbaumer,et al.  Real-time support vector classification and feedback of multiple emotional brain states , 2011, NeuroImage.

[14]  Max A. Viergever,et al.  Real-Time Decoding of Brain Responses to Visuospatial Attention Using 7T fMRI , 2011, PloS one.

[15]  Stephen LaConte,et al.  Decoding fMRI brain states in real-time , 2011, NeuroImage.

[16]  R. Goebel,et al.  A Real-Time fMRI-Based Spelling Device Immediately Enabling Robust Motor-Independent Communication , 2012, Current Biology.

[17]  M. Viergever,et al.  Navigation of a Telepresence Robot via Covert Visuospatial Attention and Real-Time fMRI , 2012, Brain Topography.

[18]  A. Kheddar,et al.  fMRI robotic embodiment: A pilot study , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[19]  Mark Chiew,et al.  Investigation of fMRI neurofeedback of differential primary motor cortex activity using kinesthetic motor imagery , 2012, NeuroImage.

[20]  Ludovico Minati,et al.  Thoughts turned into high-level commands: Proof-of-concept study of a vision-guided robot arm driven by functional MRI (fMRI) signals. , 2012, Medical engineering & physics.

[21]  Ivanei E. Bramati,et al.  Real-Time fMRI Pattern Decoding and Neurofeedback Using FRIEND: An FSL-Integrated BCI Toolbox , 2013, PloS one.

[22]  Sangkyun Lee,et al.  A toolbox for real-time subject-independent and subject-dependent classification of brain states from fMRI signals , 2013, Front. Neurosci..

[23]  Rafael Malach,et al.  Controlling an avatar by thought using real-time fMRI , 2014, Journal of neural engineering.

[24]  Nathan Intrator,et al.  An EEG Finger-Print of fMRI deep regional activation , 2014, NeuroImage.

[25]  Abderrahmane Kheddar,et al.  fMRI-Based Robotic Embodiment: Controlling a Humanoid Robot by Thought Using Real-Time fMRI , 2014, PRESENCE: Teleoperators and Virtual Environments.

[26]  Rodrigo Basilio,et al.  FRIEND Engine Framework: a real time neurofeedback client-server system for neuroimaging studies , 2015, Front. Behav. Neurosci..

[27]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[28]  Rafael Malach,et al.  High performance in brain-computer interface control of an avatar using the missing hand representation in long term amputees , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[29]  François Keith,et al.  Real-Time fMRI control of a humanoid robot using two brain Networks simultaneously: a Pilot Study , 2017, GBCIC.

[30]  S. Hallerb,et al.  Real-time fMRI neurofeedback : progress and challenges , 2017 .