Real-Time fMRI control of a humanoid robot using two brain Networks simultaneously: a Pilot Study

We have previously shown that real-time fMRI, despite the low temporal resolution of the BOLD signal, can be used for BCI navigation, using motorimagery and -execution. Here we leverage the superior spatial resolution of fMRI to implement a BCI paradigm going beyond a single brain network for control, while retaining an intuitive mapping between brain activity and BCI functionality. The experiments simulate non-trivial navigation and item selection tasks by a subject teleoperating an HRP-4 humanoid-robot. Motor actions are mapped into simple navigation commands inside a room and visual attention is mapped to direct the robot’s arm toward one of three objects placed on a table. When the correct item has been selected, the subject navigates the robot toward the experimenter in order to simulate the delivery of the object. We describe a method based on two parallel classifiers, with four and three classes (independent of the first four), offline and real-time classification results from a single-subject pilot, performing several times.