Online performance evaluation of motor imagery BCI with augmented-reality virtual hand feedback

The online performance of a motor imagery-based Brain-Computer Interface (MI-BCI) influences its effectiveness and usability in real-world clinical applications such as the restoration of motor control. The online performance depends on factors such as the different feedback techniques and motivation of the subject. This paper investigates the online performance of the MI-BCI with an augmented-reality (AR) 3D virtual hand feedback. The subject experiences the interaction with 3D virtual hands, which have been superimposed onto his real hands and displayed on the computer monitor from a first person point-of-view. While performing motor imagery, he receives continuous visual feedback from the MI-BCI in the form of different degrees of reaching and grasping actions of the 3D virtual hands with other virtual objects. The AR feedback is compared with the conventional horizontal bar feedback on 8 subjects, of whom 7 are BCI-naïve. The subjects found the AR feedback to be more engaging and motivating. Despite the higher mental workload involved in the AR feedback, their online MI-BCI performance compared to the conventional horizontal bar feedback was not affected. The results provide motivation to further develop and refine the AR feedback protocol for MI-BCI.

[1]  G. Pfurtscheller,et al.  Motor imagery and action observation: Modulation of sensorimotor brain rhythms during mental control of a brain–computer interface , 2009, Clinical Neurophysiology.

[2]  Mel Slater,et al.  Inducing a virtual hand ownership illusion through a brain–computer interface , 2009, Neuroreport.

[3]  J. Baron,et al.  Motor Imagery: A Backdoor to the Motor System After Stroke? , 2006, Stroke.

[4]  C. Wolfe,et al.  Qualitative analysis of stroke patients' motivation for rehabilitation , 2000, BMJ : British Medical Journal.

[5]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[6]  J. Wolpaw,et al.  Brain–computer interfaces in neurological rehabilitation , 2008, The Lancet Neurology.

[7]  Cuntai Guan,et al.  A clinical evaluation of non-invasive motor imagery-based brain-computer interface in stroke , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[8]  Klaus-Robert Müller,et al.  The non-invasive Berlin Brain–Computer Interface: Fast acquisition of effective performance in untrained subjects , 2007, NeuroImage.

[9]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[10]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[11]  Cuntai Guan,et al.  Multi-class filter bank common spatial pattern for four-class motor imagery BCI , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  J R Wolpaw,et al.  EEG-based communication and control: short-term role of feedback. , 1998, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[13]  J. Pineda,et al.  Learning to control brain rhythms: making a brain-computer interface possible , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[14]  G. Pfurtscheller,et al.  How many people are able to operate an EEG-based brain-computer interface (BCI)? , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  Christa Neuper,et al.  Rehabilitation with Brain-Computer Interface Systems , 2008, Computer.

[16]  G. Pfurtscheller,et al.  Brain–Computer Communication: Motivation, Aim, and Impact of Exploring a Virtual Apartment , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[17]  Horst Bischof,et al.  Toward Self-Paced Brain–Computer Communication: Navigation Through Virtual Worlds , 2008, IEEE Transactions on Biomedical Engineering.

[18]  Stefan Geyer,et al.  Imagery of voluntary movement of fingers, toes, and tongue activates corresponding body-part-specific motor representations. , 2003, Journal of neurophysiology.

[19]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[20]  Clemens Brunner,et al.  Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks , 2006, NeuroImage.

[21]  J D Bayliss,et al.  A virtual reality testbed for brain-computer interface research. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.