Development and Assessment of a Self-paced BCI-VR Paradigm Using Multimodal Stimulation and Adaptive Performance

Motor-Imagery based Brain-Computer Interfaces (BCIs) can provide alternative communication pathways to neurologically impaired patients. The combination of BCIs and Virtual Reality (VR) can provide induced illusions of movement to patients with low-level of motor control during motor rehabilitation tasks. Unfortunately, current BCI systems lack reliability and good performance levels in comparison with other types of computer interfaces. To date, there is little evidence on how BCI-based motor training needs to be designed for transferring rehabilitation improvements to real life. Based on our previous work, we showcase the development and assessment of NeuRow, a novel multiplatform immersive VR environment that makes use of multimodal stimulation through vision, sound and vibrotactile feedback and delivered through a VR Head Mounted Display. In addition, we integrated the Adaptive Performance Engine (APE), a statistical approach to optimize user control in a self-paced BCI-VR paradigm. In this paper, we describe the development and pilot assessment of NeuRow as well as its integration and assessment with APE.

[1]  G. Pfurtscheller,et al.  Graz-BCI: state of the art and clinical applications , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  D. Markland,et al.  Movement imagery ability: development and assessment of a revised version of the vividness of movement imagery questionnaire. , 2008, Journal of sport & exercise psychology.

[3]  Mark Bolas,et al.  Designing informed game-based rehabilitation tasks leveraging advances in virtual reality , 2012, Disability and rehabilitation.

[4]  Craig Hall,et al.  The MIQ-RS: A Suitable Option for Examining Movement Imagery Ability , 2007, Evidence-based complementary and alternative medicine : eCAM.

[5]  Aimee P. Reiss,et al.  Constraint-Induced Movement Therapy (CIMT): Current Perspectives and Future Directions , 2012, Stroke research and treatment.

[6]  Touradj Ebrahimi,et al.  Support vector EEG classification in the Fourier and time-frequency correlation domains , 2003, First International IEEE EMBS Conference on Neural Engineering, 2003. Conference Proceedings..

[7]  A. Pope,et al.  Biocybernetic system evaluates indices of operator engagement in automated task , 1995, Biological Psychology.

[8]  B. Dobkin.  Brain–computer interface technology as a tool to augment plasticity and outcomes for neurological rehabilitation , 2007, The Journal of physiology.

[9]  Sergi Bermúdez i Badia,et al.  Optimizing Performance of Non-Expert Users in Brain-Computer Interaction by Means of an Adaptive Performance Engine , 2015, BIH.

[10]  Antonio Frisoli,et al.  Illusory perception of arm movement induced by visuo-proprioceptive sensory stimulation and controlled by motor imagery , 2012, 2012 IEEE Haptics Symposium (HAPTICS).

[11]  Brendan Z. Allison,et al.  Could Anyone Use a BCI? , 2010, Brain-Computer Interfaces.

[12]  Mohammad Hassan Moradi,et al.  A new approach in the BCI research based on fractal dimension as feature and Adaboost as classifier , 2004, Journal of neural engineering.

[13]  J. Hattie,et al.  The Power of Feedback , 2007 .

[14]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[15]  Sergi Bermúdez i Badia,et al.  Usability and Cost-effectiveness in Brain-Computer Interaction: Is it User Throughput or Technology Related? , 2016, AH.

[16]  Michelle N. Lumicao,et al.  EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. , 2007, Aviation, space, and environmental medicine.

[17]  Matthew S. Grubb,et al.  Adult neurogenesis and functional plasticity in neuronal circuits , 2006, Nature Reviews Neuroscience.

[18]  Sergi Bermúdez i Badia,et al.  RehabNet: A distributed architecture for motor and cognitive neuro-rehabilitation , 2013, 2013 IEEE 15th International Conference on e-Health Networking, Applications and Services (Healthcom 2013).

[19]  Russell M. Taylor,et al.  VRPN: a device-independent, network-transparent VR peripheral system , 2001, VRST '01.

[20]  Takashi Hanakawa,et al.  Organizing motor imageries , 2016, Neuroscience Research.

[21]  José del R. Millán,et al.  Quantification and reduction of visual load during BCI operation , 2014, 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[22]  J. Peters,et al.  Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery , 2011, Journal of neural engineering.

[23]  P. Rossini,et al.  Post-stroke plastic reorganisation in the adult brain , 2003, The Lancet Neurology.

[24]  Doron Friedman,et al.  Brain-Computer Interfacing and Virtual Reality , 2015 .

[25]  R. Riener,et al.  Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review , 2012, Psychonomic Bulletin & Review.

[26]  H. Flor,et al.  A multimodal brain-based feedback and communication system , 2004, Experimental Brain Research.

[27]  Laura Mori,et al.  Pathophysiology of Spasticity: Implications for Neurorehabilitation , 2014, BioMed research international.

[28]  Benjamin Blankertz,et al.  Towards a Cure for BCI Illiteracy , 2009, Brain Topography.

[29]  V. Shute.  Focus on Formative Feedback , 2008 .

[30]  Christian Mühl,et al.  Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design , 2013, Front. Hum. Neurosci..

[31]  Christa Neuper,et al.  Hidden Markov models for online classification of single trial EEG data , 2001, Pattern Recognit. Lett..

[32]  G. Pfurtscheller,et al.  Motor imagery and action observation: Modulation of sensorimotor brain rhythms during mental control of a brain–computer interface , 2009, Clinical Neurophysiology.

[33]  Jianjun Meng,et al.  Combining Motor Imagery With Selective Sensation Toward a Hybrid-Modality BCI , 2014, IEEE Transactions on Biomedical Engineering.

[34]  Sergi Bermúdez i Badia,et al.  The Neurorehabilitation Training Toolkit (NTT): A Novel Worldwide Accessible Motor Training Approach for At-Home Rehabilitation after Stroke , 2012, Stroke research and treatment.

[35]  Sergi Bermúdez i Badia,et al.  NeuRow: An Immersive VR Environment for Motor-Imagery Training with the Use of Brain-Computer Interfaces and Vibrotactile Feedback , 2016, PhyCS.

[36]  Chi Thanh Vi,et al.  Continuous Tactile Feedback for Motor-Imagery Based Brain-Computer Interaction in a Multitasking Context , 2015, INTERACT.

[37]  José del R. Millán,et al.  Freeing the visual channel by exploiting vibrotactile BCI feedback , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[38]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[39]  Giuseppe Musumeci,et al.  The Role of Intrinsic Pathway in Apoptosis Activation and Progression in Peyronie's Disease , 2014, BioMed research international.

[40]  Stefan Vogt,et al.  Motor imagery during action observation modulates automatic imitation effects in rhythmical actions , 2014, Front. Hum. Neurosci..

[41]  Guillaume Gibert,et al.  OpenViBE: An Open-Source Software Platform to Design, Test, and Use BrainComputer Interfaces in Real and Virtual Environments , 2010, PRESENCE: Teleoperators and Virtual Environments.

[42]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[43]  T. Mulder.  Motor imagery and action observation: cognitive tools for rehabilitation , 2007, Journal of Neural Transmission.

[44]  Athanasios Vourvopoulos,et al.  Motor priming in virtual reality can augment motor-imagery training efficacy in restorative brain-computer interaction: a within-subject analysis , 2016, Journal of NeuroEngineering and Rehabilitation.

[45]  H. Jasper,et al.  The ten-twenty electrode system of the International Federation. The International Federation of Clinical Neurophysiology. , 1999, Electroencephalography and clinical neurophysiology. Supplement.

[46]  Anatole Lécuyer,et al.  Combining BCI with Virtual Reality: Towards New Applications and Improved BCI , 2012 .

[47]  Ruth Dickstein,et al.  Effects of integrated motor imagery practice on gait of individuals with chronic stroke: a half-crossover randomized study. , 2013, Archives of physical medicine and rehabilitation.

[48]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[49]  Febo Cincotti,et al.  Vibrotactile Feedback for Brain-Computer Interface Operation , 2007, Comput. Intell. Neurosci..

[50]  Fotis Liarokapis,et al.  Investigating the effect of user profile during training for BCI-based games , 2017, 2017 9th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games).

[51]  D. Kort,et al.  D3.3 : Game Experience Questionnaire:development of a self-report measure to assess the psychological impact of digital games , 2007 .

[52]  G. Pfurtscheller,et al.  Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. , 2005, Brain research. Cognitive brain research.

[53]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[54]  Mel Slater,et al.  Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.