Multi-Session Influence of Two Modalities of Feedback and Their Order of Presentation on MI-BCI User Training

By performing motor-imagery tasks, for example, imagining hand movements, Motor-Imagery based Brain-Computer Interfaces (MI-BCIs) users can control digital technologies, for example, neuroprosthesis, using their brain activity only. MI-BCI users need to train, usually using a unimodal visual feedback, to produce brain activity patterns that are recognizable by the system. The literature indicates that multimodal vibrotactile and visual feedback is more effective than unimodal visual feedback, at least for short term training. However, the multi-session influence of such multimodal feedback on MI-BCI user training remained unknown, so did the influence of the order of presentation of the feedback modalities. In our experiment, 16 participants trained to control a MI-BCI during five sessions with a realistic visual feedback and five others with both a realistic visual feedback and a vibrotactile one. training benefits from a multimodal feedback, in terms of performances and self-reported mindfulness. There is also a significant influence of the order presentation of the modality. Participants who started training with a visual feedback had higher performances than those who started training with a multimodal feedback. We recommend taking into account the order of presentation for future experiments assessing the influence of several modalities of feedback.

[1]  Aleksandra Vuckovic,et al.  Using a motor imagery questionnaire to estimate the performance of a Brain–Computer Interface based on object oriented motor imagery , 2013, Clinical Neurophysiology.

[2]  Mircea Ariel Schoenfeld,et al.  Mental chronometry and mental rotation abilities in stroke patients with different degrees of sensory deficit. , 2016, Restorative neurology and neuroscience.

[3]  J. Ushiba,et al.  Daily training with realistic visual feedback improves reproducibility of event-related desynchronisation following hand motor imagery , 2013, Clinical Neurophysiology.

[4]  Raymond B. Cattell,et al.  Personality Structure and the New Fifth Edition of the 16PF , 1995 .

[5]  A. Kübler,et al.  Week-long visuomotor coordination and relaxation trainings do not increase sensorimotor rhythms (SMR) based brain–computer interface performance , 2019, Behavioural Brain Research.

[6]  Motoaki Kawanabe,et al.  Toward Unsupervised Adaptation of LDA for Brain–Computer Interfaces , 2011, IEEE Transactions on Biomedical Engineering.

[7]  François-Benoît Vialatte,et al.  An Apparatus to Investigate Western Opera Singing Skill Learning Using Performance and Result Biofeedback, and Measuring its Neural Correlates , 2017, INTERSPEECH.

[8]  J. J. Zhang,et al.  Immediate and long-term effects of BCI-based rehabilitation of the upper extremity after stroke: a systematic review and meta-analysis , 2020, Journal of NeuroEngineering and Rehabilitation.

[9]  S. Vandenberg,et al.  Mental Rotations, a Group Test of Three-Dimensional Spatial Visualization , 1978, Perceptual and motor skills.

[10]  Tele Tan,et al.  3D visualization of movements can amplify motor cortex activation during subsequent motor imagery , 2015, Front. Hum. Neurosci..

[11]  J. Ushiba,et al.  Subjective Vividness of Kinesthetic Motor Imagery Is Associated With the Similarity in Magnitude of Sensorimotor Event-Related Desynchronization Between Motor Execution and Motor Imagery , 2018, Front. Hum. Neurosci..

[12]  Maureen Clerc,et al.  An analysis of performance evaluation for motor-imagery based BCI , 2013, Journal of neural engineering.

[14]  Antonio Frisoli,et al.  Effects of Continuous Kinaesthetic Feedback Based on Tendon Vibration on Motor Imagery BCI Performance , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  Shuichi Nishio,et al.  Brain-Computer Interface and Motor Imagery Training: The Role of Visual Feedback and Embodiment , 2018, Evolving BCI Therapy - Engaging Brain State Dynamics.

[16]  Gonzalo G. de Polavieja,et al.  Active control as evidence in favor of sense of ownership in the moving Virtual Hand Illusion , 2019, Consciousness and Cognition.

[17]  Kathrin Cohen Kadosh,et al.  A systematic review of the psychological factors that influence neurofeedback learning outcomes , 2019, NeuroImage.

[18]  F Lotte,et al.  Advances in user-training for mental-imagery-based BCI control: Psychological and cognitive factors and their neural correlates. , 2016, Progress in brain research.

[19]  Chi Thanh Vi,et al.  Continuous Tactile Feedback for Motor-Imagery Based Brain-Computer Interaction in a Multitasking Context , 2015, INTERACT.

[20]  T. Sollfrank,et al.  The effect of multimodal and enriched feedback on SMR-BCI performance , 2016, Clinical Neurophysiology.

[21]  V. Shute.  Focus on Formative Feedback , 2008 .

[22]  Clemens Brunner,et al.  Better than random? A closer look on BCI results , 2008 .

[23]  Fabien Lotte,et al.  Experimenters' Influence on Mental-Imagery based Brain-Computer Interface User Training , 2021, Int. J. Hum. Comput. Stud..

[24]  Brendan Z. Allison,et al.  Could Anyone Use a BCI? , 2010, Brain-Computer Interfaces.

[25]  A. Ramond,et al.  Translation and validation of the French Movement Imagery Questionnaire - Revised Second Version (MIQ-RS). , 2013, Annals of physical and rehabilitation medicine.

[26]  Guillaume Gibert,et al.  OpenViBE: An Open-Source Software Platform to Design, Test, and Use BrainComputer Interfaces in Real and Virtual Environments , 2010, PRESENCE: Teleoperators and Virtual Environments.

[27]  C. Richards,et al.  The Kinesthetic and Visual Imagery Questionnaire (KVIQ) for Assessing Motor Imagery in Persons with Physical Disabilities: A Reliability and Construct Validity Study , 2007, Journal of neurologic physical therapy : JNPT.

[28]  Febo Cincotti,et al.  Vibrotactile Feedback for Brain-Computer Interface Operation , 2007, Comput. Intell. Neurosci..

[29]  Jiping He,et al.  Journal of Neuroengineering and Rehabilitation Open Access Recent Developments in Biofeedback for Neuromotor Rehabilitation Review of Early Biofeedback Therapy Current Developments in Biofeedback in Neurorehabilitation Table 1: Function of Basic Modules in Multisensing Biofeedback Systems for Task T , 2022 .

[30]  J. Millán,et al.  Brain‐computer interfaces for post‐stroke motor rehabilitation: a meta‐analysis , 2018, Annals of clinical and translational neurology.

[31]  Paolo Bifulco,et al.  Investigating the role of combined acoustic-visual feedback in one-dimensional synchronous brain computer interfaces, a preliminary study , 2012, Medical devices.

[32]  Sriram Subramanian,et al.  Predicting Mental Imagery-Based BCI Performance from Personality, Cognitive Profile and Neurophysiological Patterns , 2015, PloS one.

[33]  Olaf Blanke,et al.  Quantifying the role of motor imagery in brain-machine interfaces , 2016, Scientific Reports.

[34]  Athanasios Vourvopoulos,et al.  Motor priming in virtual reality can augment motor-imagery training efficacy in restorative brain-computer interaction: a within-subject analysis , 2016, Journal of NeuroEngineering and Rehabilitation.

[35]  Shuichi Nishio,et al.  Effect of biased feedback on motor imagery learning in BCI-teleoperation system , 2014, Front. Syst. Neurosci..

[36]  Michitaka Hirose,et al.  Brain-Computer Interfaces, Virtual Reality, and Videogames , 2008, Computer.

[37]  J. Peters,et al.  Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery , 2011, Journal of neural engineering.

[38]  Anatole Lécuyer,et al.  Author manuscript, published in "IEEE Transactions on Computational Intelligence and AI in games (2013)" Two Brains, One Game: Design and Evaluation of a Multi-User BCI Video Game Based on Motor Imagery , 2022 .

[39]  Christian Mühl,et al.  Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design , 2013, Front. Hum. Neurosci..

[40]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[41]  Simanto Saha,et al.  Intra- and Inter-subject Variability in EEG-Based Sensorimotor Brain Computer Interface: A Review , 2020, Frontiers in Computational Neuroscience.

[42]  Fabien Lotte,et al.  A physical learning companion for Mental-Imagery BCI User Training , 2020, Int. J. Hum. Comput. Stud..

[43]  J. Lefaucheur,et al.  Relieving peripheral neuropathic pain by increasing the power-ratio of low-β over high-β activities in the central cortical region with EEG-based neurofeedback: Study protocol for a controlled pilot trial (SMRPain study) , 2020, Neurophysiologie Clinique.

[44]  F. Cabestaing,et al.  EEG neurofeedback research: A fertile ground for psychiatry? , 2019, L'Encephale.

[45]  R. Riener,et al.  Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review , 2012, Psychonomic Bulletin & Review.

[46]  Xinjun Sheng,et al.  Tactile Stimulation Improves Sensorimotor Rhythm-Based BCI Performance in Stroke Patients , 2019, IEEE Transactions on Biomedical Engineering.

[47]  M. V. Lukoyanov,et al.  The Efficiency of the Brain-Computer Interfaces Based on Motor Imagery with Tactile and Visual Feedback , 2018, Human Physiology.

[48]  Silvia Erika Kober,et al.  EEG Neurofeedback Is Under Strong Control of Psychosocial Factors , 2018, Applied Psychophysiology and Biofeedback.

[49]  F. Liarokapis,et al.  Progressive Training for Motor Imagery Brain-Computer Interfaces Using Gamification and Virtual Reality Embodiment , 2019, Front. Hum. Neurosci..