Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design

While recent research on Brain-Computer Interfaces (BCI) has highlighted their potential for many applications, they remain barely used outside laboratories. The main reason is their lack of robustness. Indeed, with current BCI, mental state recognition is usually slow and often incorrect. Spontaneous BCI (i.e., mental imagery-based BCI) often rely on mutual learning efforts by the user and the machine, with BCI users learning to produce stable ElectroEncephaloGraphy (EEG) patterns (spontaneous BCI control being widely acknowledged as a skill) while the computer learns to automatically recognize these EEG patterns, using signal processing. Most research so far was focused on signal processing, mostly neglecting the human in the loop. However, how well the user masters the BCI skill is also a key element explaining BCI robustness. Indeed, if the user is not able to produce stable and distinct EEG patterns, then no signal processing algorithm would be able to recognize them. Unfortunately, despite the importance of BCI training protocols, they have been scarcely studied so far, and used mostly unchanged for years. In this paper, we advocate that current human training approaches for spontaneous BCI are most likely inappropriate. We notably study instructional design literature in order to identify the key requirements and guidelines for a successful training procedure that promotes a good and efficient skill learning. This literature study highlights that current spontaneous BCI user training procedures satisfy very few of these requirements and hence are likely to be suboptimal. We therefore identify the flaws in BCI training protocols according to instructional design principles, at several levels: in the instructions provided to the user, in the tasks he/she has to perform, and in the feedback provided. For each level, we propose new research directions that are theoretically expected to address some of these flaws and to help users learn the BCI skill more efficiently.

[1]  M. Congedo,et al.  Open-ViBE: A Three Dimensional Platform for Real-Time Neuroscience , 2005 .

[2]  K. Müller,et al.  Psychological predictors of SMR-BCI performance , 2012, Biological Psychology.

[3]  S. Ainsworth DeFT: A Conceptual Framework for Considering Learning with Multiple Representations. , 2006 .

[4]  Eunjoon Rachel Um,et al.  Emotional design in multimedia learning. , 2012 .

[5]  C. S. Green,et al.  Action video game modifies visual selective attention , 2003, Nature.

[6]  E. Deci,et al.  Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. , 2000, The American psychologist.

[7]  Han-Jeong Hwang,et al.  Neurofeedback-based motor imagery training for brain–computer interface (BCI) , 2009, Journal of Neuroscience Methods.

[8]  Febo Cincotti,et al.  Vibrotactile Feedback for Brain-Computer Interface Operation , 2007, Comput. Intell. Neurosci..

[9]  C. Neuper,et al.  Long-term evaluation of a 4-class imagery-based brain–computer interface , 2013, Clinical Neurophysiology.

[10]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[11]  Vera Kaiser,et al.  Motivation Influences Performance in SMR-BCI , 2011 .

[12]  N. Birbaumer,et al.  Brain-computer communication: self-regulation of slow cortical potentials for verbal communication. , 2001, Archives of physical medicine and rehabilitation.

[13]  Christa Neuper,et al.  Walking by Thinking: The Brainwaves Are Crucial, Not the Muscles! , 2006, PRESENCE: Teleoperators and Virtual Environments.

[14]  Christa Neuper,et al.  Autocalibration and Recurrent Adaptation: Towards a Plug and Play Online ERD-BCI , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  D J McFarland,et al.  An EEG-based brain-computer interface for cursor control. , 1991, Electroencephalography and clinical neurophysiology.

[16]  Klaus-Robert Müller,et al.  Machine-Learning-Based Coadaptive Calibration for Brain-Computer Interfaces , 2011, Neural Computation.

[17]  Anatole Lécuyer,et al.  Combining BCI with Virtual Reality: Towards New Applications and Improved BCI , 2012 .

[18]  Tomasz M. Rutkowski,et al.  Imagery Movement Paradigm User Adaptation Improvement with Quasi-movements Phenomenon , 2011 .

[19]  Clifford Nass,et al.  The media equation - how people treat computers, television, and new media like real people and places , 1996 .

[20]  Fabien Lotte,et al.  Brain computer interface vs walking interface in VR: the impact of motor activity on spatial transfer , 2012, VRST '12.

[21]  J. Wolpaw,et al.  Towards an independent brain–computer interface using steady state visual evoked potentials , 2008, Clinical Neurophysiology.

[22]  Michitaka Hirose,et al.  Brain-Computer Interfaces, Virtual Reality, and Videogames , 2008, Computer.

[23]  Christa Neuper,et al.  Neurofeedback Training for BCI Control , 2009 .

[24]  H. Flor,et al.  A spelling device for the paralysed , 1999, Nature.

[25]  Wolfgang Rosenstiel,et al.  Neural mechanisms of brain–computer interface control , 2011, NeuroImage.

[26]  Dirk Heylen,et al.  Brain-Computer Interfacing and Games , 2010, Brain-Computer Interfaces.

[27]  Ricardo Ron-Angevin,et al.  Brain–computer interface: Changes in performance using virtual reality techniques , 2009, Neuroscience Letters.

[28]  B. Varkuti,et al.  Prediction of brain-computer interface aptitude from individual brain structure , 2013, Front. Hum. Neurosci..

[29]  N. Birbaumer,et al.  The thought translation device: a neurophysiological approach to communication in total motor paralysis , 1999, Experimental Brain Research.

[30]  Anton Nijholt,et al.  Turning Shortcomings into Challenges: Brain-Computer Interfaces for Games , 2009, INTETAIN.

[31]  D. McFarland,et al.  An auditory brain–computer interface (BCI) , 2008, Journal of Neuroscience Methods.

[32]  R. Mayer,et al.  Interactive Multimodal Learning Environments , 2007 .

[33]  Klaus-Robert Müller,et al.  Neurophysiological predictor of SMR-based BCI performance , 2010, NeuroImage.

[34]  Desney S. Tan,et al.  Brain-Computer Interfaces: Applying our Minds to Human-Computer Interaction , 2010 .

[35]  Anatole Lécuyer,et al.  Author manuscript, published in "IEEE Transactions on Computational Intelligence and AI in games (2013)" Two Brains, One Game: Design and Evaluation of a Multi-User BCI Video Game Based on Motor Imagery , 2022 .

[36]  Roderick Murray-Smith,et al.  Visually Multimodal vs. Classic Unimodal Feedback Approach for SMR-BCIs: A Comparison Study , 2011 .

[37]  Cuntai Guan,et al.  Comparison of designs towards a subject-independent brain-computer interface based on motor imagery , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[38]  Gabriel Curio,et al.  Quasi-movements: A novel motor–cognitive phenomenon , 2008, Neuropsychologia.

[39]  V. Shute Focus on Formative Feedback , 2007 .

[40]  Larry Ambrose,et al.  The power of feedback. , 2002, Healthcare executive.

[41]  M. Congedo,et al.  Open-ViBE: a 3D Platform for Real-Time Neuroscience , 2004 .

[42]  Joshua A. Granek,et al.  Extensive video-game experience alters cortical networks for complex visuomotor transformations , 2010, Cortex.

[43]  Fabien Lotte,et al.  A new feature and associated optimal spatial filter for EEG signal classification: Waveform Length , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[44]  J. Peters,et al.  Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery , 2011, Journal of neural engineering.

[45]  K.-R. Muller,et al.  The Berlin brain-computer interface: EEG-based communication without subject training , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[46]  N. Birbaumer,et al.  Predictability of Brain-Computer Communication , 2004 .

[47]  G. Pfurtscheller,et al.  Enhancement of left-right sensorimotor EEG differences during feedback-regulated motor imagery. , 1999, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[48]  Fabien Lotte,et al.  Brain-Computer Interfaces: Beyond Medical Applications , 2012, Computer.

[49]  H. Flor,et al.  A multimodal brain-based feedback and communication system , 2004, Experimental Brain Research.

[50]  Adriane B. Randolph Not All Created Equal: Individual-Technology Fit of Brain-Computer Interfaces , 2012, 2012 45th Hawaii International Conference on System Sciences.

[51]  Jukka Heikkonen,et al.  A local neural classifier for the recognition of EEG patterns associated to mental tasks , 2002, IEEE Trans. Neural Networks.

[52]  M. Grosse-Wentrup,et al.  Biased feedback in brain-computer interfaces , 2010, Journal of NeuroEngineering and Rehabilitation.

[53]  Brendan Z. Allison,et al.  Could Anyone Use a BCI? , 2010, Brain-Computer Interfaces.

[54]  J R Wolpaw,et al.  EEG-based communication and control: short-term role of feedback. , 1998, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[55]  Remo Guidieri Res , 1995, RES: Anthropology and Aesthetics.

[56]  V. Caggiano,et al.  Proprioceptive Feedback and Brain Computer Interface (BCI) Based Neuroprostheses , 2012, PloS one.

[57]  Klaus-Robert Müller,et al.  Subject independent EEG-based BCI decoding , 2009, NIPS.

[58]  D. Simons,et al.  The effects of video game playing on attention, memory, and executive control. , 2008, Acta psychologica.

[59]  Gert Pfurtscheller,et al.  Motor imagery and direct brain-computer communication , 2001, Proc. IEEE.

[60]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[61]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[62]  G. Pfurtscheller,et al.  Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. , 2005, Brain research. Cognitive brain research.

[63]  J. Pratt,et al.  Playing an Action Video Game Reduces Gender Differences in Spatial Cognition , 2007, Psychological science.

[64]  G. R. Muller,et al.  Clinical application of an EEG-based brain–computer interface: a case study in a patient with severe motor impairment , 2003, Clinical Neurophysiology.

[65]  W. A. Sarnacki,et al.  Electroencephalographic (EEG) control of three-dimensional movement , 2010, Journal of neural engineering.

[66]  Brendan Z. Allison,et al.  P300 brain computer interface: current challenges and emerging trends , 2012, Front. Neuroeng..

[67]  Christa Neuper,et al.  Rehabilitation with Brain-Computer Interface Systems , 2008, Computer.

[68]  Rosalind W. Picard,et al.  An affective model of interplay between emotions and learning: reengineering educational pedagogy-building a learning companion , 2001, Proceedings IEEE International Conference on Advanced Learning Technologies.