Towards Artificial Learning Companions for Mental Imagery-based Brain-Computer Interfaces

Mental Imagery based Brain-Computer Interfaces (MI-BCI) enable their users to control an interface, e.g., a prosthesis, by performing mental imagery tasks only, such as imagining a right arm movement while their brain activity is measured and processed by the system. Designing and using a BCI requires users to learn how to produce different and stable patterns of brain activity for each of the mental imagery tasks. However, current training protocols do not enable every user to acquire the skills required to use BCIs. These training protocols are most likely one of the main reasons why BCIs remain not reliable enough for wider applications outside research laboratories. Learning companions have been shown to improve training in different disciplines, but they have barely been explored for BCIs so far. This article aims at investigating the potential benefits learning companions could bring to BCI training by improving the feedback, i.e., the information provided to the user, which is primordial to the learning process and yet have proven both theoretically and practically inadequate in BCI. This paper first presents the potentials of BCI and the limitations of current training approaches. Then, it reviews both the BCI and learning companion literature regarding three main characteristics of feedback: its appearance, its social and emotional components and its cognitive component. From these considerations, this paper draws some guidelines, identify open challenges and suggests potential solutions to design and use learning companions for BCIs.

[1]  V. Shute.  Focus on Formative Feedback , 2008 .

[2]  M. David Merrill,et al.  First principles of instruction , 2012 .

[3]  Fabien Lotte,et al.  Peanut: Personalised Emotional Agent for Neurotechnology User-Training , 2017, GBCIC.

[4]  Fabien Lotte,et al.  Towards Explanatory Feedback for User Training in Brain-Computer Interfaces , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[5]  W. Lewis Johnson,et al.  Politeness in Tutoring Dialogs: "Run the Factory, That's What I'd Do" , 2004, Intelligent Tutoring Systems.

[6]  J. Wolpaw,et al.  Brain-computer communication: unlocking the locked in. , 2001, Psychological bulletin.

[7]  K. Koedinger,et al.  Automated, Unobtrusive, Action-by-Action Assessment of Self-Regulation During Learning With an Intelligent Tutoring System , 2010 .

[8]  R. Sitaram,et al.  How feedback, motor imagery, and reward influence brain self‐regulation using real‐time fMRI , 2016, Human brain mapping.

[9]  K. Koedinger,et al.  Example-Tracing Tutors : A New Paradigm for Intelligent Tutoring Systems , 2008 .

[10]  Winslow Burleson,et al.  Gender-Specific Approaches to Developing Emotionally Intelligent Learning Companions , 2007, IEEE Intelligent Systems.

[11]  T. Gog,et al.  Example-Based Learning: Integrating Cognitive and Social-Cognitive Research Perspectives , 2010 .

[12]  milie,et al.  Why Standard Brain-Computer Interface ( BCI ) Training Protocols Should be Changed : An Experimental Study , 2016 .

[13]  Chen-Lin C. Kulik,et al.  The Instructional Effect of Feedback in Test-Like Events , 1991 .

[14]  Russell Beale,et al.  Affective interaction: How emotional agents affect users , 2009, Int. J. Hum. Comput. Stud..

[15]  Robin I. M. Dunbar,et al.  Evolution of the Social Brain , 2003, Science.

[16]  Anastasios A. Economides,et al.  The effect of emotional feedback on behavioral intention to use computer based assessment , 2012, Comput. Educ..

[17]  N. Birbaumer,et al.  Brain-computer communication: self-regulation of slow cortical potentials for verbal communication. , 2001, Archives of physical medicine and rehabilitation.

[18]  Christian Mühl,et al.  Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design , 2013, Front. Hum. Neurosci..

[19]  Matthew C Keller,et al.  Mental Exercising Through Simple Socializing: Social Interaction Promotes General Cognitive Functioning , 2008, Personality & social psychology bulletin.

[20]  A. Kübler,et al.  Motivation modulates the P300 amplitude during brain–computer interface use , 2010, Clinical Neurophysiology.

[21]  Jérémy Frey,et al.  Teegi: tangible EEG interface , 2014, UIST.

[22]  David W. Johnson,et al.  An Educational Psychology Success Story: Social Interdependence Theory and Cooperative Learning , 2009 .

[23]  Ramón Zataraín-Cabada,et al.  Fermat: Merging Affective Tutoring Systems with Learning Social Networks , 2012, 2012 IEEE 12th International Conference on Advanced Learning Technologies.

[24]  James C. Lester,et al.  Affective Transitions in Narrative-Centered Learning Environments , 2008, J. Educ. Technol. Soc..

[25]  Fabien Lotte,et al.  Online Classification accuracy is a Poor Metric to Study Mental imagery-based BCI User Learning: an Experimental Demonstration and New Metrics , 2017, GBCIC.

[26]  Donald A. Norman,et al.  How might people interact with agents , 1994, CACM.

[27]  Anton Nijholt,et al.  Social Interaction in a Cooperative Brain-Computer Interface Game , 2011, INTETAIN.

[28]  A. Nijholt,et al.  A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges , 2014 .

[29]  T. Sollfrank,et al.  The effect of multimodal and enriched feedback on SMR-BCI performance , 2016, Clinical Neurophysiology.

[30]  Christa Neuper,et al.  Neurofeedback Training for BCI Control , 2009 .

[31]  Mariya Timofeeva,et al.  Semiotic training for brain-computer interfaces , 2016, 2016 Federated Conference on Computer Science and Information Systems (FedCSIS).

[32]  Ricardo Ron-Angevin,et al.  Brain–computer interface: Changes in performance using virtual reality techniques , 2009, Neuroscience Letters.

[33]  Fabien Lotte,et al.  Towards a Cognitive Model of MI-BCI User Training , 2017, GBCIC.

[34]  F. Binkofski,et al.  Can Machines Think? Interaction and Perspective Taking with Robots Investigated via fMRI , 2008, PloS one.

[35]  Brian R. Duffy,et al.  Anthropomorphism and the social robot , 2003, Robotics Auton. Syst..

[36]  Anatole Lécuyer,et al.  Author manuscript, published in "IEEE Transactions on Computational Intelligence and AI in games (2013)" Two Brains, One Game: Design and Evaluation of a Multi-User BCI Video Game Based on Motor Imagery , 2022 .

[37]  Roderick Murray-Smith,et al.  Visually Multimodal vs. Classic Unimodal Feedback Approach for SMR-BCIs: A Comparison Study , 2011 .

[38]  Michael Schmitz,et al.  Tangible interaction with anthropomorphic smart objects in instrumented environments , 2010 .

[39]  D. Goleman.  Emotional Intelligence: Why It Can Matter More Than IQ , 1995 .

[40]  Larry Ambrose,et al.  The power of feedback. , 2002, Healthcare executive.

[41]  Fabien Lotte,et al.  Brain-Computer Interfaces: Beyond Medical Applications , 2012, Computer.

[42]  Jarrod A. Lewis-Peacock,et al.  Closed-loop brain training: the science of neurofeedback , 2017, Nature Reviews Neuroscience.

[43]  C. Neuper,et al.  Combining Brain–Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges , 2010, Front. Neurosci..

[44]  Fabien Lotte.  Towards Usable Electroencephalography-based Brain-Computer Interfaces , 2016 .

[45]  Cuntai Guan,et al.  Brain–Computer Interface for Neurorehabilitation of Upper Limb After Stroke , 2015, Proceedings of the IEEE.

[46]  Mukesh K. Mohania,et al.  Modeling user behavior data in systems of engagement , 2017, Future Gener. Comput. Syst..

[47]  Vincent Aleven,et al.  A New Paradigm for Intelligent Tutoring Systems: Example-Tracing Tutors , 2009, Int. J. Artif. Intell. Educ..

[48]  Maureen Clerc,et al.  Brain-Computer Interfaces 1: Foundations and Methods , 2016 .

[49]  Eva Hornecker,et al.  The role of physicality in tangible and embodied interactions , 2011, INTR.

[50]  Chi Thanh Vi,et al.  Continuous Tactile Feedback for Motor-Imagery Based Brain-Computer Interaction in a Multitasking Context , 2015, INTERACT.

[51]  Antonija Mitrovic.  Modeling Domains and Students with Constraint-Based Modeling , 2010, Advances in Intelligent Tutoring Systems.

[52]  M. Doherty,et al.  Effects of cognitive feedback on performance. , 1989 .

[53]  Maureen Clerc,et al.  Brain-Computer Interfaces 2: Technology and Applications , 2016 .

[54]  Thorsten O. Zander,et al.  Detecting affective covert user states with passive brain-computer interfaces , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[55]  C. A. Sexton.  The overlooked potential for social factors to improve effectiveness of brain-computer interfaces , 2015, Front. Syst. Neurosci..

[56]  J. Keller.  An Integrative Theory of Motivation, Volition, and Performance , 2008 .

[57]  N. Palomero-Gallagher,et al.  Social reward improves the voluntary control over localized brain activity in fMRI-based neurofeedback training , 2015, Front. Behav. Neurosci..

[58]  Michael D. Robinson,et al.  Belief and feeling: evidence for an accessibility model of emotional self-report. , 2002, Psychological bulletin.

[59]  Fabien Lotte,et al.  Towards improved BCI based on human learning principles , 2015, The 3rd International Winter Conference on Brain-Computer Interface.

[60]  N. Sadato,et al.  Processing of Social and Monetary Rewards in the Human Striatum , 2008, Neuron.

[61]  Robert Rosenthal,et al.  People Studying People: Artifacts and Ethics in Behavioral Research , 1997 .

[62]  Chi-Jen Lin,et al.  Redefining the learning companion: the past, present, and future of educational agents , 2003, Comput. Educ..

[63]  Roger Azevedo,et al.  A Meta-Analysis of the Effects of Feedback in Computer-Based Instruction , 1995 .

[64]  Cynthia Breazeal,et al.  Affective Personalization of a Social Robot Tutor for Children's Second Language Skills , 2016, AAAI.

[65]  E. Deci,et al.  Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. , 2000, The American psychologist.

[66]  Sue Ellen Williams.  Teachers' Written Comments and Students' Responses: A Socially Constructed Interaction. , 1996 .

[67]  Joseph E LeDoux.  Emotion: clues from the brain. , 1995, Annual review of psychology.

[68]  V. Caggiano,et al.  Proprioceptive Feedback and Brain Computer Interface (BCI) Based Neuroprostheses , 2012, PloS one.

[69]  Susanne Narciss,et al.  How to design informative tutoring feedback for multi-media learning , 2004 .

[70]  James C. Lester,et al.  The persona effect: affective impact of animated pedagogical agents , 1997, CHI.

[71]  D. McFarland,et al.  An auditory brain–computer interface (BCI) , 2008, Journal of Neuroscience Methods.

[72]  Fabien Lotte,et al.  Human Learning for Brain-Computer Interfaces , 2016 .

[73]  A. Isen,et al.  An Influence of Positive Affect on Decision Making in Complex Situations: Theoretical Issues With Practical Implications , 2001 .

[74]  Jérémie Mattout,et al.  Brain-Computer Interfaces: A Neuroscience Paradigm of Social Interaction? A Matter of Perspective , 2012, Front. Hum. Neurosci..

[75]  Stefan Wermter,et al.  Teaching emotion expressions to a human companion robot using deep neural architectures , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[76]  Toshiyuki Kondo,et al.  Effect of instructive visual stimuli on neurofeedback training for motor imagery-based brain-computer interface. , 2015, Human movement science.

[77]  G. Pfurtscheller,et al.  Brain–Computer Communication: Motivation, Aim, and Impact of Exploring a Virtual Apartment , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[78]  Yanghee Kim,et al.  Pedagogical Agents as Learning Companions: Building Social Relations with Learners , 2005, AIED.

[79]  Yu. I. Lyubich.  Foundations and Methods , 1992 .

[80]  Gert Pfurtscheller,et al.  Motor imagery and direct brain-computer communication , 2001, Proc. IEEE.

[81]  Günter Edlinger,et al.  Social Environments, Mixed Communication and Goal-Oriented Control Application Using a Brain-Computer Interface , 2011, HCI.

[82]  Christa Neuper,et al.  Control beliefs can predict the ability to up-regulate sensorimotor rhythm during neurofeedback training , 2013, Front. Hum. Neurosci..

[83]  Maarten De Vos,et al.  Lateralization patterns of covert but not overt movements change with age: An EEG neurofeedback study , 2015, NeuroImage.

[84]  Michitaka Hirose,et al.  Brain-Computer Interfaces, Virtual Reality, and Videogames , 2008, Computer.

[85]  Christoph Bartneck,et al.  Expressive robots in education: varying the degree of social supportive behavior of a robotic tutor , 2010, CHI.

[86]  Maud Marchal,et al.  The Mind-Mirror: See your brain in action in your head using EEG and augmented reality , 2014, 2014 IEEE Virtual Reality (VR).