Brain Machine Interface System Automation Considering User Preferences and Error Perception Feedback

This paper addresses the problem of mental fatigue caused by prolonged use of Brain Machine Interface (BMI) Systems. We propose a system that gradually becomes autonomous by learning user preferences and by considering error perception feedback. As a particular application, we show that our system allows patients to control electronic appliances in a hospital room, and learns the correlation of room sensor data, brain states, and user control commands. Moreover, error perception feedback based on a brain potential called error related negativity (ERN) - that spontaneously occurs when the user perceives an error made by the system - was used to correct system's mistakes and improve its learning performance. Experimental results with volunteers demonstrate that our system reduces the level of mental fatigue, and achieves over 90% overall learning performance when error perception feedback is considered. Note to Practitioners - This paper suggests a new approach for designing BMI systems that incorporate learning capabilities and error perception feedback in order to gradually become autonomous. This approach consists in learning the relationship between sensing data from the environment-brain and user actions when controlling robotic devices. After the system is trained, can predict control commands on behalf of the user under similar conditions. If the system makes a mistake, user's error perception feedback is considered to improve the learning performance the system. In this paper, we describe the methodologies to design and build hardware and software interfaces, acquire and process brain signals, and train the system using machine learning techniques. We then provide experimental evidence that demonstrates the effectiveness of this approach to design BMI systems that gradually become autonomous.

[1]  Mel Slater,et al.  Virtual Smart Home Controlled by Thoughts , 2009, 2009 18th IEEE International Workshops on Enabling Technologies: Infrastructures for Collaborative Enterprises.

[2]  Brian Potetz Estimating the Bayes Point Using Linear Knapsack Problems , 2011, ICML.

[3]  Tatsuo Arai,et al.  BMI-based learning system for appliance control automation , 2013, 2013 IEEE International Conference on Robotics and Automation.

[4]  Luis Montesano,et al.  Single trial recognition of error-related potentials during observation of robot operation , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[5]  R Chavarriaga,et al.  Learning From EEG Error-Related Potentials in Noninvasive Brain-Computer Interfaces , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[6]  Chi Thanh Vi,et al.  Detecting error-related negativity for interaction design , 2012, CHI.

[7]  Urbano Nunes,et al.  Assisted navigation based on shared-control, using discrete and sparse human-machine interfaces , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[8]  Rajesh P. N. Rao,et al.  A Hierarchical Architecture for Adaptive Brain-Computer Interfacing , 2011, IJCAI.

[9]  L. Rönnbäck,et al.  A self-assessment questionnaire for mental fatigue and related symptoms after neurological disorders and injuries , 2010, Brain injury.

[10]  M. Coles,et al.  Performance monitoring in a confusing world: error-related brain activity, judgments of response accuracy, and types of errors. , 2000, Journal of experimental psychology. Human perception and performance.

[11]  Anatole Lécuyer,et al.  Exploring Large Virtual Environments by Thoughts Using a BrainComputer Interface Based on Motor Imagery and High-Level Commands , 2010, PRESENCE: Teleoperators and Virtual Environments.

[12]  Tom Minka,et al.  Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[13]  M. Teplan FUNDAMENTALS OF EEG MEASUREMENT , 2002 .

[14]  Ferdinando Grossi,et al.  Home smart home: brain-computer interface control for real smart home environments , 2010 .

[15]  Rajesh P. N. Rao,et al.  An adaptive brain-computer interface for humanoid robot control , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[16]  W. Markesbery,et al.  Seven cases of cerebromedullospinal disconnection , 1971, Neurology.

[17]  E. Cardiel,et al.  Communication aid for speech disabled people using Morse codification , 2002, Proceedings of the Second Joint 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society] [Engineering in Medicine and Biology.

[18]  Wanderley Cardoso Celeste,et al.  Human–machine interface based on muscular and brain signals applied to a robotic wheelchair , 2007 .

[19]  H.T. Nguyen,et al.  Effects of mental fatigue on 8–13Hz brain activity in people with spinal cord injury , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[20]  Iñaki Iturrate,et al.  Robot reinforcement learning using EEG-based reward signals , 2010, 2010 IEEE International Conference on Robotics and Automation.

[21]  H. Bekkering,et al.  Modulation of activity in medial frontal and motor cortices during error observation , 2004, Nature Neuroscience.

[22]  Iñaki Iturrate,et al.  Synchronous EEG brain-actuated wheelchair with automated navigation , 2009, 2009 IEEE International Conference on Robotics and Automation.