Task level hierarchical system for BCI-enabled shared autonomy

This paper describes a novel hierarchical system for shared control of a humanoid robot. Our framework uses a low-bandwidth Brain Computer Interface (BCI) to interpret electroencephalography (EEG) signals via Steady-State Visual Evoked Potentials (SSVEP). This BCI allows a user to reliably interact with the humanoid. Our system clearly delineates between autonomous robot operation and human-guided intervention and control. Our shared-control system leverages the ability of the robot to accomplish low level tasks on its own, while the user assists the robot with high level directions when needed. This partnership prevents fatigue of the human controller by not requiring continuous BCI control to accomplish tasks which can be automated. We have tested the system in simulation and in real physical settings with multiple subjects using a Fetch mobile manipulator. Working together, the robot and human controller were able to accomplish tasks such as navigation, pick and place, and table clean up.

[1]  Hendrik Van Brussel,et al.  Shared Autonomy for Wheel Chair Control: Attempts to Assess the User's Autonomy , 2001, AMS.

[2]  Peter K. Allen,et al.  Graspit! A versatile simulator for robotic grasping , 2004, IEEE Robotics & Automation Magazine.

[3]  Gert Pfurtscheller,et al.  Characterization of four-class motor imagery EEG data for the BCI-competition 2005 , 2005, Journal of neural engineering.

[4]  N. Crone,et al.  High-frequency gamma oscillations and human brain mapping with electrocorticography. , 2006, Progress in brain research.

[5]  M Fatourechi,et al.  A self-paced brain–computer interface system with a low false positive rate , 2008, Journal of neural engineering.

[6]  Reid G. Simmons,et al.  Robotic Systems Architectures and Programming , 2008, Springer Handbook of Robotics.

[7]  Rajesh P. N. Rao,et al.  Control of a humanoid robot by a noninvasive brain–computer interface in humans , 2008, Journal of neural engineering.

[8]  Xiaorong Gao,et al.  An online multi-channel SSVEP-based brain–computer interface using a canonical correlation analysis method , 2009, Journal of neural engineering.

[9]  Rajesh P. N. Rao,et al.  An adaptive brain-computer interface for humanoid robot control , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[10]  G. Glover.  Overview of functional magnetic resonance imaging. , 2011, Neurosurgery clinics of North America.

[11]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  Brendan Z. Allison,et al.  P300 brain computer interface: current challenges and emerging trends , 2012, Front. Neuroeng..

[13]  Sungho Jo,et al.  A Low-Cost EEG System-Based Hybrid Brain-Computer Interface for Humanoid Robot Navigation and Recognition , 2013, PloS one.

[14]  Yili Liu,et al.  EEG-Based Brain-Controlled Mobile Robots: A Survey , 2013, IEEE Transactions on Human-Machine Systems.

[15]  L. Carin,et al.  Relationship between intracortical electrode design and chronic recording function. , 2013, Biomaterials.

[16]  Siddhartha S. Srinivasa,et al.  A policy-blending formalism for shared control , 2013, Int. J. Robotics Res..

[17]  Nikhil Rasiwasia,et al.  Cluster Canonical Correlation Analysis , 2014, AISTATS.

[18]  Yili Liu,et al.  A speed and direction-based cursor control system with P300 and SSVEP , 2014, Biomed. Signal Process. Control..

[19]  Sungho Tak,et al.  Statistical analysis of fNIRS data: A comprehensive review , 2014, NeuroImage.

[20]  A. Nijholt,et al.  A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges , 2014 .

[21]  Long Chen,et al.  An online hybrid brain-computer interface combining multiple physiological signals for webpage browse , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[22]  Tzyy-Ping Jung,et al.  High-speed spelling with a noninvasive brain–computer interface , 2015, Proceedings of the National Academy of Sciences.

[23]  Robertas Damasevicius,et al.  A Prototype SSVEP Based Real Time BCI Gaming System , 2016, Comput. Intell. Neurosci..

[24]  Daniel King,et al.  Fetch & Freight : Standard Platforms for Service Robot Applications , 2016 .

[25]  Yuanqing Li,et al.  Control of a Wheelchair in an Indoor Environment Based on a Brain–Computer Interface and Automated Navigation , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[26]  Tiago H. Falk,et al.  Recent advances and open challenges in hybrid brain-computer interfacing: a technological review of non-invasive human research , 2016 .

[27]  Brent Lance,et al.  Collaborative image triage with humans and computer vision , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[28]  Gordon Cheng,et al.  A neuro-based method for detecting context-dependent erroneous robot action , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[29]  Chad DeChant,et al.  Shape completion enabled robotic grasping , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[30]  Stanislav Ivanov,et al.  Adoption of robots and service automation by tourism and hospitality companies , 2017 .

[31]  Joseph DelPreto,et al.  Correcting robot mistakes in real time using EEG signals , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[32]  Brendan Z. Allison,et al.  A feasibility study on SSVEP-based interaction with motivating and immersive virtual and augmented reality , 2017, ArXiv.

[33]  Sylvain Baillet,et al.  Magnetoencephalography for brain electrophysiology and imaging , 2017, Nature Neuroscience.