Control of a humanoid robot by a noninvasive brain–computer interface in humans

We describe a brain-computer interface for controlling a humanoid robot directly using brain signals obtained non-invasively from the scalp through electroencephalography (EEG). EEG has previously been used for tasks such as controlling a cursor and spelling a word, but it has been regarded as an unlikely candidate for more complex forms of control owing to its low signal-to-noise ratio. Here we show that by leveraging advances in robotics, an interface based on EEG can be used to command a partially autonomous humanoid robot to perform complex tasks such as walking to specific locations and picking up desired objects. Visual feedback from the robot's cameras allows the user to select arbitrary objects in the environment for pick-up and transport to chosen locations. Results from a study involving nine users indicate that a command for the robot can be selected from four possible choices in 5 s with 95% accuracy. Our results demonstrate that an EEG-based brain-computer interface can be used for sophisticated robotic interaction with the environment, involving not only navigation as in previous applications but also manipulation and transport of objects.

[1]  Dana H. Ballard,et al.  Recognizing Evoked Potentials in a Virtual Environment , 1999, NIPS.

[2]  Z. Keirn,et al.  A new mode of communication between man and his surroundings , 1990, IEEE Transactions on Biomedical Engineering.

[3]  A. Graser,et al.  Low level control in a semi-autonomous rehabilitation robotic system via a Brain-Computer Interface , 2007, 2007 IEEE 10th International Conference on Rehabilitation Robotics.

[4]  Touradj Ebrahimi,et al.  Spatial filters for the classification of event-related potentials , 2006, ESANN.

[5]  H. Jasper,et al.  The ten-twenty electrode system of the International Federation. The International Federation of Clinical Neurophysiology. , 1999, Electroencephalography and clinical neurophysiology. Supplement.

[6]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[7]  J. Wolpaw,et al.  A P300 event-related potential brain–computer interface (BCI): The effects of matrix size and inter stimulus interval on performance , 2006, Biological Psychology.

[8]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[9]  M. Thulasidas,et al.  Robust classification of EEG signal for brain-computer interface , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[10]  Dean J Krusienski,et al.  A comparison of classification techniques for the P300 Speller , 2006, Journal of neural engineering.

[11]  Dawn M. Taylor,et al.  Direct Cortical Control of 3D Neuroprosthetic Devices , 2002, Science.

[12]  Nicholas G. Hatsopoulos,et al.  Brain-machine interface: Instant neural control of a movement signal , 2002, Nature.

[13]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[14]  N. Birbaumer,et al.  The thought translation device: a neurophysiological approach to communication in total motor paralysis , 1999, Experimental Brain Research.

[15]  G. Pfurtscheller,et al.  ‘Thought’ – control of functional electrical stimulation to restore hand grasp in a patient with tetraplegia , 2003, Neuroscience Letters.

[16]  Reinhold Scherer,et al.  Steady-state visual evoked potential (SSVEP)-based communication: impact of harmonic frequency components , 2005, Journal of neural engineering.

[17]  R. Andersen,et al.  Cognitive Control Signals for Neural Prosthetics , 2004, Science.

[18]  José del R. Millán,et al.  Noninvasive brain-actuated control of a mobile robot by human EEG , 2004, IEEE Transactions on Biomedical Engineering.

[19]  Jerald D. Kralik,et al.  Real-time prediction of hand trajectory by ensembles of cortical neurons in primates , 2000, Nature.

[20]  G Pfurtscheller,et al.  Current trends in Graz Brain-Computer Interface (BCI) research. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[21]  E. W. Sellers,et al.  Toward enhanced P300 speller performance , 2008, Journal of Neuroscience Methods.

[22]  N. Birbaumer,et al.  BCI2000: a general-purpose brain-computer interface (BCI) system , 2004, IEEE Transactions on Biomedical Engineering.

[23]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.