Brain controlled robots

In January 2008, Duke University and the Japan Science and Technology Agency (JST) publicized their successful control of a brain‐machine interface for a humanoid robot by a monkey brain across the Pacific Ocean. The activities of a few hundred neurons were recorded from a monkey's motor cortex in Miguel Nicolelis's lab at Duke University, and the kinematic features of monkey locomotion on a treadmill were decoded from neural firing rates in real time. The decoded information was sent to a humanoid robot, CB‐i, in ATR Computational Neuroscience Laboratories located in Kyoto, Japan. This robot was developed by the JST International Collaborative Research Project (ICORP) as the “Computational Brain Project.” CB‐i's locomotion‐like movement was video‐recorded and projected on a screen in front of the monkey. Although the bidirectional communication used a conventional Internet connection, its delay was suppressed below one over several seconds, partly due to a video‐streaming technique, and this encouraged the monkey's voluntary locomotion and influenced its brain activity. This commentary introduces the background and future directions of the brain‐controlled robot.

[1]  A P Georgopoulos,et al.  On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[2]  Yasuharu Koike,et al.  Prediction of arm trajectory from a small number of neuron activities in the primary motor cortex , 2006, Neuroscience Research.

[3]  F. Tong,et al.  Decoding Seen and Attended Motion Directions from Activity in the Human Visual Cortex , 2006, Current Biology.

[4]  Jun Morimoto,et al.  Learning CPG Sensory Feedback with Policy Gradient for Biped Locomotion for a Full-Body Humanoid , 2005, AAAI.

[5]  Jun Morimoto,et al.  Learning CPG-based biped locomotion with a policy gradient method , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[6]  Dawn M. Taylor,et al.  Direct Cortical Control of 3D Neuroprosthetic Devices , 2002, Science.

[7]  Stefan Schaal,et al.  A Model of Smooth Pursuit based on Learning of the Target Dynamics Using Only Retinal Signals , 2001 .

[8]  Joshua G. Hale,et al.  Using Humanoid Robots to Study Human Behavior , 2000, IEEE Intell. Syst..

[9]  Masa-aki Sato,et al.  Hierarchical Bayesian estimation for MEG inverse problem , 2004, NeuroImage.

[10]  Stefan Schaal,et al.  Biomimetic gaze stabilization based on feedback-error-learning with nonparametric regression networks , 2001, Neural Networks.

[11]  Jun Morimoto,et al.  CB: A Humanoid Research Platform for Exploring NeuroScience , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[12]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[13]  Masa-aki Sato,et al.  Reconstruction of Temporal Movement from Single-trial Non-invasive Brain Activity: A Hierarchical Bayesian Method , 2007, ICONIP.

[14]  Ales Ude,et al.  Online tracking and mimicking of human movements by a humanoid robot , 2003, Adv. Robotics.

[15]  Jun Morimoto,et al.  Modulation of simple sinusoidal patterns by a coupled oscillator model for biped walking , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[16]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[17]  Yasuharu Koike,et al.  PII: S0893-6080(96)00043-3 , 1997 .

[18]  Miguel A. L. Nicolelis,et al.  Actions from thoughts , 2001, Nature.

[19]  Joshua G. Hale,et al.  "Sticky Hands": learning and generalization for cooperative physical interactions with a humanoid robot , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[20]  E. Todorov.  Direct cortical control of muscle activation in voluntary arm movements: a model , 2000, Nature Neuroscience.

[21]  Christopher G. Atkeson,et al.  Adapting human motion for the control of a humanoid robot , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[22]  E. Fetz.  Operant Conditioning of Cortical Unit Activity , 1969, Science.

[23]  Christopher G. Atkeson,et al.  Constructive Incremental Learning from Only Local Information , 1998, Neural Computation.

[24]  Christopher G. Atkeson,et al.  Methods for Motion Generation and Interaction with a Humanoid Robot: Case Studies of Dancing and Catching , 2000 .

[25]  Jun Nakanishi,et al.  Control, Planning, Learning, and Imitation with Dynamic Movement Primitives , 2003 .

[26]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[27]  Gordon Cheng,et al.  Learning tasks from observation and practice , 2004, Robotics Auton. Syst..

[28]  Ales Ude,et al.  Programming full-body movements for humanoid robots by observation , 2004, Robotics Auton. Syst..

[29]  Stefan Schaal,et al.  A Kendama learning robot based on a dynamic optimization theory , 1995, Proceedings 4th IEEE International Workshop on Robot and Human Communication.

[30]  M. Kawato,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[31]  Jun Morimoto,et al.  Learning from demonstration and adaptation of biped locomotion , 2004, Robotics Auton. Syst..

[32]  Stefan Schaal,et al.  A model of smooth pursuit in primates based on learning the target dynamics , 2005, Neural Networks.

[33]  Mitsuo Kawato,et al.  From ‘Understanding the Brain by Creating the Brain’ towards manipulative neuroscience , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[34]  Mitsuo Kawato,et al.  Feedback-error-learning neural network for trajectory control of a robotic manipulator , 1988, Neural Networks.

[35]  Gordon Cheng,et al.  Learning to Act from Observation and Practice , 2004, Int. J. Humanoid Robotics.