Creating the brain and interacting with the brain: an integrated approach to understanding the brain

In the past two decades, brain science and robotics have made gigantic advances in their own fields, and their interactions have generated several interdisciplinary research fields. First, in the ‘understanding the brain by creating the brain’ approach, computational neuroscience models have been applied to many robotics problems. Second, such brain-motivated fields as cognitive robotics and developmental robotics have emerged as interdisciplinary areas among robotics, neuroscience and cognitive science with special emphasis on humanoid robots. Third, in brain–machine interface research, a brain and a robot are mutually connected within a closed loop. In this paper, we review the theoretical backgrounds of these three interdisciplinary fields and their recent progress. Then, we introduce recent efforts to reintegrate these research fields into a coherent perspective and propose a new direction that integrates brain science and robotics where the decoding of information from the brain, robot control based on the decoded information and multimodal feedback to the brain from the robot are carried out in real time and in a closed loop.

[1]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[2]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[3]  M. Kawato,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[4]  Aaron M. Dollar,et al.  Lower Extremity Exoskeletons and Active Orthoses: Challenges and State-of-the-Art , 2008, IEEE Transactions on Robotics.

[5]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[6]  Stanislas Dehaene,et al.  Human Brain Project , 2014 .

[7]  Jun Morimoto,et al.  Brain-controlled exoskeleton robot for BMI rehabilitation , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[8]  P. Derambure,et al.  Does post-movement beta synchronization reflect an idling motor cortex? , 2001, Neuroreport.

[9]  Ziv M. Williams,et al.  Selective enhancement of associative learning by microstimulation of the anterior caudate , 2006, Nature Neuroscience.

[10]  Christopher G. Atkeson,et al.  Adapting human motion for the control of a humanoid robot , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[11]  J. Wolpaw,et al.  Mu and Beta Rhythm Topographies During Motor Imagery and Actual Movements , 2004, Brain Topography.

[12]  Kenji Doya,et al.  Brain mechanism of reward prediction under predictable and unpredictable environmental dynamics , 2006, Neural Networks.

[13]  Jun Morimoto,et al.  Learning from demonstration and adaptation of biped locomotion , 2004, Robotics Auton. Syst..

[14]  Mitsuo Kawato,et al.  MOSAIC for Multiple-Reward Environments , 2008 .

[15]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[16]  Shigeki Toyama,et al.  Development of Wearable-Agri-Robot ∼mechanism for agricultural work∼ , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  S. Schultz Principles of Neural Science, 4th ed. , 2001 .

[18]  Joshua G. Hale,et al.  Using Humanoid Robots to Study Human Behavior , 2000, IEEE Intell. Syst..

[19]  Christa Neuper,et al.  Level of participation in robotic-assisted treadmill walking modulates midline sensorimotor EEG rhythms in able-bodied subjects , 2012, NeuroImage.

[20]  Jun Morimoto,et al.  Decoding the ERD/ERS: influence of afferent input induced by a leg assistive robot , 2014, Front. Syst. Neurosci..

[21]  E. Fetz Operant Conditioning of Cortical Unit Activity , 1969, Science.

[22]  Stefan Schaal,et al.  A Model of Smooth Pursuit based on Learning of the Target Dynamics Using Only Retinal Signals , 2001 .

[23]  Shigenobu Kobayashi,et al.  An Analysis of Actor/Critic Algorithms Using Eligibility Traces: Reinforcement Learning with Imperfect Value Function , 1998, ICML.

[24]  Darwin G. Caldwell,et al.  Learning and Reproduction of Gestures by Imitation , 2010, IEEE Robotics & Automation Magazine.

[25]  Mitsuo Kawato,et al.  Feedback-error-learning neural network for trajectory control of a robotic manipulator , 1988, Neural Networks.

[26]  Miguel A. L. Nicolelis,et al.  Brain–machine interfaces: past, present and future , 2006, Trends in Neurosciences.

[27]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[28]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[29]  Emery N. Brown,et al.  The BRAIN Initiative: developing technology to catalyse neuroscience discovery , 2015, Philosophical Transactions of the Royal Society B: Biological Sciences.

[30]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[31]  J. Morimoto,et al.  A Biologically Inspired Biped Locomotion Strategy for Humanoid Robots: Modulation of Sinusoidal Patterns by a Coupled Oscillator Model , 2008, IEEE Transactions on Robotics.

[32]  Ales Ude,et al.  Online tracking and mimicking of human movements by a humanoid robot , 2003, Adv. Robotics.

[33]  Weiping Li,et al.  Applied Nonlinear Control , 1991 .

[34]  G Pfurtscheller,et al.  Real-time EEG analysis with subject-specific spatial patterns for a brain-computer interface (BCI). , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[35]  F. Lacquaniti,et al.  From Spinal Central Pattern Generators to Cortical Network: Integrated BCI for Walking Rehabilitation , 2012, Neural plasticity.

[36]  Yasuharu Koike,et al.  PII: S0893-6080(96)00043-3 , 1997 .

[37]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[38]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[39]  S. Amari Dynamics of pattern formation in lateral-inhibition type neural fields , 1977, Biological Cybernetics.

[40]  Christopher G. Atkeson,et al.  Methods for Motion Generation and Interaction with a Humanoid Robot: Case Studies of Dancing and Catching , 2000 .

[41]  Michael I. Jordan,et al.  An internal model for sensorimotor integration. , 1995, Science.

[42]  Ana-Maria Cebolla,et al.  Biological oscillations for learning walking coordination: dynamic recurrent neural network functionally models physiological central pattern generator , 2013, Front. Comput. Neurosci..

[43]  R T Constable,et al.  Orbitofrontal cortex neurofeedback produces lasting changes in contamination anxiety and resting-state connectivity , 2013, Translational Psychiatry.

[44]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[45]  Christopher G. Atkeson,et al.  Constructive Incremental Learning from Only Local Information , 1998, Neural Computation.

[46]  Yoshiaki Hayashi,et al.  An EMG-Based Control for an Upper-Limb Power-Assist Exoskeleton Robot , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[47]  Karl Johan Åström,et al.  Adaptive Control , 1989, Embedded Digital Control with Microcontrollers.

[48]  Wilson J. Rugh,et al.  Analytical Framework for Gain Scheduling , 1990, 1990 American Control Conference.

[49]  Jun Morimoto,et al.  Hierarchical reinforcement learning for motion learning: learning 'stand-up' trajectories , 1998, Adv. Robotics.

[50]  Klaus-Robert Müller,et al.  A regularized discriminative framework for EEG analysis with application to brain–computer interface , 2010, NeuroImage.

[51]  Kazuhito Yokoi,et al.  Biped walking pattern generation by a simple three-dimensional inverted pendulum model , 2003, Adv. Robotics.

[52]  Terrence J. Sejnowski,et al.  Variational Learning for Switching State-Space Models , 2001 .

[53]  Anna Devor,et al.  The BRAIN Initiative. , 2014, Neurophotonics.

[54]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[55]  John D E Gabrieli,et al.  Control over brain activation and pain learned by using real-time functional MRI. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[56]  Joshua G. Hale,et al.  "Sticky Hands": learning and generalization for cooperative physical interactions with a humanoid robot , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[57]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[58]  Jun Morimoto,et al.  Modulation of simple sinusoidal patterns by a coupled oscillator model for biped walking , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[59]  Jun Morimoto,et al.  Learning CPG-based biped locomotion with a policy gradient method , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[60]  E. D’Angelo The human brain project. , 2012, Functional neurology.

[61]  Steven H. Strogatz,et al.  Nonlinear Dynamics and Chaos , 2024 .

[62]  J. Ushiba,et al.  Effects of neurofeedback training with an electroencephalogram-based brain-computer interface for hand paralysis in patients with chronic stroke: a preliminary case series study. , 2011, Journal of rehabilitation medicine.

[63]  Ales Ude,et al.  Programming full-body movements for humanoid robots by observation , 2004, Robotics Auton. Syst..

[64]  A. Kral,et al.  Erratum to “Development of Brainstem-Evoked Responses in Congenital Auditory Deprivation” , 2012, Neural Plasticity.

[65]  Miguel A. L. Nicolelis,et al.  Actions from thoughts , 2001, Nature.

[66]  A. Ijspeert,et al.  From Swimming to Walking with a Salamander Robot Driven by a Spinal Cord Model , 2007, Science.

[67]  Giulio Sandini,et al.  Developmental robotics: a survey , 2003, Connect. Sci..

[68]  Kenji Doya,et al.  What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? , 1999, Neural Networks.

[69]  Stefan Schaal,et al.  A model of smooth pursuit in primates based on learning the target dynamics , 2005, Neural Networks.

[70]  Gordon Cheng,et al.  Learning tasks from observation and practice , 2004, Robotics Auton. Syst..

[71]  Yijun Wang,et al.  Implementation of a Brain-Computer Interface Based on Three States of Motor Imagery , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[72]  Mitsuo Kawato,et al.  A computational model of four regions of the cerebellum based on feedback-error learning , 2004, Biological Cybernetics.

[73]  K. Doya,et al.  A Neural Correlate of Reward-Based Behavioral Learning in Caudate Nucleus: A Functional Magnetic Resonance Imaging Study of a Stochastic Decision Task , 2004, The Journal of Neuroscience.

[74]  Jonathan R Wolpaw,et al.  Brain–computer interfaces as new brain output pathways , 2007, The Journal of physiology.

[75]  R. Goebel,et al.  Real-Time Functional Magnetic Resonance Imaging Neurofeedback for Treatment of Parkinson's Disease , 2011, The Journal of Neuroscience.

[76]  Stefan Schaal,et al.  A Kendama learning robot based on a dynamic optimization theory , 1995, Proceedings 4th IEEE International Workshop on Robot and Human Communication.

[77]  Michael I. Jordan,et al.  Dimensionality Reduction for Supervised Learning with Reproducing Kernel Hilbert Spaces , 2004, J. Mach. Learn. Res..

[78]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[79]  Mitsuo Kawato,et al.  From ‘Understanding the Brain by Creating the Brain’ towards manipulative neuroscience , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[80]  Masayuki Inaba,et al.  Learning by watching: extracting reusable task knowledge from visual observation of human performance , 1994, IEEE Trans. Robotics Autom..

[81]  Masa-aki Sato,et al.  Sparse estimation automatically selects voxels relevant for the decoding of fMRI activity patterns , 2008, NeuroImage.

[82]  Masaki Ogino,et al.  Cognitive Developmental Robotics: A Survey , 2009, IEEE Transactions on Autonomous Mental Development.

[83]  Jun Morimoto,et al.  XoR: Hybrid drive exoskeleton robot that can balance , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[84]  Jun Morimoto,et al.  Extraction of primitive representation from captured human movements and measured ground reaction force to generate physically consistent imitated behaviors , 2013, Neural Networks.

[85]  S. Grillner Neurobiological bases of rhythmic motor acts in vertebrates. , 1985, Science.

[86]  Molly M. Huntsman,et al.  Pathological Plasticity in Fragile X Syndrome , 2012, Neural plasticity.

[87]  Takahiro Kagawa,et al.  Gait pattern generation for a power-assist device of paraplegic gait , 2009, RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication.

[88]  L. Cohen,et al.  Brain–computer interfaces: communication and restoration of movement in paralysis , 2007, The Journal of physiology.

[89]  Jun Morimoto,et al.  The eMOSAIC model for humanoid robot control , 2012, Neural Networks.

[90]  G. Rizzolatti,et al.  The mirror neuron system. , 2009, Archives of neurology.

[91]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[92]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[93]  Christa Neuper,et al.  Non-invasive control of neuroprostheses for the upper extremity: Temporal coding of brain patterns , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[94]  Joel L. Davis,et al.  Adaptive Critics and the Basal Ganglia , 1995 .

[95]  R Chavarriaga,et al.  Learning From EEG Error-Related Potentials in Noninvasive Brain-Computer Interfaces , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[96]  Jun Morimoto,et al.  Low-dimensional feature extraction for humanoid locomotion using kernel dimension reduction , 2008, 2008 IEEE International Conference on Robotics and Automation.

[97]  Gordon Cheng,et al.  Learning to Act from Observation and Practice , 2004, Int. J. Humanoid Robotics.

[98]  Jun Morimoto,et al.  Task-Specific Generalization of Discrete and Periodic Dynamic Movement Primitives , 2010, IEEE Transactions on Robotics.

[99]  Mitsuo Kawato,et al.  Heterarchical reinforcement-learning model for integration of multiple cortico-striatal loops: fMRI examination in stimulus-action-reward association learning , 2006, Neural Networks.

[100]  Hiroshi Kobayashi,et al.  Muscle Suit Development and Factory Application , 2009, Int. J. Autom. Technol..

[101]  Mitsuo Kawato,et al.  MOSAIC Model for Sensorimotor Learning and Control , 2001, Neural Computation.

[102]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[103]  Satinder Singh Transfer of Learning by Composing Solutions of Elemental Sequential Tasks , 1992, Mach. Learn..

[104]  Jun Morimoto,et al.  CB: A Humanoid Research Platform for Exploring NeuroScience , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[105]  Kenji Doya,et al.  Reinforcement Learning in Continuous Time and Space , 2000, Neural Computation.

[106]  M. Gazzaniga,et al.  Cognitive Neuroscience: The Biology of the Mind , 1998 .

[107]  G. Pfurtscheller,et al.  ‘Thought’ – control of functional electrical stimulation to restore hand grasp in a patient with tetraplegia , 2003, Neuroscience Letters.

[108]  M. Kawato,et al.  Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning. , 2006, Journal of neurophysiology.

[109]  Stefan Schaal,et al.  Biomimetic gaze stabilization based on feedback-error-learning with nonparametric regression networks , 2001, Neural Networks.

[110]  D Normile New Institute Seen as Brains Behind Big Boost in Spending , 1997, Science.

[111]  Jun Morimoto,et al.  Learning CPG-based Biped Locomotion with a Policy Gradient Method: Application to a Humanoid Robot , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[112]  Timothy E. J. Behrens,et al.  Optimal decision making and the anterior cingulate cortex , 2006, Nature Neuroscience.

[113]  Jun Morimoto,et al.  Learning Biped Locomotion , 2007, IEEE Robotics & Automation Magazine.

[114]  J. V. Basmajian,et al.  Control and Training of Individual Motor Units , 1963, Science.

[115]  Yasuhisa Hasegawa,et al.  Intention-based walking support for paraplegia patients with Robot Suit HAL , 2007, Adv. Robotics.

[116]  Jun Morimoto,et al.  Minimax Differential Dynamic Programming: An Application to Robust Biped Walking , 2002, NIPS.

[117]  Takeo Watanabe,et al.  Perceptual learning incepted by decoded fMRI neurofeedback without stimulus presentation , 2012 .